I am receiving data over UDP in PHP.
After passing the buffer through bin2hex function in PHP I end up with something resembling the Raw Data as indicated below in the example
Here is my PHP code:
$socket = socket_create(AF_INET, SOCK_DGRAM, SOL_UDP);
socket_bind($socket, "0.0.0.0" , 20500);
$from = '';
$port = 0;
socket_recvfrom($socket, $buf, 512, 0, $from, $port);
$file = 'testudp.txt';
// Open the file to get existing content
$current = file_get_contents($file);
// Append a new log line(s) to file
$current .= "New Connection " . date('Y-m-d H:i:s') . " \n" . bin2hex($buf) . "\n";
// Write the contents back to the file
file_put_contents($file, $current);
echo "Received connection from remote address $from and remote port $port" . PHP_EOL;
Which results in something like 83 05 01 02 03 04 05 01 01 01 02 00 01 4F B4 64 88 4F B4 64 88 13 BF 71 A8 BA 18 A5 06 00 00 1333 00 00 00 00 11 11 02 ...
but without the spaces.
Example of Data and what it ought to translate to: note that the translation only begins after ther first 9 bytes, then address 2, then skips 2 again.
Raw Data:
83 05 01 02 03 04 05 01 01 01 02 00 01 4F B4 64 88 4F B4 64
88 13 BF 71 A8 BA 18 A5 06 00 00 1333 00 00 00 00 11 11 02
33 44 44 55 55 66 77 88 99 10 11 ?? 00 ??
Decoded:
-------Message Header--------
01 Service Type, for this message 1 = Acknowledged Request
02 Message Type, for this message 2 = Event Report
-------Event Report----------
4FB46488 Update Time (5/17/12 # 2:38pm)
4FB46488 Time of Fix (5/17/12 # 2:38pm)
13BF71A8 Latitude (33.1313576)
BA18A506 Longitude (-117.2790010)
00001333 Altitude
00000000 Speed
1111 Heading
02 Satellites
33 Fix Status
4444 Carrier
5555 RSSI
66 Comm State
77 HDOP
88 Inputs
99 Unit Status
10 Event Index
11 Event Code
?? Accums (# of 4-byte Accum List values)
00 Spare
?? Accum List (Varies depending on the # of Accums reporting)
I have tried various decoders hexadecimal to ascii, I've also tried unpack() in PHP all to no avail.
How do you translate the data into something resembling the example via PHP?
i.e. how would BA18A506 end up giving me the data (unformatted or not) to correspond to Longitude (-117.2790010) in a human readable form?
EDIT:
The docs specify Note that all bytes in multi-byte fields are transmitted in Net Endian format (Big Endian) where the most significant bits are transmitted first. For example, for a 32-bit field, bits 31-24 are transmitted first, 16-23 second, 8-15 third and 0-7 last
if that helps at all.
unpack() or unpack() per substr() of the binary data should in the end get you where you want. But as HamZa mentions, you have to know what format it is...
$test = "\x06\xA5\x18\xBA";
var_dump(unpack('l',$test));
result:
-1172790010
Just add the . on the correct place
Sample code to unpack first part (extend / correct where needed)
Change your code to recieve into a buffer instead of a file:
socket_recv($socket, $buf, 512, 0);
Then you can use the buffer $buf (for now i enter the sample data you provided)
// Now we use some test data
$buf = "\x83\x05\x01\x02\x03\x04\x05\x01\x01\x01\x02\x00\x01\x4F\xB4\x64\x88\x4F\xB4\x64\x88\x13\xBF\x71\xA8\xBA\x18\xA5\x06\x00\x00\x13\x33\x00\x00\x00\x00\x11\x11\x02\x33\x44\x44\x55\x55\x66\x77\x88\x99\x10\x11";
// field definition
$fields = array();
$fields['header'] = array(9, 'c*');
$fields['serviceType'] = array(1, 'c');
$fields['messageType'] = array(1, 'c');
$fields['skip'] = array(2, 'c');
$fields['updateTime'] = array(4, 'l');
$fields['fixTime'] = array(4, 'l');
$fields['longitude'] = array(4, 'l');
$fields['latitude'] = array(4, 'l');
$fields['altitude'] = array(4, 'l');
$fields['speed'] = array(4, 'l');
// etc.
$values = array();
$start = 0;
foreach($fields as $type=>$setting) {
list($len, $format) = $setting;
$values[$type] = unpack($format, strrev(substr($buf, $start, $len)));
$start += $len;
}
var_dump($values);
Will output something like:
["serviceType"]=>
array(1) {
[1]=>
int(1)
}
["messageType"]=>
array(1) {
[1]=>
int(2)
}
["skip"]=>
array(1) {
[1]=>
int(1)
}
["updateTime"]=>
array(1) {
[1]=>
int(1337222280)
}
["fixTime"]=>
array(1) {
[1]=>
int(1337222280)
}
["longitude"]=>
array(1) {
[1]=>
int(331313576)
}
["latitude"]=>
array(1) {
[1]=>
int(-1172790010)
}
["altitude"]=>
array(1) {
[1]=>
int(4915)
}
["speed"]=>
array(1) {
[1]=>
int(0)
}
Just divide latitude and longitude by 10000000 (10M) to get the real longitude and latitude
Related
PHP has a method hash_hmac that computes the HMAC signature of a given string using a given key and algorithm. But HMAC technically operates on binary data, and PHP takes all its params here as strings. How does it convert those strings to binary data?
Short answer: String encoding is just metadata attached to a lump of binary data. PHP strings are just the lump, you have to keep track of the rest.
Long answer:
PHP takes the Honey Badger approach to native string encodings, in other words, "PHP don't care". You give it a sequence of bytes, it stores them. It has no concept of encoding until you want to use a function that cares about it. Even then you need to explicitly declare the input and output encodings, otherwise PHP will go with its configured default which is usually not what anyone actually wants.
function nice_hex($in) {
return implode(' ', str_split(bin2hex($in), 2));
}
$utf8 = "You owe me €5.";
$utf16le = mb_convert_encoding($utf8, 'utf-16le', 'utf-8');
$utf16be = mb_convert_encoding($utf8, 'utf-16be', 'utf-8');
$iso88591 = mb_convert_encoding($utf8, 'iso-8859-1', 'utf-8');
$cp1252 = mb_convert_encoding($utf8, 'cp1252', 'utf-8');
var_dump(
$utf8,
nice_hex($utf8),
hash_hmac('md5', $utf8, 'foo'),
$utf16le,
nice_hex($utf16le),
hash_hmac('md5', $utf16le, 'foo'),
$utf16be,
nice_hex($utf16be),
hash_hmac('md5', $utf16be, 'foo'),
$iso88591,
nice_hex($iso88591),
hash_hmac('md5', $iso88591, 'foo'),
$cp1252,
nice_hex($cp1252),
hash_hmac('md5', $cp1252, 'foo')
);
Output:
string(16) "You owe me €5."
string(47) "59 6f 75 20 6f 77 65 20 6d 65 20 e2 82 ac 35 2e"
string(32) "7724135d91c43906f8730a26dcd76ffb"
string(28) "You owe me � 5."
string(83) "59 00 6f 00 75 00 20 00 6f 00 77 00 65 00 20 00 6d 00 65 00 20 00 ac 20 35 00 2e 00"
string(32) "f4a2347b4a1336dae1db21554c54b9e2"
string(28) "You owe me �5."
string(83) "00 59 00 6f 00 75 00 20 00 6f 00 77 00 65 00 20 00 6d 00 65 00 20 20 ac 00 35 00 2e"
string(32) "b0c1a98d8b853e6568bae513d764a029"
string(14) "You owe me ?5."
string(41) "59 6f 75 20 6f 77 65 20 6d 65 20 3f 35 2e"
string(32) "301a0fb55e23285904413323d10cc774"
string(14) "You owe me �5."
string(41) "59 6f 75 20 6f 77 65 20 6d 65 20 80 35 2e"
string(32) "fa1ee73d39e1a70fe2cde7a8c5bbf0ba"
And the reason why that all looks like it does is because:
StackOverflow uses UTF-8.
My editor uses UTF-8.
My console uses UTF-8.
The fact that PHP doesn't care about string encoding lets me produce arbitrarily-encoded trash output like the above quite easily.
Additional recommended reading: UTF-8 all the way through
Fun Fact: One of the reasons why PHP6 never ended up happening was because they wanted to include native multibyte string encoding but no one could agree on what flavor it should be. Eventually they just scrapped the whole thing and left it up to us the same as it was in PHP5.
It's just UTF-8 (for string literals).
You can put whatever encoding you want in a string, hash_hmac() doesn't use any specific encoding, just whatever encoding your string has.
Here's an example from Wikipedia using UTF-8 encoding and running a HMAC algorithm over the binary:
HMAC_MD5("key", "The quick brown fox jumps over the lazy dog") = 80070713463e7749b90c2dc24911e275
And here's the result of the equivalent PHP code, which gets the same response:
php > echo hash_hmac('md5', "The quick brown fox jumps over the lazy dog", "key");
80070713463e7749b90c2dc24911e275
I am attempting to replicate a PHP hash generation function in Node. This hash is used as part of an API. The PHP version creates the correct output that is accepted by the system. The Node version creates a different output despite what I believe to be the same inputs on the functions.
Is this because there is some fundamentally different way the PHP and Node HMAC functions work? Or is it because of some quirk with character encoding that I am just missing? Or have I just simply messed up something else?
PHP Code
$url = 'https://example.com/api/endpoint';
$user = 'apiuser';
// Example key
$key = '+raC8YR2F+fZypNJ5q+CAlqLFqNN1AlAfWwkwJLcI7jrAvppjRPikWp523G/u0BLSpN9+2LusJvpSwrfU9X2uA==';
$timestamp = gmdate('D, d M Y H:i:s T', 1543554184); // gmdate('D, d M Y H:i:s T');
$hashdata = "GET\n$url\n$user\n$timestamp\n";
print_r($hashdata);
/*
GET
https://example.com/api/endpoint
apiuser
Fri, 30 Nov 2018 05:03:04 GMT
*/
$decoded_key = base64_decode($key);
print_r(unpack('H*', $decoded_key));
// Array ( [1] => fab682f1847617e7d9ca9349e6af82025a8b16a34dd409407d6c24c092dc23b8eb02fa698d13e2916a79db71bfbb404b4a937dfb62eeb09be94b0adf53d5f6b8 )
$generated_hash = hash_hmac('sha256', $hashdata, $decoded_key, true);
$encoded_hash = base64_encode($generated_hash);
print_r($encoded_hash);
// vwdT8XhtSA1q+JvAfsRpJumfI4pemoaNFbjjc5JFsvw=
Node.js Code
crypto = require('crypto');
moment = require('moment-timezone');
let url = 'https://example.com/api/endpoint';
let api_user = 'apiuser';
// Example key
let api_key = '+raC8YR2F+fZypNJ5q+CAlqLFqNN1AlAfWwkwJLcI7jrAvppjRPikWp523G/u0BLSpN9+2LusJvpSwrfU9X2uA==';
let timestamp = moment.tz(1543554184 * 1000, 'GMT').format('ddd, DD MMM YYYY HH:mm:ss z'); // moment.tz(new Date(), 'GMT').format('ddd, DD MMM YYYY HH:mm:ss z');
let hash_data = 'GET\n' + url + '\n' + api_user + '\n' + timestamp + '\n';
console.log($hashdata);
/*
GET
https://example.com/api/endpoint
apiuser
Fri, 30 Nov 2018 05:03:04 GMT
*/
let decoded_key = Buffer.from(api_key, 'base64').toString('utf8');
console.log(Buffer.from(api_key, 'base64'));
// <Buffer fa b6 82 f1 84 76 17 e7 d9 ca 93 49 e6 af 82 02 5a 8b 16 a3 4d d4 09 40 7d 6c 24 c0 92 dc 23 b8 eb 02 fa 69 8d 13 e2 91 6a 79 db 71 bf bb 40 4b 4a 93 ... >
const hmac = crypto.createHmac('sha256', decoded_key);
hmac.update(hash_data);
// Not sure which should be closest to PHP
// Or if there is a difference
let encoded_hash = hmac.digest('base64');
// let encoded_hash = Buffer(hmac.digest('binary')).toString('base64');
console.log(encoded_hash);
// hmac.digest('base64') == eLLVC9cUvq6Ber6t9TBTihSoq+2VWIMUJKiL4/fIj3s=
// Buffer(hmac.digest('binary')).toString('base64') == eLLVC9cUvq6Ber6t9TBTihSoq+2VWIMUJKiL4/fIj3s=
Everything besides the HMAC functions output seems to be the same.
OS: Windows 10 - 64 Bit
Node.js Version: v10.13.0
PHP Version: 7.2.7
I can get the correct result in Node.js by keeping decoded_key a Buffer, and sending it directly as a Buffer to crypto.createHmac:
let decoded_key = Buffer.from(api_key, 'base64');
const hmac = crypto.createHmac('sha256', decoded_key);
This is supported , see crypto.createHmac:
key <string> | <Buffer> | <TypedArray> | <DataView>
Result is vwdT8XhtSA1q+JvAfsRpJumfI4pemoaNFbjjc5JFsvw= - same as PHP.
Working example: https://repl.it/repls/DisguisedBlankTechnologies
The problem must be with .toString('utf8'). I didn't find another encoding the works as a string, but it works just as well as a Buffer.
For completeness, another option supported by the Crypto module:
const hmac = crypto.createHmac('sha256', decoded_key);
hmac.write(hash_data);
hmac.end();
let encoded_hash = hmac.read().toString('base64');
Working example: https://repl.it/repls/LightcoralUnwelcomeProfessionals
I'm trying to learn binary and create a simple WebM parser in PHP based on Matroska.
I read TimecodeScale, MuxingAppm WritingApp, etc. with unpack(format, data). My problem is when I reach Duration (0x4489) in Segment Information (0x1549a966) I must read a float and based on TimecodeScale convert it to seconds: 261.564s->00:04:21.564 and I don't know how.
This is a sample sequence:
`2A D7 B1 83 0F 42 40 4D 80 86 67 6F 6F 67 6C 65 57 41 86 67 6F 6F 67 6C 65 44 89 88 41 0F ED E0 00 00 00 00 16 54 AE 6B`
TimecodeScale := 2ad7b1 uint [ def:1000000; ]
MuxingApp := 4d80 string; ("google")
WritingApp := 5741 string; ("google")
Duration := 4489 float [ range:>0.0; ]
Tracks := 1654ae6b container [ card:*; ]{...}
I must read a float after (0x4489) and return 261.564s.
The duration is a double precision floating point value (64-bits) represented in the IEEE 754 format. If you want to see how the conversion is done check this.
The TimecodeScale is the timestamp scale in nanoseconds.
In php you can do:
$bin = hex2bin('410fede000000000');
$timecode_scale = 1e6;
// endianness
if (unpack('S', "\xff\x00")[1] === 0xff) {
$bytes = unpack('C8', $bin);
$bytes = array_reverse($bytes);
$bin = implode('', array_map('chr', $bytes));
}
$duration = unpack('d', $bin)[1];
$duration_s = $duration * $timecode_scale / 1e9;
echo "duration=${duration_s}s\n";
Result:
duration=261.564s
I read bytes from a file and process them. Afterwards I would like to save the packed bytes.
What is the recommended+generic way to convert an array with mixed objects/types to a byte string? In my case: array with int and string, pack types a,C,x.
A simplified example:
// $bytes = fread($handle, 100);
$bytes = "437XYZ25.011001DBEFORE ....";
$unpackString = "a3CPN/x8spare/CDSC/x4spare/a32OPT";
$unpacked = unpack($unpackString, $bytes);
var_dump($unpacked);
/*
array(3) {
["CPN"]=> string(3) "437"
["DSC"]=> int(49)
["OPT"]=> string(32) "BEFORE "
}
*/
// example of processing
$unpacked["DSC"] = 12;
$unpacked["OPT"] = "AFTER ";
// pack + write the result
// $packString = "a3x8Cx4a32";
$packTypes = ["a3","x8","C","x4","a32"];
$packFields = [ $unpacked["CPN"], null, $unpacked["DSC"], null, $unpacked["OPT"] ];
// ...
update: in the simplified example I have replaced $packString with $packTypes and $packFields to make sure it is clear what content belongs where and with what type.
I suppose what you're looking for is a way to call pack, which accepts arguments with an associative array as you have in your example. For this, we can use call_user_func_array which calls a function by its name and provides its arguments from a given array.
$bytes = "437XYZ25.011001DBEFORE ....";
$unpackString = "a3CPN/x8spare/CDSC/x4spare/a32OPT";
$unpacked = unpack($unpackString, $bytes);
// example of processing
$unpacked["DSC"] = 12;
$unpacked["OPT"] = "AFTER ";
// pack + write the result
$packTypes = ["a3", "x8", "C", "x4", "a32"];
$packFields = [$unpacked["CPN"], null, $unpacked["DSC"], null, $unpacked["OPT"]];
$packString = "";
$packArguments = [];
for ($i = 0; $i < count($packTypes); $i++){
$packString .= $packTypes[$i];
if ($packFields[$i] !== null){
// the null bytes don't use an argument
$packArguments[] = $packFields[$i];
}
}
// put packString as the first argument
array_unshift($packArguments, $packString);
$output = call_user_func_array("pack", $packArguments);
And $output would then be:
00000000 34 33 37 00 00 00 00 00 00 00 00 0c 00 00 00 00 |437.............|
00000010 41 46 54 45 52 20 20 20 20 20 20 20 20 20 20 20 |AFTER |
00000020 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 | |
00000030
How can I encode strings on UTF-16BE format in PHP? For "Demo Message!!!" the encoded string should be '00440065006D006F0020004D00650073007300610067006'. Also, I need to encode Arabic characters to this format.
First of all, this is absolutly not UTF-8, which is just a charset (i.e. a way to store strings in memory / display them).
WHat you have here looks like a dump of the bytes that are used to build each characters.
If so, you could get those bytes this way :
$str = utf8_encode("Demo Message!!!");
for ($i=0 ; $i<strlen($str) ; $i++) {
$byte = $str[$i];
$char = ord($byte);
printf('%02x ', $char);
}
And you'd get the following output :
44 65 6d 6f 20 4d 65 73 73 61 67 65 21 21 21
But, once again, this is not UTF-8 : in UTF-8, like you can see in the example I've give, D is stored on only one byte : 0x44
In what you posted, it's stored using two Bytes : 0x00 0x44.
Maybe you're using some kind of UTF-16 ?
EDIT after a bit more testing and #aSeptik's comment : this is indeed UTF-16.
To get the kind of dump you're getting, you'll have to make sure your string is encoded in UTF-16, which could be done this way, using, for example, the mb_convert_encoding function :
$str = mb_convert_encoding("Demo Message!!!", 'UTF-16', 'UTF-8');
Then, it's just a matter of iterating over the bytes that make this string, and dumping their values, like I did before :
for ($i=0 ; $i<strlen($str) ; $i++) {
$byte = $str[$i];
$char = ord($byte);
printf('%02x ', $char);
}
And you'll get the following output :
00 44 00 65 00 6d 00 6f 00 20 00 4d 00 65 00 73 00 73 00 61 00 67 00 65 00 21 00 21 00 21
Which kind of looks like what youy posted :-)
(you just have to remove the space in the call to printf -- I let it there to get an easier to read output=)
E.g. by using the mbstring extension and its mb_convert_encoding() function.
$in = 'Demo Message!!!';
$out = mb_convert_encoding($in, 'UTF-16BE');
for($i=0; $i<strlen($out); $i++) {
printf("%02X ", ord($out[$i]));
}
prints
00 44 00 65 00 6D 00 6F 00 20 00 4D 00 65 00 73 00 73 00 61 00 67 00 65 00 21 00 21 00 21
Or by using iconv()
$in = 'Demo Message!!!';
$out = iconv('iso-8859-1', 'UTF-16BE', $in);
for($i=0; $i<strlen($out); $i++) {
printf("%02X ", ord($out[$i]));
}