use PHP to unpack, process and pack array - php

I read bytes from a file and process them. Afterwards I would like to save the packed bytes.
What is the recommended+generic way to convert an array with mixed objects/types to a byte string? In my case: array with int and string, pack types a,C,x.
A simplified example:
// $bytes = fread($handle, 100);
$bytes = "437XYZ25.011001DBEFORE ....";
$unpackString = "a3CPN/x8spare/CDSC/x4spare/a32OPT";
$unpacked = unpack($unpackString, $bytes);
var_dump($unpacked);
/*
array(3) {
["CPN"]=> string(3) "437"
["DSC"]=> int(49)
["OPT"]=> string(32) "BEFORE "
}
*/
// example of processing
$unpacked["DSC"] = 12;
$unpacked["OPT"] = "AFTER ";
// pack + write the result
// $packString = "a3x8Cx4a32";
$packTypes = ["a3","x8","C","x4","a32"];
$packFields = [ $unpacked["CPN"], null, $unpacked["DSC"], null, $unpacked["OPT"] ];
// ...
update: in the simplified example I have replaced $packString with $packTypes and $packFields to make sure it is clear what content belongs where and with what type.

I suppose what you're looking for is a way to call pack, which accepts arguments with an associative array as you have in your example. For this, we can use call_user_func_array which calls a function by its name and provides its arguments from a given array.
$bytes = "437XYZ25.011001DBEFORE ....";
$unpackString = "a3CPN/x8spare/CDSC/x4spare/a32OPT";
$unpacked = unpack($unpackString, $bytes);
// example of processing
$unpacked["DSC"] = 12;
$unpacked["OPT"] = "AFTER ";
// pack + write the result
$packTypes = ["a3", "x8", "C", "x4", "a32"];
$packFields = [$unpacked["CPN"], null, $unpacked["DSC"], null, $unpacked["OPT"]];
$packString = "";
$packArguments = [];
for ($i = 0; $i < count($packTypes); $i++){
$packString .= $packTypes[$i];
if ($packFields[$i] !== null){
// the null bytes don't use an argument
$packArguments[] = $packFields[$i];
}
}
// put packString as the first argument
array_unshift($packArguments, $packString);
$output = call_user_func_array("pack", $packArguments);
And $output would then be:
00000000 34 33 37 00 00 00 00 00 00 00 00 0c 00 00 00 00 |437.............|
00000010 41 46 54 45 52 20 20 20 20 20 20 20 20 20 20 20 |AFTER |
00000020 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 | |
00000030

Related

PHP encoding problem: how to encode base64 encoded uint8 bytes to hex

My problem is that I was some time ago base64 encoding random bytes from openssl sha256 in C (as uint8_t), feeding them into a shell script and using the output.
What I can recreate from my data now is:
Content of file.txt:
uvjWEHTUk1LnzVZul9ynRpezWfKYN3bvlx103wxACxo
test#test:~# base64 -d file.txt | od -t x1
0000000 ba f8 d6 10 74 d4 93 52 e7 cd 56 6e 97 dc a7 46
0000020 97 b3 59 f2 98 37 76 ef 97 1d 74 df 0c 40 0b 1a
The output is the same as calling in PHP:
echo bin2hex(base64_decode("uvjWEHTUk1LnzVZul9ynRpezWfKYN3bvlx103wxACxo="));
baf8d61074d49352e7cd566e97dca74697b359f2983776ef971d74df0c400b1a
What I did all the time in shell and need to do now in PHP is the following:
Again, same content of file.txt:
uvjWEHTUk1LnzVZul9ynRpezWfKYN3bvlx103wxACxo
test#test:~# base64 -d file.txt | od -t x8
0000000 5293d47410d6f8ba 46a7dc976e56cde7
0000020 ef763798f259b397 1a0b400cdf741d97
My problem here: what is now the equal procedure in PHP (to od -t x8 in shell)?
I tried pack / unpack / bin2hex / ... and can't get the same result.
I'm trying to get a string with this content:
"5293d47410d6f8ba46a7dc976e56cde7ef763798f259b3971a0b400cdf741d97"
from a starting point of base64_decode("uvjWEHTUk1LnzVZul9ynRpezWfKYN3bvlx103wxACxo="). Any ideas?
If x8 is what you really need, which is 8 bytes, then the implementation would be as simple as
<?php
$str = 'uvjWEHTUk1LnzVZul9ynRpezWfKYN3bvlx103wxACxo';
$bin = base64_decode($str);
if (strlen($bin) % 8 !== 0) {
throw new \RuntimeException('data length should be divisible by 8');
}
$result = '';
for ($i = 0; $i < strlen($bin); $i += 8) {
for ($j = $i + 7; $j >= $i; --$j) {
$result .= bin2hex($bin[$j]);
}
}
echo $result;
It iterates over blocks of 8 bytes, then dumps them in reverse order each.
Ideone: https://ideone.com/hBanqi

How to read a float from binary WebM file?

I'm trying to learn binary and create a simple WebM parser in PHP based on Matroska.
I read TimecodeScale, MuxingAppm WritingApp, etc. with unpack(format, data). My problem is when I reach Duration (0x4489) in Segment Information (0x1549a966) I must read a float and based on TimecodeScale convert it to seconds: 261.564s->00:04:21.564 and I don't know how.
This is a sample sequence:
`2A D7 B1 83 0F 42 40 4D 80 86 67 6F 6F 67 6C 65 57 41 86 67 6F 6F 67 6C 65 44 89 88 41 0F ED E0 00 00 00 00 16 54 AE 6B`
TimecodeScale := 2ad7b1 uint [ def:1000000; ]
MuxingApp := 4d80 string; ("google")
WritingApp := 5741 string; ("google")
Duration := 4489 float [ range:>0.0; ]
Tracks := 1654ae6b container [ card:*; ]{...}
I must read a float after (0x4489) and return 261.564s.
The duration is a double precision floating point value (64-bits) represented in the IEEE 754 format. If you want to see how the conversion is done check this.
The TimecodeScale is the timestamp scale in nanoseconds.
In php you can do:
$bin = hex2bin('410fede000000000');
$timecode_scale = 1e6;
// endianness
if (unpack('S', "\xff\x00")[1] === 0xff) {
$bytes = unpack('C8', $bin);
$bytes = array_reverse($bytes);
$bin = implode('', array_map('chr', $bytes));
}
$duration = unpack('d', $bin)[1];
$duration_s = $duration * $timecode_scale / 1e9;
echo "duration=${duration_s}s\n";
Result:
duration=261.564s

Translate UDP packet data into human-readable form in PHP

I am receiving data over UDP in PHP.
After passing the buffer through bin2hex function in PHP I end up with something resembling the Raw Data as indicated below in the example
Here is my PHP code:
$socket = socket_create(AF_INET, SOCK_DGRAM, SOL_UDP);
socket_bind($socket, "0.0.0.0" , 20500);
$from = '';
$port = 0;
socket_recvfrom($socket, $buf, 512, 0, $from, $port);
$file = 'testudp.txt';
// Open the file to get existing content
$current = file_get_contents($file);
// Append a new log line(s) to file
$current .= "New Connection " . date('Y-m-d H:i:s') . " \n" . bin2hex($buf) . "\n";
// Write the contents back to the file
file_put_contents($file, $current);
echo "Received connection from remote address $from and remote port $port" . PHP_EOL;
Which results in something like 83 05 01 02 03 04 05 01 01 01 02 00 01 4F B4 64 88 4F B4 64 88 13 BF 71 A8 BA 18 A5 06 00 00 1333 00 00 00 00 11 11 02 ...
but without the spaces.
Example of Data and what it ought to translate to: note that the translation only begins after ther first 9 bytes, then address 2, then skips 2 again.
Raw Data:
83 05 01 02 03 04 05 01 01 01 02 00 01 4F B4 64 88 4F B4 64
88 13 BF 71 A8 BA 18 A5 06 00 00 1333 00 00 00 00 11 11 02
33 44 44 55 55 66 77 88 99 10 11 ?? 00 ??
Decoded:
-------Message Header--------
01 Service Type, for this message 1 = Acknowledged Request
02 Message Type, for this message 2 = Event Report
-------Event Report----------
4FB46488 Update Time (5/17/12 # 2:38pm)
4FB46488 Time of Fix (5/17/12 # 2:38pm)
13BF71A8 Latitude (33.1313576)
BA18A506 Longitude (-117.2790010)
00001333 Altitude
00000000 Speed
1111 Heading
02 Satellites
33 Fix Status
4444 Carrier
5555 RSSI
66 Comm State
77 HDOP
88 Inputs
99 Unit Status
10 Event Index
11 Event Code
?? Accums (# of 4-byte Accum List values)
00 Spare
?? Accum List (Varies depending on the # of Accums reporting)
I have tried various decoders hexadecimal to ascii, I've also tried unpack() in PHP all to no avail.
How do you translate the data into something resembling the example via PHP?
i.e. how would BA18A506 end up giving me the data (unformatted or not) to correspond to Longitude (-117.2790010) in a human readable form?
EDIT:
The docs specify Note that all bytes in multi-byte fields are transmitted in Net Endian format (Big Endian) where the most significant bits are transmitted first. For example, for a 32-bit field, bits 31-24 are transmitted first, 16-23 second, 8-15 third and 0-7 last
if that helps at all.
unpack() or unpack() per substr() of the binary data should in the end get you where you want. But as HamZa mentions, you have to know what format it is...
$test = "\x06\xA5\x18\xBA";
var_dump(unpack('l',$test));
result:
-1172790010
Just add the . on the correct place
Sample code to unpack first part (extend / correct where needed)
Change your code to recieve into a buffer instead of a file:
socket_recv($socket, $buf, 512, 0);
Then you can use the buffer $buf (for now i enter the sample data you provided)
// Now we use some test data
$buf = "\x83\x05\x01\x02\x03\x04\x05\x01\x01\x01\x02\x00\x01\x4F\xB4\x64\x88\x4F\xB4\x64\x88\x13\xBF\x71\xA8\xBA\x18\xA5\x06\x00\x00\x13\x33\x00\x00\x00\x00\x11\x11\x02\x33\x44\x44\x55\x55\x66\x77\x88\x99\x10\x11";
// field definition
$fields = array();
$fields['header'] = array(9, 'c*');
$fields['serviceType'] = array(1, 'c');
$fields['messageType'] = array(1, 'c');
$fields['skip'] = array(2, 'c');
$fields['updateTime'] = array(4, 'l');
$fields['fixTime'] = array(4, 'l');
$fields['longitude'] = array(4, 'l');
$fields['latitude'] = array(4, 'l');
$fields['altitude'] = array(4, 'l');
$fields['speed'] = array(4, 'l');
// etc.
$values = array();
$start = 0;
foreach($fields as $type=>$setting) {
list($len, $format) = $setting;
$values[$type] = unpack($format, strrev(substr($buf, $start, $len)));
$start += $len;
}
var_dump($values);
Will output something like:
["serviceType"]=>
array(1) {
[1]=>
int(1)
}
["messageType"]=>
array(1) {
[1]=>
int(2)
}
["skip"]=>
array(1) {
[1]=>
int(1)
}
["updateTime"]=>
array(1) {
[1]=>
int(1337222280)
}
["fixTime"]=>
array(1) {
[1]=>
int(1337222280)
}
["longitude"]=>
array(1) {
[1]=>
int(331313576)
}
["latitude"]=>
array(1) {
[1]=>
int(-1172790010)
}
["altitude"]=>
array(1) {
[1]=>
int(4915)
}
["speed"]=>
array(1) {
[1]=>
int(0)
}
Just divide latitude and longitude by 10000000 (10M) to get the real longitude and latitude

Get Least Significant Bit in PHP from Hex

I have a file with hex code and I need to get all the least significant bits from every byte in the file, concatenate them, split them in groups of 8 and then convert the bytes into ASCII. My problem is to extract the LSB from every byte.
The hex file looks like this (but is much longer):
58 00 00 1F 58 00 00 00 00 00 00 00 00 00 00 1C 22 23 1F 26 25 1E 2C 26 20 31 2B 22 38 2F 26 42 36 25 47 37 24 49 39 22
My code looks like this:
<?php
// Read file, remove spaces
$file = implode('', explode(' ', file_get_contents('vogel.hex')));
// Save LSB
$bits = array();
for ($i = 1; $i < strlen($file); ++$i)
{
$bits[] = $file[$i] & 1;
}
// Split LSB array into chunks of 8 bits.
$bytes = array_chunk($bits, 8);
// Implode byte arrays, convert to decimal, convert to ASCII.
foreach ($bytes as $byte)
{
echo chr(bindec(implode('', $byte)));
}
?>
I think that the splitting and converting part should work correctly, but I think I made a mistake when extracting the LSB. Can someone provide an example how I can extract the LSB?
I slightly edited my code, so that I start reading the bits at position 1. Then the decimal representation is within the ASCII range and the script outputs an actual ASCII character.
You could simply build up the bit string within the for loop, skipping the entire array procedure:
$bits = '';
for ($i = 1; $i < strlen($file); $i++) {
$bits .= (($file[$i] & 1) == 0) ? '0' : '1';
if ($i % 8 == 0) {
echo bindec($bits);
$bits = '';
}
}
Of course, you'd have some dangling bits if the input file's size isn't a multiple of 8.
A lot of programmers might cringe at your solution, but it works just fine with both ASCII and EBCDIC. I don't know of any other character set one might possibly be using with PHP.
The least significant bit of the character digit is the same as the value it represents. So your code will work. But it really deserves to have a comment explaining it relies on the least significant bit of the ASCII/EBCDIC display codes being the same as the digit.
what about shifting the hex left for hexvalue/2 times?
$hex = 0x05;
$shiftVal = (0 + $hex)/2;
echo $hex>>$shiftVal;//should output 1
another approach is to convert the hex to a number and see if it's odd or even:
$hex = 0xad;
echo $hex%2;

PHP utf encoding problem

How can I encode strings on UTF-16BE format in PHP? For "Demo Message!!!" the encoded string should be '00440065006D006F0020004D00650073007300610067006'. Also, I need to encode Arabic characters to this format.
First of all, this is absolutly not UTF-8, which is just a charset (i.e. a way to store strings in memory / display them).
WHat you have here looks like a dump of the bytes that are used to build each characters.
If so, you could get those bytes this way :
$str = utf8_encode("Demo Message!!!");
for ($i=0 ; $i<strlen($str) ; $i++) {
$byte = $str[$i];
$char = ord($byte);
printf('%02x ', $char);
}
And you'd get the following output :
44 65 6d 6f 20 4d 65 73 73 61 67 65 21 21 21
But, once again, this is not UTF-8 : in UTF-8, like you can see in the example I've give, D is stored on only one byte : 0x44
In what you posted, it's stored using two Bytes : 0x00 0x44.
Maybe you're using some kind of UTF-16 ?
EDIT after a bit more testing and #aSeptik's comment : this is indeed UTF-16.
To get the kind of dump you're getting, you'll have to make sure your string is encoded in UTF-16, which could be done this way, using, for example, the mb_convert_encoding function :
$str = mb_convert_encoding("Demo Message!!!", 'UTF-16', 'UTF-8');
Then, it's just a matter of iterating over the bytes that make this string, and dumping their values, like I did before :
for ($i=0 ; $i<strlen($str) ; $i++) {
$byte = $str[$i];
$char = ord($byte);
printf('%02x ', $char);
}
And you'll get the following output :
00 44 00 65 00 6d 00 6f 00 20 00 4d 00 65 00 73 00 73 00 61 00 67 00 65 00 21 00 21 00 21
Which kind of looks like what youy posted :-)
(you just have to remove the space in the call to printf -- I let it there to get an easier to read output=)
E.g. by using the mbstring extension and its mb_convert_encoding() function.
$in = 'Demo Message!!!';
$out = mb_convert_encoding($in, 'UTF-16BE');
for($i=0; $i<strlen($out); $i++) {
printf("%02X ", ord($out[$i]));
}
prints
00 44 00 65 00 6D 00 6F 00 20 00 4D 00 65 00 73 00 73 00 61 00 67 00 65 00 21 00 21 00 21
Or by using iconv()
$in = 'Demo Message!!!';
$out = iconv('iso-8859-1', 'UTF-16BE', $in);
for($i=0; $i<strlen($out); $i++) {
printf("%02X ", ord($out[$i]));
}

Categories