I have a device witch use binary format style config, and i have to generate that files on-the-fly.
File structure must consist of a number of configuration settings (1 per parameter) each of the form:
Type
Length
Value
where:
Type: is a single-octet identifier which defines the parameter
Length: is a single octet containing the length of the value field in octets (not including type and length fields)
Value: is from one to 254 octets containing the specific value for the parameter
I have a corresponding table
Type_code[int] => { Type_length[int] => Value[int/string/hex/etc.] }
How to parse that table to that binary format?
And, second way, how to parse that binary file, to php array format back?
There's the pack/unpack functions that can translate between various binary/hex/octal/string formats. Read a chunk of the file, extract necessary bits with unpack, and work from there.
$fh = fopen('data.txt', 'rb'); // b for binary-safe
// read 2 bytes, extract code/length, then read $length more bytes to get $value
while(($data = fread($fh, 2)) !== EOF)) {
list($code, $length) = unpack('CC', $data);
$data = fread($fh, $length);
// do stuff
}
fclose($fh);
Related
i am using (shuchkin/simplexlsxgen) to generate xlsx file which works fine.
My data contain decimal numbers such as (0.19) which are written in the SQL DB, but instead writting them with dott, the (simplexlsxgen) convert them to (0,19) comma decimal numbers.
is there a way to prevent changing the decimal Dott to comma, before generating?
Thanks in advance.
//convert test_bulk.csv to Xlsx
function csvToArray($csvFile)
{
$file_to_read = fopen($csvFile, 'r');
while (!feof($file_to_read))
{
$lines[] = fgetcsv($file_to_read, 1000, ';');
}
fclose($file_to_read);
return $lines;
}
//read the csv file into an array
$csvFile = 'test_bulk.csv';
$csv = csvToArray($csvFile);
Shuchkin\SimpleXLSXGen::fromArray($csv)->saveAs('final_bulk.xlsx');
by adding "\0". before the targeted string, it will turn it to be a RAW STRING. which will make this super Tool write it without changing the decimal dott to comma.
you all gotta try this amazing fast effective tool by the PHP Excel Old school Master (Sergey Shuchkin).
https://github.com/shuchkin
I'm trying to parse a Binary File in PHP which is an attachment of a Document in a NoSQL DB. However, in my tests, if the size of a file is of 1MB, the unpacking lasts for around 12-15 seconds. The file contains information about speed from a sensor.
The binary file converted to hexadecimal is structured as follow:
BB22 1100 0015 XXXX ...
BB22 1300 0400 20FB 5900 25FB 5910 ... 20FB 5910
BB22 1100 0015 ...
BB22 1300 0400 20FB 5700 25FB 5810 ... 20FB 5912
BB22 1300 0400 20FB 5700 25FB 5810 ... 20FB 5912
...
The marker BB22 1100 contains the sensor specification, while 0015 refers to the size of that information.
The marker BB22 1300 contains other data plus the actual speed from the sensor. The next two bytes 0400 represent the length of that chunk, which is of 1024 bytes.
I'm only interested in the speed which are the values e.g. 5900 5910 5910 5700 5810 ...
My approach is as follow:
$file = fopen($url, 'r', false, authenticationContext($url));
$result = stream_get_contents($file, -1);
fclose($file);
$hex_result = bin2hex($result);
$markerData = 'bb2213';
$sensorDataUnpack= "sspeed"; // signed int16
while(($pos = strpos($hex_result, $markerData, $pos)) !== FALSE){
$pos=$pos+4;
for ($j=4; $j<1028; $j=$j+4) {
$d = unpack($sensorDataUnpack, substr($result, $pos/2+$j+2));
$sensorData[] = $d;
}
}
I converted the results from binary to hexadecimal because it wasn't working for me to get the positions properly. Anyway, I believe this code can be very much improved, any ideas?.
This should be fast, but without test data I wasn't able to test it.
The key points are these:
Open the URL as binary, and use the fread() to help in positioning and in slicing up the data to parts.
Use the unpack both for parsing the headers and the bodies of the entries as well.
Use the asterisk * repeater to quickly parse the big bodies for signed shorts.
Use array_values() to convert the associative array to a simple array with numeric keys (like: 0, 1, 2, ...).
Update: I solved the endianness and bitness problem around the marker comparison by using the "H4" pack format to get a hexa string in big endian order.
$sensorData = array();
$file = fopen($url, 'rb', false, authenticationContext($url));
while (($header = fread($file, 6)) !== false) {
$fields = unpack("H4marker/ssize", $header);
$body = fread($file, $fields["size"] * 2);
if ($body === false) {
throw new Exception("import: data stream unexpectedly ended.");
}
if ($fields["marker"] == "BB221300") {
$data = array_values(unpack("s*", $body));
// Store only every second value.
for ($i = 1; $i < count($data); $i+=2) {
$sensorData[] = $data[$i];
}
}
}
fclose($file);
I'm trying to write a binary file from hex string.
For example, if my hex string is C27EF0EC, then the hex file should contain ASCII characters for C2, 7E, F0 and EC.
How do I do this in PHP?
Here's what I've tried:
$s="";
for ($i=0; $i<count($h); $i++) {
$s+=pack("C*", "0x".$h[$i]);
}
$f2=fopen("codes0", "wb+");
fwrite($f2, $s);
So the first thing you need to do is turn your single string into an array of two-character strings with str_split.
$hex_bytes = str_split($h, 2);
Then you want to convert each of those values from a hexadecimal string to the corresponding number with hexdec.
$code_array = array_map(hexdec, $hex_bytes);
Then you want the byte value corresponding to each of those character codes, which you can get with chr:
$char_array = array_map(chr, $code_array);
Finally, you want to join all those bytes together into a single string, which you can do with implode.
$s = implode($char_array);
You can use the steps above in that order, or you can put it all together into one expression like this:
$s = implode(array_map(chr, array_map(hexdec, str_split($h,2))));
Note that as soon as you get a value above 0x7F it's no longer "ASCII".
Assuming that array $binary is a previously constructed array bytes (like monochrome bitmap pixels in my case) that you want written to the disk in this exact order, the below code worked for me on an AMD 1055t running ubuntu server 10.04 LTS.
I iterated over every kind of answer I could find on the Net, checking the output (I used either shed or vi, like in this answer) to confirm the results.
<?php
$fp = fopen($base.".bin", "w");
$binout=Array();
for($idx=0; $idx < $stop; $idx=$idx+2 ){
if( array_key_exists($idx,$binary) )
fwrite($fp,pack( "n", $binary[$idx]<<8 | $binary[$idx+1]));
else {
echo "index $idx not found in array \$binary[], wtf?\n";
}
}
fclose($fp);
echo "Filename $base.bin had ".filesize($base.".bin")." bytes written\n";
?>
I have a set of ZLIB compressed / base64 encoded strings (done in a C program) that are stored in a database. I have written a small PHP page that should retrieve these values and plot them (the string originally was a list of floats).
Chunk of C program that compresses/encodes:
error=compress2(comp_buffer, &comp_length,(const Bytef*)data.mz ,(uLongf)length,Z_DEFAULT_COMPRESSION); /* compression */
if (error != Z_OK) {fprintf(stderr,"zlib error..exiting"); exit(EXIT_FAILURE);}
mz_binary=g_base64_encode (comp_buffer,comp_length); /* encoding */
(Example) of original input format:
292.1149 8379.5928
366.1519 101313.3906
367.3778 20361.8105
369.1290 17033.3223
375.4355 1159.1841
467.3191 8445.3926
Each column was compressed/encoded as a single string. To reconstruct the original data i am using the following code:
//$row[4] is retrieved from the DB and contains the compressed/encoded string
$mz = base64_decode($row[4]);
$unc_mz = gzuncompress($mz);
echo $unc_mz;
Yet this gives me the following output:
f6jEÍ„]EšiSE#IEfŽ
Could anyone give me a tip/hint about what I might be missing?
------ Added Information -----
I feel that the problem comes from the fact that currently php views $unc_mz as a single string while in reality i would have to re-construct an array containing X lines (this output was from a 9 line file) but... no idea how to do that assignment.
The C program that did that went roughly like this:
uncompress( pUncompr , &uncomprLen , (const Bytef*)pDecoded , decodedSize );
pToBeCorrected = (char *)pUncompr;
for (n = 0; n < (2 * peaksCount); n++) {
pPeaks[n] = (RAMPREAL) ((float *) pToBeCorrected)[n];
}
where peaksCount would be the amount of 'lines' in the input file.
EDIT (15-2-2012): The problem with my code was that I was not reconstructing the array, the fixed code is as follows (might be handy if someone needs a similar snippet):
while ($row = mysql_fetch_array($result, MYSQL_NUM)) {
$m< = base64_decode($row[4]);
$mz_int = gzuncompress($int);
$max = strlen($unc_mz);
$counter = 0;
for ($i = 0; $i < $max; $i = $i + 4) {
$temp= substr($unc_mz,$i,4);
$temp = unpack("f",$temp);
$mz_array[$counter] = $temp[1];
$counter++;
}
The uncompressed string has to be chopped into chunks corresponding to the length of a float, unpack() then reconstructs the float data from teh binary 'chunk'. That's the simplest description that I can give for the above snippet.
compress2() produces the zlib format (RFC 1950). I would have to guess that something called gzuncompress() is expecting the gzip format (RFC 1952). So gzuncompress() would immediately fail upon not finding a gzip header.
You would need to use deflateInit2() in zlib to request that deflate() produce gzip-formatted output, or find or provide a different function in PHP that expects the zlib format.
I have a binary file that is all 8 bit integers. I have tried to use the php unpack() functions but I cant get any of the arguments to work for 1 byte integers. I have tried to combine the data with a dummy byte so that I can use the 'n'/'v' arguments. I am working with a windows machine to do this. Ultimately I would like a function to return an array of integers based on a string of 8 bit binary integers. The code I have tried is below -
$dat_handle = "intergers.dat";
$dat_file = fopen($dat_handle, "rb");
$dat_data = fread($dat_file, 1);
$dummy = decbin(0);
$combined = $dummy.$dat_data;
$result = unpack("n", $combined);
What your looking for is the char datatype. Now there are two version of this, signed (lowercase c) and unsigned (uppercase C). Just use the one that's correct for your data.
<?php
$byte = unpack('c', $byte);
?>
Also, if the data file is just a bunch of bytes and nothing else, and you know it's length, you can do this. (If the length is 16 signed chars in a row.)
<?php
$bytes = unpack('c16', $byte);
?>
If you don't know how many bytes will be in the file, but you know there is only going to be bytes you can use the asterisk code to read until EOF.
<?php
$bytes = unpack('c*', $byte);
?>
The following should do what you want (ord):
$dat_handle = "intergers.dat";
$dat_file = fopen($dat_handle, "rb");
$dat_data = ord(fread($dat_file, 1));
What you are trying to do is retrieve the integer value of the single byte. Because you are reading in single bytes at a time, you will always have exactly one valid ASCII character. ord returns the binary value of that one character.