How can I pack/unpack floats in big endian byte order with php?
I got this far with an unpack function, but I'm not sure if this would even work.
function unpackFloat ($float) {
$n = unpack ('Nn');
$n = $n['n'];
$sign = ($n >> 31);
$exponent = ($n >> 23) & 0xFF;
$fraction = $n & 0x7FFFFF;
}
After thinking about it for a while I found a pretty easy solution, to use the opposite byte order from the one pack('f') uses.
unpack
unpack('fdat', strrev(substr($data, 0, 4)));
pack
strrev(pack('f', $data));
PHP 7.2 introduced the option to pack floating point numbers with big endian byte order directly:
// float
$bytes = pack('G', 3.1415);
// double precision float
$bytes = pack('E', 3.1415);
https://www.php.net/manual/en/function.pack.php
Related
I need to do some work on data contained in legacy files. For this purpose, I need to read and write Turbo Pascal's 6-byte (48 bit) floating point numbers, from PHP. The Turbo Pascal data type is commonly known as real48 (specs).
I have the following php code to read the format:
/**
* Convert Turbo Pascal 48-bit (6 byte) real to a PHP float
* #param binary 48-bit real (in binary) to convert
* #return float number
*/
function real48ToDouble($real48) {
$byteArray = array_values( unpack('C*', $real48) );
if ($byteArray[0] == 0) {
return 0; // Zero exponent = 0
}
$exponent = $byteArray[0] - 129;
$mantissa = 0;
for ($b = 1; $b <= 4; $b++) {
$mantissa += $byteArray[$b];
$mantissa /= 256;
}
$mantissa += ($byteArray[5] & 127);
$mantissa /= 128;
$mantissa += 1;
if ($byteArray[5] & 128) { // Sign bit check
$mantissa = -$mantissa;
}
return $mantissa * pow(2, $exponent);
}
(adapted from)
Now I need to do the reverse: write the data type.
Note:
I'm aware of the answer to the question Convert C# double to Delphi Real48, but it seems awfully hacky and I would think a much cleaner solution is possible. AND my machine does not natively support 64-bits.
On a second look, the method posted in the answer to Convert C# double to Delphi Real48 cleaned up pretty nicely.
For future reference:
/**
* Convert a PHP number [Int|Float] to a Turbo Pascal 48-bit (6 byte) real byte representation
* #param float number to convert
* #return binary 48-bit real
*/
function doubleToReal48($double) {
$byteArray = array_values( unpack('C*', pack('d', $double)) ); // 64 bit double as array of integers
$real48 = array(0, 0, 0, 0, 0, 0);
// Copy the negative flag
$real48[5] |= ($byteArray[7] & 128);
// Get the exponent
$n = ($byteArray[7] & 127) << 4;
$n |= ($byteArray[6] & 240) >> 4;
if ($n == 0) { // Zero exponent = 0
return pack('c6', $real48[0], $real48[1], $real48[2], $real48[3], $real48[4], $real48[5]);
}
$real48[0] = $n - 1023 + 129;
// Copy the Mantissa
$real48[5] |= (($byteArray[6] & 15) << 3); // Get the last 4 bits
$real48[5] |= (($byteArray[5] & 224) >> 5); // Get the first 3 bits
for ($b = 4; $b >= 1; $b--) {
$real48[$b] = (($byteArray[$b+1] & 31) << 3); // Get the last 5 bits
$real48[$b] |= (($byteArray[$b] & 224) >> 5); // Get the first 3 bits
}
return pack('c6', $real48[0], $real48[1], $real48[2], $real48[3], $real48[4], $real48[5]);
}
I'm reading 6 bytes in little endian from a binary file using
$data = fread($fp, 6);
unpack("V", $data);
The result is 1152664389 and in HEX it is 0x44B44345
Now this result is in little endian and has a decimal number. In Delphi, I was able to get the decimal number using this function:
var
myint:integer;
s:single;
begin
myint:= 1152664389; // same as myint:= $44B44345;
s:= PSingle(#myint)^;
and output of s is 1442.102.. This is exactly the number im looking for...
I searched for a way to do it in PHP but I just got lost.
Please help,
Thanks
Okay, I was a bit confused. I wanted to actually retrieve a little endian number from a binary file then convert it to float.
So my steps were:
1) Read bytes and unpack using $number = unpack("V", $data);
2) Convert $number to decimal using dechex.
3) Then use the following function to convert hex to float:
function hexTo32Float($strHex) {
$v = hexdec($strHex);
$x = ($v & ((1 << 23) - 1)) + (1 << 23) * ($v >> 31 | 1);
$exp = ($v >> 23 & 0xFF) - 127;
return $x * pow(2, $exp - 23);
}
Thanks again.
I'm trying to convert a 64-bit float to a 64-bit integer (and back) in php. I need to preserve the bytes, so I'm using the pack and unpack functions. The functionality I'm looking for is basically Java's Double.doubleToLongBits() method. http://docs.oracle.com/javase/7/docs/api/java/lang/Double.html#doubleToLongBits(double)
I managed to get this far with some help from the comments on the php docs for pack():
function encode($int) {
$int = round($int);
$left = 0xffffffff00000000;
$right = 0x00000000ffffffff;
$l = ($int & $left) >>32;
$r = $int & $right;
return unpack('d', pack('NN', $l, $r))[1];
}
function decode($float) {
$set = unpack('N2', pack('d', $float));
return $set[1] << 32 | $set[2];
}
And this works well, for the most part...
echo decode(encode(10000000000000));
100000000
echo encode(10000000000000);
1.1710299640683E-305
But here's where it gets tricky...
echo decode(1.1710299640683E-305);
-6629571225977708544
I have no idea what's wrong here. Try it for yourself: http://pastebin.com/zWKC97Z7
You'll need 64-bit PHP on linux. This site seems to emulate that setup: http://www.compileonline.com/execute_php_online.php
$x = encode(10000000000000);
var_dump($x); //float(1.1710299640683E-305)
echo decode($x); //10000000000000
$y = (float) "1.1710299640683E-305";
var_dump($y); //float(1.1710299640683E-305)
echo decode($y); //-6629571225977708544
$z = ($x == $y);
var_dump($z); //false
http://www.php.net/manual/en/language.types.float.php
... never trust
floating number results to the last digit, and do not compare floating
point numbers directly for equality. If higher precision is necessary,
the arbitrary precision math functions and gmp functions are
available. For a "simple" explanation, see the » floating point guide
that's also titled "Why don’t my numbers add up?"
It is working properly, the only problem in this case is in logic of:
echo decode(1.1710299640683E-305);
You can't use "rounded" and "human readable" output of echo function to decode the original value (because you are loosing precision of this double then).
If you will save the return of encode(10000000000000) to the variable and then try to decode it again it will works properly (you can use echo on 10000000000000 without loosing precision).
Please see the example below which you can execute on PHP compiler as well:
<?php
function encode($int) {
$int = round($int);
$left = 0xffffffff00000000;
$right = 0x00000000ffffffff;
$l = ($int & $left) >>32;
$r = $int & $right;
return unpack('d', pack('NN', $l, $r))[1];
}
function decode($float) {
$set = unpack('N2', pack('d', $float));
return $set[1] << 32 | $set[2];
}
echo decode(encode(10000000000000)); // untouched
echo '<br /><br />';
$encoded = encode(10000000000000);
echo $encoded; // LOOSING PRECISION!
echo ' - "human readable" version of encoded int<br /><br />';
echo decode($encoded); // STILL WORKS - HAPPY DAYS!
?>
If you have a reliable fixed decimal point, like in my case and the case of currency, you can multiply your float by some power of 10 (ex. 100 for dollars).
function encode($float) {
return (int) $float * pow(10, 2);
}
function decode($str) {
return bcdiv($str, pow(10, 2), 2);
}
However, this doesn't work for huge numbers and doesn't officially solve the problem.
Seems like it's impossible to convert from an integer to a float string and back without losing the original integer value in php 5.4
I am creating and then writing data to a file (a new 'ESRI Shape file') using PHP, fopen, fseek, pack etc. The file spec is here http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf.
The file spec states that the data written needs to be in a combination of the following:
Integer: Signed 32-bit integer (4 bytes) - Big Endian
Integer: Signed 32-bit integer (4 bytes) - Little Endian
Double: Signed 64-bit IEEE double-precision floating point number (8 bytes) - Little Endian
I cant seem to find a pack() format that allows for these formats. I don't want to use a machine dependent format as this code may be running on a variety of platforms.
Can anyone advise on what format (or combination of formats) I need to use for these 3 formats?
Many thanks,
Steve
You could check the endianness of the machine running the code and reverse the bytes manually as necessary. The code below should work, but you will only be able to convert one int or float at a time.
define('BIG_ENDIAN', pack('L', 1) === pack('N', 1));
function pack_int32s_be($n) {
if (BIG_ENDIAN) {
return pack('l', $n); // that's a lower case L
}
return strrev(pack('l', $n));
}
function pack_int32s_le($n) {
if (BIG_ENDIAN) {
return strrev(pack('l', $n));
}
return pack('l', $n); // that's a lower case L
}
function pack_double_be($n) {
if (BIG_ENDIAN) {
return pack('d', $n);
}
return strrev(pack('d', $n));
}
function pack_double_le($n) {
if (BIG_ENDIAN) {
return strrev(pack('d', $n));
}
return pack('d', $n);
}
If PHP doesn't support it, you could implement your own.
function pack_int32be($i) {
if ($i < -2147483648 || $i > 2147483647) {
die("Out of bounds");
}
return pack('C4',
($i >> 24) & 0xFF,
($i >> 16) & 0xFF,
($i >> 8) & 0xFF,
($i >> 0) & 0xFF
);
}
function pack_int32le($i) {
if ($i < -2147483648 || $i > 2147483647) {
die("Out of bounds");
}
return pack('C4',
($i >> 0) & 0xFF,
($i >> 8) & 0xFF,
($i >> 16) & 0xFF,
($i >> 24) & 0xFF
);
}
The double-precision LE is much harder. Supporting quad-precision system would involve packing the number using d, converting it to a binary string, splitting the binary into fields, truncating the fields to the right size if they're too large, concatenating the fields, then converting from binary to bytes.
I have some large HEX values that I want to display as regular numbers, I was using hexdec() to convert to float, and I found a function on PHP.net to convert that to decimal, but it seems to hit a ceiling, e.g.:
$h = 'D5CE3E462533364B';
$f = hexdec($h);
echo $f .' = '. Exp_to_dec($f);
Output: 1.5406319846274E+19 = 15406319846274000000
Result from calc.exe = 15406319846273791563
Is there another method to convert large hex values?
As said on the hexdec manual page:
The function can now convert values
that are to big for the platforms
integer type, it will return the value
as float instead in that case.
If you want to get some kind of big integer (not float), you'll need it stored inside a string. This might be possible using BC Math functions.
For instance, if you look in the comments of the hexdec manual page, you'll find this note
If you adapt that function a bit, to avoid a notice, you'll get:
function bchexdec($hex)
{
$dec = 0;
$len = strlen($hex);
for ($i = 1; $i <= $len; $i++) {
$dec = bcadd($dec, bcmul(strval(hexdec($hex[$i - 1])), bcpow('16', strval($len - $i))));
}
return $dec;
}
(This function has been copied from the note I linked to; and only a bit adapted by me)
And using it on your number:
$h = 'D5CE3E462533364B';
$f = bchexdec($h);
var_dump($f);
The output will be:
string '15406319846273791563' (length=20)
So, not the kind of big float you had ; and seems OK with what you are expecting:
Result from calc.exe =
15406319846273791563
Hope this help ;-)
And, yes, user notes on the PHP documentation are sometimes a real gold mine ;-)
hexdec() switches from int to float when the result is too large to be represented as an int. If you want arbitrarily long values, you're probably going to have to roll your own conversion function to change the hex string to a GMP integer.
function gmp_hexdec($n) {
$gmp = gmp_init(0);
$mult = gmp_init(1);
for ($i=strlen($n)-1;$i>=0;$i--,$mult=gmp_mul($mult, 16)) {
$gmp = gmp_add($gmp, gmp_mul($mult, hexdec($n[$i])));
}
return $gmp;
}
print gmp_strval(gmp_hexdec("D5CE3E462533364B"));
Output: 15406319846273791563
$num = gmp_init( '0xD5CE3E462533364B' ); // way to input a number in gmp
echo gmp_strval($num, 10); // display value in decimal
That's the module to use. Convert it to a function and then use on your numbers.
Note: provide these hex numbers as strings so:
$num = "0x348726837469972346"; // set variable
$gmpnum = gmp_init("$num"); // gmp number format
echo gmp_strval($gmpnum, 10); // convert to decimal and print out
1.5406319846274E+19 is a limited representation of you number. You can have a more complete one by using printf()
printf("%u\n", hexdec($h));
...will output "15406319846273792000". PHP uses floats for such big numbers, so you may lose a bit of precision. If you have to work with arbitrary precision numbers, you may try the bcmath extension. By splitting the hex into two 32-bit words (which should be safe on most systems) you should be able to get more precision. For instance:
$f = bcadd(bcmul(hexdec(substr($h, 0, -8)), 0x100000000), hexdec(substr($h, 8)));
...would set $f to 15406319846273791563.
Convert HEX to DEC is easy.. But, reconstruct back hexadecimal number is very hard.
Try to use base_convert ..
$hexadecimal = base_convert(2826896153644826, 10, 16);
// result: a0b0c0d0e0f1a
Run into this issue while storing 64-bit keys in MySQL database. I was able to get a bit perfect conversion to a 64-bit signed integer (PHP limitation) using a few binary operators: (This code is 16x faster than bchexdec function and resulting variables are using half the memory on average).
function x64toSignedInt($k){
$left = hexdec(substr($k,0,8));
$right = hexdec(substr($k,8,8));
return (int) ($left << 32) | $right;
}
MySQL signed BIGINT datatype is a great match for this as an index or storage in general. HEX(column) is a simple way to convert it back to HEX within the SQL query for use elsewhere.
This solution also uses the BC Math Functions. However, an algorithm is used which does without the bcpow function. This function is a bit shorter and faster than the accepted solution, tested on PHP 7.4.
function hexDecBc(string $hex) : string
{
for ($dec = '0', $i = 0; $i < strlen($hex); $i++) {
$dec = bcadd(bcmul($dec,'16'),(string)hexdec($hex[$i]));
}
return $dec;
}
Make sure to enable gmp extension. ext-gmp
$number = gmp_strval(gmp_init('0x03....')); // outputs: 1234324....
Doesn't intval(var, base) take care of it?
From the PHP Manual.