I have a modified MD5 hash function which I am using in PHP and VB.NET. When I run the PHP code on my local server (WAMP) I get a different result to the VB version. I have tried running the script on phpfiddle which gives the same result as the VB version.
I am thinking the problem could lie with my PHP settings on the WAMP server?
If I run the script below on my PC running WAMP the result I get is:
e5c35f7c3dea80fc68a4031582f34c25
When I run the exact same script on phpfiddle or php sandbox the result I get is (this is the expected result):
6337a43e8cd36058e80ae8cb4f465998
Setting aside for a moment the fact that what you are doing here sounds like a bad approach what ever the actual problem is that you are trying to solve, here is a direct answer to the question.
As I already outlined in a comment above, the root cause of the problems you are having is that PHP has no concept of unsigned integers, and it handles this by converting numbers that overflow the bounds of an integer to floating point (which doesn't play nice with bitwise operations). This means that, on 32-bit systems, your code won't work correctly, as MD5 works with unsigned 32-bit integers.
You will need to ensure that your code is "binary safe" - so that all numbers are represented as if they were unsigned 32-bit integers.
To do this you will need to re-implement the addition operator, and (with your current implementation) the bindec()/hexdec() functions. It's worth noting that your current approach to certain procedures is very inefficient - all that converting to/from hex strings, and places where binary is represented as ASCII strings - but I'll gloss over that for now while I show you how to quick-fix your current implementation.
Firstly let's take a look at the addition operation:
private function binarySafeAddition($a, $b)
{
// NB: we don't actually need 64 bits, theoretically we only need 33
// but 40 bit integers are confusing enough, and 33 bits is unrepresentable
$a = "\x00\x00\x00\x00" . pack('N', $a);
$b = "\x00\x00\x00\x00" . pack('N', $b);
$carry = $a & $b;
$result = $a ^ $b;
while ($carry != "\x00\x00\x00\x00\x00\x00\x00\x00") {
$shiftedcarry = $this->leftShiftByOne($carry);
$carry = $result & $shiftedcarry;
$result ^= $shiftedcarry;
}
return current(unpack('N', substr($result, 4)));
}
private function leftShiftByOne($intAsStr)
{
$p = unpack('N2', $intAsStr);
return pack('N2', ($p[1] << 1) | (($p[2] >> 31) & 0x00000001), $p[2] << 1);
}
private function add()
{
$result = 0;
foreach (func_get_args() as $i => $int) {
$result = $this->binarySafeAddition($result, $int);
}
return $result;
}
The real nuts-and-bolts of this routine is shamelessly stolen from here. There's also a helper function to perform the left-shift, because PHP doesn't let you left-shift strings, and a convenience wrapper function, to allow us to add an arbitrary number of operands together in a single clean call.
Next lets look at the bindec() and hexdec() replacements:
private function binarySafeBinDec($bin)
{
$bits = array_reverse(str_split($bin, 1));
$result = 0;
foreach ($bits as $position => $bit) {
$result |= ((int) $bit) << $position;
}
return $result;
}
private function binarySafeHexDec($hex)
{
$h = str_split(substr(str_pad($hex, 8, '0', STR_PAD_LEFT), -8), 2);
return (hexdec($h[0]) << 24) | (hexdec($h[1]) << 16) | (hexdec($h[2]) << 8) | hexdec($h[3]);
}
Hopefully these are reasonably self explanatory, but feel free to ask about anything you don't understand.
We also need to replace all those 0xffffffff hex literals with a binary safe implementation, as these will also result in a float on 32-bit systems. Here is a safe way to get the right-most 32 bits set in an integer, that will work on 32- and 64-bit systems:
private $right32;
public function __construct()
{
$this->right32 = ~((~0 << 16) << 16);
}
There's one other method we need to re-implement, and that's rotate(). This is because it uses a right-shift, and this shifts a copy of the sign bit on from the right. This means that the left-hand side of the rotated block will end up with all it's bits set, and this is obviously not what we want. We can overcome this by creating a number with only the target bits for the right-hand side set, and ANDing the right-hand side operand with it:
private function rotate ($decimal, $bits)
{
return dechex(($decimal << $bits) | (($decimal >> (32 - $bits)) & (~(~0 << $bits) & $this->right32)));
}
When you put all this together you come up with something like this, which works for me on 32- and 64-bit systems.
Related
I'm trying to write an algorithm to convert a 64Bit integer to 32Bit via truncation, and return a value accurately representing a 32Bit value. The obvious problem is that integers in PHP are only 64Bit (barring 32Bit systems).
I first tried the following:
$x = $v & 0xFFFFFFFF
So if $v = PHP_INT_MAX, $x is 4294967295, when really I want -1. I know the reason for this is that when represented as a 64Bit integer, PHP prepends the zeros to the last 32 bits, and thats why I get a positive number.
My solution so far is this:
function convert64BitTo32Bit(int $v): int
{
$v &= 0xFFFFFFFF;
if ($v & 0x80000000) {
return $v | 0xFFFFFFFF00000000;
}
return $v;
}
And I'm pretty sure its right. The problem I have with this is that it requires that if statement to inspect whether the truncated number is negative. I was really hoping for a bitwise only solution. It may not be possible, but I thought I'd ask.
EDIT:
The problem can be simplified to just part of the solution. i.e. if the first bit is 1, make all bits 1, if first bit is 0 make all bits 0.
# example assumes input is the LSBs of an 8Bit integer.
# scale for 64Bit and the same solution should work.
op(1010) = 1111
op(0101) = 0000
op(0000) = 0000
op(1000) = 1111
op(0001) = 0000
I would be able to use the LSBs to derive this value, then mask it onto the MSBs of the 64Bit integer. This is what I'm trying to figure out now, though I'm trying to avoid creating a monster equation.
There's probably a more elegant/efficient way using bitwise operations, but you can also force the conversion with pack() and unpack(). Eg: Pack as unsigned, unpack as signed.
function overflow32($in) {
return unpack('l', pack('i', $in & 0xFFFFFFFF))[1];
}
var_dump( overflow32(pow(2,33)-1) ); // int(-1)
I'm curious what you're applying this to, because I had to break a hash function with this same thing some years ago when I moved an app from a 32 bit machine to a 64 bit machine, but I can't remember what it was.
Edit: Wow for some reason I remembered Two's Complement being hard. Literally just invert and add one.
function overflow32($in) {
if( $in & 0x80000000 ) {
return ( ( ~$in & 0xFFFFFFFF) + 1 ) * -1;
} else {
return $in & 0xFFFFFFFF;
}
}
var_dump( kludge32(pow(2,33)-1) );
Edit 2: I saw that you want to extend this to arbitrary bit lengths, in which case you just need to calculate the masks instead of explicitly setting them:
function overflow_bits($in, $bits=32) {
$sign_mask = 1 << $bits-1;
$clamp_mask = ($sign_mask << 1) - 1;
var_dump(
decbin($in),
decbin($sign_mask),
decbin($clamp_mask)
);
if( $in & $sign_mask ) {
return ( ( ~$in & $clamp_mask) + 1 ) * -1;
} else {
return $in & $clamp_mask;
}
}
var_dump(
overflow_bits(pow(2, 31), 32),
overflow_bits(pow(2, 15), 16),
overflow_bits(pow(2, 7), 8)
);
I left in the debug var_dump()s for output flavor:
string(32) "10000000000000000000000000000000"
string(32) "10000000000000000000000000000000"
string(32) "11111111111111111111111111111111"
string(16) "1000000000000000"
string(16) "1000000000000000"
string(16) "1111111111111111"
string(8) "10000000"
string(8) "10000000"
string(8) "11111111"
int(-2147483648)
int(-32768)
int(-128)
I think I've cracked it:
function overflow32Bit(int $x): int
{
return ($x & 0xFFFFFFFF) | ((($x & 0xFFFFFFFF) >> 31) * ((2 ** 32) - 1) << 32);
}
var_dump(overflow32Bit(PHP_INT_MAX)); // -1 correct
var_dump(overflow32Bit(PHP_INT_MAX - 1)); // -2 correct
var_dump(overflow32Bit(PHP_INT_MIN)); // 0 correct
var_dump(overflow32Bit((2 ** 31) - 1)); // 2147483647 correct
var_dump(overflow32Bit((2 ** 31))); // -2147483647 correct
var_dump(overflow32Bit(0xFFFFFFFF)); // -1 correct
var_dump(overflow32Bit(0x7FFFFFFF)); // 2147483647 correct
The solution was actually staring me in the face. Get the value of the first bit, then multiply it by the max value of unsigned 32bit integer.
If someone can come up with a better or shorter solution, I'll also accept that.
PS. This is only for 32Bit, i also intend to use this proof for 16Bit and 8Bit.
Expanded, $x is input, $z is output.
$x = PHP_INT_MAX;
$y = (2 ** 32) - 1;
$z = ($x & $y) | ((($x & $y) >> 31) * ($y << 32));
var_dump($z);
For a project I need to read in information from MQTT. The payload is filled with protobuf information, that needs to be converted.
For a certain value I receive 5.6904566139035E-28 as float. Using http://www.exploringbinary.com/floating-point-converter/ I can convert this when I tick single and raw hexadecimal value, then I receive 12345678, the value I should have (I know what is sent).
But now I need to do that conversion in PHP. I haven't any idea how this could be done. After some reading I figured out it is a Floating Point, but how to convert this like done on that website.
Is there someone that can help me with this!
Thanks a lot!
With the quite cryptic pack and unpack functions, it can be done in a one-liner:
function rawSingleHex($num) {
return strrev(unpack('h*', pack('f', $num))[1]);
}
This "packs" the number as its binary representation, then "unpacks" it in an array with one element: the binary representation in hexadecimal format. This format has the digits in the reversed order, so the function reverses that in the final result.
Call it by passing the floating point number:
echo rawSingleHex(5.6904566139035E-28);
Output:
12345678
Without pack/pack
(this was my original answer, but with the first option being available, this is not the advised way to proceed)
The binary format is explained in Wikipedia's article on the Single-precision floating-point format.
Here is a PHP function that implements the described procedure:
function rawSingleHex($num) {
if ($num == 0) return '00000000';
// set sign bit, and add another, higher one, which will be stripped later
$sign = $num < 0 ? 0x300 : 0x200;
$significant = abs($num);
$exponent = floor(log($significant, 2));
// get 24 most significant binary bits before the comma:
$significant = round($significant / pow(2, $exponent-23));
// exponent has exponent-bias format:
$exponent += 127;
// format: 1 sign bit + 8 exponent bits + 23 significant bits,
// without left-most "1" of significant
$bin = substr(decbin($sign + $exponent), 1) .
substr(decbin($significant), 1);
// assert that result has correct number of bits:
if (strlen($bin) !== 32) {
return "unexpected error";
}
// convert binary representation to hex, with exactly 8 digits
return str_pad(dechex(bindec($bin)), 8, "0", STR_PAD_LEFT);
}
It outputs the same as in the first solution.
I'm trying to convert a 64-bit float to a 64-bit integer (and back) in php. I need to preserve the bytes, so I'm using the pack and unpack functions. The functionality I'm looking for is basically Java's Double.doubleToLongBits() method. http://docs.oracle.com/javase/7/docs/api/java/lang/Double.html#doubleToLongBits(double)
I managed to get this far with some help from the comments on the php docs for pack():
function encode($int) {
$int = round($int);
$left = 0xffffffff00000000;
$right = 0x00000000ffffffff;
$l = ($int & $left) >>32;
$r = $int & $right;
return unpack('d', pack('NN', $l, $r))[1];
}
function decode($float) {
$set = unpack('N2', pack('d', $float));
return $set[1] << 32 | $set[2];
}
And this works well, for the most part...
echo decode(encode(10000000000000));
100000000
echo encode(10000000000000);
1.1710299640683E-305
But here's where it gets tricky...
echo decode(1.1710299640683E-305);
-6629571225977708544
I have no idea what's wrong here. Try it for yourself: http://pastebin.com/zWKC97Z7
You'll need 64-bit PHP on linux. This site seems to emulate that setup: http://www.compileonline.com/execute_php_online.php
$x = encode(10000000000000);
var_dump($x); //float(1.1710299640683E-305)
echo decode($x); //10000000000000
$y = (float) "1.1710299640683E-305";
var_dump($y); //float(1.1710299640683E-305)
echo decode($y); //-6629571225977708544
$z = ($x == $y);
var_dump($z); //false
http://www.php.net/manual/en/language.types.float.php
... never trust
floating number results to the last digit, and do not compare floating
point numbers directly for equality. If higher precision is necessary,
the arbitrary precision math functions and gmp functions are
available. For a "simple" explanation, see the » floating point guide
that's also titled "Why don’t my numbers add up?"
It is working properly, the only problem in this case is in logic of:
echo decode(1.1710299640683E-305);
You can't use "rounded" and "human readable" output of echo function to decode the original value (because you are loosing precision of this double then).
If you will save the return of encode(10000000000000) to the variable and then try to decode it again it will works properly (you can use echo on 10000000000000 without loosing precision).
Please see the example below which you can execute on PHP compiler as well:
<?php
function encode($int) {
$int = round($int);
$left = 0xffffffff00000000;
$right = 0x00000000ffffffff;
$l = ($int & $left) >>32;
$r = $int & $right;
return unpack('d', pack('NN', $l, $r))[1];
}
function decode($float) {
$set = unpack('N2', pack('d', $float));
return $set[1] << 32 | $set[2];
}
echo decode(encode(10000000000000)); // untouched
echo '<br /><br />';
$encoded = encode(10000000000000);
echo $encoded; // LOOSING PRECISION!
echo ' - "human readable" version of encoded int<br /><br />';
echo decode($encoded); // STILL WORKS - HAPPY DAYS!
?>
If you have a reliable fixed decimal point, like in my case and the case of currency, you can multiply your float by some power of 10 (ex. 100 for dollars).
function encode($float) {
return (int) $float * pow(10, 2);
}
function decode($str) {
return bcdiv($str, pow(10, 2), 2);
}
However, this doesn't work for huge numbers and doesn't officially solve the problem.
Seems like it's impossible to convert from an integer to a float string and back without losing the original integer value in php 5.4
I have some large HEX values that I want to display as regular numbers, I was using hexdec() to convert to float, and I found a function on PHP.net to convert that to decimal, but it seems to hit a ceiling, e.g.:
$h = 'D5CE3E462533364B';
$f = hexdec($h);
echo $f .' = '. Exp_to_dec($f);
Output: 1.5406319846274E+19 = 15406319846274000000
Result from calc.exe = 15406319846273791563
Is there another method to convert large hex values?
As said on the hexdec manual page:
The function can now convert values
that are to big for the platforms
integer type, it will return the value
as float instead in that case.
If you want to get some kind of big integer (not float), you'll need it stored inside a string. This might be possible using BC Math functions.
For instance, if you look in the comments of the hexdec manual page, you'll find this note
If you adapt that function a bit, to avoid a notice, you'll get:
function bchexdec($hex)
{
$dec = 0;
$len = strlen($hex);
for ($i = 1; $i <= $len; $i++) {
$dec = bcadd($dec, bcmul(strval(hexdec($hex[$i - 1])), bcpow('16', strval($len - $i))));
}
return $dec;
}
(This function has been copied from the note I linked to; and only a bit adapted by me)
And using it on your number:
$h = 'D5CE3E462533364B';
$f = bchexdec($h);
var_dump($f);
The output will be:
string '15406319846273791563' (length=20)
So, not the kind of big float you had ; and seems OK with what you are expecting:
Result from calc.exe =
15406319846273791563
Hope this help ;-)
And, yes, user notes on the PHP documentation are sometimes a real gold mine ;-)
hexdec() switches from int to float when the result is too large to be represented as an int. If you want arbitrarily long values, you're probably going to have to roll your own conversion function to change the hex string to a GMP integer.
function gmp_hexdec($n) {
$gmp = gmp_init(0);
$mult = gmp_init(1);
for ($i=strlen($n)-1;$i>=0;$i--,$mult=gmp_mul($mult, 16)) {
$gmp = gmp_add($gmp, gmp_mul($mult, hexdec($n[$i])));
}
return $gmp;
}
print gmp_strval(gmp_hexdec("D5CE3E462533364B"));
Output: 15406319846273791563
$num = gmp_init( '0xD5CE3E462533364B' ); // way to input a number in gmp
echo gmp_strval($num, 10); // display value in decimal
That's the module to use. Convert it to a function and then use on your numbers.
Note: provide these hex numbers as strings so:
$num = "0x348726837469972346"; // set variable
$gmpnum = gmp_init("$num"); // gmp number format
echo gmp_strval($gmpnum, 10); // convert to decimal and print out
1.5406319846274E+19 is a limited representation of you number. You can have a more complete one by using printf()
printf("%u\n", hexdec($h));
...will output "15406319846273792000". PHP uses floats for such big numbers, so you may lose a bit of precision. If you have to work with arbitrary precision numbers, you may try the bcmath extension. By splitting the hex into two 32-bit words (which should be safe on most systems) you should be able to get more precision. For instance:
$f = bcadd(bcmul(hexdec(substr($h, 0, -8)), 0x100000000), hexdec(substr($h, 8)));
...would set $f to 15406319846273791563.
Convert HEX to DEC is easy.. But, reconstruct back hexadecimal number is very hard.
Try to use base_convert ..
$hexadecimal = base_convert(2826896153644826, 10, 16);
// result: a0b0c0d0e0f1a
Run into this issue while storing 64-bit keys in MySQL database. I was able to get a bit perfect conversion to a 64-bit signed integer (PHP limitation) using a few binary operators: (This code is 16x faster than bchexdec function and resulting variables are using half the memory on average).
function x64toSignedInt($k){
$left = hexdec(substr($k,0,8));
$right = hexdec(substr($k,8,8));
return (int) ($left << 32) | $right;
}
MySQL signed BIGINT datatype is a great match for this as an index or storage in general. HEX(column) is a simple way to convert it back to HEX within the SQL query for use elsewhere.
This solution also uses the BC Math Functions. However, an algorithm is used which does without the bcpow function. This function is a bit shorter and faster than the accepted solution, tested on PHP 7.4.
function hexDecBc(string $hex) : string
{
for ($dec = '0', $i = 0; $i < strlen($hex); $i++) {
$dec = bcadd(bcmul($dec,'16'),(string)hexdec($hex[$i]));
}
return $dec;
}
Make sure to enable gmp extension. ext-gmp
$number = gmp_strval(gmp_init('0x03....')); // outputs: 1234324....
Doesn't intval(var, base) take care of it?
From the PHP Manual.
I'm getting some odd results where 2 identical functions (one in PHP and one in javascript) are returning different results.
The input for both of these lines of code is identical:
a = 4653896912;
b = 13;
I have double checked the variable types and both variables are numbers in JS and integers in PHP.
The line of code for PHP is this:
$a = $a >> $b;
For Javascript it's this:
a = a >> b;
You'd expect a to have the same value after both, but I'm getting the following:
PHP: $a = 568102
JS: a = 43814
Which has completely baffled me at this point.
Turns out this is definitely an issue of PHP using 64 bit integers and JS only using 32 bit. The problem I face now is that I need to get PHP to use 32-bit integers for these calculations. I found a function someone else wrote that looks like it should work, but it doesn't seem to be changing the output at all for me.
private static function toInt32(&$x) {
$z = hexdec(80000000);
$y = (int) $x;
if($y ==- $z && $x <- $z){
$y = (int) ((-1) * $x);
$y = (-1) * $y;
}
$x = $y;
}
The below code demonstrates masking the upper 32 bits of the number to retrieve only the lower 32 bits to use in your calculations. 4294967295 is 2^32 - 1. I think that if you mask all values that could be greater than 32 bits in this manner, then you can get the same results from your php and javascript.
<?php
$php_a = 4653896912;
$php_b = 13;
//convert $php_a into a 32 bit val
$php_a = $php_a & 4294967295;
$a = $php_a >> $php_b;
echo "PHP: \$a = $a <br />";
?>
<script type="text/javascript">
var a = 4653896912;
var b = 13;
var a = a >> b;
alert('Javascript A value is ' + a);
</script>
4653896912 is more than 32 bits.. unpredictable results are likely. I get $a = 43814 for PHP, but that is actually 358929617 >> 13, so in all likelihood PHP is doing 64 bit operations but JavaScript is only 32 bit.
I believe it's because you're a is above the limit of a 32-bit signed integer for [PHP][1].
The highest value possible is about 2 million, and a is over 4 billion.
When you're rolling over because of space limitations, results can be unpredictable (or at least, very difficult to figure out).
If your server is on a 64-bit version of PHP then it'll max out much higher than than, but javascript is limited by what the end-user is running.
You can read up on PHP on their integers page.