The goal is to store the hash on a mysql database, using INT (not BIGINT or MEDIUMINT).
md5('string', true) returns binary data, 16 bytes of hash. I thought i could grep the first 4 bytes and convert it to an INT(32bit/4bytes) integer, but i don't know how to do it.
What do you suggest? Thanks.
Use crc32, it will return a 32bit int.
var_dump (crc32 ("hello world"));
var_dump (crc32 ("world hello"));
output
int(222957957)
int(1292159901)
PHP: crc32 - Manual
Generates the cyclic redundancy checksum polynomial of 32-bit lengths
of the str. This is usually used to validate the integrity of data
being transmitted.
Because PHP's integer type is signed, and many crc32 checksums will
result in negative integers, you need to use the "%u" formatter of
sprintf() or printf() to get the string representation of the unsigned
crc32 checksum.
The php hash() function will be even better than crc32():
hash("crc32b", $str);
And if an integer is required:
intval(hash("crc32b", $str), 16);
Compared to the plain crc32(), it will not be affected by the signed/unsigned integer inconsistency between 32-bit and 64-bit systems (see PHP docs for the details: crc32)
ord($hash[0]) * 16777216 + ord($hash[1]) * 65536 + ord($hash[2]) * 256 + ord($hash[3]) ;
Or:
unpack("L", substr($hash,0,4));
But Filip Roséen's solution is better.
Another alternative is to use a map to link your hash to integer, such as Redis.
L43F34FLK34FL3K4 ==> 1
LKZLCKCLK32 ==> 2
etc....
Here is Daniel J. Bernstein hash that returns an 32bit integer.
function djb_hash($str) {
for ($i = 0, $h = 5381, $len = strlen($str); $i < $len; $i++) {
$h = (($h << 5) + $h + ord($str[$i])) & 0x7FFFFFFF;
}
return $h;
}
Related
I am trying to convert a hex string into a signed integer.
I am able to easily transfer it into an unsigned value with hexdec() but this does not give a signed value.
Edit:
code in VB - the two "AA" hex values are representative.
Dim bs(2) As Byte
bs(1) = "AA"
bs(2) = "AA"
Dim s As Short
s = BitConverter.ToInt16(bs, 1)
Check out this comment via php.net:
hexdec() returns unsigned integers. For example hexdec("FFFFFFFE") returns 4294967294, not -2. To convert to signed 32-bit integer you may do:
<?php
echo reset(unpack("l", pack("l", hexdec("FFFFFFFE"))));
?>
As said on the hexdec manual page :
The function can now convert values
that are to big for the platforms
integer type, it will return the value
as float instead in that case.
If you want to get some kind of big integer (not float), you'll need it stored inside a string... This might be possible using BC Math functions.
For instance, if you look in the comments of the hexdec manual page, you'll find this note
If you adapt that function a bit, to avoid a notice, you'll get :
function bchexdec($hex)
{
$dec = 0;
$len = strlen($hex);
for ($i = 1; $i <= $len; $i++) {
$dec = bcadd($dec, bcmul(strval(hexdec($hex[$i - 1])), bcpow('16', strval($len - $i))));
}
return $dec;
}
(This function has be copied from the note I linked to ; and only a bit adapted by me)
And using it on your number :
$h = 'D5CE3E462533364B';
$f = bchexdec($h);
var_dump($f);
The output will be :
string '15406319846273791563' (length=20)
So, not the kind of big float you had ; and seems OK with what you are expecting :
Result from calc.exe =
15406319846273791563
Hope this help ;-)
And, yes, user notes on the PHP documentation are sometimes a real gold mine ;-)
I've been trying to find a decent answer to this question and so I wrote this function which works well for a Hex string input, returning a signed decimal value.
public function hex_to_signed_int($hex_str){
$dec = hexdec($hex_str);
//test is value negative
if($dec & pow(16,strlen($hex_str))/2 ){ return $dec-pow(16,strlen($hex_str));}
return $dec;
}
I have some large HEX values that I want to display as regular numbers, I was using hexdec() to convert to float, and I found a function on PHP.net to convert that to decimal, but it seems to hit a ceiling, e.g.:
$h = 'D5CE3E462533364B';
$f = hexdec($h);
echo $f .' = '. Exp_to_dec($f);
Output: 1.5406319846274E+19 = 15406319846274000000
Result from calc.exe = 15406319846273791563
Is there another method to convert large hex values?
As said on the hexdec manual page:
The function can now convert values
that are to big for the platforms
integer type, it will return the value
as float instead in that case.
If you want to get some kind of big integer (not float), you'll need it stored inside a string. This might be possible using BC Math functions.
For instance, if you look in the comments of the hexdec manual page, you'll find this note
If you adapt that function a bit, to avoid a notice, you'll get:
function bchexdec($hex)
{
$dec = 0;
$len = strlen($hex);
for ($i = 1; $i <= $len; $i++) {
$dec = bcadd($dec, bcmul(strval(hexdec($hex[$i - 1])), bcpow('16', strval($len - $i))));
}
return $dec;
}
(This function has been copied from the note I linked to; and only a bit adapted by me)
And using it on your number:
$h = 'D5CE3E462533364B';
$f = bchexdec($h);
var_dump($f);
The output will be:
string '15406319846273791563' (length=20)
So, not the kind of big float you had ; and seems OK with what you are expecting:
Result from calc.exe =
15406319846273791563
Hope this help ;-)
And, yes, user notes on the PHP documentation are sometimes a real gold mine ;-)
hexdec() switches from int to float when the result is too large to be represented as an int. If you want arbitrarily long values, you're probably going to have to roll your own conversion function to change the hex string to a GMP integer.
function gmp_hexdec($n) {
$gmp = gmp_init(0);
$mult = gmp_init(1);
for ($i=strlen($n)-1;$i>=0;$i--,$mult=gmp_mul($mult, 16)) {
$gmp = gmp_add($gmp, gmp_mul($mult, hexdec($n[$i])));
}
return $gmp;
}
print gmp_strval(gmp_hexdec("D5CE3E462533364B"));
Output: 15406319846273791563
$num = gmp_init( '0xD5CE3E462533364B' ); // way to input a number in gmp
echo gmp_strval($num, 10); // display value in decimal
That's the module to use. Convert it to a function and then use on your numbers.
Note: provide these hex numbers as strings so:
$num = "0x348726837469972346"; // set variable
$gmpnum = gmp_init("$num"); // gmp number format
echo gmp_strval($gmpnum, 10); // convert to decimal and print out
1.5406319846274E+19 is a limited representation of you number. You can have a more complete one by using printf()
printf("%u\n", hexdec($h));
...will output "15406319846273792000". PHP uses floats for such big numbers, so you may lose a bit of precision. If you have to work with arbitrary precision numbers, you may try the bcmath extension. By splitting the hex into two 32-bit words (which should be safe on most systems) you should be able to get more precision. For instance:
$f = bcadd(bcmul(hexdec(substr($h, 0, -8)), 0x100000000), hexdec(substr($h, 8)));
...would set $f to 15406319846273791563.
Convert HEX to DEC is easy.. But, reconstruct back hexadecimal number is very hard.
Try to use base_convert ..
$hexadecimal = base_convert(2826896153644826, 10, 16);
// result: a0b0c0d0e0f1a
Run into this issue while storing 64-bit keys in MySQL database. I was able to get a bit perfect conversion to a 64-bit signed integer (PHP limitation) using a few binary operators: (This code is 16x faster than bchexdec function and resulting variables are using half the memory on average).
function x64toSignedInt($k){
$left = hexdec(substr($k,0,8));
$right = hexdec(substr($k,8,8));
return (int) ($left << 32) | $right;
}
MySQL signed BIGINT datatype is a great match for this as an index or storage in general. HEX(column) is a simple way to convert it back to HEX within the SQL query for use elsewhere.
This solution also uses the BC Math Functions. However, an algorithm is used which does without the bcpow function. This function is a bit shorter and faster than the accepted solution, tested on PHP 7.4.
function hexDecBc(string $hex) : string
{
for ($dec = '0', $i = 0; $i < strlen($hex); $i++) {
$dec = bcadd(bcmul($dec,'16'),(string)hexdec($hex[$i]));
}
return $dec;
}
Make sure to enable gmp extension. ext-gmp
$number = gmp_strval(gmp_init('0x03....')); // outputs: 1234324....
Doesn't intval(var, base) take care of it?
From the PHP Manual.
I'm having a problem with converting binary strings to signed integers
If you call decbin('-40'), php will output 1111111111111111111111111111111111111111111111111111111111011000
But if you call bindec(decbin('-40')), it will output 1.84467440737E+19 (or something similar, which is obviously not -40) because it "sees the most significant bit as another order of magnitude rather than as the sign bit" - php manual
Is there a way to convert a binary 64 bit binary string (much like the one output by decbin) string into a signed integer?
From the documentation, you cannot use bindec
bindec() interprets all binary_string values as unsigned integers. This is because bindec() sees the most significant bit as another order of magnitude rather than as the sign bit.
base_convert appears to ignore signing altogether.
If you know that your incoming string will always be 64 bit binary and you are not on a 32 bit system, it's quite easy to write a custom function.
Check if the string is 64 characters long.
Check if the most significant bit is a 1.
Flip all bits
Add 1
Negate
Here's a quick one I knocked together.
function bindec2($bin)
{
if (strlen($bin) == 64 && $bin[0] == '1') {
for ($i = 0; $i < 64; $i++) {
$bin[$i] = $bin[$i] == '1' ? '0' : '1';
}
return (bindec($bin) + 1) * -1;
}
return bindec($bin);
}
Use pack and unpack to convert between int and binary string.
// 'i' means signed integer
var_dump(unpack('i', pack('i', '-40'))); // gives you -40
The goal is to store the hash on a mysql database, using INT (not BIGINT or MEDIUMINT).
md5('string', true) returns binary data, 16 bytes of hash. I thought i could grep the first 4 bytes and convert it to an INT(32bit/4bytes) integer, but i don't know how to do it.
What do you suggest? Thanks.
Use crc32, it will return a 32bit int.
var_dump (crc32 ("hello world"));
var_dump (crc32 ("world hello"));
output
int(222957957)
int(1292159901)
PHP: crc32 - Manual
Generates the cyclic redundancy checksum polynomial of 32-bit lengths
of the str. This is usually used to validate the integrity of data
being transmitted.
Because PHP's integer type is signed, and many crc32 checksums will
result in negative integers, you need to use the "%u" formatter of
sprintf() or printf() to get the string representation of the unsigned
crc32 checksum.
The php hash() function will be even better than crc32():
hash("crc32b", $str);
And if an integer is required:
intval(hash("crc32b", $str), 16);
Compared to the plain crc32(), it will not be affected by the signed/unsigned integer inconsistency between 32-bit and 64-bit systems (see PHP docs for the details: crc32)
ord($hash[0]) * 16777216 + ord($hash[1]) * 65536 + ord($hash[2]) * 256 + ord($hash[3]) ;
Or:
unpack("L", substr($hash,0,4));
But Filip Roséen's solution is better.
Another alternative is to use a map to link your hash to integer, such as Redis.
L43F34FLK34FL3K4 ==> 1
LKZLCKCLK32 ==> 2
etc....
Here is Daniel J. Bernstein hash that returns an 32bit integer.
function djb_hash($str) {
for ($i = 0, $h = 5381, $len = strlen($str); $i < $len; $i++) {
$h = (($h << 5) + $h + ord($str[$i])) & 0x7FFFFFFF;
}
return $h;
}
I'm pretty inexperienced in PHP so this may be obvious to some of you, but if I call md5($mystring,true) in PHP it says it returns a "raw binary format with a length of 16". So what is that? Is it an array? An array of what? How do I read the individual bits and bytes of that return value?
None of the examples I can find online use it without going straight into base64_encode() or something. I just want to be able to check the fifth bit, or the third byte, for example.
var_dump(md5("string", TRUE));
"Raw binary format" means a string (as strings are binary-safe in PHP):
string(16) "�\����= �(��^{!"
If you want to read a byte out of that, use the $string[5] offset syntax for that. Or to extract a single bit out of the fifth byte for example:
$bit4_from_5th_byte = ord($result[5]) & (1 << 4);
It returns it as a "string" with each Character being a byte of the output. What you might consider instead is using the hex output of md5() without the second parameter and converting substrings of it to numbers using intval() with 16 as the base parameter. Like so:
$hash = md5($raw);
$byte = intval(substr($hash, 0, 2), 16);
An MD5 hash is a 128bit number, e.g. 16 bytes of raw binary data. For legibility, PHP's md5() function defaults to outputting this as a human readable string, where those 16 binary bytes get converted into a 32 character alpha-numeric string.
When you specify that second true parameter for MD5, you're telling PHP to return that raw 128bit number instead of the human readable version.
The RAW output of MD5 is the plain binary string of the hash. You cannot print it to the screen since it's not encoded as actual ASCII characters but binary numbers. This is only useful if you need to store or transfer the hash in a binary format.
To get the individual bits:
$md5 = md5( "b", true );
$l = strlen( $md5 );
$bits = "";
for( $i = 0; $i < $l; ++$i ) {
$num = ord( $md5[$i] );
for( $j = 7; $j >= 0; --$j ) {
$bits .= ( $num & ( 1 << $j ) ) ? "1" : "0";
}
}
echo $bits."\n";
//10010010111010110101111111111110111001101010111000101111111011000011101011010111000111000111011101110101001100010101011110001111
echo asciibin2hex( $bits ) == md5("b"); // true
So we go byte by byte, and since it is in string form, we need to convert it to ASCII number by ord(). Now go through the byte, bit by bit and see which are turned on and which are turned off and concatenate to the bit string. Go to next byte and rinse and repeat until all 128 bits are read.
Read third bit ( from the left ) in third byte:
ord( $md5[2] ) & ( 1 << ( 8 - 3 ) )
returns 1 if the bit is turned on, 0 otherwise