How should a crc32 be stored in MySQL? - php

I'm creating a crc32 in PHP and need to store it in a field in a MySQL database. After reading about how there is a concern about the results on a 32-bit vs 64-bit machine, I'm wondering how this number should be stored. This is how I'm treating the crc32 in PHP to get the same result on either bitsize machine:
<?php
$checksum = crc32("The quick brown fox jumped over the lazy dog.");
// On a 32-bit system it prints -2103228862 instead of
// 2191738434 which is correct and what prints on a 64-bit system.
// See the php.net manual page on crc32 for more information about
// 32-bit vs 64-bit.
echo "checksum without printf formatting: " . $checksum . "\n";
printf("%u\n", $checksum);
$string = sprintf("%u", $checksum);
echo $string . "\n";
?>
Output (on a 64-bit machine is):
checksum without printf formatting: 2191738434
2191738434
2191738434
How should this number be stored on MySQL? Here are a few choices I've come up with so far:
`hash1` CHAR(10) NOT NULL ,
`hash2` varchar(32) NOT NULL,
`hash3` int unsigned NOT NULL,
It looks like I should go with:
`hash4` BIGINT UNSIGNED NOT NULL ,

You can store the values in MySQL as INT UNSIGNED which occupies 4 bytes (i.e. 32 bits).
To insert the values into the database, you must use sprintf() with %u format on 32 bit machines:
$hash = crc32("The quick brown fox jumped over the lazy dog.");
$stmt = $db->prepare('INSERT INTO mytable VALUES (:hash)');
$stmt->execute(array(
':hash' => sprintf('%u', $hash),
));
Update
You could also make sure that you're always working with int32 types (signed long) on both 32 and 64-bit platforms. Currently, you can only accomplish this by using pack() and unpack():
echo current(unpack('l', pack('l', $hash)));
// returns -2103228862 on both 32-bit and 64-bit platforms
The idea for this was contributed by mindplay.dk

Related

When I try to insert ip2long Integer, the wrong Integer gets inserted into table

My code:
$ip = explode(',', $_SERVER['HTTP_X_FORWARDED_FOR'][0]); // Gives an IP address string. Example: "176.10.99.207"
$ipL = ip2long($ip); // An integer, which for this example would be 2953470927.
if( $stmt_insert_ip = $ip_link->prepare("INSERT INTO dataTable(ip,datetime) VALUES(?,?)") ){
$stmt_insert_ip->bind_param('is',$ipL,date("Y-m-d H:i:s"));
}
But when I check my MySQL table, the newly-inserted value under the ip column is 2147483647, which is "127.255.255.255".
When I run echo($ipL);, however, I get 2953470927.
What the H-E-Double hockey sticks is going on here?
I assume you have used INT as datatype in your database. To make this story short just change datatype of ip column to BIGINT and problem will be solved.
INT range (-2147483648, 2147483647)
BIGINT range (-9223372036854775808, 9223372036854775807)
2953470927 goes bit over the range of signed INT.
Refference: MySQL: Integer types
Unfortunately depending on your system (32-bit vs 64-bit) PHP returns different values.
on a 32-bit system it uses signed integers, and on a 64-bit machine it uses unsigned integers, so you'll need to store the data accordingly.
Based on what you returned it seems like you're using a 64-bit machine, so change your mysql column to an unsigned integer, or a signed or unsigned bigint

BIGINT in MySQL Changes a my number? Why does it do this?

I'll try and keep this simple. I'm using a BIGINT data type on a MySQL database table. I have a function generating a unique random number that may span anywhere from 12 digits to 13 digest long.
When I insert into the database with a digit that is 12 digits in length, it enters it just fine,
but when I use a value that is 13 digits or longer, it seems like it rounds up or something. here is the
php
$defaultText = 'some string'; //just some string
$web_id = 12;
$element_id = 23112182735363; //case 1 (doesn't work)
//$element_id = 2311218333205; //case 2, does work ()
mysql_query("INSERT INTO tableName (web_id, element_id, content)
VALUES ($web_id, $element_id, '".mysql_real_escape_string($defaultText)."')");
results:
in case one, it inserts a slightly different number, usually rounds up for some reason.
in case two, it inserts the number just fine! Maybe someone can help shed some light on this mystery! Thanks again!
the big int datatype:
bigint(200)
Numbers lose their precision from PHP_INT_MAX onwards. See also: http://www.php.net/manual/en/reserved.constants.php#constant.php-int-max
After that they are turned into floats which have limited precision and causes weird rounding issues. The only way to support BIGINT in PHP is by using strings.
I assumed you were talking about a 32-bit server.
But in my server, PHP seemed not lose the precision.
echo(PHP_INT_MAX . "<br/>");
$big_int = -6174803190685607000;
echo($big_int . '<br/>');
output
9223372036854775807<br/>-6174803190685607000<br/>
Sadly I still got the precision losing. I guessed it might because i used 'i' in prepare statement of mysqli, but I could not prove it.

PHP: Converting big integer from string to int with overflow

PHP does not support unsigned ints. Is there a way to convert a string representation of an unsigned integer into a signed integer with overflow?
Example:
On a 32 bit system, PHP can store int values <= 2147483647. I want a way to convert the string "2147483648" to integer, causing it to overflow to -2147483648 instead of being reduced to 2147483647.
Why do I want to do this?
I store IPv4 addresses in a database as unsigned int (32 bits). I want to do binary operations on the addresses in PHP to check for subnets. This needs to be done on every request, so it needs to be quick. Therefore it seems better to store the IP address as an unsigned int rather than storing a string which will have to be converted back and forth.
here is the workaround.
<?php
$unsignedString = "3000000000";
echo "unsigned string: ".$unsignedString."<br />";
$signedInt = intval(doubleval($unsignedString));
echo "signed int: ".$signedInt."<br />";
?>
The fastest and easiest way to do this is to get your RDBMS to do it somehow.
You can find out what the size of an integer is in PHP by checking the value of the predefined constant PHP_INT_SIZE. This will be 4 if running on a 32-bit system, 8 if running on a 64-bit system.
I suggest that you populate a variable with for example
$smallIntegers = intval(PHP_INT_SIZE == 4);
and then in your query use something like this:
SELECT
...,
CASE
WHEN :smallIntegers: = 1 AND IPAddress > 2147483647 THEN IPAddress - 4294967296
ELSE IPAddress
END AS IPAddress,
...
...
Could you store the IP address in an integer array, one segment of the IP to each part of the array so the max value would end up as 255 stored in anyone one integer:
$ipAddress = array(127, 0, 0, 1);

Unsigned int to signed in php

It appears, that in 32bit OS ip2long returns signed int, and in 64bit OS unsigned int is returned.
My application is working on 10 servers, and some are 32bit and some are 64bit, so I need all them to work same way.
In PHP documentation there is a trick to make that result always unsigned, but since I got my database already full of data, I want to have it signed.
So how to change an unsigned int into a signed one in PHP?
PHP does not support unsigned integers as a type, but what you can do is simply turn the result of ip2long into an unsigned int string by having sprintf interpret the value as unsigned with %u:
$ip="128.1.2.3";
$signed=ip2long($ip); // -2147417597 in this example
$unsigned=sprintf("%u", $signed); // 2147549699 in this example
Edit, since you really wanted it to be signed even on 64 bit systems - here's how you'd convert the 64 bit +ve value to a 32 bit signed equivalent:
$ip = ip2long($ip);
if (PHP_INT_SIZE == 8)
{
if ($ip>0x7FFFFFFF)
{
$ip-=0x100000000;
}
}
Fwiw, if you're using MySQL it's usually a lot easier and cleaner if you just pass in the IPs as strings to the database, and let MySQL do the conversion using INET_ATON() (when INSERTing/UPDAT(E)'ing) and INET_NTOA() (when SELECTing). MySQL does not have any of the problems described here.
Examples:
SELECT INET_NTOA(ip_column) FROM t;
INSERT INTO t (ip_column) VALUES (INET_ATON('10.0.0.1'));
The queries are also much more readable.
Note that you can not mix INET_NTOA()/INET_ATON() in MySQL with ip2long()/long2ip() in PHP, since MySQL uses an INT UNSIGNED datatype, while PHP uses a signed integer. Mixing signed and unsigned integers will seriously mess up your data!
interpreting an integer value as signed int on 32 and 64 bit systems:
function signedint32($value) {
$i = (int)$value;
if (PHP_INT_SIZE > 4) // e.g. php 64bit
if($i & 0x80000000) // is negative
return $i - 0x100000000;
return $i;
}
-Misunderstood problem, see Paul Dixon's answer above.
64bit unsigned integers are not technically supported in PHP5. It will use the native type. To convert to a 32bit signed int from a 64bit signed int without losing the high bit, you could mask and then type cast the output:
$ip_int = ip2long($ip);
if (PHP_INT_SIZE == 8) // 64bit native
{
$temp_int = (int)(0x7FFFFFFF & $ip_int);
$temp_int |= (int)(0x80000000 & ($ip_int >> 32));
$ip_int = $temp_int;
}
On a 64 bit system, printing this value ($ip_int) will display an 'unsigned' integer since we've removed the high bit. However this should allow you to take the output and store it how you wish.
public function unsigned2signed($num) { // converts unsigned integer to signed
$res = pack('i',$num); // change to 'l' to handle longs
$res = unpack('i',$res)[1];
return $res;
}
unsigned-int is a php library to convert signed integers to unsigned. (Disclaimer: I'm the author)
use Oct8pus\Unsigned\UInt32;
require_once './vendor/autoload.php';
$uint32 = new UInt32(-2147483648);
echo $uint32;
-2147483648 > 0x80000000 (2147483648)
The repository is here https://github.com/8ctopus/unsigned-int

PHP strptime format bug?

I am wrestling with a php 5.2.6 problem. An api we use returns dates in this format DDMMYYYYHHMM. Exactly that format, fixed length, no delimiters. However, in my experimentation, this format seems to break strptime, which returns a false (fail) when I feed it a date in this format. It can reproduced, at least on my system, with this example:
$format = "%d%m%Y%H%M"; echo print_r(strptime(strftime($format,1225405967),$format),true);
If I add any character between the date and the time, it works, even a space. So, this DOES work:
$format = "%d%m%Y %H%M"; echo print_r(strptime(strftime($format,1225405967),$format),true);
Am I missing something obvious?
edit: further to this and owing to the results indicated by the comments, this seems to be platform specific. I can reproduce it on the Macs running OSX Leopard in the office but the Linux boxes parse it fine. I assume it is a bug or idiosyncrasy of the strptime in the underlying C library in the *nix of OSX.
This function is locale-dependent. Have you tried setting different locale? (see setlocale())
If you think the problem is on MacOS X and in the C library, then you could produce a test case to demonstrate it. For example, I ran the code below on MacOS X 10.4.11 (PPC, G4, 32-bit), and got the output:
Now: 1225573977
Formatted (12): 011120082112
End: 0xBFFFF553 (Buffer: 0xBFFFF547)
Then: year = 2008, month = 11, day = 1, hour = 21, minute = 12
Reformatted (12): 011120082112
The code I used is:
#include <time.h>
#include <stdio.h>
int main(void)
{
time_t now = time(0);
struct tm *tm = gmtime(&now);
char format[] = "%d%m%Y%H%M";
char buffer1[64];
char buffer2[64];
size_t f_len = strftime(buffer1, sizeof(buffer1), format, tm);
struct tm then;
char *end = strptime(buffer1, format, &then);
size_t p_len = strftime(buffer2, sizeof(buffer2), format, &then);
printf("Now: %ld\n", (long)now);
printf("Formatted (%lu): %s\n", (unsigned long)f_len, buffer1);
printf("End: 0x%08lX (Buffer: 0x%08lX)\n", (unsigned long)end, (unsigned long)buffer1);
printf("Then: year = %d, month = %d, day = %d, hour = %d, minute = %d\n",
then.tm_year + 1900, then.tm_mon + 1, then.tm_mday, then.tm_hour, then.tm_min);
printf("Reformatted (%lu): %s\n", (unsigned long)p_len, buffer2);
return(0);
}
On the basis of what I see, there is no bug in strptime() in the version I'm using. We can debate about the merits of the non-existent error checking, and of the casts versus C99 <inttypes.h> notations in the printing, but I think the code is accurate enough. I used gmtime() instead of localtime(); I doubt if that was a factor in not reproducing the problem.
Maybe you should look at the PHP test suite?
Maybe you should split your rather complex expression into pieces to detect where the error occurs in PHP?
Nothing obvious since both versions work fine in PHP 5.2.0. I can't readily check 5.2.6 at the moment, though. That will have to wait until I get home.
It's very likely that this is isolated to Macs only, as this hasn't even been implemented on Windows yet (although I've passed that suggestion along).
5.2.6
Fatal error: Call to undefined function strptime() in F:\htdocs\strptime.php on line 5
I would go ahead and submit a bug at bugs.php.net.
I confirm:
$p = strptime("09.02.2002", "%d.%m.%Y");
var_dump($p);
Output:
array(9) {
["tm_sec"]=>
int(0)
["tm_min"]=>
int(0)
["tm_hour"]=>
int(0)
["tm_mday"]=>
int(9)
["tm_mon"]=>
int(1)
["tm_year"]=>
int(102)
["tm_wday"]=>
int(0)
["tm_yday"]=>
int(0)
["unparsed"]=>
string(0) ""
}
OS X Leopard, PHP 5.3.2

Categories