I have a table in a MySQL database with a BIT(64) row, which is a bitmask of 64 flags.
Using PHP and mysqli, I have an integer $n, which is in the range [0, 64). I have already written a function (see Appendix I) to set the nth bit of a byte array (i.e. string in PHP).
When passing the abovementioned byte-array (string) to MySQL, it seems that bitmask & ?, where ? is mysqli::bind_paramed as the byte array (param type is "s"), does not compare the bits as expected.
For example, using this query:
SELECT id FROM my_table WHERE (bitmask & ?) > 0
upon this table:
CREATE TABLE my_table (id INT PRIMARY KEY, bitmask BIT(64));
How can this be fixed?
I thought of passing a bin2hex and UNHEX() it, but this doesn't seem to fix the problem.
Appendix I
public static function setNthBit(int $n, int $bytes = 8) : string{
$offset = $n >> 3;
$byteArray = str_repeat("\0", $bytes);
$byteArray{$bytes - 1 - $offset} = chr(1 << ($n & 7));
return $byteArray;
}
It appears that the problem originates from that MySQL does not support bitwise comparison of strings. When I pass the bitmask as the parameter directly, MySQL would try to interpret as an integer, e.g. trying to interpret bitmask & '123' as bitmask & 123. I'm unsure how MySQL interprets a bunch of binary characters like \xFF or \x00 -- it just won't work, probably interpreted as 0.
I solved this by passing the bin2hex of the bitmask in the input and then CONV(?, 16, 10) in the query. CONV() will express the data in decimal, which MySQL will interpret as, hopefully, something like a BIGINT, which can be successfully bitwise-compared with a BIT(64) row.
Related
As mentioned in https://www.sqlite.org/datatype3.html :
INTEGER: The value is a signed integer, stored in 1, 2, 3, 4, 6, or 8
bytes depending on the magnitude of the value.
The problem is that the statement below gives the desired result as the bound values are comparatively smaller (e.g, $roll_no = 1111111111), however, the execution of statement fetches no result when the bound value is bigger(e.g, $roll_no =3333333333) whereas the SQLite table already holds record with that bigger value.
Is it because the parameter value is truncated or any other reason? What is to be done to get rid of it?
$stmt1 = $db->prepare("select sname,reg_no from student where roll_no=:roll_no");
$stmt1->bindParam(':roll_no', $roll_no, SQLITE3_INTEGER);
See the PDO driver of PHP:
https://github.com/php/php-src/search?q=SQLITE_INTEGER&unscoped_q=SQLITE_INTEGER
#if LONG_MAX <= 2147483647
if (val > ZEND_LONG_MAX || val < ZEND_LONG_MIN) {
ZVAL_STRINGL(data, (char *)sqlite3_column_text(stmt, column), sqlite3_column_bytes(stmt, column));
It supports returning larger integers, but only as strings.
There's no way to have 3333333333 as native integer on PHPs end (32-bit versions). Such it would become a float before it even reaches SQLite.
What you should do is not trying to bind it as integer. Use a string. SQL does type casting of its own. You could likely keep the column as INTEGER on SQLites end even. It's only PHP you have to work around. Or you know, do the overdue upgrade.
I have created a table in MySQL with a binary field name 'active' and set to NULL by default.
But when I want to update it with this command:
$update_users = $bdd -> query("UPDATE users SET `active` = 1 WHERE `id` = '$data1' LIMIT 1") or die(mysql_error());
The field is updated to 31 ! and not to 1.
I have also tried
SET `active` = true
but same result.
p.s: I have set 'active' using phpMyAdmin interface (no SQL statement), but here are the values of this field: Type => binary(1), Null => yes, Defaut => NULL
0x31 == 49 is the ASCII code of character '1'.
The value is stored in the column but, because of its type, it is returned in a special format: each byte from the value is displayed in its hexadecimal representation (2 uppercase hex digits). Depending on the MySQL client you use, the 0x hexadecimal prefix may or may not be present in the output.
As the documentation explains, the BINARY type contains strings (similar to CHAR types). That's why the numeric value you want to insert (1) is handled as a string and it becomes the character '1').
In order to store the number 1 into a BINARY column you have to store the character '0x01' in it:
UPDATE users SET `active` = 0x01 WHERE `id` = '$data1' LIMIT 1
Or you better use the TINYINT type instead.
I think you are confusing BINARY. According to the documentation:
The BINARY and VARBINARY types are similar to CHAR and VARCHAR, except that they contain binary strings rather than nonbinary strings. That is, they contain byte strings rather than character strings. This means they have the binary character set and collation, and comparison and sorting are based on the numeric values of the bytes in the values.
From looking at your question I assume what you really want is something like a boolean value (0 or 1) and not binary data.
In this case either use TINYINT(1) or BIT(1).
Say i have three bit flags in a status, stored in mysql as an integer
Approved Has Result Finished
0|1 0|1 0|1
Now i want to find the rows with status: Finished = 1, Has Result = 1 and Approved = 0.
Then the data:
0 "000"
1 "001"
3 "011"
7 "111"
Should produce
false
false
true
false
Can I do something like? (in mysql)
status & "011" AND ~((bool) status & "100")
Can't quite figure out how to query "Approved = 0".
Or should i completely drop using bit flags, and split these into separate columns?
The reasoning for using bit flags is, in part, for mysql performance.
Use ints instead of binary text. Instead of 011, use 3.
To get approved rows:
SELECT
*
FROM
`foo`
WHERE
(`status` & 4)
or approved and finished rows:
SELECT
*
FROM
`foo`
WHERE
(`status` & 5)
or finished but not accepted:
SELECT
*
FROM
`foo`
WHERE
(`status` & 1)
AND
(`status` ^ 4)
"Finished = 1, Has Result = 1 and Approved = 0" could be as simple as status = 3.
Something I liked to do when I began programming was using powers of 2 as flags given a lack of boolean or bit types:
const FINISHED = 1;
const HAS_RESULT = 2;
const APPROVED = 4;
Then you can check like this:
$status = 5; // 101
if ($status & FINISHED) {
/*...*/
}
EDIT:
Let me expand on this:
Can't quite figure out how to query "Approved = 0".
Or should i completely drop using bit flags, and split these into separate columns?
The reasoning for using bit flags is, in part, for mysql performance.
The issue is, you are not using bitwise flags. You are using a string which "emulates" a bitwise flag and sort of makes it hard to actually do proper flag checking. You'd have to convert it to its bitwise representation and then do the checking.
Store the flag value as an integer and declare the flag identifiers that you will then use to do the checking. A TINYINT should give you 7 possible flags (minus the most significant bit used for sign) and an unsigned TINYINT 8 possible flags.
I am trying to match a md5 has (generated through php) to its original value in a SQLExpress database.
I am using the following function in my SQL query
master.sys.fn_varbintohexsubstring(0, HASHBYTES('MD5', 'ID'), 1, 0)
Where 'ID' is the field in the database.
However they both seem to return different values for the md5 hash. I have been using '12290' as a static value to test this.
php md5() returns: 0bd81786a8ec6ae9b22cbb3cb4d88179
The following SQL Statement returns the same output:
DECLARE #password VARCHAR(255)
SET #password = master.sys.fn_varbintohexsubstring(0, HASHBYTES('MD5', '12290'), 1, 0)
SELECT #password
Yet when I run the following statement from the table:
SELECT ID, master.sys.fn_varbintohexsubstring(0, HASHBYTES('MD5', CONVERT(NVARCHAR(255), ID)), 1, 0) AS temp
FROM Clients
ORDER BY ID ASC
The 'temp' value matching to the 'ID' value of 12290 returns: 1867dce5f1ee1ddb46ff0ccd1fc58e03
Any help on the matter would be much appreciated!
Thanks
Python helped me to help you.
>>> from hashlib import md5
>>> md5('1\x002\x002\x009\x000\x00').digest().encode('hex')
'1867dce5f1ee1ddb46ff0ccd1fc58e03'
NVARCHAR is Unicode type and it seems from the above experiment that '12990' is stored as UTF-16LE in your database: '1\02\09\09\00\0'.
Assuming that the data encoding in the PHP is UTF-8 data and you don't want to change the existing data in the database, this is how you can fix your PHP script:
<?php
$password = '12290';
$hash = md5(mb_convert_encoding($password, 'UTF-16LE', 'UTF-8')) . "\n";
echo $hash;
?>
Output:
susam#swift:~$ php utf16le-hash.php
1867dce5f1ee1ddb46ff0ccd1fc58e03
In case the data in PHP is in some other encoding such as ASCII, ISO-8859-1, etc. you can change the third argument to mb_convert_encoding accordingly. The list of all supported encodings is available at: http://www.php.net/manual/en/mbstring.supported-encodings.php
Also, see http://www.php.net/manual/en/function.mb-convert-encoding.php
I don't have SQL server to test this on, but the CONVERT command might be creating the NVARCHAR with 240-odd trailing blanks (as you have specified NVARCHAR(255))
Try setting the NVARCHAR to the length of the ID to test:
ARE #password VARCHAR(255)
SET #password = master.sys.fn_varbintohexsubstring(0, HASHBYTES('MD5', CONVERT(NVARCHAR(5), '12290')), 1, 0)
SELECT #password
Try with different lengths in the CONVERT - is there any difference?
One of two things is most likely the problem:
Either the ID column in that row has a value not exactly equal to '12290' (e.g. extra whitespace)
Or the CONVERT function produces such a value
In any case, a standard debugging approach would be to use an SQL query to SELECT the string lengths of that ID field and the return value of CONVERT; if either is not equal to 5, you found the error.
Alternatively you can perform a dump of the table in question including data, and look at the generated INSERT statement to see what the database says the value in that column is.
It appears, that in 32bit OS ip2long returns signed int, and in 64bit OS unsigned int is returned.
My application is working on 10 servers, and some are 32bit and some are 64bit, so I need all them to work same way.
In PHP documentation there is a trick to make that result always unsigned, but since I got my database already full of data, I want to have it signed.
So how to change an unsigned int into a signed one in PHP?
PHP does not support unsigned integers as a type, but what you can do is simply turn the result of ip2long into an unsigned int string by having sprintf interpret the value as unsigned with %u:
$ip="128.1.2.3";
$signed=ip2long($ip); // -2147417597 in this example
$unsigned=sprintf("%u", $signed); // 2147549699 in this example
Edit, since you really wanted it to be signed even on 64 bit systems - here's how you'd convert the 64 bit +ve value to a 32 bit signed equivalent:
$ip = ip2long($ip);
if (PHP_INT_SIZE == 8)
{
if ($ip>0x7FFFFFFF)
{
$ip-=0x100000000;
}
}
Fwiw, if you're using MySQL it's usually a lot easier and cleaner if you just pass in the IPs as strings to the database, and let MySQL do the conversion using INET_ATON() (when INSERTing/UPDAT(E)'ing) and INET_NTOA() (when SELECTing). MySQL does not have any of the problems described here.
Examples:
SELECT INET_NTOA(ip_column) FROM t;
INSERT INTO t (ip_column) VALUES (INET_ATON('10.0.0.1'));
The queries are also much more readable.
Note that you can not mix INET_NTOA()/INET_ATON() in MySQL with ip2long()/long2ip() in PHP, since MySQL uses an INT UNSIGNED datatype, while PHP uses a signed integer. Mixing signed and unsigned integers will seriously mess up your data!
interpreting an integer value as signed int on 32 and 64 bit systems:
function signedint32($value) {
$i = (int)$value;
if (PHP_INT_SIZE > 4) // e.g. php 64bit
if($i & 0x80000000) // is negative
return $i - 0x100000000;
return $i;
}
-Misunderstood problem, see Paul Dixon's answer above.
64bit unsigned integers are not technically supported in PHP5. It will use the native type. To convert to a 32bit signed int from a 64bit signed int without losing the high bit, you could mask and then type cast the output:
$ip_int = ip2long($ip);
if (PHP_INT_SIZE == 8) // 64bit native
{
$temp_int = (int)(0x7FFFFFFF & $ip_int);
$temp_int |= (int)(0x80000000 & ($ip_int >> 32));
$ip_int = $temp_int;
}
On a 64 bit system, printing this value ($ip_int) will display an 'unsigned' integer since we've removed the high bit. However this should allow you to take the output and store it how you wish.
public function unsigned2signed($num) { // converts unsigned integer to signed
$res = pack('i',$num); // change to 'l' to handle longs
$res = unpack('i',$res)[1];
return $res;
}
unsigned-int is a php library to convert signed integers to unsigned. (Disclaimer: I'm the author)
use Oct8pus\Unsigned\UInt32;
require_once './vendor/autoload.php';
$uint32 = new UInt32(-2147483648);
echo $uint32;
-2147483648 > 0x80000000 (2147483648)
The repository is here https://github.com/8ctopus/unsigned-int