Debug PDO mySql insert NULL into database instead of empty - php

I am trying to dynamically insert 'NULL' into the database using PDO.
TABLE STRUCTURE:
CREATE TABLE IF NOT EXISTS `Fixes` (
`Id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'PK',
`CurrencyId` int(11) NOT NULL COMMENT 'FK',
`MetalId` int(11) NOT NULL COMMENT 'FK',
`FixAM` decimal(10,5) NOT NULL,
`FixPM` decimal(10,5) DEFAULT NULL,
`TimeStamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`Id`),
KEY `CurrencyId` (`CurrencyId`),
KEY `MetalId` (`MetalId`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci AUTO_INCREMENT=13 ;
PHP / PDO QUERY:
$sql = 'UPDATE
Fixes
SET
FixAM = :fixAM,
FixPM = :fixPM
WHERE
MetalId IN (SELECT Id FROM Metals WHERE Name = :metal) AND
CurrencyId IN (SELECT Id FROM Currencies Where Id = :currency)';
$stmt = $db->prepare($sql);
for ($i = 0; $i<3; $i++) {
$stmt->execute(array(
':metal' => 'Silver',
':fixAM' => $fix['FixAM'][$i],
':fixPM' => $fix['FixPM'][$i],
':currency' => ($i+1))
);
}
e.g. sometimes, the value for $fix['FixPM'][$i] is sometimes 'NULL'. How do I insert this into the database? When I run the query and then view the data in the database, this record shows 0.0000, and not null.
How do I insert NULL values using PDO? provides a few solutions.
I dont think I can use $stmt->execute(array( ':v1' => null, ':v2' => ... )) as per example because sometimes the item is null, and sometimes not. As such, I need to refer to the variable I have created $fix['FixPM'][$i] and make that null as and when needed
Thanks in advance.

This appears to me to be a(n unreported?) bug in PDO's prepared statement emulation:
the implementation of PDOStatement::execute() eventually invokes pdo_parse_params();
that, in turn, attempts to quote/escape values based on the relevant parameter's data type (as indicated by the $data_type arguments to PDOStatement::bindValue() and PDOStatement::bindParam()—all parameters provided as $input_parameters to PDOStatement::execute() are treated as PDO::PARAM_STR, as stated in the documentation of that function);
string-typed values are escaped/quoted by calling the relevant database driver's quoter() method irrespective of whether they are null: in the case of PDO_MySQL, that's mysql_handle_quoter(), which (eventually) passes the value to either mysqlnd_cset_escape_quotes() or mysql_cset_escape_slashes(), depending on the server's NO_BACKSLASH_ESCAPES SQL mode;
given a null argument, both of those functions return an empty string.
My opinion is that, prior to switching on the parameter's type (in step 2 above), pdo_parse_params() should set the type to PDO::PARAM_NULL if the value is null. However, some might argue that this would prevent type-specific handling of null values where appropriate, in which case the string case (in step 3 above) should definitely handle null values before proceeding with a call to the driver's quoter() method.
As an interim workaround, disabling prepared statement emulation is usually for the best anyway:
$db->setAttribute(PDO::ATTR_EMULATE_PREPARES, FALSE);

Related

Prepared Statement does not use expected index

I have a very large table of IOT sample that I'm trying to run a relativly simple query against. Running the query normally using the MySql CLI returns a result in ~0.07 seconds. If I first prepare the query either via PDO or by running a SQL PREPARE statement then the request takes over a minute.
I've enabled the the optimizer trace feature, and it looks like when the statement is prepared, MySql ignores the index that it should use and does a file sort of the whole table. I'd like any insight if I am doing something wrong or if this looks like a MySql bug.
The table itself contains over 100 million samples, and at least 300 thousand are associated with the device being queried here. I ran these tests with MySql 8.0.23, but when I upgraded to 8.0.25 the issues persisted.
Table definition (some data rows ommited)
Create Table: CREATE TABLE `samples` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`organization_id` int unsigned NOT NULL,
`device_id` int unsigned NOT NULL,
`created_at` timestamp NULL DEFAULT NULL,
`raw_reading` int DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `samples_organization_id_foreign` (`organization_id`),
KEY `samples_reverse_device_id_created_at_organization_id_index` (`device_id`,`created_at` DESC,`organization_id`),
CONSTRAINT `samples_device_id_foreign` FOREIGN KEY (`device_id`) REFERENCES `devices` (`id`) ON DELETE RESTRICT ON UPDATE CASCADE,
CONSTRAINT `samples_organization_id_foreign` FOREIGN KEY (`organization_id`) REFERENCES `organizations` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=188315314 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
Sql That runs in < 1s
select *
from `samples`
where `samples`.`device_id` = 5852
and `samples`.`device_id` is not null
and `id` != 188315308
order by `created_at` desc
limit 1;
Sql That runs in over a minute
prepare test_prep from 'select * from `samples` where `samples`.`device_id` = ? and `samples`.`device_id` is not null and `id` != ? order by `created_at` desc limit 1';
set #a = 5852;
set #b = 188315308;
execute test_prep using #a, #b;
Trace for the non prepared SQL can be found at my gist, but the relevant part is
{
"reconsidering_access_paths_for_index_ordering": {
"clause": "ORDER BY",
"steps": [
],
"index_order_summary": {
"table": "`samples`",
"index_provides_order": true,
"order_direction": "asc",
"index": "samples_reverse_device_id_created_at_organization_id_index",
"plan_changed": false
}
}
},
Trace for the prepared query can be found at my other gist, but the relevant part is
{
"reconsidering_access_paths_for_index_ordering": {
"clause": "ORDER BY",
"steps": [
],
"index_order_summary": {
"table": "`samples`",
"index_provides_order": false,
"order_direction": "undefined",
"index": "samples_reverse_device_id_created_at_organization_id_index",
"plan_changed": false
}
}
},
The index you want to use is not that bad:
`samples_reverse_device_id_created_at_organization_id_index`
(`device_id`,`created_at` DESC,`organization_id`)
However, is not a covering index. If the query performance is really important, I would add an index that covers the filtering predicate at least. Your don't need a real covering index since you are retrieving all columns. I would try:
create index ix1 on samples (device_id, created_at, id);
EDIT
Another trick that could promote the index usage is to delay the predicate id != 188315308 as much as possible. If you know that this predicate will be matched by at least one row in the first 100 rows produced by the rest of the predicates you can try rephrasing your query as:
select *
from (
select *
from `samples`
where `samples`.`device_id` = 5852
order by `created_at` desc
limit 100
) x
where `id` != 188315308
order by `created_at` desc
limit 1
Get rid of this, since the = 5852 assures that it will be false:
and `samples`.`device_id` is not null
Then your index, or this one, should work fine.
INDEX(device_id, created_at, id)
Do not use #variables; the Optimizer seems to not look at the value they contain. That is, instead of
set #a = 5852;
set #b = 188315308;
execute test_prep using #a, #b;
Simply do
execute test_prep using 5852, 188315308;
Consider writing a bug report at bugs.mysql.com
I suspect "order_direction": "undefined" is part of the problem.
Not full solution, but a workaround. I added an index on just my timestamp and that seems to satisfy the optimizer.
KEY `samples_created_at_index` (`created_at` DESC),
I'm going to try to clean up a minimal test case and post it over on MySql bugs. I'll add a followup here if anything comes of that.

BindValue does not accept false value

I have a text data type in a database that would like to enter true or false values. If I do using Mysqli or PDO with BindParam works correctly it adds 1 or 0 but when I try to use BindValue it only works true. False value is exchanged for an empty space.
try{
$conn = new PDO("mysql:host=localhost;dbname=name_db", "root", "");
$conn->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$sql = "INSERT INTO upload_meta (video_id, upload_key, upload_value) VALUES (:video_id,:upload_key,:upload_value)";
$temp = $conn->prepare($sql);
$temp->bindValue(':video_id', 11111111);
$temp->bindValue(':upload_key', 'exodo');
$temp->bindValue(':upload_value', false);
$temp->execute();
}catch(PDOException $e){
echo $e->getMessage();
}
This field will receive values of various types needing to be text.
CREATE TABLE `upload_meta` (
`meta_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`video_id` int(11) unsigned NOT NULL,
`upload_key` varchar(255) DEFAULT NULL,
`upload_value` varchar(255) DEFAULT NULL,
PRIMARY KEY (`meta_id`),
KEY `video_id` (`video_id`),
KEY `index_upload_key` (`upload_key`(191)),
CONSTRAINT `upload_meta_ibfk_1` FOREIGN KEY (`video_id`) REFERENCES `video` (`id_video`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
According to the documentation
The answer is in the documentation for bindParam:
Unlike PDOStatement::bindValue(), the variable is bound as a reference and will only be evaluated at the time that PDOStatement::execute() is called.
And execute
call PDOStatement::bindParam() to bind PHP variables to the parameter markers: bound variables pass their value as input and receive the output value, if any, of their associated parameter markers
Your case
Your database structure expects upload_value as varchar which is basically text/string. When you use bindParam it is working because it is passing the value of true or false i.e., 1 or 0
But when you use bindValue the reference is passed and then evaluated at the time of execution, and hence true is getting converted into 1 (string/text), but false is evaluated to "empty" string.
Solution
Either use bindParam or if you want to use bindValue you should update your database structure to accept boolean values for upload_value instead of varchar
A 5 min guide: https://w3guy.com/php-pdostatement-bindparam-bindvalue/

MySQL INSERT IGNORE Adding 1 to Non-Indexed column

I'm building a small report in a PHP while loop.
The query I'm running inside the while() loop is this:
INSERT IGNORE INTO `tbl_reporting` SET datesubmitted = '2015-05-26', submissiontype = 'email', outcome = 0, totalcount = totalcount+1
I'm expecting the totalcount column to increment every time the query is run.
But the number stays at 1.
The UNIQUE index composes the first 3 columns.
Here's the Table Schema:
CREATE TABLE `tbl_reporting` (
`datesubmitted` date NOT NULL,
`submissiontype` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL,
`outcome` tinyint(1) unsigned NOT NULL DEFAULT '0',
`totalcount` mediumint(5) unsigned NOT NULL DEFAULT '0',
UNIQUE KEY `datesubmitted` (`datesubmitted`,`submissiontype`,`outcome`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
When I modify the query into a regular UPDATE statement:
UPDATE `tbl_reporting` SET totalcount = totalcount+1 WHERE datesubmitted = '2015-05-26' AND submissiontype = 'email' AND outcome = 1
...it works.
Does INSERT IGNORE not allow adding numbers? Or is my original query malformed?
I'd like to use the INSERT IGNORE, otherwise I'll have to query for the original record first, then insert, then eventually update.
Think of what you're doing:
INSERT .... totalcount=totalcount+1
To calculate totalcount+1, the DB has to retrieve the current value of totalcount... which doesn't exist yet, because you're CREATING a new record, and there is NO existing data to retrieve the "old" value from.
e.g. you're trying eat your cake before you ever went to the store to buy the ingredients, let alone mix/bake them.

Php PDO: Why does Inserting NULL yields to 0 [duplicate]

This question already has answers here:
Insert NULL instead of empty string with PDO
(3 answers)
Closed last month.
I searched any possible help that can be found online but still the problem with INSERT NULL using PHP PDO persists.
The script is a csvupload script originally came from here Import CSV into MySQL
To make the story short, Let me present the possible cause..
if($linearray[4]=='Unknown')
$linearray[4]=null;
$linemysql = implode("','",$linearray);
$linemysql = "'".$linemysql."'";
$setsu->setAttribute(PDO::ATTR_EMULATE_PREPARES, false);
$tsuika = $setsu->prepare("INSERT INTO tablename (SubAgentID, BookID, AgentID, SubAgentName, Risk, Area, CurrentBalance) VALUES ($linemysql)");
$tsuika -> bindValue(':Risk', null, PDO::PARAM_INT);
$tsuika ->execute();
Looking the code above, I explicitly set the field values on the prepare statment.
On phpmyadmin the Risk field accepts NULL, set the default value to NULL, and has no problems. But when doing INSERT with PHP the value it gets is 0. Why?
Before Inserting, I echoed it and if the field $linearray[4] contains Unknown, it converts it to NULL yielding, '' for that part.
table structure
CREATE TABLE IF NOT EXISTS `subagentdb` (
`SubAgentID` int(10) NOT NULL AUTO_INCREMENT,
`BookID` int(10) NOT NULL,
`AgentID` int(10) NOT NULL,
`SubAgentName` varchar(30) NOT NULL,
`Risk` float DEFAULT NULL,
`Area` varchar(20) NOT NULL,
`CurrentBalance` float NOT NULL,
PRIMARY KEY (`SubAgentID`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 AUTO_INCREMENT=1 ;
You're binding the value explicitly as PDO::PARAM_INT. Whatever value you pass will be cast to an int because of that. null casts to 0.
To actually bind an SQL NULL value, you need to bind the value as PDO::PARAM_NULL.
just use PDO::PARAM_NULL instead of PDO::PARAM_INT ? I think the NULL is converted to 0 (INT) instead of null value
If you back-up a and then restore the table, the null value becomes 0. The only way I found to correct this is after the create table, add "UPDATE table SET foo to null WHERE foo = 0".

Extracting data from a column using mysqli::bind_result()

My query returns with null in my php code , but when I enter the same query into phpmyadmin it returns the row to which it belongs. Here is the database I am using
CREATE TABLE `payment`.`users`(
`u_id` int(12) NOT NULL AUTO_INCREMENT,
`email` varchar(255) NOT NULL,
`passwd` varchar(100) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY (`email`)
) ENGINE=MyISAM;
and here is the code i am using that is returning null when it clearly works in phpmyadmin.
function getUserId($email, $passwd) {
$mysqli = db_connect();
$query = "SELECT `u_id` FROM `payment`.`users` WHERE `email`='$email' AND `passwd`='$passwd' ORDER BY 1";
if ($stmt = $mysqli->prepare($query)) {
/* execute query */
$stmt->execute();
$stmt->bind_result($u_id);
while ($stmt->fetch()) {
return $u_id;
}
}
}
The thing is that you are using variables in your php code to set the values. In phpMyAdmin you're inserting values directly, therefore the problem may be in the values inserted.
First of all use PDO's bindParam() or mysqli's bind_param() statements as they sanitize inputs and help you avoid SQL Injections.
Second good thing about using prepared statements and binding params is that you can specify the type of the data being binded to to the query which in most cases will fix such problems. Though in your case you're probably inserting strings.

Categories