load data infile and unbuffered queries error - php

I'm experimenting with loading a table from a file, and having difficulty. The code below is trying to take an existing database and copy it to a temporary table, then replace the original with imported data from a .csv file, and then I've got more work to do comparing the two tables before I let go of the temporary one. (Hints welcome if I should do this a different way). I get the error:
'SQLSTATE[HY000]: General error: 2014 Cannot execute queries while
other unbuffered queries are active. Consider using
PDOStatement::fetchAll()...'
I've tried many of the suggestions from similar questions, but haven't cracked it yet. Thanks for your help! Here's my code:
<?php
//database connection
$data_source = 'mysql:host=localhost;dbname=myDB';
$db_user = 'root';
$db_password = 'pass';
$conn = new PDO($data_source, $db_user, $db_password,
array(PDO::ATTR_EMULATE_PREPARES => false,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
PDO::ATTR_PERSISTENT));
if ( isset($_GET['submit']) ){
$stmt = $conn->prepare("CREATE TEMPORARY TABLE mfsw_dupe AS SELECT * FROM mfsw_test");
$stmt->execute();
$stmt = $conn->prepare("TRUNCATE mfsw_test");
$stmt->execute();
$stmt = $conn->prepare("LOAD DATA LOCAL INFILE 'C:\\xampp\htdocs\assets\mfsw_test.csv' INTO TABLE mfsw_test
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES");
$stmt->execute();
}
?>

After trying all the recommended solutions to this problem, I found that the answer was to set the PDO::ATTR_EMULATE_PREPARES option to true.
That got rid of "unbuffered queries" error, but it then started reporting a "LOAD DATA LOCAL INFILE forbidden" error on the LOAD query.
The solution to that was to set the PDO::MYSQL_ATTR_USE_BUFFERED_QUERY option to true as well.
In short, your initial connection should look like this:
$conn = new PDO($data_source, $db_user, $db_password,
array(PDO::ATTR_EMULATE_PREPARES => true,
PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
PDO::ATTR_PERSISTENT));
I don't understand why these options are necessary, but they worked for me.

I have had exactly the same problem, tried everything mentioned above and a lot more and still couldn't avoid the "Cannot execute queries while other unbuffered queries are active" error.
However, after several hours of googling, I got it working - instead of using a prepared statement, or just query, I used 'exec' to run the 'LOAD DATA LOCAL INFILE' statement:
$stmt = $conn->exec($sql_import_csv);
I didn't need any of the aforementioned changes to the PDO options - this is my options array:
$options = array(
PDO::ATTR_EMULATE_PREPARES => false,
PDO::MYSQL_ATTR_SSL_CA => "xxxxxxxx.crt.pem",
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
PDO::MYSQL_ATTR_LOCAL_INFILE => true
);
Hope this helps someone else out there.

Related

PDO or AJAX not working when I try to connect [duplicate]

I'm following this tutorial, I'm currently around minute 04:00 and I want to make a connection with my MySQL database through PDO. But my webpage will always give "Could not connect." when I'm trying to make the connection.
When I used PHPStorms inside Database program, I had to change my serverTimezone to Europe/Amsterdam and then I was able to connect to my db.
I tried to add the port number in the 'new PDO()' code.
I tried to change the timezone in the code and on my MySQL server but it gives this error;
mysql> SET GLOBAL time_zone = 'Europe/Amsterdam';
ERROR 1298 (HY000): Unknown or incorrect time zone: 'Europe/Amsterdam'
<?php
try {
$pdo = new PDO('mysql:host=localhost:dbname=mytodo', 'root', '');
} catch (PDOException $e) {
die('Could not connect.');
}
$statement = $pdo->prepare('select * from todos');
$statement->execute();
var_dump($statement->fetchAll());
require 'index.view.php';
Extra information:
mysql> SELECT ##global.time_zone, ##session.time_zone;
+--------------------+---------------------+
| ##global.time_zone | ##session.time_zone |
+--------------------+---------------------+
| SYSTEM | SYSTEM |
+--------------------+---------------------+
The tutorial you are following is very outdated.
A code connecting to PDO has to follow certain rules explained in my article How to connect to MySQL with PDO. In brief
it should configure error reporting mode
it should set the connection charset, the right way
it shouldn't catch it's own errors to report them
it should follow correct DSN syntax, without any extra decorations
it also coud set a couple useful settings
So the code should be like this
$host = '127.0.0.1';
$db = 'mytodo';
$user = 'root';
$pass = '';
$charset = 'utf8mb4';
$options = [
\PDO::ATTR_ERRMODE => \PDO::ERRMODE_EXCEPTION,
\PDO::ATTR_DEFAULT_FETCH_MODE => \PDO::FETCH_ASSOC,
\PDO::ATTR_EMULATE_PREPARES => false,
];
$dsn = "mysql:host=$host;dbname=$db;charset=$charset";
$pdo = new \PDO($dsn, $user, $pass, $options);
Here you can see the proper connection code that does a lot of things. Among them, it tells you what the actual problem is when your code cannot connect to database.
You have error in code near localhost
Use semicolon in place of colon
Write this instead:
$pdo=new PDO('mysql:host=localhost;dbname=mytodo','root','');

Can't connect to MySQL with PDO

I'm following this tutorial, I'm currently around minute 04:00 and I want to make a connection with my MySQL database through PDO. But my webpage will always give "Could not connect." when I'm trying to make the connection.
When I used PHPStorms inside Database program, I had to change my serverTimezone to Europe/Amsterdam and then I was able to connect to my db.
I tried to add the port number in the 'new PDO()' code.
I tried to change the timezone in the code and on my MySQL server but it gives this error;
mysql> SET GLOBAL time_zone = 'Europe/Amsterdam';
ERROR 1298 (HY000): Unknown or incorrect time zone: 'Europe/Amsterdam'
<?php
try {
$pdo = new PDO('mysql:host=localhost:dbname=mytodo', 'root', '');
} catch (PDOException $e) {
die('Could not connect.');
}
$statement = $pdo->prepare('select * from todos');
$statement->execute();
var_dump($statement->fetchAll());
require 'index.view.php';
Extra information:
mysql> SELECT ##global.time_zone, ##session.time_zone;
+--------------------+---------------------+
| ##global.time_zone | ##session.time_zone |
+--------------------+---------------------+
| SYSTEM | SYSTEM |
+--------------------+---------------------+
The tutorial you are following is very outdated.
A code connecting to PDO has to follow certain rules explained in my article How to connect to MySQL with PDO. In brief
it should configure error reporting mode
it should set the connection charset, the right way
it shouldn't catch it's own errors to report them
it should follow correct DSN syntax, without any extra decorations
it also coud set a couple useful settings
So the code should be like this
$host = '127.0.0.1';
$db = 'mytodo';
$user = 'root';
$pass = '';
$charset = 'utf8mb4';
$options = [
\PDO::ATTR_ERRMODE => \PDO::ERRMODE_EXCEPTION,
\PDO::ATTR_DEFAULT_FETCH_MODE => \PDO::FETCH_ASSOC,
\PDO::ATTR_EMULATE_PREPARES => false,
];
$dsn = "mysql:host=$host;dbname=$db;charset=$charset";
$pdo = new \PDO($dsn, $user, $pass, $options);
Here you can see the proper connection code that does a lot of things. Among them, it tells you what the actual problem is when your code cannot connect to database.
You have error in code near localhost
Use semicolon in place of colon
Write this instead:
$pdo=new PDO('mysql:host=localhost;dbname=mytodo','root','');

Uploading image to INSERT in MariaDB table fails with PDO::PARAM_LOB

My problem is that when I use the PDO::PARAM_LOB parameter in the bindParam() method the photo sent from the html form isn’t inserted in the MariaDB table. phpMyAdmin subsequently shows the table column ‘photo’ to be empty.
The image is successfully inserted in the table when I don’t use the PDO::PARAM_LOB but this causes a Notice: Array to string conversion in C:\xampp\htdocs\TestSite\php-sellform-handler.php on line 80 to display to the user on the web page.
There are other fields on the same form (which for simplicity I haven’t shown) which upload text and use PDO::PARAM_STR. This text data is always successfully inserted in the same table.
How do I get insert to work using PDO::PARAM_LOB?
The nearest problem to mine that I can find on S/O is PHP/PDO/MySQL: inserting into MEDIUMBLOB stores bad data
However that involves the image actually being inserted in the table, but in a corrupted form.
I’m using Apache/2.4.29 (Win32) OpenSSL/1.1.0g PHP/7.2.2 and 10.1.30-MariaDB
Here is my code:
$host = '127.0.0.1';
$db = 'test_db2';
$user = 'root';
$pass = '';
$charset = 'utf8mb4';
$dsn = "mysql:host=$host;dbname=$db;charset=$charset";
$opt = [PDO::ATTR_ERRMODE =>PDO::ERRMODE_EXCEPTION,
PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC,
PDO::ATTR_EMULATE_PREPARES => false
];
$pdo=new PDO($dsn, $user, $pass, $opt);
if (isset($_FILES ['photo'])) {
$photo=$_FILES['photo'];
$stmt_user = $pdo->prepare("INSERT INTO users_table1(photo) VALUES(:photo)");
$stmt_user->bindParam(':photo', $photo, PDO::PARAM_LOB);
$stmt_user->execute();
}
Thanks for any help!

HTTP 504 timeouts by running this PHP script

I have a PHP script below that I'm running every hour using a cronjob. Sometimes this script runs without any errors and other times it throws a 504 Gateway Timeout.
Can someone please help to improve the execution and efficiency of this script? I believe it can be written cleaner, but not sure where to start.
This is on a shared server and I do not have much control over the settings. Max timeout is 120.
Cronjob:
curl www.mysite.com/apps/script.php
PHP script:
<?php
set_time_limit(500);
$databasehost = "localhost";
$databasename = "foodb";
$databasetable = "footable";
$databaseusername="foouser";
$databasepassword = "foopass";
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "inventory.csv";
if(!file_exists($csvfile)) {
die("File not found. Make sure you specified the correct path.");
}
try {
$pdo = new PDO("mysql:host=$databasehost;dbname=$databasename",
$databaseusername, $databasepassword,
array(
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
)
);
$sql = "TRUNCATE TABLE $databasetable";
$command = $pdo->prepare($sql);
$command->execute();
echo "Removed records from $databasename.\n<br />";
} catch (PDOException $e) {
die("database connection failed: ".$e->getMessage());
}
$affectedRows = $pdo->exec("
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$databasetable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator)."IGNORE 1 LINES");
echo "Loaded a total of $affectedRows records from this csv file.\n";
/****/
try {
$pdo = new PDO("mysql:host=$databasehost;dbname=$databasename",
$databaseusername, $databasepassword,
array(
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
)
);
$sql = $pdo->exec("
UPDATE wp37_pmxi_posts
JOIN wp37_postmeta USING (post_id)
JOIN inventoryImport ON wp37_pmxi_posts.unique_key = inventoryImport.sku
SET meta_value = inventoryImport.qty
WHERE meta_key = '_stock'");
echo "Updated $affectedRows records after the import.\n";
} catch (PDOException $e) {
die("database connection failed: ".$e->getMessage());
}
?>
Thanks for your time and I appreciate any help.
I had the exact same issue last week and even though I am on a VPS server with a full root access, I didn't want to mess about with the files in the etc/ folder or php.ini or any other PHP settings...
So I came up with my own solution..
Please note this solution and this answer intended to just give you and other people an idea of how you could solve this issue without fiddling with server settings...
The solution was quite simple actually...
First i created an extra column in the MYSQL database with a pre-defined value of 0. You can name it anything you want.
What I did next was basically LIMIT my SELECT to 100.
something like:
SELECT * FROM mytable WHERE newColumn=0 LIMIT 100
And then I did whatever i wanted with this 100 selected items and every time I did that i UPDATE the database and changed the newColumn=1
And I repeated the same process until there is no more data in the MYSQL database with newColumn=0.
once I reach this stage, I UPDATE the table again and reset the newColumn back to 0.
Note that I run the process above on cron job.
This way your PHP won't get killed as you are only selecting 100 rows from the MYSQL.
hope this helps.

PHP PDO INSERT query running twice

I've got this code with PHP PDO and I've got an INSERT query that runs twice. This code should insert the value 'Hello World' into the database. However, my code inserts the value twice into the database.
How do I fix this? I've stumbled upon other people with the same problem, but they have problems with a loop or they've used the explode function that messed something up. I don't have any of that, just one connection block and an insert query, and yet it does this weird thing.
The code:
// DB connect configuration
$user = 'user';
$pass = 'password';
// Database connection
$dsn = "mysql:host=localhost;dbname=pdotest;charset=utf8";
$opt = array(
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC
);
$conn = new PDO($dsn, $user, $pass, $opt);
// Data to insert
$data = 'Test and test and test';
// Insert data into database
$sql = "INSERT INTO tabletwo (rowTwo) VALUES (?)";
$q = $conn->prepare($sql);
$q->execute(array($data));
It's happening because of some Javascript code for page transitions. The code can be found here: http://www.fasw.ws/faswwp/non-jquery-page-transitions-lightweight/. I have no idea how to fix this though.

Categories