php PDO insert batch multiple rows with placeholders - php

I am looking to do multiple inserts using PHP PDO.
The closest answer I have found is this one
how-to-insert-an-array-into-a-single-mysql-prepared-statement
However the example thats been given uses ?? instead of real placeholders.
I have looked at the examples on the PHP doc site for place holders
php.net pdo.prepared-statements
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
$stmt->bindParam(':name', $name);
$stmt->bindParam(':value', $value);
Now lets say I wanted to achieve the above but with an array
$valuesToInsert = array(
0 => array('name' => 'Robert', 'value' => 'some value'),
1 => array('name' -> 'Louise', 'value' => 'another value')
);
How would I go about it with PDO and multiple inserts per transaction?
I imagine it would start of with a loop?
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
foreach($valuesToInsert as $insertRow){
// now loop through each inner array to match binded values
foreach($insertRow as $column => value){
$stmt->bindParam(":{$column}", value);
}
}
$stmt->execute();
However the above does not work but hopefully will demonstrate what im trying to achieve

First of all, ? symbols are real place-holders (most drivers allow to use both syntaxes, positional and named place-holders). Secondly, prepared statements are nothing but a tool to inject raw input into SQL statements—the syntax of the SQL statement itself is unaffected. You already have all the elements you need:
How to insert multiple rows with a single query
How to generate SQL dynamically
How to use prepared statements with named place-holders.
It's fairly trivial to combine them all:
$sql = 'INSERT INTO table (memberID, programID) VALUES ';
$insertQuery = [];
$insertData = [];
$n = 0;
foreach ($data as $row) {
$insertQuery[] = '(:memberID' . $n . ', :programID' . $n . ')';
$insertData['memberID' . $n] = $memberid;
$insertData['programID' . $n] = $row;
$n++;
}
if (!empty($insertQuery)) {
$sql .= implode(', ', $insertQuery);
$stmt = $db->prepare($sql);
$stmt->execute($insertData);
}

I'm assuming you are using InnoDB so this answer is only valid for that engine (or any other transaction-capable engine, meaning MyISAM isn't included).
By default InnoDB runs in auto-commit mode. That means each query is treated as its own contained transaction.
To translate that to something us mortals can understand, it means that every INSERT query you issue will force hard-disk to commit it by confirming it wrote down the query information.
Considering how mechanical hard-disks are super slow since their input-output operation per second is low (if I'm not mistaken, the average is 300ish IO's), it means your 50 000 queries will be - well, super slow.
So what do you do? You commit all of your 50k queries in a single transaction. It might not be the best solution for various purposes but it'll be fast.
You do it like this:
$dbh->beginTransaction();
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
foreach($valuesToInsert as $insertRow)
{
// now loop through each inner array to match bound values
foreach($insertRow as $column => value)
{
$stmt->bindParam(":$column", value);
$stmt->execute();
}
}
$dbh->commit();

A little modifications in solution provided by N.B
$stmt->execute() should be outside of inner loop because you may have one or more columns that need to bind before calling $stmt->execute() else you 'll get exception "Invalid parameter number: number of bound variables does not match number of token".
2nd "value" variable were missing dollar signs.
function batchinsert($sql,$params){
try {
db->beginTransaction();
$stmt = db->prepare($sql);
foreach($params as $row)
{
// now loop through each inner array to match bound values
foreach($row as $column => $value)
{
$stmt->bindParam(":$column", $value);
}
$stmt->execute();
}
db->commit();
} catch(PDOExecption $e) {
$db->rollback();
}
}
Test:
$sql = "INSERT INTO `test`(`name`, `value`) VALUES (:name, :value)" ;
$data = array();
array_push($data, array('name'=>'Name1','value'=>'Value1'));
array_push($data, array('name'=>'Name2','value'=>'Value2'));
array_push($data, array('name'=>'Name3','value'=>'Value3'));
array_push($data, array('name'=>'Name4','value'=>'Value4'));
array_push($data, array('name'=>'Name5','value'=>'Value5'));
batchinsert($sql,$data);

Your code was actually ok, but had a problem in $stmt->bindParam(":$column", value); It should be $stmt->bindValue(":{$column}", $value); and it will work perfectly. This will assist others in future.
Full code:
foreach($params as $row)
{
// now loop through each inner array to match bound values
foreach($row as $column => $value)
{
$stmt->bindValue(":{$column}", $value); //EDIT
}
// Execute statement to add to transaction
$stmt->execute();
}

Move execute inside of the loop.
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
foreach($valuesToInsert as $insertRow)
{
$stmt->execute($insertRow);
}
If you experience any problems with this such recommended way, you have to ask a question, describing these certain problems.

Related

Import 75,000 records from CVS to mysql faster [duplicate]

I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.

PHP array to mysql [duplicate]

I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.

How to perform multiple mysqli_multi_queries? [duplicate]

I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.

how should I go about inserting this array from $_POST to php to mysqli [duplicate]

I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.

Trying to create a query function for PDO

As I am writing a script, I am typing $db->prepare()'s and $stmt->bindParam()'s constantly. I am looking for a way to consolidate it all in a function. This is what I have so far.
$sql = "SELECT (name, email) FROM users WHERE VALUES (:name, :email)"
$values = array(':name' => 'my_name', ':email' => 'blahblah#example.com', );
function db_query($sql, $values) {
global $db; //Database object
$stmt = $db->prepare($sql);
foreach($values as $placeholder => $value) {
$stmt->bindParam($placeholder, $value);
}
$stmt->execute();
$result = $stmt->fetchAll(PDO::FETCH_ASSOC);
return $result;
}
Would this be sufficient for most queries? Is this a secure way to do this?
I just want the query to run and return whatever it returns (NULL, values, error, etc...).
Thanks.
Your code will not work as expected because bindParam() binds the placeholder name (first argument) to the variable reference present in the second argument.
Using your example, this would result in all parameters set to blahblah#example.com as it is the last $value in the loop.
As mentioned in the comments, simply use $stmt->execute($values). See http://php.net/manual/en/pdostatement.execute.php
If you really want to continue with your loop, use PDOStatement::bindValue() instead.

Categories