PHP array to mysql [duplicate] - php

I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?

You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.

Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();

Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}

You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.

Related

Import 75,000 records from CVS to mysql faster [duplicate]

I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.

Setting an Array inside a prepared SQL statement

I was told that "There is no way to bind an array to an SQL statement using prepared statements" but I have done it. I am having trouble recreating it though.
I have a statement that updates the database:
if (isset($_POST['printRow'])){
$ids = "";
foreach ($_POST['checkbox'] as $rowid)
{
if(!empty($ids)) $ids .= ',';
$ids .= $rowid;
$_SESSION['ids'] = $ids;
}
}
Here I forgot to post the WORKING code:
$stmt = $conn->prepare("UPDATE just_ink SET deleted=1 WHERE ID IN( " . $ids . ")");
$stmt->execute();
But I still have the following problem:
Where $ids can be either one or multiple ids.
So here is the problem, if I try to take $ids and set a SESSION with it
($_SESSION['ids'] = $ids;)
For use on another page.
On the next page I want to select data using $_SESSION['ids'] so,
$stmt = $conn->prepare("SELECT * FROM just_ink WHERE ID IN( " . $_SESSION['ids'] . ")");
$stmt->execute();
But this doesn't work. Any ideas why?
It doesn't work, because, as you correctly said, you can't bind an array to an SQL statement using prepared statements.
The correct way to bind an array is to create a string of placeholders (question marks) and then bind params in a loop.
Let's say you have an array of necessary ID's called $checkboxes. First, we need to create a string that we will use in our query to bind required params. If $checkboxes has 3 items, our string will look like
$placeholder = "?,?,?";
For this we can use str_repeat function to create a string, where every but last element will add ?, part to placeholder. For last element we need to concatenate single question mark.
$placeholder = str_repeat('?,', count($checkboxes)-1).'?';
Now we need to form and prepare a query that will contain our placeholders:
$query = 'SELECT * FROM just_ink WHERE ID IN (".$placeholder.")';
$stmt = $conn->prepare($query);
To bind every ID to its placeholder we use bindParam method in a loop:
for ($i=0; $i<count($checkboxes); $i++) {
$stmt->bindParam($i+1, ($checkboxes[$i]); #position is 1-indexed
}
$stmt->execute();
You can use arrays with mysqli prepared statements by using call_user_func_array
Your code would end up something like this
$varArray = array();
$questionArray = array();
foreach ($_POST['checkbox'] as $daNumber=>$daValue) {
$questionArray[] = "?";
//We're declaring these as strings, if they were ints, they would be i
$varArray[0] .= 's';
//These must be relational variables. The ampersand is vry important.
$varArray[] = &$_POST['checkbox'][$daNumber];
}
//comma separated series of questionmarks
$allDaQuestions = implode(', ', $questionArray);
$query = "SELECT * FROM just_ink WHERE ID IN ($allDaQuestions)";
$stmt = $conn->prepare($query);
//Where the magic happens
call_user_func_array(array(&$stmt, 'bind_param'), $varArray);
//continue with your regularly scheduled broadcast
$stmt->execute();
//etc.
did you set session_start() at the beginning of the file? you can't use $_SESSION if you don't do that first

How to perform multiple mysqli_multi_queries? [duplicate]

I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.

how should I go about inserting this array from $_POST to php to mysqli [duplicate]

I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.

php PDO insert batch multiple rows with placeholders

I am looking to do multiple inserts using PHP PDO.
The closest answer I have found is this one
how-to-insert-an-array-into-a-single-mysql-prepared-statement
However the example thats been given uses ?? instead of real placeholders.
I have looked at the examples on the PHP doc site for place holders
php.net pdo.prepared-statements
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
$stmt->bindParam(':name', $name);
$stmt->bindParam(':value', $value);
Now lets say I wanted to achieve the above but with an array
$valuesToInsert = array(
0 => array('name' => 'Robert', 'value' => 'some value'),
1 => array('name' -> 'Louise', 'value' => 'another value')
);
How would I go about it with PDO and multiple inserts per transaction?
I imagine it would start of with a loop?
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
foreach($valuesToInsert as $insertRow){
// now loop through each inner array to match binded values
foreach($insertRow as $column => value){
$stmt->bindParam(":{$column}", value);
}
}
$stmt->execute();
However the above does not work but hopefully will demonstrate what im trying to achieve
First of all, ? symbols are real place-holders (most drivers allow to use both syntaxes, positional and named place-holders). Secondly, prepared statements are nothing but a tool to inject raw input into SQL statements—the syntax of the SQL statement itself is unaffected. You already have all the elements you need:
How to insert multiple rows with a single query
How to generate SQL dynamically
How to use prepared statements with named place-holders.
It's fairly trivial to combine them all:
$sql = 'INSERT INTO table (memberID, programID) VALUES ';
$insertQuery = [];
$insertData = [];
$n = 0;
foreach ($data as $row) {
$insertQuery[] = '(:memberID' . $n . ', :programID' . $n . ')';
$insertData['memberID' . $n] = $memberid;
$insertData['programID' . $n] = $row;
$n++;
}
if (!empty($insertQuery)) {
$sql .= implode(', ', $insertQuery);
$stmt = $db->prepare($sql);
$stmt->execute($insertData);
}
I'm assuming you are using InnoDB so this answer is only valid for that engine (or any other transaction-capable engine, meaning MyISAM isn't included).
By default InnoDB runs in auto-commit mode. That means each query is treated as its own contained transaction.
To translate that to something us mortals can understand, it means that every INSERT query you issue will force hard-disk to commit it by confirming it wrote down the query information.
Considering how mechanical hard-disks are super slow since their input-output operation per second is low (if I'm not mistaken, the average is 300ish IO's), it means your 50 000 queries will be - well, super slow.
So what do you do? You commit all of your 50k queries in a single transaction. It might not be the best solution for various purposes but it'll be fast.
You do it like this:
$dbh->beginTransaction();
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
foreach($valuesToInsert as $insertRow)
{
// now loop through each inner array to match bound values
foreach($insertRow as $column => value)
{
$stmt->bindParam(":$column", value);
$stmt->execute();
}
}
$dbh->commit();
A little modifications in solution provided by N.B
$stmt->execute() should be outside of inner loop because you may have one or more columns that need to bind before calling $stmt->execute() else you 'll get exception "Invalid parameter number: number of bound variables does not match number of token".
2nd "value" variable were missing dollar signs.
function batchinsert($sql,$params){
try {
db->beginTransaction();
$stmt = db->prepare($sql);
foreach($params as $row)
{
// now loop through each inner array to match bound values
foreach($row as $column => $value)
{
$stmt->bindParam(":$column", $value);
}
$stmt->execute();
}
db->commit();
} catch(PDOExecption $e) {
$db->rollback();
}
}
Test:
$sql = "INSERT INTO `test`(`name`, `value`) VALUES (:name, :value)" ;
$data = array();
array_push($data, array('name'=>'Name1','value'=>'Value1'));
array_push($data, array('name'=>'Name2','value'=>'Value2'));
array_push($data, array('name'=>'Name3','value'=>'Value3'));
array_push($data, array('name'=>'Name4','value'=>'Value4'));
array_push($data, array('name'=>'Name5','value'=>'Value5'));
batchinsert($sql,$data);
Your code was actually ok, but had a problem in $stmt->bindParam(":$column", value); It should be $stmt->bindValue(":{$column}", $value); and it will work perfectly. This will assist others in future.
Full code:
foreach($params as $row)
{
// now loop through each inner array to match bound values
foreach($row as $column => $value)
{
$stmt->bindValue(":{$column}", $value); //EDIT
}
// Execute statement to add to transaction
$stmt->execute();
}
Move execute inside of the loop.
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
foreach($valuesToInsert as $insertRow)
{
$stmt->execute($insertRow);
}
If you experience any problems with this such recommended way, you have to ask a question, describing these certain problems.

Categories