I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.
Related
I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.
How do I properly re-execute a prepared statement using different integer values?
There's something deathly wrong with explicit and implicit binding PDO::PARAM_INT when reusing an ODBC prepared statement.
CREATE TABLE mytab (
col INT,
something VARCHAR(20)
);
Works : multiple strings
$pdoDB = new PDO('odbc:Driver=ODBC Driver 13 for SQL Server;
Server='.DATABASE_SERVER.';
Database='.DATABASE_NAME,
DATABASE_USERNAME,
DATABASE_PASSWORD
);
$pdoDB->setAttribute( PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION );
$values = ['here','are','some','values'];
$sql = "INSERT INTO mytab (something) VALUES (:something)";
$stmt = $pdoDB->prepare($sql);
foreach ($values as $value)
$stmt->execute(['something'=>$value]);
Works : single integer
$values = [42];
$sql = "INSERT INTO mytab (col) VALUES (:col)";
$stmt = $pdoDB->prepare($sql);
foreach ($values as $value)
$stmt->execute(['col'=>$value]);
Does Not Work : multiple integers
$values = [1,3,5,7,11];
$sql = "INSERT INTO mytab (col) VALUES (:col)";
$stmt = $pdoDB->prepare($sql);
foreach ($values as $value)
$stmt->execute(['col'=>$value]);
It actually successfully inserts the first record 1 but fails when it tries to reuse the statement on the next execute.
PHP Fatal error: Uncaught PDOException: SQLSTATE[22018]: Invalid character value for cast specification: 206 [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Operand type clash: text is incompatible with int (SQLExecute[206] at /build/php7.0-lPMnpS/php7.0-7.0.8/ext/pdo_odbc/odbc_stmt.c:260)
I'm connecting from 64-bit Ubuntu 16.04 running PHP 7.0.8 using the Microsoft® ODBC Driver 13 (Preview) for SQL Server®
I have tried wrapping the whole thing in PDO::beginTransaction and PDO::commit
I've also tried using PDOStatement::bindParam but it throws the exact same error.
Works
$values = [1];
$sql = "INSERT INTO mytab (col) VALUES (:col)";
$stmt = $pdoDB->prepare($sql);
foreach ($values as $value){
$stmt->bindParam('col', $value, PDO::PARAM_INT);
$stmt->execute();
}
Does Not Work
$values = [1,2];
$sql = "INSERT INTO mytab (col) VALUES (:col)";
$stmt = $pdoDB->prepare($sql);
foreach ($values as $value){
$stmt->bindParam('col', $value, PDO::PARAM_INT);
$stmt->execute();
}
I think it's interesting to note that I am getting the exact same error as this unanswered question using PHP 5.6.9. However, they are not able to execute even one statement, so I'm wondering if there's been a partial patch considering the exact line throwing the error has moved from odbc_stmt.c:254 to odbc_stmt.c:260
Workaround
If I prepare the statement inside the loop, then it works just fine. But I've read that this is very inefficient and I should be able to reuse the statement. I'm particularly worried about using this with massive datasets. Is this OK? Is there something better that I can do?
$values = [1,3,5,7,9,11];
$sql = "INSERT INTO mytab (col) VALUES (:col)";
foreach ($values as $value){
$stmt = $pdoDB->prepare($sql);
$stmt->execute(['col'=>$value]);
}
In case of prepared statements you have to use bindParam outside of loop, usually.
bindParam is a single step
setting bound variables is a repeatable step (loop)
you have to run execute for each repetition
I guess, something like that would work:
$stmt = $pdoDB->prepare("INSERT INTO mytab (col, key) VALUES (:col, :key)");
// bind params (by reference)
$stmt->bindParams(":col", $col, PDO::PARAM_STR); //bind variable $col
$stmt->bindParams(":key", $key, PDO::PARAM_INT); //bind variable $key
$values = ['here','are','some','values'];
foreach ($values as $i => $value) {
$col = $value; //set col
$key = $i; //set key
$stmt->execute();
}
Assuming I have an array as follows:
$array = array('first_value',
'second_value',
'thrid_value', 'and so on');
And a Column in which I'd want to insert those values, but each value in a separate row.
Would it it be possible to do that?
Obviously there are some answers to this one would be just loop thru the array elements and for every loop execute an insert statement, but that just seems unwise.
Or given that I'd have an ID column, that would help a lot(but I don't).
The amount of data to be introduced is not terribly large so the loop is perfectly viable, I just wanna make sure there isn't some easier way to do this that I may not be aware of.
You could use prepared statements; the first query will send the SQL statement and the subsequent calls will only send the data, thereby reducing the load:
$stmt = $db->prepare('INSERT INTO mytable (colname) VALUES (?)');
foreach ($array as $value) {
$stmt->execute(array($value));
}
If you're using PDO, such as the above example, make sure to disable prepared statement emulation.
// connect to database and store the resource in $connection
$array = array('first_value',
'second_value',
'thrid_value', 'and so on');
foreach($array as $value)
{
$value=mysqli_real_escape_string($connection,$value);
mysqli_query($connection,"INSERT INTO yourTABLE(columnName) VALUES('$value')");
}
You can put them all into a single INSERT statement with multiple VALUES lists.
$values = implode(',', array_map(function($v) use ($mysqli) {
return "'" . $mysqli->real_escape_string($v) . "'"; },
$array));
$query = "INSERT INTO yourTable (Column) VALUES $values";
$mysqli->execute($query) or die ($mysqli->error);
From mysql manual for insert,you may try this:
INSERT INTO yourtable (column_name) VALUES (value_a), (value_b), (value_c);
$array = array('first_value','second_value','third_value');
$SQL = "INSERT INTO `table` (column) VALUES('".implode("'),('",$array)."')";
OR
$values = '';
foreach($array as $val){
$values .= !empty($values)? ",('{$val}')" : "('{$val}')";
}
$SQL = "INSERT INTO `table` (column) VALUES{$values}";
This question already has answers here:
PDO Prepared Inserts multiple rows in single query
(21 answers)
Closed 9 years ago.
i have an array from a post:
//this are arrays
$name = $_POST["name"];
$age = $_POST["age"];
$date = $_POST["date"];
i have an insert query in PDO:
$stmt = $db->prepare("INSERT INTO staff (name, age, address) VALUES (:name, :age, :address)");
my question how can i generate/or achieve a query using php that looks like:
INSERT INTO staff (name, age, address) VALUES
($name[0], $age[0], $address[0]), ($name[1], $age[1], $address[1])... etc
I try to keep server load low.
It is not possible with prepared statements.
If you want to insert a variable amount of records in one query then the SQL query must be generated dynamically, which makes it impossible to prepare it beforehand.
If you use the database driver's escape_string function (that would be quote() for MySQL) then you can build the query and still get the security that prepared statements give you.
$query = "INSERT INTO table (a, b, c) VALUES";
$tuple = " ('%s', '%s', '%s'),";
foreach ($items as $item) {
$a = $db->quote($item['a']);
$b = $db->quote($item['b']);
$c = $db->quote($item['c']);
$query .= sprintf($tuple, $a, $b, $c);
}
$query = rtrim($query, ","); // Remove trailing comma
// Use the query...
Addendum:
If you are using prepared statements then you can insert records separately and not have to worry about inserting them in one go. That is actually the whole point of prepared statement.
Unlike good old SQL queries, which are a one-step process and are just sent and then forgotten...
$db->query("INSERT INTO table (a, b, c) VALUES ('a', 'b', 'c')");
...prepared statements are a two-step process.
First, you create the prepared statement. This statement is then sent to the database, telling it that "this is what you should be expecting". The database will often also optimize the query to make it faster. This step then gives you back a statement handle (often called $stmt in PHP documentation).
$stmt = $db->prepare('INSERT INTO table (a, b, c) VALUES (:a, :b, :c)');
Second, with this handle you can then proceed to insert stuff:
foreach ($records as $record) {
$stmt->execute(array(
':a' => $record['a'],
':b' => $record['b'],
':c' => $record['c'],
));
}
Because the database already knows what to expect it can optimize the speed of the INSERTs, meaning that you do not have to go through the stuff that I mentioned above this addendum.
Wikipedia actually has a quite nice writeup of prepared statements.
You could prepare the appropriate statement, i.e. something like:
$sql = "INSERT INTO staff (name, age, address) VALUES ";
$values = array();
for ($i = 1; $i <= $count; $i++) {
$values[] = "(:name$i, :age$i, :address$i)"
}
$stmt = $db->prepare($sql . implode(', ', $values));
But, imho, you'll probably be just as well off lopping through each set of values. It's not much of a burden for the server, and your code will be simpler.
If you are using PDO, you should prepare your statement, and execute it with different values:
$stmt = $db->prepare("INSERT INTO staff (name,age,address) VALUES (:name,:age,:address)");
for($i=0;$i<count($name);$i++)
{
$stmt->execute(array(
":name" => $name[$i],
":age" => $age[$i],
":address" => $address[$i]));
}
However, if you would like to put it into a single query, you should use $db->exec():
$query = "INSERT INTO staff (name,age,address) VALUES ";
$values = array();
for($i=0;$i<count($names);$i++)
{
array_push($values, "('" . $name[$i] . "','" . $age[$i] . "','" . $address . "')");
}
$db->exec($query . implode("," . $values));
Be warned this method, contrary to the prepare and execute method, is vulnerable since it does not validate the input
Add the delayed keyword to the insert (insert delayed ...) to reduce serverload, and just loop over the prepared statement with all values. MySQL will queue it internally anyway as separate insert statements if you do a big bunch of inserts, the extended syntax is just syntactic sugar.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Can PHP PDO Statements accept the table name as parameter?
I have a function in my class which is doing some trouble. Here the function
function insert($table,$column = array(),$value = array())
{
$array1 = implode(",", $column);
$array2 = implode(",", $value);
try
{
$sql = $this->connect->prepare("INSERT INTO :table (:date1) VALUES (:date2)");
$sql->bindParam(':table',$table, PDO::PARAM_STR);
$sql->bindParam(':data1',$array1, PDO::PARAM_STR);
$sql->bindParam(':data2',$array2, PDO::PARAM_STR);
$sql->execute();
}
catch(PDOException $e)
{
echo $e->getMessage();
}
}
I call the function with:
-> insert('coupons',array('categorie','name','link','code','id'),array('test11','test','test','test','NULL'));
The error I get is :
Warning: PDOStatement::execute() [pdostatement.execute]: SQLSTATE[HY093]: Invalid parameter number: parameter was not defined in C:\xampp\htdocs\MYFRAMEWORK\lib\database.class.php on line 46
Line 46 is :
$sql->execute();
So now I don't really see where the issue is. Any pointers?
PDOs bind value data, not table and column names.
You are misunderstanding the use of bindings. You cannot bind table and column names with PDO. You bind data to insert INTO those columns. You need to construct the SQL to include the table names and columns using string operations.
Format the data
I've renamed your $column and $value to $column_array, $value_array to make it clear what they are, and assumed that each is a simple array: $column_array = array('column1', 'column2', ...) etc.
$placeholders = array_map(function($col) { return ":$col"; }, $column_array);
$bindvalues = array_combine($placeholders , $value_array);
$placeholders now looks like this:
$placeholders = array(
':column1',
':column2',
...
);
$bindvalues now looks like this:
$bindvalues = array(
':column1'=>'value1',
':column2'=>'value2',
...
);
Build, prepare, execute
$sql = $this->connect->prepare("INSERT INTO $table (" .implode(",", $column_array) .") VALUES (". implode(",", $placeholders) . ")";
This will give you a prepared statement of the form:
$sql = INSERT INTO table_name (column1, column2, ...) VALUES (:column1, :column2, ...)
You can then execute the prepared statement and pass the $values as an argument.
$sql->execute($bindValues);
Note:
One caveat that must be mentioned. Make sure that your original data has been sanitized against SQL Injection. PDO's take care of that for the bound values, but if you are constructing the columns from, say, $_POST data this is vulnerable and needs to be sanitized.
The query is not constructed properly, the values are missing surrounding ' quotes.
When you do an implode, the array2 looks like,
test1,test,test... //and so on.
It needs to be 'test1','test','test'... in-order to be properly binded inside the Insert query.
$sql = $this->connect->prepare("INSERT INTO :table (:date1) VALUES (:date2)");
Also, you have typos in the parameter names, date1 instead of data1 and date2 instead of data2.