I'm trying to combine an SQLite transaction and prepared statement to get the best insert speed for thousands of records. However, all the inserted lines are empty.
Printing the variables before inserting shows that they have the correct data and there are no errors.
$db->beginTransaction();
$insert_stmt = $db->prepare("INSERT INTO `table` VALUES (:id, :value2, :value3, :value4)");
$insert_stmt->bindValue(":id", $id);
$insert_stmt->bindValue(":value2", $value2);
$insert_stmt->bindValue(":value3", $value3);
$insert_stmt->bindValue(":value4", $value4);
foreach ($records as $record)
{
$id = $record["id"];
$value2 = $record["value2"];
$value3 = $record["value3"];
$value4 = $record["value4"];
$insert_stmt->execute();
print_r($db->errorInfo()); // print errors
}
$db->commit();
What's wrong with the code?
How can I get better output? Printing the prepared statements before executing for example, to see if there's something wrong with it.
You have to put the bindValue calls into your loop.
The values get copied into the statement when you execute bindValue. Whatever happens to the variable you used to do that afterwards does not matter.
Related
I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.
I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.
Here's my dilemma, I'm at a loss. I'm trying to add multiple values from an array into a database by looping through them with a prepared mysql statement:
If I "execute" the statement within the foreach loop it only adds the
last entry from the array.
If I "execute" it outside of the loop it inserts 3 individual rows to the database. I would like all columns filled out under one created now() date and row.
If I prepare the $stmt before the foreach loop, then it errors undefined variable. So I've put it after defining in the loop, even though in w3schools it shows that you can define the "bind_param" variables prior to setting parameters, which I don't understand either. Thanks!
foreach ($products as $v) {
$product_was = $v['name'];
$product_now = $v['name'];
$was = $v['was'];
$now = $v['now'];
$stmt = $conn->prepare("INSERT INTO $table_name (date_created, $product_was, $product_now) VALUES (now(), ?, ?)");
$stmt->bind_param("ss", $was, $now);
$stmt->execute();
}
$stmt->close();
$conn->close();
I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.
This question already has answers here:
Can I bind an array to an IN() condition in a PDO query?
(23 answers)
Closed 8 years ago.
This is my code:
if(isset($_POST['abc']))
{
$things['abc'] = mysqli_real_escape_string($connect, implode("','", $_POST['abc']));
$result = mysqli_query($connect, "SELECT * FROM this_list WHERE abc_column IN ('{$things['abc']}')");
if (!$result)
{
echo "Error fetching results: " . mysqli_error();
}
else
{
while ($row = mysqli_fetch_array($result))
{
$abc[] = $row['description'];
}
}
}
The above code uses mysqli_real_escape_string(), and $things is an array with checkbox values that is received via POST. This array contains the list of strings separated by comma that I am using in the query.
When I was searching on the net, I noticed that some people say mysqli_real_escape_string() may prevent sql injection, I was thinking maybe prepared statement for checkbox values might be more safer against sql injection.
I have used prepared statement with separate parameters to prevent sql injection. But I am stuck on this one and I dont know how to change the above code to a prepare() statement since it uses an array $things['abc']. I tried searching and everytime I search array in prepared statement, I am getting info on Java, etc.. Can someone enlighten me on how I can do this with php please?
EDIT:
After the help from onetrickpony code below, this is what I have now:
if(isset($_POST['abc']))
{
$ph = rtrim(str_repeat('?,', count($_POST['abc'])), ',');
$query = sprintf("SELECT col1 FROM abc_table WHERE col2 IN (%s)", $ph);
$stmt = mysqli_prepare($connect, $query);
// bind variables
$params = array();
foreach($_POST['abc'] as $v)
$params[] = &$v;
array_unshift($params, $stmt, str_repeat('s', count($_POST['abc']))); // s = string type
call_user_func_array('mysqli_stmt_bind_param', $params);
mysqli_stmt_execute($stmt);
// Get the data result from the query.
mysqli_stmt_bind_result($stmt, $col1);
/* fetch values and store them to each variables */
while (mysqli_stmt_fetch($stmt)) {
$name[] = $col1;
echo $name;
}
//loop to echo and see whats stored in the array above
foreach($name as $v) {
echo $v;
}
// Close the prepared statement.
$stmt->close();
}
In the above code, the sqli method for prepare statement seems to work which is great. However, when I use the mysqli_stmt_bind_result(), the $name[] array inside the while loop only seems to print the last row.
UPDATE:
onetrickpony's code with the mysqli method for using php array in a Prepared Statement worked fine and it was a very good approach he had suggested. However, I have been having nightmare with the second half of the code which is trying to get the fetched array results to work. After trying for more than a day, I have given up on that and I have made the switch to PDO. Again onetrickpony's advise below was totally worth it. Making the switch to PDO made the code so much easier and simpler and couldnt believe it.
Try this:
// build placeholder string (?,?...)
$ph = rtrim(str_repeat('?,', count($_POST['abc'])), ',');
$query = sprintf("SELECT * FROM this_list WHERE abc_column IN (%s)", $ph);
$stm = mysqli_prepare($connect, $query);
// bind variables (see my notes below)
$params = array();
foreach($_POST['abc'] as $v)
$params[] = &$v;
// s = string type
array_unshift($params, $stm, str_repeat('s', count($_POST['abc'])));
call_user_func_array('mysqli_stmt_bind_param', $params);
mysqli_stmt_execute($stm);
It appears that mysqli_stmt_bind_param cannot be called multiple times to bind multiple variables. And even worse, it requires referenced variables. I'd recommend you switch to PDO, just because of these limitations that force you to write ugly code :)