Related
I'm looking for a SQL-injection-secure technique to insert a lot of rows (ca. 2000) at once with PHP and MySQLi.
I have an array with all the values that have to be include.
Currently I'm doing that:
<?php
$array = array("array", "with", "about", "2000", "values");
foreach ($array as $one)
{
$query = "INSERT INTO table (link) VALUES ( ?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$stmt->execute();
$stmt->close();
}
?>
I tried call_user_func_array(), but it caused a stack overflow.
What is a faster method to do this (like inserting them all at once?), but still secure against SQL injections (like a prepared statement) and stack overflows?
You should be able to greatly increase the speed by putting your inserts inside a transaction. You can also move your prepare and bind statements outside of your loop.
$array = array("array", "with", "about", "2000", "values");
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt ->bind_param("s", $one);
$mysqli->query("START TRANSACTION");
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
$mysqli->query("COMMIT");
I tested this code with 10,000 iterations on my web server.
Without transaction: 226 seconds.
With transaction: 2 seconds.
Or a two order of magnitude speed increase, at least for that test.
Trying this again, I don't see why your original code won't work with minor modifications:
$query = "INSERT INTO table (link) VALUES (?)";
$stmt = $mysqli->prepare($query);
$stmt->bind_param("s", $one);
foreach ($array as $one) {
$stmt->execute();
}
$stmt->close();
Yes, you can build a single big query manually, with something like:
$query = "";
foreach ($array as $curvalue) {
if ($query)
$query .= ",";
$query .= "('" . $mysqli->real_escape_string($curvalue) . "')";
}
if ($query) {
$query = "INSERT INTO table (link) VALUES " . $query;
$mysqli->query($query);
}
You should first convert your array into a string. Given that it is an array of strings (not a two-dimentional array), you can use the implode function.
Please be aware that each value should be enclosed into parenthesis and properly escaped to ensure a correct INSERT statement and to avoid the risk of an SQL injection. For proper escaping you can use the quote method of the PDOConnection -- assuming you're connecting to MySQL through PDO. To perform this operation on every entry of your array, you can use array_map.
After escaping each value and imploding them into a single string, you need to put them into the INSERT statement. This can be done with sprintf.
Example:
<?php
$connection = new PDO(/*...*/);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$dataToBeSaved = [
'some',
'data',
'with "quotes"',
'and statements\'); DROP DATABASE facebook_main; --'
];
$connection->query(
sprintf(
'INSERT INTO table (link) VALUES %s',
implode(',',
// for each entry of the array
array_map(function($entry) use ($connection) {
// escape it and wrap it in parenthesis
return sprintf('(%s)', $connection->quote($entry));
}, $dataToBeSaved)
)
)
);
Note: depending on the amount of records you're willing to insert into the database, you may want to split them into several INSERT statements.
I have some method to insert some data into a database like this:
public function register($username, $email, $hashedPassword, $activationCode)
{
try {
$conn = Database::getConnection();
// Connect and create the PDO object
$conn->exec('SET CHARACTER SET utf8'); // Sets encoding UTF-8
// Define and prepare an INSERT statement
$sql = 'INSERT INTO users (username, email, pass, reset_token, dateAdded )
VALUES (:username, :pass, :email, :token, now())';
$sqlprep = $conn->prepare($sql);
// Adds value with bindParam
$sqlprep->bindParam(':username', $username, PDO::PARAM_STR);
$sqlprep->bindParam(':email', $email, PDO::PARAM_STR);
$sqlprep->bindParam(':pass', $hashedPassword);
$sqlprep->bindParam(':token', $activationCode);
// If the query is successfully executed, output the value of the last insert id
if ($sqlprep->execute()) {
//echo 'Succesfully added the row with id='. $conn->lastInsertId();
$this->result = true;
}
$conn = null; // Disconnect
} catch (PDOException $e) {
include('../views/error.php');
include('../views/admin/includes/footer.php');
exit();
}
}
The problem is I think it's not a good method if I have so many arguments for my function to enter into a database. So is it any good way I can enter a lot of fields just by using 1 parameter but still using bindParam? Since I see a lot of examples is only using prepare without bindParam. I think I can use an array, but I don't know the proper way to do it. So I need some help how I can do it.
since you want keep your bindparam i suggest you use input like this:
$input = array('username' => $username, 'activationHash' => $activationHash);
and in your bindParam add a code like this:
public function register($input){
//code
$sqlprep->bindParam(':username', $input['username'], PDO::PARAM_STR);
//other
}
hope this will solve your problem
https://stackoverflow.com/a/10060755/1747411
Check second example, you have to repeat values with binds
e.g
VALUES (:username1, :pass1, :email1, :token1, now()), (:username2, :pass2, :email2, :token2, now())
and bindParam with loop
You can insert the params as an array into $sqlprep->execute($param_array)
Or, simply passing each param into an array inside execute, like this: $sqlprep->execute(array($param1, $param2))
Update:
Pass values into $input as an array:
$input = array('username' => $username, 'activationHash' => $activationHash); //and so on
Now on the model side,
You can bind these values to params using foreach loop like this:
foreach ($values as $key => $value) {
$sqlprep->bindParam(':' . $key, $value , PDO::PARAM_STR);
}
I am looking to do multiple inserts using PHP PDO.
The closest answer I have found is this one
how-to-insert-an-array-into-a-single-mysql-prepared-statement
However the example thats been given uses ?? instead of real placeholders.
I have looked at the examples on the PHP doc site for place holders
php.net pdo.prepared-statements
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
$stmt->bindParam(':name', $name);
$stmt->bindParam(':value', $value);
Now lets say I wanted to achieve the above but with an array
$valuesToInsert = array(
0 => array('name' => 'Robert', 'value' => 'some value'),
1 => array('name' -> 'Louise', 'value' => 'another value')
);
How would I go about it with PDO and multiple inserts per transaction?
I imagine it would start of with a loop?
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
foreach($valuesToInsert as $insertRow){
// now loop through each inner array to match binded values
foreach($insertRow as $column => value){
$stmt->bindParam(":{$column}", value);
}
}
$stmt->execute();
However the above does not work but hopefully will demonstrate what im trying to achieve
First of all, ? symbols are real place-holders (most drivers allow to use both syntaxes, positional and named place-holders). Secondly, prepared statements are nothing but a tool to inject raw input into SQL statements—the syntax of the SQL statement itself is unaffected. You already have all the elements you need:
How to insert multiple rows with a single query
How to generate SQL dynamically
How to use prepared statements with named place-holders.
It's fairly trivial to combine them all:
$sql = 'INSERT INTO table (memberID, programID) VALUES ';
$insertQuery = [];
$insertData = [];
$n = 0;
foreach ($data as $row) {
$insertQuery[] = '(:memberID' . $n . ', :programID' . $n . ')';
$insertData['memberID' . $n] = $memberid;
$insertData['programID' . $n] = $row;
$n++;
}
if (!empty($insertQuery)) {
$sql .= implode(', ', $insertQuery);
$stmt = $db->prepare($sql);
$stmt->execute($insertData);
}
I'm assuming you are using InnoDB so this answer is only valid for that engine (or any other transaction-capable engine, meaning MyISAM isn't included).
By default InnoDB runs in auto-commit mode. That means each query is treated as its own contained transaction.
To translate that to something us mortals can understand, it means that every INSERT query you issue will force hard-disk to commit it by confirming it wrote down the query information.
Considering how mechanical hard-disks are super slow since their input-output operation per second is low (if I'm not mistaken, the average is 300ish IO's), it means your 50 000 queries will be - well, super slow.
So what do you do? You commit all of your 50k queries in a single transaction. It might not be the best solution for various purposes but it'll be fast.
You do it like this:
$dbh->beginTransaction();
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
foreach($valuesToInsert as $insertRow)
{
// now loop through each inner array to match bound values
foreach($insertRow as $column => value)
{
$stmt->bindParam(":$column", value);
$stmt->execute();
}
}
$dbh->commit();
A little modifications in solution provided by N.B
$stmt->execute() should be outside of inner loop because you may have one or more columns that need to bind before calling $stmt->execute() else you 'll get exception "Invalid parameter number: number of bound variables does not match number of token".
2nd "value" variable were missing dollar signs.
function batchinsert($sql,$params){
try {
db->beginTransaction();
$stmt = db->prepare($sql);
foreach($params as $row)
{
// now loop through each inner array to match bound values
foreach($row as $column => $value)
{
$stmt->bindParam(":$column", $value);
}
$stmt->execute();
}
db->commit();
} catch(PDOExecption $e) {
$db->rollback();
}
}
Test:
$sql = "INSERT INTO `test`(`name`, `value`) VALUES (:name, :value)" ;
$data = array();
array_push($data, array('name'=>'Name1','value'=>'Value1'));
array_push($data, array('name'=>'Name2','value'=>'Value2'));
array_push($data, array('name'=>'Name3','value'=>'Value3'));
array_push($data, array('name'=>'Name4','value'=>'Value4'));
array_push($data, array('name'=>'Name5','value'=>'Value5'));
batchinsert($sql,$data);
Your code was actually ok, but had a problem in $stmt->bindParam(":$column", value); It should be $stmt->bindValue(":{$column}", $value); and it will work perfectly. This will assist others in future.
Full code:
foreach($params as $row)
{
// now loop through each inner array to match bound values
foreach($row as $column => $value)
{
$stmt->bindValue(":{$column}", $value); //EDIT
}
// Execute statement to add to transaction
$stmt->execute();
}
Move execute inside of the loop.
$stmt = $dbh->prepare("INSERT INTO REGISTRY (name, value) VALUES (:name, :value)");
foreach($valuesToInsert as $insertRow)
{
$stmt->execute($insertRow);
}
If you experience any problems with this such recommended way, you have to ask a question, describing these certain problems.
Is there a way I can put these bindParam statements into one statement?
$q = $dbc -> prepare("INSERT INTO accounts (username, email, password) VALUES (:username, :email, :password)");
$q -> bindParam(':username', $_POST['username']);
$q -> bindParam(':email', $_POST['email']);
$q -> bindParam(':password', $_POST['password']);
$q -> execute();
I was using mysqli prepared before where it was possible, I switched to PDO for assoc_array support. On the php.net website for PDO it shows them on seperate lines, and in all examples I have seen it is on seperate lines.
Is it possible?
Example 2 on the execute page is what you want:
$sth->execute(array(':calories' => $calories, ':colour' => $colour));
You may want to look at the other examples too. With question mark parameters, it would be:
$q = $dbc -> prepare("INSERT INTO accounts (username, email, password) VALUES (?, ?, ?)");
$q->execute(array($_POST['username'], $_POST['email'], $_POST['password']));
If those are the only columns, you can just write:
$q = $dbc -> prepare("INSERT INTO accounts VALUES (?, ?, ?)");
$q->execute(array($_POST['username'], $_POST['email'], $_POST['password']));
helper function is a function that makes you help to avoid writing bunch of repetitive code every time you want to run a query.
This is called "programming" and there is almost none of it on this site, at least under "PHP" tag.
While many peiople thinks that programming stands for copy/pasting chunks of code from manual examples, it's somewhat different.
Although it's hard to learn but really worth it, especially if you're devoting yourself to web-developing.
As you can see, no accepted answer did no real help for you, as you still have to write something like
$sth->execute(array(':username' => $_POST['username'],
':email' => $_POST['email']
':password' => $_POST['password']);
as many times as many fields in your table, which makes not much difference from your initial approach, still makes you write each field name FOUR times.
But being a programmer, you can use powers of programming. A loop, for example - one of cornerstone programming operators.
Every time you see repetitions, you know there should be a loop.
for example, you can set up a list of fields, naming them only once.
And let a program do the rest.
Say, such a function like this one
function pdoSet($fields, &$values, $source = array()) {
$set = '';
$values = array();
if (!$source) $source = &$_POST;
foreach ($fields as $field) {
if (isset($source[$field])) {
$set.="`$field`=:$field, ";
$values[$field] = $source[$field];
}
}
return substr($set, 0, -2);
}
being given an array of field names, it can produce both insert statement and data array for you. Programmatically. So, your code become no more than these 3 short lines:
$fields = array('username', 'email', 'password');
$stmt = $dbh->prepare("INSERT INTO accounts SET ".pdoSet($fields,$values));
$stmt->execute($values);
Your Common Sense is totally right that the aim of coding is to save typing... but his solution doesn't help with the BindParams bit. I couldn't find anything else about this online, so here's something I finally just persuaded to work - I hope it's useful for someone!
//First, a function to add the colon for each field value.
function PrepareString($array){
//takes array (title,author);
//and returns the middle bit of pdo update query :title,:author etc
foreach($array as $k =>$v){
$array[$k]=':'.$v;
}
return implode(', ', $array);
}
Then...
function PdoInsert($table_name,$array){
$db = new PDO(); //however you create your own pdo
//get $fields and $vals for statement
$fields_vals=array_keys($array);
$fields=implode(',',$fields_vals);
$vals=PrepareString($fields_vals);
$sql = "INSERT INTO $table_name($fields) VALUES ($vals)";
$qwe=$db->prepare($sql);
foreach ($array as $k =>$v ){
//add the colon to the key
$y=':'.$k;
//god knows why it doesn't like $qwe->bindParam($y,$v,PDO::PARAM_STR);
// but it really doesn't! So we refer back to $array.
//add checks for different binding types here
(see PDO::PARAM_INT is important in bindParam?)
$qwe->bindParam($y,$array[$k],PDO::PARAM_STR);
}
if ($qwe->execute()==true){
return $db->lastInsertId();
}
else {
return $db->errorCode();
}
}
Then you can insert anything by doing
PdoInsert('MyTableName',array('field1'=>$value1,'field2'=>$value2...));
Having previously sanitized your values of course.
+1 to Matthew Flaschen for the accepted answer, but I'll show you another tip. If you use SQL parameters with names the same as the entries in $_POST, you could take advantage of the fact that $_POST is already an array:
$q->execute($_POST);
The SQL parameter names are prefixed with a colon (:) but the keys in the $_POST array are not. But modern versions of PDO account for this - you no longer need to use colon prefixes in the keys in the array you pass to execute().
But you should be careful that anyone can add extra parameters to any web request, and you should get only the subset of $_POST params that match parameters in your query.
$q = $dbc -> prepare("INSERT INTO accounts (username, email, password)
VALUES (:username, :email, :password)");
$params = array_intersect_key($_POST, array("username"=>1,"email"=>1,"password"=>1));
$q->execute($params);
Personally, I prefer to use a wrapper function for all of pdo, which simplifies the code necessary substantially.
For example, to run bound queries (well, all my queries), I do this:
$iterable_resultset = query("INSERT INTO accounts (username, email, password) VALUES (:username, :email, :password)", array(':username'=>'bob', ':email'=>'bob#example.com', ':password'=>'bobpassword'));
Note that not only is the sql simply a string, but it's actually a reusable string, as you can simply pass the sql as a string and change the array of variables to pass in if you want to perform a similar insert right after that one (not applicable to this situation, but applicable to other sql use cases).
The code that I use to create this wrapper function is as below:
/**
* Run bound queries on the database.
*
* Use: query('select all from players limit :count', array('count'=>10));
* Or: query('select all from players limit :count', array('count'=>array(10, PDO::PARAM_INT)));
*
* Note that it returns foreachable resultset object unless an array is specifically requested.
**/
function query($sql, $bindings=array(), $return_resultset=true) {
DatabaseConnection::getInstance(); // Gets a singleton database connection
$statement = DatabaseConnection::$pdo->prepare($sql); // Get your pdo instance, in this case I use a static singleton instance. You may want to do something simpler.
foreach ($bindings as $binding => $value) {
if (is_array($value)) {
$first = reset($value);
$last = end($value);
// Cast the bindings when something to cast to was sent in.
$statement->bindParam($binding, $first, $last);
} else {
$statement->bindValue($binding, $value);
}
}
$statement->execute();
if ($return_resultset) {
return $statement; // Returns a foreachable resultset
} else {
// Otherwise returns all the data an associative array.
return $statement->fetchAll(PDO::FETCH_ASSOC);
}
}
// Wrapper to explicitly & simply get a multi-dimensional array.
function query_array($sql_query, $bindings=array()) {
return query($sql_query, $bindings, false); // Set return_resultset to false to return the array.
}
As noted in the comments, you'd want to use your own method for setting up a database connection and getting an initialized pdo, but in general it allows your bound sql to be cut down to just a single line.
Thanks for looking. All helpful answers/comments are up voted.
In php, you can use NOW() like this:
mysql_query("INSERT INTO tablename (id, value, time_created)
VALUES ('{$id}', '{$value}', NOW())");
How can I do the same thing in PDO. When I bind like this, I get an error:
$stmt->bindParam(':time_added', NOW(), PDO::PARAM_STR);
Is it the PDO:PARAM_STR?
Because nobody has explicitly answered the question, I'll add the correct answer for the sake of completeness.
$stmt = $pdoDb->prepare('INSERT INTO tablename (id, value, time_created) VALUES (:id, :value, NOW())');
// either bind each parameter explicitly
$stmt->bindParam(':id', $id); // PDOStatement::bindValue() is also possibly
$stmt->bindParam(':value', $value);
$stmt->execute();
// or bind when executing the statement
$stmt->execute(array(
':id' => $id,
':value' => $value
));
Presuming your PDO statement is correct you could do something like this:
$date = date('Y-m-d H:i:s');
$stmt->bindParam(':time_added', $date, PDO::PARAM_STR);
None of the answers solve the question as I see it!
So there are some of my findings:
there is NO WAY how to force PDO to pass MySQL function call as a query value - so there is no way to do simple wrapper that will be able to use NOW() or any other function as passed values. Every time you need something like that, you need manually change the query, so the function call is part of the query string. :-(
I'm using function that tests given values for MySQL function I am using and modifies the query itself, but it is not a good solution to my opinion... :-}
This might be useful to some of you, maybe not. I was confronted with the same problem as Ollie Saunders was. I'm pretty new to php/mysql, and most of all PDO. I was able to solve the problem with the following:
$active = 0;
$id = NULL;
$query = "INSERT
INTO tbl_user(ID_user, firstname, lastname, email, password, active, create_date)
VALUES (?,?,?,?,?,?,NOW())";
if($stmt=$this->conn->prepare($query)) {
$stmt->bind_param('issssi', $id, $firstname, $lastname, $email, $password, $active);
$stmt->execute();
}
and guess what it works! Hope to have helped here. Any comments are welcome. Try it and tell me if it worked for you, or if you have any additions.
To answer Elmo's question, you can create a PDO wrapper that allows for SQL functions like NOW(). You just need to pass an additional argument with the columns that you want to use SQL functions for. Here's mine:
function pInsertFunc($action, $table, $values, $sqlfunctions)
{
global $pdb;
// There's no way to pass an SQL function like "NOW()" as a PDO parameter,
// so this function builds the query string with those functions. $values
// and $sqlfunctions should be key => value arrays, with column names
// as keys. The $values values will be passed in as parameters, and the
// $sqlfunction values will be made part of the query string.
$value_columns = array_keys($values);
$sqlfunc_columns = array_keys($sqlfunctions);
$columns = array_merge($value_columns, $sqlfunc_columns);
// Only $values become ':paramname' PDO parameters.
$value_parameters = array_map(function($col) {return (':' . $col);}, $value_columns);
// SQL functions go straight in as strings.
$sqlfunc_parameters = array_values($sqlfunctions);
$parameters = array_merge($value_parameters, $sqlfunc_parameters);
$column_list = join(', ', $columns);
$parameter_list = join(', ', $parameters);
$query = "$action $table ($column_list) VALUES ($parameter_list)";
$stmt = $pdb->prepare($query);
$stmt->execute($values);
}
Use it like this:
$values = array(
'ID' => NULL,
'name' => $username,
'address' => $address,
);
$sqlfuncs = array(
'date' => 'NOW()',
);
pInsertFunc("INSERT INTO", "addresses", $values, $sqlfuncs);
The query string that results looks like this:
INSERT INTO addresses (ID, name, address, date) VALUES (:ID, :name, :address, NOW())
other than NOW() i also utilize the "timestamp" type column and set its default to CURRENT_TIMESTAMP .. so i just pass nothing for that field and time is automatically set. maybe not exactly what ur looking for.