Im trying to loop through several arrays to insert data into a mysql database. And Im trying to bind the data so that I can loop through it. There can be a various number of columns to which data is bound.
It appears that the data Im binding is not being processed as expected and the insert ultimately fails.
I have a columns array that stores the column names and data types. I also have a values array that stores the values that are to be inserted. Sample data:
$colArr = array (
array('i', 'ID'),
array('s', 'Date')
);
$valArr = array(
array(1, 'now()'),
array(2, 'now()'),
);
//I create my type and query strings as well as the array referencing the columns for binding.
$valStrForQry = rtrim(str_repeat('?, ', count($v['colArr'])), ', '); //result: '?, ?'
$params = array();
$colsForQry = '';
$typeStr = '';
$cntr = 0;
foreach ($colArr as $cols) {
$colsForQry .= $cols[1] . ', ';
$typeStr .= $cols[0];
$params[] = &$valArr[$cntr][1];
$cntr++;
}
$colsForQry = rtrim($colsForQry, ', '); //result: 'ID, Date'
$qry = 'INSERT INTO table (' . $colsForQry . ') VALUES (' . $valStrForQry . ')';
$stmt = $mysqli->prepare($qry);
//Bind the parameters.
call_user_func_array(array($stmt, 'bind_param'), array_merge(array($typeStr), $params));
//Loop through the values array, assign them using eval, and execute the statement. Im open to suggestions if theres a better way to do this.
foreach ($valArr as $vals) {
$cntr = 0;
foreach ($colArr as $c) {
eval('$' . $c[1] . ' = ' . $vals[$cntr] . ';');
$cntr++;
}
if ($stmt->execute() === FALSE) {
//show $stmt->error for this iteration
} else {
//show success for this iteration
}
}
The first iteration results in a successful insertion of incorrect data. That is, the inserted ID is 0, not 1, and no other info is inserted. The second iteration (and all consecutive ones) results in the following error message: Duplicate entry '0' for key 'PRIMARY'
What am I doing wrong here, is it the eval or something else? Im not sure how to figure this one out.
Instead of continuing to try to get the existing code working, I'm going to suggest a KISS starting point, without the prepare(), the eval(), or the bind_param().
$cols = ['ID', 'Date'];
$vals = [
[1, '\'now()\''],
[2, '\'now()\''],
];
foreach ($vals as $val)
{
$sql = 'INSERT INTO table (' . implode($cols, ', ') . ') VALUES (' . implode($val, ', ') . ')';
// exec here
}
To make this a bit safer, you'll probably want to escape all the values before the implode, or before/as they are put into the array you're working with. The existing code is, IMHO, trying to be too "clever" to do something so simple.
Alternately, you may want to consider switching to using the PDO library instead of mysqli. PDO supports binding of named parameters on a per-parameter basis, which could be done in a loop without the eval().
Someone else may get the provided "clever" solution working instead of course.
Does laravel have an update batch functionality similar to Codeigniter?
Codeigniter uses $this->db->update_batch('mytable', $data, 'title'); to do a batch update. More info could be found here.
But as for laravel's update, it seems that it could only do a single transaction. I feel that this is kind of bad when you have multiple rows to update wherein it will be inside a for loop. Something similar to this:
foreach ($rows => $row) {
DB::table('users')->where('id', $row['row_id'])->update(['votes' => 1]);
}
For atleast you get the picture, right?
If you'll look into this code, your database could get knock out pretty much as it keeps on connecting unlike the update_batch(), only a single transaction is being throw.
TL;DR - it is not clear that the CodeIgniter (CI) method is more efficient than a series of UPDATE queries. In addition, the CI method is less clear than a looped series of UPDATEs.
It is not correct to say that Laravel "keeps on connecting" - it will open a connection at the start of the request and keep using that connection until the request is finished. I think what you mean to say that you are sending a whole lot of queries to the server.
It is true that doing this:
INSERT INTO table VALUES (1, ...);
INSERT INTO table VALUES (2, ...);
INSERT INTO table VALUES (3, ...);
...
INSERT INTO table VALUES (n, ...);
is going to be less efficient than doing this:
INSERT INTO table VALUES (1, ...),
(2, ...),
(3, ...),
...
(n, ...);
But this is not a simple INSERT. Look at the code generated by the CI library in the link you posted. What we're looking at is the difference in efficiency between:
UPDATE table SET value=1 WHERE id=1;
UPDATE table SET value=2 WHERE id=2;
UPDATE table SET value=3 WHERE id=3;
...
UPDATE table SET value=n WHERE id=n;
and
UPDATE table
SET value = CASE
WHEN id = 1 THEN 1
WHEN id = 2 THEN 2
WHEN id = 3 THEN 3
...
WHEN id = n THEN n
ELSE value END
WHERE id IN (1, 2, 3, ..., n);
I'm no SQL expert, but I'm not convinced that the second is more efficient than the first. That long chain of WHEN clauses is going to be processed by the SQL server - a bunch of short UPDATE statements is (in my opinion) going to be more efficient, particularly when you're using an indexed column in the WHERE clause.
Finally, looking at the CI documentation, that update_batch method makes some assumptions about the structure of the data you pass to it - for example, that the first item in the array is the key for the update statements. That (to me at least) is not going to be clear when you look at your code six months from now.
maybe it will help
trait InsertOrUpdate {
static function InsertOrUpdate(array $rows) {
$table = DB::getTablePrefix().with(new self)->getTable();
if (empty($table)) {
return false;
}
$maxRowData = DB::table($table)->where('id', DB::raw('(select max(`id`) from ' . $table . ')'))->first();
if (! empty($maxRowData)) {
$maxId = $maxRowData->id;
$result = DB::statement('ALTER TABLE ' . $table . ' AUTO_INCREMENT = ' . $maxId . ';');
}
$tableColumns = DB::getSchemaBuilder()->getColumnListing($table);
$datetime = Carbon::now()->toDateTimeString();
if (in_array('created_at', $tableColumns)) {
foreach ($rows as $key => $row) {
$rows[$key]['created_at'] = $datetime;
$rows[$key]['updated_at'] = $datetime;
}
}
$first = reset($rows);
$columns = implode(',',
array_map(function($value) {
return "$value";
},
array_keys($first))
);
$values = implode(',', array_map(function($row) {
return '('.implode( ',',
array_map(function($value) { return '"' . str_replace('"', '""', $value) . '"'; }, $row)
).')';
} , $rows)
);
$updates = '';
if (in_array('updated_at', $tableColumns)) {
unset($first['created_at']);
unset($first['updated_at']);
$first['deleted_at'] = NULL;
$updateString = '(CASE WHEN ';
$lastClolumn = count($first);
$columnNum = 1;
foreach (array_keys($first) as $column) {
$updateString .= $column . ' <> VALUES(' . $column . ')';
if ($columnNum != $lastClolumn) {
$updateString .= ' OR ';
}
$columnNum++;
}
$updateString .= ' THEN \'' . $datetime . '\' ELSE `updated_at` END), ';
$updates .= 'updated_at = ' . $updateString;
}
$updates .= implode(',',
array_map(function($value) {return "$value = VALUES($value)"; } , array_keys($first) )
);
$sql = "INSERT INTO {$table}({$columns}) VALUES {$values} ON DUPLICATE KEY UPDATE {$updates};";
return DB::statement($sql);
}
Look into database transactions...
https://laravel.com/docs/5.2/database#database-transactions
PDO seems to require a lot of repetition if you want to use named parameters. I was looking for a way to make it simpler, using a single instance of column/data pairs -- without having to re-type column names or even variable names multiple times.
I'm answering this question myself because I wrote a function that I think does this pretty elegantly, and basically, I wanted to show it off (and help people looking to do the same).
I'm not at all sure if I'm the first one to think of this, or if there are any issues I didn't foresee. Feel free to let me know, or supply your own solution, if you have something better.
Starting from #equazcion's answer, but using slightly different code method:
function bindFields($fields) {
return implode(",", array_map(function ($f) { return "`$f`=:$f"; },
array_keys($fields)));
}
Or if you want traditional INSERT syntax instead of the MySQL-specific INSERT...SET syntax:
function bindFields($fields) {
return "(" . implode(",", array_map(function ($f) { return "`$f`"; },
array_keys($fields))) . ")"
. " VALUES (" . implode(",", array_map(function ($f) { return ":$f"; },
array_keys($fields))) . ")";
}
function bindFields($fields){
end($fields);
$lastField = key($fields);
$bindString = ' ';
foreach($fields as $field => $data){
$bindString .= $field . '=:' . $field;
$bindString .= ($field === $lastField ? ' ' : ',');
}
return $bindString;
}
Supply the data to be inserted using a single associative array. Then, use bindFields() on that array, to generate a string of column = :column pairs for the MySQL query:
$data = array(
'first_column' => 'column data string',
'second_column' => 'another column data string',
'another_column' => 678,
'one_more_field' => 'something'
);
$query = "INSERT INTO tablename SET" . bindFields($data);
$link = new PDO("mysql:host='your-hostname.com';dbname='your_dbname'", 'db_username', 'db_pass');
$prepared = $link->prepare($query);
$prepared->execute($data);
bindFields($data) output:
first_column=:first_column,second_column=:second_column,another_column=:another_column,one_more_field=:one_more_field
EDIT:
Thank you so much for your answers, you really amaze me with so much wisdom :)
I am trying to relay on TuteC's code a bit changed, but can't figure how to make it work properly:
$valor = $_POST['valor'];
$post_vars = array('iphone3g1', 'iphone3g2', 'nome', 'iphone41', 'postal', 'apelido');
foreach($post_vars as $var) {
$$var = "'" . mysql_real_escape_string($_POST[$var]). "', ";
}
$sql = "INSERT INTO clientes (iphone3g1, iphone3g2, nome, iphone41, postal, apelido, valor) VALUES ($$var '$valor')";
$query= mysql_query($sql);
I know there's a bit of cheating on the code, i would need to use substring so the $$var wouldn't output a "," at the end where i need the values, instead i tried to insert a variable that is a value ($valor = $_POST['valor'];)
What is going wrong?
And for the others who tried to help me, thank you very much, i am learning so much with you here at stackoverflow.
I have a form with several field values, when trying to write a php file that reads those values it came out a mostruosity:
$codigounico= md5(uniqid(rand()));
$modelo=$_POST['selectName'];
$serial=$_POST['serial'];
$nif=$_POST['nif'];
$iphone3g1=$_POST['iphone3g1'];
$iphone3g2=$_POST['iphone3g2'];
$iphone3g3=$_POST['iphone3g3'];
$iphone3g4=$_POST['iphone3g4'];
$iphone3gs1=$_POST['iphone3gs1'];
$iphone3gs2=$_POST['iphone3gs2'];
$iphone3gs3=$_POST['iphone3gs3'];
$iphone3gs4=$_POST['iphone3gs4'];
$iphone41=$_POST['iphone41'];
$iphone42=$_POST['iphone42'];
$iphone43=$_POST['iphone43'];
$iphone44=$_POST['iphone44'];
$total=$_POST['total'];
$valor=$_POST['valor'];
$nome=$_POST['nome'];
$apelido=$_POST['apelido'];
$postal=$_POST['postal'];
$morada=$_POST['morada'];
$notas=$_POST['notas'];
$sql="INSERT INTO clientes (postal, morada, nome, apelido, name, serial, iphone3g1, iphone3g2, iphone3g3, iphone3g4, total, valor, iphone3gs1, iphone3gs2, iphone3gs3, iphone3gs4, iphone41, iphone42, iphone43, iphone44, nif, codigounico, Notas)VALUES('$postal', '$morada', '$nome', '$apelido', '$modelo', '$serial', '$iphone3g1', '$iphone3g2', '$iphone3g3', '$iphone3g4', '$total', '$valor', '$iphone3gs1', '$iphone3gs2', '$iphone3gs3', '$iphone3gs4', '$iphone41', '$iphone42', '$iphone43', '$iphone44', '$nif', '$codigounico', '$notas')";
$result=mysql_query($sql);
This is a very dificult code to maintain,
can I make my life easier?
To restrict which POST variables you "import", you can do something like:
$post_vars = array('iphone3g1', 'iphone3g2', '...');
foreach($post_vars as $var) {
$$var = mysql_real_escape_string($_POST[$var]);
}
EDIT: Changed addslashes by mysql_real_escape_string (thanks #Czechnology).
The issue I see is repetition of the same names four times over. This is how I would reduce it to two occurrences (you could drop it to one with more finagling).
$sql = 'INSERT INTO clientes (postal, morada, nome, apelido, name, serial, iphone3g1, iphone3g2, iphone3g3, iphone3g4, total, valor, iphone3gs1, iphone3gs2, iphone3gs3, iphone3gs4, iphone41, iphone42, iphone43, iphone44, nif, codigounico, Notas) VALUES(:postal, :morada, :nome, :apelido, :modelo, :serial, :iphone3g1, :iphone3g2, :iphone3g3, :iphone3g4, :total, :valor, :iphone3gs1, :iphone3gs2, :iphone3gs3, :iphone3gs4, :iphone41, :iphone42, :iphone43, :iphone44, :nif, :codigounico, :notas)';
preg_match_all('/:(\w+)/', $sql, $inputKeys);
$tokens = $inputKeys[0];
$values = array_map($inputKeys[1], function($k){
return mysql_real_escape_string($_POST[$k]);
});
$sql = str_replace($tokens, $values, $sql);
$result = mysql_query($sql);
Depending on how you want to separate your logic, a reversed approach might be more useful, where you would specify the array of key names and iterate over that to generate the SQL string.
<?php
$inputKeys = array('postal', 'morada', 'nome', 'apelido', 'name', 'serial', 'iphone3g1', 'iphone3g2', 'iphone3g3', 'iphone3g4', 'total', 'valor', 'iphone3gs1', 'iphone3gs2', 'iphone3gs3', 'iphone3gs4', 'iphone41', 'iphone42', 'iphone43', 'iphone44', 'nif', 'codigounico', 'Notas');
$keyList = '(' . implode(',', $inputKeys) . ')';
$valueList = 'VALUES (';
foreach ($inputKeys as $k) {
$valueList .= mysql_real_escape_string($_POST[$k]);
$valueList .= ',';
}
$valueList = rtrim($valueList, ',');
$valueList .= ')';
$sql = 'INSERT INTO clientes '.$keyList.' '.$valueList;
$result = mysql_query($sql);
This approach drops the occurrences of the keys to one and will probably more naturally with your application.
TuteC had a good aim but failed in details.
It makes me wonder, why noone has a ready made solution, but had to devise it on-the-fly. Nobody faced the same problem before?
And why most people trying to solve only part of the problem, getting variables only.
The goal is not to get variables.
The goal is to get a query. So, get yourself a query.
//quite handy way to define an array, saves you from typing zillion quotes
$fields = explode(" ","postal morada nome apelido name serial iphone3g1 iphone3g2 iphone3g3 iphone3g4 total valor iphone3gs1 iphone3gs2 iphone3gs3 iphone3gs4 iphone41 iphone42 iphone43 iphone44 nif codigounico Notas");
$sql = "INSERT INTO clientes SET ";
foreach ($fields as $field) {
if (isset($_POST[$field])) {
$sql.= "`$field`='".mysql_real_escape_string($_POST[$field])."', ";
}
}
$sql = substr($set, 0, -2);
This code will create you a query without boring repeating the same field name many times.
But that's still not all improvements you can make.
A really neat thing is called a function.
function dbSet($fields) {
$set = '';
foreach ($fields as $field) {
if (isset($_POST[$field])) {
$set.="`$field`='".mysql_real_escape_string($_POST[$field])."', ";
}
}
return substr($set, 0, -2);
}
put this function into your code library being included into all your scripts (you have one, don't you?)
and then use it for both insert and update queries:
$_POST['codigounico'] = md5(uniqid(rand()));//a little hack to add custom field(s)
if ($action=="update") {
$id = intval($_POST['id']);
$sql = "UPDATE $table SET ".dbSet($fields)." WHERE id = $id";
}
if ($action=="insert") {
$sql = "INSERT $table SET ".dbSet($fields);
}
So, your code become extremely short and reliable and even reusable.
The only thing you have to change to handle another table is $fields array.
It seems your database is not well planned as it contains seemingly repetitive fields (iphone*). You have to normalize your database.
The same approach to use with prepared statements can be found in this my question: Insert/update helper function using PDO
You could use a rather ugly part of PHP called variable variables, but it is generally considered a poor coding practice. You could include your database escaping at the same time. The code would look something like:
foreach($_POST as $key => $value){
$$key = mysql_real_escape_string($value);
}
The variable variables manual section says they do not work with superglobals like $_PATH, but I think it may work in this case. I am not somewhere where I can test right now.
PHP: extract
Be careful though and make sure you clean the data before using it.
$set = array();
$keys = array('forename', 'surname', 'email');
foreach($keys as $val) {
$safe_value = mysqli_escape_string($db, $_POST[$val]);
array_push($set, "$val='$safe_value'");
}
$set_query = implode(',', $set);
Then make your MySQL query something like UPDATE table SET $set_query WHERE... or INSERT INTO table SET $set_query.
If you need to validate, trim, etc, do it before the above code like this:
$_POST["surname"] = trim($_POST["surname"];
Actually, you could make your life easier by making your code a bit more complicated - escape the input before inserting into the database!
$sql =
"INSERT INTO clientes SET
"postal = '" . mysql_real_escape_string($_POST['postal']) . "', ".
"morada = '" . mysql_real_escape_string($_POST['morada']) . "', ".
...
First, I recommend you to create a key-value array like this:
$newClient = array(
'codigounico' => md5(uniqid(rand())),
'postal' => $_POST['postal'],
'modelo' => $_POST['selectName'],
...
);
In this array key is the column name your MySQL table.
In the code you've provided not every field is copied right from POST array (some are calculated, and some keys of the POST aren't equal with the tables column names), so you should use a flexible method.
You should still specify all columns and values but only once so code is still maintainable and you won't have any security errors if someone sends you a broken POST. As for me it looks more like configuration than coding.
Then I recommend you to write a function similar to this:
function buildInsertQuery($tableName, $keyValue) {
$result = '';
if (!empty($keyValue)) {
$delimiter = ', ';
$columns = '';
$values = '';
foreach ($keyValue as $key => $value) {
$columns .= $key . $delimiter;
$values .= mysql_real_escape_string($value) . $delimiter;
}
$columns = substr($columns, 0, -length($delimiter));
$values = substr($values, 0, -length($delimiter));
$result = 'INSERT INTO `' . $tableName . '` (' . $columns . ') VALUES (' . $values . ')';
}
return $result;
}
And then you can simply build your query with just one function call:
$query = buildInsertQuery('clientes', $newClient);