Related
So, I have a following js and php:
JS:
var names = jQuery('#name').val();
data : {'action':'AJAX' , name:names },
The #name values are "mike,sean,steve"
PHP:
global $wpdb;
$names = $_POST['name'];
$table = $wpdb->prefix . 'my_name';
$RSS_UPDATE = $wpdb->get_col("SELECT update_number FROM $table WHERE id_name IN ($names)");
//update_number are int (example: 0,3,1,2)
$name = explode(',', $names);
if ( $RSS_UPDATE ){
foreach ( $RSS_UPDATE as $RSS_SINGLE ){
$RSS_ROW_NEW = $RSS_SINGLE + 1;
$wpdb->update($table, array('update_number' => $RSS_ROW_NEW),array( 'id_name' => $name));
}
}
So, few things:
what I am trying to achieve:
With the input values, get corresponding update_number. Then increase each value by "1" and update the same column with the new value.
Errors
Unknown column 'Array' in 'where clause' for query SELECT update_number FROM wp_my_name WHERE id_name IN (Array)
Just in general, something is not right...
Can someone help me out?
Thank you.
EDIT:
Does this look right?
if(!empty($_POST['name'])) {
$names = $_POST['name']; //array
$table = $wpdb->prefix . 'rh_subs';
$query = "SELECT update_number FROM $table WHERE id_name = %s";
$RSS_UPDATE = $wpdb->get_results($wpdb->prepare($query, $names));
if(!empty($RSS_UPDATE)) {
foreach($RSS_UPDATE as $RSS_SINGLE) { // for each row
$RSS_ROW_NEW = $RSS_SINGLE->update_number + 1;
$wpdb->update($table, array('update_number' => $RSS_ROW_NEW),array('id_name' => $RSS_SINGLE->id_name));
}
}
}
Just what I've said in the comments in your earlier post, since you're taking multiple inputs, you'll need to use the WHERE IN clause.
The simple example would be like this:
$_POST['name']; // these are comma delimited string of names
// "mike,sean,steve"
So in essence, you'll need to construct them inside a WHERE IN clause like this:
WHERE id_name IN ('mike', 'sean', 'steve')
The unsafe and dirtiest way would be to just explode - put quotations on the strings - implode it back together with comma again:
$names = array_map(function($e){
return "'$e'";
}, explode(',', $test));
$names = implode(',', $names);
// 'mike','sean','steve' // SATISFIES WHERE IN CLAUSE
// BUT UNSAFE!
So in order to do this safely, use the wpdb prepared statements. (This could get you started).
if(!empty($_POST['name'])) {
$names = explode(',', $_POST['name']); // explode the comma delimited string into an array
$table = $wpdb->prefix . 'my_name';
$stringPlaceholders = implode(', ', array_fill(0, count($names), '%s')); // create placeholders for the query statement, this will generate
$statement = $wpdb->prepare("SELECT update_number, id_name FROM $table WHERE id_name IN ($stringPlaceholders)", $names); // create the statement using those placeholders
$RSS_UPDATE = $wpdb->get_results($statement); // execute
// fetch resuls
if(!empty($RSS_UPDATE)) {
foreach($RSS_UPDATE as $RSS_SINGLE) { // for each row
$RSS_ROW_NEW = $RSS_SINGLE->update_number + 1;
$wpdb->update($table, array('update_number' => $RSS_ROW_NEW),array('id_name' => $RSS_SINGLE->id_name));
}
}
}
Note: Of course you can get creative yourself. I think you could combine the UPDATE and WHERE IN clause so that you'll just execute all of this once.
First of all, It seems that $_POST['name'] returns an array.
You can view what exactly you are getting in $_POST['name'] by:
var_dump($_POST['name'], true);
Also For the id_name, if they are like these "mike,sean,steve" then you should do this for adding quotes for strings and the escaping issue so that they can be like this "'mike','sean','steve'" as you are using a WHERE IN clause:
$names = $_POST['name'];
if(!is_array($names)) $names = explode(",",$names);
$new_names = array();
foreach($names as $name){
$name = get_magic_quotes_gpc() ? stripslashes($name) : $name;
$new_names[] = "'".mysql_real_escape_string($name)."'";
}
$names = implode(",", $new_names);
I have the following array which I would like to insert into mysql database.
Item[0] = Soccer
Item[1] = Rugby
Item[2] = Football
Item[3] = Netball
Item[4] = Hockey
I am using the following function to insert into the database, located in functions.php:
//Capture items
function item($register_data)
{
array_walk($register_data,'array_clean');
$fields = ' '.implode(',',array_keys($register_data)).' ';
$data = '\''.implode('\',\'',$register_data).'\'';
//Insert user Data into the database
$query = "INSERT INTO items ($fields) VALUES ($data)";
mysql_query($query);
}
Now this is how I insert:
for($i =0;$i<4;$i++)
{
$item = array($i = item[i]);
}
//Call the function to insert into the database
item($item);
This method doesn't seem to work. Please assist
PHP has a built in serialize & deserialize functions (http://php.net/manual/en/function.serialize.php) for that. You can pass any object or array to it, and it will return a serialized string, which you can store in a single field in the database.
Since your items table has two columns, one of which is id (auto increment) you're going to want to skip over it and only specify the second column ItemName. This is described in the mysql documentation on this page.
Lets say you want to insert 5 sports in this table:
$sports[0] = "Soccer";
$sports[1] = "Rugby";
$sports[2] = "Football";
$sports[3] = "Netball";
$sports[4] = "Hockey";
You'll want the query to look something along these lines:
INSERT INTO items (ItemName) VALUES ('Soccer'), ('Rugby'), ('Football'), ('Netball'), ('Hockey');
The code would look something like this:
function insert($insert_data)
{
$fields = implode(', ', $insert_data);
$data = '(\''.implode('\'), (\'', $insert_data).'\')';
//mysql_query("INSERT INTO items (ItemName) VALUES $data;");
echo "INSERT INTO items (ItemName) VALUES $data;";
}
$sports = array();
$sports[0] = "Soccer";
$sports[1] = "Rugby";
$sports[2] = "Football";
$sports[3] = "Netball";
$sports[4] = "Hockey";
insert($sports);
You may also want to sanitize the strings first and take a look at mysqli since mysql is pretty outdated.
I am looking for the fastest way to INSERT 1-3,000 rows into a MySQL database using PHP. My current solution is taking around 42 seconds to insert the rows which I think that could be much faster.
I am using a self-written DB class, the insert() method takes two params (string) $table and (array) $vars. The $items array is an associative array where the key is the column name in the table and the value is the value to insert. This works really well for because I sometimes have 30 columns in a table and already have the data there in an array. The insert() method is below:
function insert($table,$vars) {
if(empty($this->sql_link)){
$this->connection();
}
$cols = array();
$vals = array();
foreach($vars as $key => $value) {
$cols[] = "`" . $key . "`";
$vals[] = "'" . $this->esc($value) . "'";
}
//join the columns and values to insert into sql
$fields = join(', ', $cols);
$values = join(', ', $vals);
$insert = mysql_query("INSERT INTO `$table` ($fields) VALUES ($values);", $this->sql_link);
return $insert;
}
It should be self-explanatory but basically I take the keys and values from $vars and create an INSERT statement. It works, I think the problem I am having is sending the queries one at a time.
Should I build a long query string?
INSERT INTO table (field, field2, etc) VALUES (1, 2, ect);INSERT INTO table (field, field2, etc) VALUES (1, 2, ect);INSERT INTO table (field, field2, etc) VALUES (1, 2, ect);INSERT INTO table (field, field2, etc) VALUES (1, 2, ect);INSERT INTO table (field, field2, etc) VALUES (1, 2, ect); and send it all at one time? If so can this handle 3,000 insert statements in one call?
Is there another way I am not looking at? Any info is appreciated.
Thanks
The most performant way is to use the multiple-row insert syntax:
INSERT INTO table (field, field2, etc) VALUES (1, 2, etc),(1, 2, etc),(1, 2, etc);
Manual:
INSERT statements that use VALUES syntax can insert multiple rows. To do this, include multiple lists of column values, each enclosed within parentheses and separated by commas. Example:
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
The values list for each row must be enclosed within parentheses.
Two ways of improve insertion speeds:
At the start, before any INSERT, do a mysql_query("START TRANSACTION"); or a simpler mysql_query("BEGIN");. At the end, do a mysql_query("COMMIT");. These two lines, speeds up the bulk insertion a 5-10x performance.
If the table backend is MyISAM (NOT InnoDB), do the INSERTs followed with the word DELAYED. For example, instead of INSERT INTO table use INSERT DELAYED INTO table for an aditional 10-15x speed-up.
If you combine the 2 methods, is posible to achieve a speed-up of 100 times.
Mysql can import data directly from a file which can significantly speed up importing data. See:
LOAD DATA INFILE Syntax
<?php
$data = "data/fullz.txt";
$db = new PDO("sqlite:db/ssninfo.db");
$db->beginTransaction();
$stmt = $db->prepare('INSERT INTO ssninfo (fname,lname,ssn,address,city,state,zip,phone,birth,email) VALUES (?,?,?,?,?,?,?,?,?,?)');
if($file=fopen($data, "r")){
while(!feof($file)){
$line = fgets($file);
$part = explode('|', $line);
$stmt->execute($part);
}
}
$db->commit();
As usual, it depends; you don't even mention which engine you're using, which is a big determinant. But I've found the MySQL manual guidance pretty reliable.
http://dev.mysql.com/doc/refman/5.0/en/insert-speed.html
Auto discovering the maximum ammount of inserts.
to insert that kind of ammounts (3000) there should not be any problem of doing something like (assuming you use pdo):
$stmt = $dbh->prepare("INSERT INTO yourtable(name, id) VALUES " . str_repeat('(?,?),', $amountOfRows - 1) . '(?,?)');
You can improve that to make create generic way to create big statements like the one above for tables with different ammount of fields:
$fields = array("name", "id");
$fieldList = implode(", ", $fields);
$params = '(' . str_repeat('?,', count($fields) - 1) . '?)';
$values = str_repeat($params . ',', $ammountOfRows - 1) . $params;
$stmt = $dbh->prepare("INSERT INTO $table($fieldList) VALUES " . $values);
but the problem with the above solution is that wont work with any combination of rows and ammount of fields.
Seems to be that mysql is not only limited by the ammount of rows but also the ammount of parameters is taken into account.
But you dont want to be changing your code whenever a new mysql release changes the limit of the parameters, rows or even the size of the sql sentence.
So, a much better approach to create a generic way to generate big statements would be trying to feat the underlaying database engine:
/**
* Creates an insert sql with the maximum allowed of parameters
* #param string $table
* #param string $attributeList
* #param int &$ammountInserts returns the ammount of inserts
* #return \PDOStatement
*/
public static function getBiggestInsertStatement($table, $attributeList, $max, &$ammountInserts)
{
$previousSize = null;
$size = 10;
$sql = 'INSERT INTO ' . $table . '(' . implode(',', $attributeList) . ') values ';
$return = null;
$params = '(' . str_repeat('?,', count($attributeList) - 1) . '?)';
do {
try {
$previousSize = $size;
$values = str_repeat($params . ',', $size - 1) . $params;
$return = Db::getInstance()->prepare($sql . $values);
if ($size > $max) {
$values = str_repeat($params . ',', $max - 1) . $params;
$return = Db::getInstance()->prepare($sql . $values);
$ammountInserts = $max;
break;
}
$ammountInserts = $size;
$size *= 2;
} catch(\Exception $e) {
}
} while($previousSize != $size);
return $return;
}
One thing that you must have in mind is that since you dont know that limits the query could be able to push a lower ammount of items that all that you need to insert.
So you would have to create a strategy like the one below to succesfuly achieve insert them all in any possible scenario:
$insert = Db::getBiggestInsertStatement($table, array('field1','field2'), $numrows, $maximumInserts);
$i = 0;
$values = array();
for ($j = 0; $j < $numrows; $j++) {
if ($i === $maximumInserts) {
$insert->execute($values);
$i = 0;
$values = array();
}
$values[] = "value1" . $j;
$values[] = "value2" . $j;
$i++;
});
if ($i > 0) {
$insertRemaining = Db::getBiggestInsertStatement($table, array('field1', 'field2'), $i, $maximumInserts);
$insertRemaining->execute($values);
}
I have tried to insert in a table with a single column 1000000 rows, and it's done within seconds, agains minutes that would take to insert them one by one.
The standard technique for speeding up bulk inserts in to use a prepared SQL statement inside a loop inside a transaction. That will make it pretty well optimal. After that you could try tweaking it in various ways, but you are probably wasting your time.
I have two arrays with anywhere from 1 to 5 set values. I want to insert these values into a table with two columns.
Here's my current query, given to me in another SO question:
INSERT INTO table_name (country, redirect)
VALUES ('$country[1]', '$redirect[1]'),
('$country[2]', '$redirect[2]'),
('$country[3]', '$redirect[3]'),
('$country[4]', '$redirect[4]'),
('$country[5]', '$redirect[5]')
ON DUPLICATE KEY UPDATE redirect=VALUES(redirect)
I'm a little concerned however with what happens if some of these array values aren't set, as I believe the above assumes there's 5 sets of values (10 values in total), which definitely isn't certain. If a value is null/0 does it automatically skip it?
Would something like this work better, would it be a lot more taxing on resources?
for($i = 0, $size = $sizeof($country); $i <= size; $i++) {
$query = "INSERT INTO table_name (country, redirect) VALUES ('$country[$i]', '$redirect[$i]) ON DUPLICATE KEY UPDATE redirect='$redirect[$i]'";
$result = mysql_query($query);
}
Questions highlighted in bold ;). Any answers would be very much appreciated :) :)!!
Do something like this:
$vals = array()
foreach($country as $key => $country_val) {
if (empty($country_val) || empty($redirect[$key])) {
continue;
}
$vals[] = "('" . mysql_real_escape_string($country_val) . "','" . mysql_real_escape_string($redirect[$key]) . "')";
}
$val_string = implode(',', $vals);
$sql = "INSERT INTO .... VALUES $val_string";
That'll built up the values section dynamically, skipping any that aren't set. Note, however, that there is a length limit to mysql query strings, set by the max_allowed_packet setting. If you're building a "huge" query, you'll have to split it into multiple smaller ones if it exceeds this limit.
If you are asking whether php will automatically skip inserting your values into the query if it is null or 0, the answer is no. Why dont you loop through the countries, if they have a matching redirect then include that portion of the insert statement.. something like this: (not tested, just showing an example). It's one query, all values. You can also incorporate some checking or default to null if they do not exist.
$query = "INSERT INTO table_name (country, redirect) VALUES ";
for($i = 0, $size = $sizeof($country); $i <= size; $i++) {
if(array_key_exists($i, $country && array_key_exists($i, $redirect)
if($i + 1 != $size){
$query .= "('".$country[$i]."', '".$redirect[$i]).",";
} else $query .= "('".$country[$i]."', '".$redirect[$i].")";
}
}
$query .= " ON DUPLICATE KEY UPDATE redirect=VALUES(redirect);"
$result = mysql_query($query);
EDIT:
Thank you so much for your answers, you really amaze me with so much wisdom :)
I am trying to relay on TuteC's code a bit changed, but can't figure how to make it work properly:
$valor = $_POST['valor'];
$post_vars = array('iphone3g1', 'iphone3g2', 'nome', 'iphone41', 'postal', 'apelido');
foreach($post_vars as $var) {
$$var = "'" . mysql_real_escape_string($_POST[$var]). "', ";
}
$sql = "INSERT INTO clientes (iphone3g1, iphone3g2, nome, iphone41, postal, apelido, valor) VALUES ($$var '$valor')";
$query= mysql_query($sql);
I know there's a bit of cheating on the code, i would need to use substring so the $$var wouldn't output a "," at the end where i need the values, instead i tried to insert a variable that is a value ($valor = $_POST['valor'];)
What is going wrong?
And for the others who tried to help me, thank you very much, i am learning so much with you here at stackoverflow.
I have a form with several field values, when trying to write a php file that reads those values it came out a mostruosity:
$codigounico= md5(uniqid(rand()));
$modelo=$_POST['selectName'];
$serial=$_POST['serial'];
$nif=$_POST['nif'];
$iphone3g1=$_POST['iphone3g1'];
$iphone3g2=$_POST['iphone3g2'];
$iphone3g3=$_POST['iphone3g3'];
$iphone3g4=$_POST['iphone3g4'];
$iphone3gs1=$_POST['iphone3gs1'];
$iphone3gs2=$_POST['iphone3gs2'];
$iphone3gs3=$_POST['iphone3gs3'];
$iphone3gs4=$_POST['iphone3gs4'];
$iphone41=$_POST['iphone41'];
$iphone42=$_POST['iphone42'];
$iphone43=$_POST['iphone43'];
$iphone44=$_POST['iphone44'];
$total=$_POST['total'];
$valor=$_POST['valor'];
$nome=$_POST['nome'];
$apelido=$_POST['apelido'];
$postal=$_POST['postal'];
$morada=$_POST['morada'];
$notas=$_POST['notas'];
$sql="INSERT INTO clientes (postal, morada, nome, apelido, name, serial, iphone3g1, iphone3g2, iphone3g3, iphone3g4, total, valor, iphone3gs1, iphone3gs2, iphone3gs3, iphone3gs4, iphone41, iphone42, iphone43, iphone44, nif, codigounico, Notas)VALUES('$postal', '$morada', '$nome', '$apelido', '$modelo', '$serial', '$iphone3g1', '$iphone3g2', '$iphone3g3', '$iphone3g4', '$total', '$valor', '$iphone3gs1', '$iphone3gs2', '$iphone3gs3', '$iphone3gs4', '$iphone41', '$iphone42', '$iphone43', '$iphone44', '$nif', '$codigounico', '$notas')";
$result=mysql_query($sql);
This is a very dificult code to maintain,
can I make my life easier?
To restrict which POST variables you "import", you can do something like:
$post_vars = array('iphone3g1', 'iphone3g2', '...');
foreach($post_vars as $var) {
$$var = mysql_real_escape_string($_POST[$var]);
}
EDIT: Changed addslashes by mysql_real_escape_string (thanks #Czechnology).
The issue I see is repetition of the same names four times over. This is how I would reduce it to two occurrences (you could drop it to one with more finagling).
$sql = 'INSERT INTO clientes (postal, morada, nome, apelido, name, serial, iphone3g1, iphone3g2, iphone3g3, iphone3g4, total, valor, iphone3gs1, iphone3gs2, iphone3gs3, iphone3gs4, iphone41, iphone42, iphone43, iphone44, nif, codigounico, Notas) VALUES(:postal, :morada, :nome, :apelido, :modelo, :serial, :iphone3g1, :iphone3g2, :iphone3g3, :iphone3g4, :total, :valor, :iphone3gs1, :iphone3gs2, :iphone3gs3, :iphone3gs4, :iphone41, :iphone42, :iphone43, :iphone44, :nif, :codigounico, :notas)';
preg_match_all('/:(\w+)/', $sql, $inputKeys);
$tokens = $inputKeys[0];
$values = array_map($inputKeys[1], function($k){
return mysql_real_escape_string($_POST[$k]);
});
$sql = str_replace($tokens, $values, $sql);
$result = mysql_query($sql);
Depending on how you want to separate your logic, a reversed approach might be more useful, where you would specify the array of key names and iterate over that to generate the SQL string.
<?php
$inputKeys = array('postal', 'morada', 'nome', 'apelido', 'name', 'serial', 'iphone3g1', 'iphone3g2', 'iphone3g3', 'iphone3g4', 'total', 'valor', 'iphone3gs1', 'iphone3gs2', 'iphone3gs3', 'iphone3gs4', 'iphone41', 'iphone42', 'iphone43', 'iphone44', 'nif', 'codigounico', 'Notas');
$keyList = '(' . implode(',', $inputKeys) . ')';
$valueList = 'VALUES (';
foreach ($inputKeys as $k) {
$valueList .= mysql_real_escape_string($_POST[$k]);
$valueList .= ',';
}
$valueList = rtrim($valueList, ',');
$valueList .= ')';
$sql = 'INSERT INTO clientes '.$keyList.' '.$valueList;
$result = mysql_query($sql);
This approach drops the occurrences of the keys to one and will probably more naturally with your application.
TuteC had a good aim but failed in details.
It makes me wonder, why noone has a ready made solution, but had to devise it on-the-fly. Nobody faced the same problem before?
And why most people trying to solve only part of the problem, getting variables only.
The goal is not to get variables.
The goal is to get a query. So, get yourself a query.
//quite handy way to define an array, saves you from typing zillion quotes
$fields = explode(" ","postal morada nome apelido name serial iphone3g1 iphone3g2 iphone3g3 iphone3g4 total valor iphone3gs1 iphone3gs2 iphone3gs3 iphone3gs4 iphone41 iphone42 iphone43 iphone44 nif codigounico Notas");
$sql = "INSERT INTO clientes SET ";
foreach ($fields as $field) {
if (isset($_POST[$field])) {
$sql.= "`$field`='".mysql_real_escape_string($_POST[$field])."', ";
}
}
$sql = substr($set, 0, -2);
This code will create you a query without boring repeating the same field name many times.
But that's still not all improvements you can make.
A really neat thing is called a function.
function dbSet($fields) {
$set = '';
foreach ($fields as $field) {
if (isset($_POST[$field])) {
$set.="`$field`='".mysql_real_escape_string($_POST[$field])."', ";
}
}
return substr($set, 0, -2);
}
put this function into your code library being included into all your scripts (you have one, don't you?)
and then use it for both insert and update queries:
$_POST['codigounico'] = md5(uniqid(rand()));//a little hack to add custom field(s)
if ($action=="update") {
$id = intval($_POST['id']);
$sql = "UPDATE $table SET ".dbSet($fields)." WHERE id = $id";
}
if ($action=="insert") {
$sql = "INSERT $table SET ".dbSet($fields);
}
So, your code become extremely short and reliable and even reusable.
The only thing you have to change to handle another table is $fields array.
It seems your database is not well planned as it contains seemingly repetitive fields (iphone*). You have to normalize your database.
The same approach to use with prepared statements can be found in this my question: Insert/update helper function using PDO
You could use a rather ugly part of PHP called variable variables, but it is generally considered a poor coding practice. You could include your database escaping at the same time. The code would look something like:
foreach($_POST as $key => $value){
$$key = mysql_real_escape_string($value);
}
The variable variables manual section says they do not work with superglobals like $_PATH, but I think it may work in this case. I am not somewhere where I can test right now.
PHP: extract
Be careful though and make sure you clean the data before using it.
$set = array();
$keys = array('forename', 'surname', 'email');
foreach($keys as $val) {
$safe_value = mysqli_escape_string($db, $_POST[$val]);
array_push($set, "$val='$safe_value'");
}
$set_query = implode(',', $set);
Then make your MySQL query something like UPDATE table SET $set_query WHERE... or INSERT INTO table SET $set_query.
If you need to validate, trim, etc, do it before the above code like this:
$_POST["surname"] = trim($_POST["surname"];
Actually, you could make your life easier by making your code a bit more complicated - escape the input before inserting into the database!
$sql =
"INSERT INTO clientes SET
"postal = '" . mysql_real_escape_string($_POST['postal']) . "', ".
"morada = '" . mysql_real_escape_string($_POST['morada']) . "', ".
...
First, I recommend you to create a key-value array like this:
$newClient = array(
'codigounico' => md5(uniqid(rand())),
'postal' => $_POST['postal'],
'modelo' => $_POST['selectName'],
...
);
In this array key is the column name your MySQL table.
In the code you've provided not every field is copied right from POST array (some are calculated, and some keys of the POST aren't equal with the tables column names), so you should use a flexible method.
You should still specify all columns and values but only once so code is still maintainable and you won't have any security errors if someone sends you a broken POST. As for me it looks more like configuration than coding.
Then I recommend you to write a function similar to this:
function buildInsertQuery($tableName, $keyValue) {
$result = '';
if (!empty($keyValue)) {
$delimiter = ', ';
$columns = '';
$values = '';
foreach ($keyValue as $key => $value) {
$columns .= $key . $delimiter;
$values .= mysql_real_escape_string($value) . $delimiter;
}
$columns = substr($columns, 0, -length($delimiter));
$values = substr($values, 0, -length($delimiter));
$result = 'INSERT INTO `' . $tableName . '` (' . $columns . ') VALUES (' . $values . ')';
}
return $result;
}
And then you can simply build your query with just one function call:
$query = buildInsertQuery('clientes', $newClient);