I'm new to PHP. I'm trying to split up my array (done successfully) and submit them as individual submisions into my MYSQL Table (via FOR loops). However, using the code below, it only submits the LAST object in the array. And thus defeats the whole purpose of the desired function. Can anybody show me a way to avoid this?
$directives = "(John Doe, Directive, Time)(Jane Doe, Directive2, Time2)";
$action = explode("(" ,str_replace (")","",$directives));
for ($i = 0; $i < count($action); ++$i){
$newdata = explode("," , $action[$i]);
$name = $newdata[0];
$namesplit = explode(" " , $name);
$actiondo = $newdata[1];
$date = $newdata[2];
$sqlaction="INSERT INTO TABLE (Firstname, lastname, Description, Action, date)
VALUES
('$namesplit[0]','$namesplit[1]',' Directive','$actiondo','$date')";
}
You are overwriting your query in $sqlaction with each loop iteration, thus this is not useful.
You also don't need a query for each row. You can build up a query that would allow insert of all rows with a single query. Something like this:
$query = 'INSERT INTO TABLE (Firstname, lastname, Description, Action, date)
VALUES ';
for ($i = 0; $i < count($action); ++$i){
$newdata = explode("," , $action[$i]);
$name = $newdata[0];
$namesplit = explode(" " , $name);
$actiondo = $newdata[1];
$date = $newdata[2];
$query .= "('" . $namesplit[0] . "','" . $namesplit[1] . "'Directive','" . $actiondo . "','" . $date . "'),";
}
$query = rtrim(',', $query);
// not shown - execute query using $query
First, you would have to put the code that actually puts stuff in the database inside the loop in order to do lots of individual submissions. All your code currently does is just set the same string over and over. That's why only the last one gets inserted.
Second, why don't you just do one insert? You can have multiple values parentheses, you know. Like this:
VALUES (x, y, z), (x2, y2, z2), (x3, y3, z3)
Third, your design is just asking for trouble because you're making so many assumptions about your data. Imagine that there's a person like "James Earl Jones" in your list. Here's how it would go:
$namesplit = explode(" ", 'James Earl Jones');
echo $namesplit[0]; // = James
echo $namesplit[1]; // = Earl
Oops, now his name is just "James Earl."
To avoid this kind of thing, you have to check the data carefully before inserting it, and you really need to think about what might happen if there are extra commas or spaces every time you do those explodes.
Related
So the problem is, when uploading to mysql the database is only uploading the last value in the array.
if (substr($avsnitt["serier"], 0, 16) === 'http://random'){
// Create DOM from URL or file
$html = file_get_html($avsnitt["serier"]);
$array_title = array();
$array_link = array();
foreach($html->find('div[class=entry]') as $element){
foreach ($element->find('a') as $text) {
$array_title[] = $text->plaintext;
}
foreach ($element->find('a') as $test) {
$array_link[] = $test->href;
}
$count_name = count($array_title);
for($i=0; $i<$count_name; $i++){
$_array_title = mysql_escape_string($array_title[$i]);
$_array_link = mysql_escape_string($array_link [$i]);
print_r($_array_title);
print_r($_array_link);
$sql2 = "INSERT INTO episodes (name, ID, link) VALUES ('" . #$_array_title. "','" . #$avsnitt["ID"] . "', '" . #$_array_link . "');";
mysqli_query($CON, $sql2);
}
}
}
i'm new to php mysql so i dont relly know how arrays is uploaded to mysql, fast answers will be appreciated thanks.
I think, your database-field "link" is type string.
You want to insert your variable $_array_link, witch is type array.
So try serialize($_array_link) to "convert" array to string.
I think, the # are not nessessary, without it is better to read.
$sql2 = "INSERT INTO episodes (name, ID, link)
VALUES ('" . #$_array_title. "','" . #$avsnitt["ID"] . "', '" . serialize($_array_link). "');";
Later, you want to use the variable from the database. So after you SELECT your data, use unserialize() to get the array back.
problem was the database ID was primery key only allowing one withe a specific ID
I am looking for the fastest way to INSERT 1-3,000 rows into a MySQL database using PHP. My current solution is taking around 42 seconds to insert the rows which I think that could be much faster.
I am using a self-written DB class, the insert() method takes two params (string) $table and (array) $vars. The $items array is an associative array where the key is the column name in the table and the value is the value to insert. This works really well for because I sometimes have 30 columns in a table and already have the data there in an array. The insert() method is below:
function insert($table,$vars) {
if(empty($this->sql_link)){
$this->connection();
}
$cols = array();
$vals = array();
foreach($vars as $key => $value) {
$cols[] = "`" . $key . "`";
$vals[] = "'" . $this->esc($value) . "'";
}
//join the columns and values to insert into sql
$fields = join(', ', $cols);
$values = join(', ', $vals);
$insert = mysql_query("INSERT INTO `$table` ($fields) VALUES ($values);", $this->sql_link);
return $insert;
}
It should be self-explanatory but basically I take the keys and values from $vars and create an INSERT statement. It works, I think the problem I am having is sending the queries one at a time.
Should I build a long query string?
INSERT INTO table (field, field2, etc) VALUES (1, 2, ect);INSERT INTO table (field, field2, etc) VALUES (1, 2, ect);INSERT INTO table (field, field2, etc) VALUES (1, 2, ect);INSERT INTO table (field, field2, etc) VALUES (1, 2, ect);INSERT INTO table (field, field2, etc) VALUES (1, 2, ect); and send it all at one time? If so can this handle 3,000 insert statements in one call?
Is there another way I am not looking at? Any info is appreciated.
Thanks
The most performant way is to use the multiple-row insert syntax:
INSERT INTO table (field, field2, etc) VALUES (1, 2, etc),(1, 2, etc),(1, 2, etc);
Manual:
INSERT statements that use VALUES syntax can insert multiple rows. To do this, include multiple lists of column values, each enclosed within parentheses and separated by commas. Example:
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
The values list for each row must be enclosed within parentheses.
Two ways of improve insertion speeds:
At the start, before any INSERT, do a mysql_query("START TRANSACTION"); or a simpler mysql_query("BEGIN");. At the end, do a mysql_query("COMMIT");. These two lines, speeds up the bulk insertion a 5-10x performance.
If the table backend is MyISAM (NOT InnoDB), do the INSERTs followed with the word DELAYED. For example, instead of INSERT INTO table use INSERT DELAYED INTO table for an aditional 10-15x speed-up.
If you combine the 2 methods, is posible to achieve a speed-up of 100 times.
Mysql can import data directly from a file which can significantly speed up importing data. See:
LOAD DATA INFILE Syntax
<?php
$data = "data/fullz.txt";
$db = new PDO("sqlite:db/ssninfo.db");
$db->beginTransaction();
$stmt = $db->prepare('INSERT INTO ssninfo (fname,lname,ssn,address,city,state,zip,phone,birth,email) VALUES (?,?,?,?,?,?,?,?,?,?)');
if($file=fopen($data, "r")){
while(!feof($file)){
$line = fgets($file);
$part = explode('|', $line);
$stmt->execute($part);
}
}
$db->commit();
As usual, it depends; you don't even mention which engine you're using, which is a big determinant. But I've found the MySQL manual guidance pretty reliable.
http://dev.mysql.com/doc/refman/5.0/en/insert-speed.html
Auto discovering the maximum ammount of inserts.
to insert that kind of ammounts (3000) there should not be any problem of doing something like (assuming you use pdo):
$stmt = $dbh->prepare("INSERT INTO yourtable(name, id) VALUES " . str_repeat('(?,?),', $amountOfRows - 1) . '(?,?)');
You can improve that to make create generic way to create big statements like the one above for tables with different ammount of fields:
$fields = array("name", "id");
$fieldList = implode(", ", $fields);
$params = '(' . str_repeat('?,', count($fields) - 1) . '?)';
$values = str_repeat($params . ',', $ammountOfRows - 1) . $params;
$stmt = $dbh->prepare("INSERT INTO $table($fieldList) VALUES " . $values);
but the problem with the above solution is that wont work with any combination of rows and ammount of fields.
Seems to be that mysql is not only limited by the ammount of rows but also the ammount of parameters is taken into account.
But you dont want to be changing your code whenever a new mysql release changes the limit of the parameters, rows or even the size of the sql sentence.
So, a much better approach to create a generic way to generate big statements would be trying to feat the underlaying database engine:
/**
* Creates an insert sql with the maximum allowed of parameters
* #param string $table
* #param string $attributeList
* #param int &$ammountInserts returns the ammount of inserts
* #return \PDOStatement
*/
public static function getBiggestInsertStatement($table, $attributeList, $max, &$ammountInserts)
{
$previousSize = null;
$size = 10;
$sql = 'INSERT INTO ' . $table . '(' . implode(',', $attributeList) . ') values ';
$return = null;
$params = '(' . str_repeat('?,', count($attributeList) - 1) . '?)';
do {
try {
$previousSize = $size;
$values = str_repeat($params . ',', $size - 1) . $params;
$return = Db::getInstance()->prepare($sql . $values);
if ($size > $max) {
$values = str_repeat($params . ',', $max - 1) . $params;
$return = Db::getInstance()->prepare($sql . $values);
$ammountInserts = $max;
break;
}
$ammountInserts = $size;
$size *= 2;
} catch(\Exception $e) {
}
} while($previousSize != $size);
return $return;
}
One thing that you must have in mind is that since you dont know that limits the query could be able to push a lower ammount of items that all that you need to insert.
So you would have to create a strategy like the one below to succesfuly achieve insert them all in any possible scenario:
$insert = Db::getBiggestInsertStatement($table, array('field1','field2'), $numrows, $maximumInserts);
$i = 0;
$values = array();
for ($j = 0; $j < $numrows; $j++) {
if ($i === $maximumInserts) {
$insert->execute($values);
$i = 0;
$values = array();
}
$values[] = "value1" . $j;
$values[] = "value2" . $j;
$i++;
});
if ($i > 0) {
$insertRemaining = Db::getBiggestInsertStatement($table, array('field1', 'field2'), $i, $maximumInserts);
$insertRemaining->execute($values);
}
I have tried to insert in a table with a single column 1000000 rows, and it's done within seconds, agains minutes that would take to insert them one by one.
The standard technique for speeding up bulk inserts in to use a prepared SQL statement inside a loop inside a transaction. That will make it pretty well optimal. After that you could try tweaking it in various ways, but you are probably wasting your time.
I have two arrays with anywhere from 1 to 5 set values. I want to insert these values into a table with two columns.
Here's my current query, given to me in another SO question:
INSERT INTO table_name (country, redirect)
VALUES ('$country[1]', '$redirect[1]'),
('$country[2]', '$redirect[2]'),
('$country[3]', '$redirect[3]'),
('$country[4]', '$redirect[4]'),
('$country[5]', '$redirect[5]')
ON DUPLICATE KEY UPDATE redirect=VALUES(redirect)
I'm a little concerned however with what happens if some of these array values aren't set, as I believe the above assumes there's 5 sets of values (10 values in total), which definitely isn't certain. If a value is null/0 does it automatically skip it?
Would something like this work better, would it be a lot more taxing on resources?
for($i = 0, $size = $sizeof($country); $i <= size; $i++) {
$query = "INSERT INTO table_name (country, redirect) VALUES ('$country[$i]', '$redirect[$i]) ON DUPLICATE KEY UPDATE redirect='$redirect[$i]'";
$result = mysql_query($query);
}
Questions highlighted in bold ;). Any answers would be very much appreciated :) :)!!
Do something like this:
$vals = array()
foreach($country as $key => $country_val) {
if (empty($country_val) || empty($redirect[$key])) {
continue;
}
$vals[] = "('" . mysql_real_escape_string($country_val) . "','" . mysql_real_escape_string($redirect[$key]) . "')";
}
$val_string = implode(',', $vals);
$sql = "INSERT INTO .... VALUES $val_string";
That'll built up the values section dynamically, skipping any that aren't set. Note, however, that there is a length limit to mysql query strings, set by the max_allowed_packet setting. If you're building a "huge" query, you'll have to split it into multiple smaller ones if it exceeds this limit.
If you are asking whether php will automatically skip inserting your values into the query if it is null or 0, the answer is no. Why dont you loop through the countries, if they have a matching redirect then include that portion of the insert statement.. something like this: (not tested, just showing an example). It's one query, all values. You can also incorporate some checking or default to null if they do not exist.
$query = "INSERT INTO table_name (country, redirect) VALUES ";
for($i = 0, $size = $sizeof($country); $i <= size; $i++) {
if(array_key_exists($i, $country && array_key_exists($i, $redirect)
if($i + 1 != $size){
$query .= "('".$country[$i]."', '".$redirect[$i]).",";
} else $query .= "('".$country[$i]."', '".$redirect[$i].")";
}
}
$query .= " ON DUPLICATE KEY UPDATE redirect=VALUES(redirect);"
$result = mysql_query($query);
I have a form for registering for weekly summer camps. There are checkboxes to select which camp the person is signing up for that look like this:
<div class="pscdiv"><input type="checkbox" class="psc" name="camps[]" value="psc_1"/><label for="psc_1">AM - Week 1: Camp Description</label></div>
There are about 30 of them total. What I'm trying to do is take the $_POST['camps'] variable on the next page and break it into something I can insert into a MySQL table that has a structure like this:
regid | psc_1 | psc_2 | psc_3 | ...
My code:
if(!empty($_POST['camps'])) {
$boxes=$_POST['camps'];
while (list ($k,$camp) = #each ($boxes)) {
$camp_string .= "'$camp',";
}
}
// The above to create a comma separated string
$camp_string = (substr($camp_string,-1) == ',') ? substr($camp_string, 0, -1) : $camp_string;
// To remove the trailing comma
$newreg_camps = mysql_query("INSERT INTO camps_registered (regid,$camp_string) VALUES ($regid,$camp_string)");
The column names (other than regid which I specify earlier in the script) are the same as the data being put into them. It was the easiest thing I could think of at the time.
I'm not sure what I'm missing.
--
Update#1 (in reference to a comment below)
if(!empty($_POST['camps'])) {
$box=$_POST['camps'];
while (list ($key,$val) = #each ($box)) {
$camp_totals += $campcost[$val];
echo "$campdesc[$val] - $$campcost[$val]";
}
}
Why not name the checkboxes something different like the names of the columns, so that your $_POST comes back as:
// print_r($_POST);
array(
'regID' => 1,
'camp_filmore' => 1,
'camp_greenwald' => 1
'camp_idunno' => 1
);
From there it's fairly simple to build your query:
$query = "INSERT INTO registered (".implode(array_keys($_POST), ", ") . ") ".
"VALUES (" . implode(array_values($_POST), ", ").")";
You should obviously check to make sure the values of $_POST are properly escaped and sanitized before inserting.
A nice trick to remove the trailing comma would be.
$camp_string = rtrim($camp_string,',');
Anyway your query is not formated correctly.
INSERT INTO camps_registered (regid,'psc_1', 'psc_2', 'psc_3') VALUES ($regid,psc_1', 'psc_2', 'psc_3')
You cannot have single quotes in the first bracket
I think you should be looking at implode and you can easily add addition strings to it...
so e.g.
$comma_seperated = implode(',', $boxes);
$subscribed = $regid . ',' . $comma_seperated;
EDIT:
Thank you so much for your answers, you really amaze me with so much wisdom :)
I am trying to relay on TuteC's code a bit changed, but can't figure how to make it work properly:
$valor = $_POST['valor'];
$post_vars = array('iphone3g1', 'iphone3g2', 'nome', 'iphone41', 'postal', 'apelido');
foreach($post_vars as $var) {
$$var = "'" . mysql_real_escape_string($_POST[$var]). "', ";
}
$sql = "INSERT INTO clientes (iphone3g1, iphone3g2, nome, iphone41, postal, apelido, valor) VALUES ($$var '$valor')";
$query= mysql_query($sql);
I know there's a bit of cheating on the code, i would need to use substring so the $$var wouldn't output a "," at the end where i need the values, instead i tried to insert a variable that is a value ($valor = $_POST['valor'];)
What is going wrong?
And for the others who tried to help me, thank you very much, i am learning so much with you here at stackoverflow.
I have a form with several field values, when trying to write a php file that reads those values it came out a mostruosity:
$codigounico= md5(uniqid(rand()));
$modelo=$_POST['selectName'];
$serial=$_POST['serial'];
$nif=$_POST['nif'];
$iphone3g1=$_POST['iphone3g1'];
$iphone3g2=$_POST['iphone3g2'];
$iphone3g3=$_POST['iphone3g3'];
$iphone3g4=$_POST['iphone3g4'];
$iphone3gs1=$_POST['iphone3gs1'];
$iphone3gs2=$_POST['iphone3gs2'];
$iphone3gs3=$_POST['iphone3gs3'];
$iphone3gs4=$_POST['iphone3gs4'];
$iphone41=$_POST['iphone41'];
$iphone42=$_POST['iphone42'];
$iphone43=$_POST['iphone43'];
$iphone44=$_POST['iphone44'];
$total=$_POST['total'];
$valor=$_POST['valor'];
$nome=$_POST['nome'];
$apelido=$_POST['apelido'];
$postal=$_POST['postal'];
$morada=$_POST['morada'];
$notas=$_POST['notas'];
$sql="INSERT INTO clientes (postal, morada, nome, apelido, name, serial, iphone3g1, iphone3g2, iphone3g3, iphone3g4, total, valor, iphone3gs1, iphone3gs2, iphone3gs3, iphone3gs4, iphone41, iphone42, iphone43, iphone44, nif, codigounico, Notas)VALUES('$postal', '$morada', '$nome', '$apelido', '$modelo', '$serial', '$iphone3g1', '$iphone3g2', '$iphone3g3', '$iphone3g4', '$total', '$valor', '$iphone3gs1', '$iphone3gs2', '$iphone3gs3', '$iphone3gs4', '$iphone41', '$iphone42', '$iphone43', '$iphone44', '$nif', '$codigounico', '$notas')";
$result=mysql_query($sql);
This is a very dificult code to maintain,
can I make my life easier?
To restrict which POST variables you "import", you can do something like:
$post_vars = array('iphone3g1', 'iphone3g2', '...');
foreach($post_vars as $var) {
$$var = mysql_real_escape_string($_POST[$var]);
}
EDIT: Changed addslashes by mysql_real_escape_string (thanks #Czechnology).
The issue I see is repetition of the same names four times over. This is how I would reduce it to two occurrences (you could drop it to one with more finagling).
$sql = 'INSERT INTO clientes (postal, morada, nome, apelido, name, serial, iphone3g1, iphone3g2, iphone3g3, iphone3g4, total, valor, iphone3gs1, iphone3gs2, iphone3gs3, iphone3gs4, iphone41, iphone42, iphone43, iphone44, nif, codigounico, Notas) VALUES(:postal, :morada, :nome, :apelido, :modelo, :serial, :iphone3g1, :iphone3g2, :iphone3g3, :iphone3g4, :total, :valor, :iphone3gs1, :iphone3gs2, :iphone3gs3, :iphone3gs4, :iphone41, :iphone42, :iphone43, :iphone44, :nif, :codigounico, :notas)';
preg_match_all('/:(\w+)/', $sql, $inputKeys);
$tokens = $inputKeys[0];
$values = array_map($inputKeys[1], function($k){
return mysql_real_escape_string($_POST[$k]);
});
$sql = str_replace($tokens, $values, $sql);
$result = mysql_query($sql);
Depending on how you want to separate your logic, a reversed approach might be more useful, where you would specify the array of key names and iterate over that to generate the SQL string.
<?php
$inputKeys = array('postal', 'morada', 'nome', 'apelido', 'name', 'serial', 'iphone3g1', 'iphone3g2', 'iphone3g3', 'iphone3g4', 'total', 'valor', 'iphone3gs1', 'iphone3gs2', 'iphone3gs3', 'iphone3gs4', 'iphone41', 'iphone42', 'iphone43', 'iphone44', 'nif', 'codigounico', 'Notas');
$keyList = '(' . implode(',', $inputKeys) . ')';
$valueList = 'VALUES (';
foreach ($inputKeys as $k) {
$valueList .= mysql_real_escape_string($_POST[$k]);
$valueList .= ',';
}
$valueList = rtrim($valueList, ',');
$valueList .= ')';
$sql = 'INSERT INTO clientes '.$keyList.' '.$valueList;
$result = mysql_query($sql);
This approach drops the occurrences of the keys to one and will probably more naturally with your application.
TuteC had a good aim but failed in details.
It makes me wonder, why noone has a ready made solution, but had to devise it on-the-fly. Nobody faced the same problem before?
And why most people trying to solve only part of the problem, getting variables only.
The goal is not to get variables.
The goal is to get a query. So, get yourself a query.
//quite handy way to define an array, saves you from typing zillion quotes
$fields = explode(" ","postal morada nome apelido name serial iphone3g1 iphone3g2 iphone3g3 iphone3g4 total valor iphone3gs1 iphone3gs2 iphone3gs3 iphone3gs4 iphone41 iphone42 iphone43 iphone44 nif codigounico Notas");
$sql = "INSERT INTO clientes SET ";
foreach ($fields as $field) {
if (isset($_POST[$field])) {
$sql.= "`$field`='".mysql_real_escape_string($_POST[$field])."', ";
}
}
$sql = substr($set, 0, -2);
This code will create you a query without boring repeating the same field name many times.
But that's still not all improvements you can make.
A really neat thing is called a function.
function dbSet($fields) {
$set = '';
foreach ($fields as $field) {
if (isset($_POST[$field])) {
$set.="`$field`='".mysql_real_escape_string($_POST[$field])."', ";
}
}
return substr($set, 0, -2);
}
put this function into your code library being included into all your scripts (you have one, don't you?)
and then use it for both insert and update queries:
$_POST['codigounico'] = md5(uniqid(rand()));//a little hack to add custom field(s)
if ($action=="update") {
$id = intval($_POST['id']);
$sql = "UPDATE $table SET ".dbSet($fields)." WHERE id = $id";
}
if ($action=="insert") {
$sql = "INSERT $table SET ".dbSet($fields);
}
So, your code become extremely short and reliable and even reusable.
The only thing you have to change to handle another table is $fields array.
It seems your database is not well planned as it contains seemingly repetitive fields (iphone*). You have to normalize your database.
The same approach to use with prepared statements can be found in this my question: Insert/update helper function using PDO
You could use a rather ugly part of PHP called variable variables, but it is generally considered a poor coding practice. You could include your database escaping at the same time. The code would look something like:
foreach($_POST as $key => $value){
$$key = mysql_real_escape_string($value);
}
The variable variables manual section says they do not work with superglobals like $_PATH, but I think it may work in this case. I am not somewhere where I can test right now.
PHP: extract
Be careful though and make sure you clean the data before using it.
$set = array();
$keys = array('forename', 'surname', 'email');
foreach($keys as $val) {
$safe_value = mysqli_escape_string($db, $_POST[$val]);
array_push($set, "$val='$safe_value'");
}
$set_query = implode(',', $set);
Then make your MySQL query something like UPDATE table SET $set_query WHERE... or INSERT INTO table SET $set_query.
If you need to validate, trim, etc, do it before the above code like this:
$_POST["surname"] = trim($_POST["surname"];
Actually, you could make your life easier by making your code a bit more complicated - escape the input before inserting into the database!
$sql =
"INSERT INTO clientes SET
"postal = '" . mysql_real_escape_string($_POST['postal']) . "', ".
"morada = '" . mysql_real_escape_string($_POST['morada']) . "', ".
...
First, I recommend you to create a key-value array like this:
$newClient = array(
'codigounico' => md5(uniqid(rand())),
'postal' => $_POST['postal'],
'modelo' => $_POST['selectName'],
...
);
In this array key is the column name your MySQL table.
In the code you've provided not every field is copied right from POST array (some are calculated, and some keys of the POST aren't equal with the tables column names), so you should use a flexible method.
You should still specify all columns and values but only once so code is still maintainable and you won't have any security errors if someone sends you a broken POST. As for me it looks more like configuration than coding.
Then I recommend you to write a function similar to this:
function buildInsertQuery($tableName, $keyValue) {
$result = '';
if (!empty($keyValue)) {
$delimiter = ', ';
$columns = '';
$values = '';
foreach ($keyValue as $key => $value) {
$columns .= $key . $delimiter;
$values .= mysql_real_escape_string($value) . $delimiter;
}
$columns = substr($columns, 0, -length($delimiter));
$values = substr($values, 0, -length($delimiter));
$result = 'INSERT INTO `' . $tableName . '` (' . $columns . ') VALUES (' . $values . ')';
}
return $result;
}
And then you can simply build your query with just one function call:
$query = buildInsertQuery('clientes', $newClient);