I'm inserting multiple rows in a table, and I get this message:
MySQL server has gone away
My Query:
INSERT INTO table
(a,b,c,d,e,f,g,h,i,j,k)
VALUES(1,2,3,4,5,6,7,8,9,10,11),(1,2,3,4,5,6,7,8,9,10,11), ...
ON DUPLICATE KEY UPDATE
c=VALUES(c),
d=VALUES(d),
e=VALUES(e),
f=VALUES(f),
g=VALUES(g),
h=VALUES(h),
i=VALUES(i),
j=VALUES(j)
Is it because I stuffed too many values inside a single query? (There are like 5000 pairs of values from a array which I implode with ,).
If this is the reason - then should I insert each row one by one? Is it slower than inserting them all at once?
The PHP code:
foreach($data as &$entry)
$entry = "('".implode("','", array(
$entry->ID,
addslashes($entry->field_1),
addslashes($entry->field_2),
...
))."')";
$data = implode(',', $data);
$query = "... VALUES{$data} ON ..."
$data is a array of STD type objects...
edit again :)
So I tried splitting my $data into smaller arrays of 100 elements each:
$data_chunks = array_chunk($data, 100);
foreach($data_chunks as $data_chunk)
insert_into_db($data_chunk);
and it works, I don't get that error anymore...
So that means the issue was the very long query string...
Now I'm even more confused:
Is there a length limit of the query, or maybe PHP arguments in general?
Is there any difference between inserting row by row than inserting multiple rows? Is it worth the array_chunk() ?
it could be that your query is taking too long to complete, mysql times out and closes the connection. You can alter the system variables to wait longer.
http://dev.mysql.com/doc/refman/5.0/en/gone-away.html
I think your problem is with *max_allowed_packet*, although the error seems to point in different direction. Try doing as suggested here: http://dev.mysql.com/doc/refman/5.5/en/packet-too-large.html
Or, before making any changes to mysql configuration, simply strlen() your query and find out how long(in bytes) it actually is.
Related
I've restructuring my code to better fit the application and the one problem I can't seem to wrap my head around is deleting multiple rows with an array I'm providing to the back-end.
The query I'm using is as follows
$items = json_decode(file_get_contents('php://input'))
//the item is passed as an object {"values": ["string1", "string2", "string3"]}
$values = implode(",", $items->values);
$sql = "DELETE FROM products WHERE value IN($values)";
Before the changes this query was used to delete with ids instead of looking up specific values in rows.
$ids = json_decode(file_get_contents('php://input'))
$values= implode(",", $ids->ids);
$sql = "DELETE FROM products WHERE id IN($values)";
This worked without a hitch, but trying to run the new query keeps throwing the error
unknown column in where clause
I've tried multiple different approaches but can't get this to work.
help would be much appreciated.
I am trying to figure out how to delete all ids in the database that do not exist in an array. I have been trying to use NOT IN in my query but I am not sure why it wont work when running it in a script the same way it works when I manually enter it into mysql. Here is an example.
mysqli_query($con, "DELETE FROM table WHERE id NOT IN ($array)");
$array is a list of ids from a json api. I use CURL to fetch the ids and I am trying to delete all ids in the database that do not match the ids in $array.
First I use another simple CURL script to scrape the apis and insert the ids found into the database and what I am trying to do here is basically make a link/data checker.
If the ids in the database are not found in the array when rechecking them then I want them deleted.
I thought that the query above would work perfect but for some reason it doesn't. When the query is ran from a script the mysql log shows the queries being ran as this.
Example:
DELETE FROM table WHERE id NOT IN ('166')
or this when I am testing multiple values.
DELETE FROM table WHERE id NOT IN ('166', '253', '3324')
And what happens is it deletes every row in the table every time. I don't really understand because if I copy/paste the same query from the log and run it manually myself it works perfect.
I have been trying various ways of capturing the array data such as array_column, array_map, array_search and various functions I have found but the end result is always the same.
For right now, just for testing I am using these 2 bits of code for testing 2 different apis which gives me the same sql query log output as above. The functions used are just a couple random ones that I found.
//$result is the result from CURL using json_decode
function implode_r($g, $p) {
return is_array($p) ?
implode($g, array_map(__FUNCTION__, array_fill(0, count($p), $g), $p)) :
$p;
}
foreach ($result['data'] as $info){
$ids = implode_r(',', $info['id']);
mysqli_query($con, "DELETE FROM table WHERE id NOT IN ($ids)");
}
And
$arrayLength = count($result);
for($i = 0; $i < $arrayLength; $i++) {
mysqli_query($con, "DELETE FROM table WHERE id NOT IN ('".$result[$i]['id']."')");
}
If anyone knows what is going on i'd appretiate the help or any suggestions on how to achieve the same result. I am using php 7 and mysql 5.7 with innodb tables if that helps.
It probably doesn't work because your IN value is something like this
IN('1,2,3,4');
When what you want is this
IN('1','2','3','4')
OR
IN( 1,2,3,4)
To get this with implode include the quotes like this
$in = "'".implode("','", $array)."'";
NOTE whenever directly inputting variables into SQL there is security Implications to consider such as SQLInjection. if the ID's are from a canned source you're probably ok, but I like to sanitize them anyway.
You can't mix array and string.
This works:
mysqli_query($con, "DELETE FROM table WHERE id NOT IN (".implode(',', $ids).")");
i'm relatively new to coding and I need a little help. I'm basically trying to loop through an entry in a mySQL database and push any new entry into an array , so that it only comes up once in my array.
// SQL query
$response = $bdd->query('SELECT serie_bd FROM inventaire_bd');
//creating array to group all elements from the db so that they do not repeat
$serie_bd_groupe=array();
while($data_collected = $response->fetch())
{
if(array_key_exists($data_collected,$serie_bd_groupe)==false)
{
array_push($data_collected,$serie_bd_groupe);
}
}
Will this work? - it seems like the loop will just stay stuck after it comes accross an entry a second time because the if statement wont execute itself.
Also in the future, are their any php equivalent to jsfiddle.net so i can test code syntaxically?
Thank you for your time
Your array keys will be default integers, so you don't want to check those. Instead of this:
if(array_key_exists($data_collected,$serie_bd_groupe)==false)
you should do this:
if(!(in_array($data_collected,$serie_bd_groupe)))
http://php.net/manual/en/function.in-array.php
On the other hand, if you're expecting your collected data to be the array key rather than value, you'd do something like this, instead of your array_push:
$serie_bd_groupe[$data_collected] = 1;
then your key check would work.
If you are looking for UNIQUE values (serie_bd) from your database, update your query to include "DISTINCT" like this:
$bdd->query('SELECT DISTINCT serie_bd FROM inventaire_bd');
On the other hand, I think you are looking for http://phpfiddle.org/
I am developing an application in PHP for which I need to implement a big file handler.
Reading and writing the file is not a problem, but checking the content of the file is a problem.
I built a recursive function which checks whether or not a variable is already used in the same document.
private function val_id($id){
if(!isset($this->id)){
$this->id = array();
}
if(in_array($id, $this->id)){
return $this->val_id($id+1);
}else{
$this->id[] = $id;
return $id;
}
}
When in_array($id,$this->id) returns FALSE, the $id will be added to $this->id (array which contains all used ids) and returns a valid id.
When this returns TRUE, it returns the same function with parameter $id+1
Since we are talking about over 300000 records a time, PHP won't not to be able to store such big arrays. It seems to quit writing lines in the documents I generate when this array gets too big. But I don't receive any error messages like that.
Since the generated documents are SQL files with multiple rows INSERT another solution could be to check if the id already exists in the database. Can MySQL catch these exceptions and try these entries again with adding 1 to id? How?
How do you think I need to solve this problem?
Kind regards,
Wouter
make error messages to appear.
increase memory_limit
instead of values store the parameter in the key - so you'll be able to use isset($array[$this->id]) instead of in_array()
Use INSERT IGNORE to disable duplicate key check in mysql and remove your key check in php. Your statement could look like this.
INSERT IGNORE INTO tbl_name SET key1 = 1, col1 = 'value1'
If you want to add 1 to the id always you could use ON DUPLICATE KEY to increment your key by one:
INSERT INTO table (a,b,c) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE c=c+1;
Why should 30.000 records be a problem? Each record in a standard PHP array takes 144 bytes, for 30.000 that would mean 4218,75 kByte. No big deal.
Otherwise, Your Common Sense's idea with the array-key is worth a thought, because it's faster.
I am trying to input multiple pieces of data through a form and all the data will be separated by (,). I plan to use this data to find the corresponding id for further processing through an sql query.
Below is the code I use.
$key_code = explode(",", $keyword);
//$key_count = count($key_code);
$list = "'". implode("','", $key_code) ."'";
//$row_count = '';
$sql4= "SELECT key_id FROM keyword WHERE key_code IN (".$list.")";
if(!$result4 = mysql_query($sql4, $connect)) {
mysql_close($connect);
$error = true;
}else{
//$i = 0;
while($row = mysql_fetch_array($result4)) {
$keyword_id[] = $row['key_id'];
//$i++;
}
//return $keyword_id;
}
The problem i see is that keyword_id[0] is the only element that contains any data (the data is accurate). Even if I input multiple values through the aforementioned form.
I thought it might be an error in the sql but I echo'ed it and it looks like:
SELECT key_id FROM keyword WHERE key_code IN ('WED','WATER','WASTE')
The values in the brackets are exactly what I inputted.
I even tried to figure out how many rows are being returned by the query and it shows only 1. I assume something is wrong with my query but I cannot figure where.
Any help will be greatly appreciated.
Edit: Alright Solved the problem. Thanks to suggestions made I copied and pasted the $sql_query I had echo'ed on the website into mysql console; which resulted in only 1 row being retrieved. After taking a closer look I realized that there was a whitespace between ' and the second word. I believe the problem starts when I input the key_code as:
WED, WATER, WASTE
Instead inputting it as
WED,WATER,WASTE
fixes the problem. I think I should make it so that it works both ways though.
Anyway, thank you for the help.
I am pretty sure that the query is ok. How many rows do you get with just
SELECT key_id FROM keyword
I think that there is just one line that matches your WHERE.
Check the query directly in the database(with phpmyadmin, or in the mysql console), however this query seems to be working as you may assumed. If it returns only 1 row when you use it directly in the db, then maybe there is only one row in your table wich matches this query.