php+mysql: insert a php array into mysql - php

I have an array with 30000 plus entries that need to go into a MySQL table.
What is the best practice? From here? Lets say [0], [1] and [2] in the database would be 'title', 'type' and 'customer'
Is it add key names matching the column names in the table and call som "magic" function? Or build the query manually...
Array
(
[0] => Array
(
[0] => 2140395946
[1] => 1SAP
[2] => 0041451463
)
[1] => Array
(
[0] => 2140411607
[1] => 2SAP
[2] => 0041411940
)
[2] => Array
(
[0] => 2140706194
[1] => 4SAP
[2] => 0041411943
)
etc. etc.
UPDATE - based on answers
Thanks for the answers.
The solution would normally be to manually create the SQL-string and all rows can be inserted as one:
INSERT INTO `tx_opengate_table` (`machine` ,`customer` ,`type`)
VALUES
('m123', 'dfkj45', 'A'),
('m137', 'kfkj49', 'A'), "repeat this line for each entry in the array"
... ... ...
('m654321', '34dgf456', 'C4') "end with out comma, or remove last comma"
;
Special for TYPO3
I happen to do this in the CMS TYPO3 and just came across a new function added not that long ago:
//Insert new rows
$table = 'tx_opengate_stuff';
$fields = array('machine','type','customer');
$lines = "array as given above"
$GLOBALS['TYPO3_DB']->exec_INSERTmultipleRows($table,$fields,$lines);

I would say just build it yourself. You can set it up like this:
$query = "INSERT INTO x (a,b,c) VALUES ";
foreach ($arr as $item) {
$query .= "('".$item[0]."','".$item[1]."','".$item[2]."'),";
}
$query = rtrim($query,",");//remove the extra comma
//execute query
Don't forget to escape quotes if it's necessary.
Also, be careful that there's not too much data being sent at once. You may have to execute it in chunks instead of all at once.

Magic function? I'm guessing you mean some sort of DB abstraction layer? IMHO that would just double your work.
Just build the query manually looping through array[] INSERT'ing values as you go.

$statement = "INSERT INTO table (title, type, customer) VALUES ";
foreach( $data as $row) {
$statement .= ' ("' . implode($row, '","') . '")';
}
UPDATE: Changed explode to implode (I always get those confused).

You will have to build the query manually if you want the best performance during this operation. If you would iteratively add everything using PDO or some abstarction layer, you will have 30000+ insert queries.
Use foreach to iterate over the arraay, build one nested INSERT query that does all the work at once, and just send it to the server.

Related

how to store php multi dimensional array data into database in their fields [duplicate]

This question already has answers here:
Best way to INSERT many values in mysqli?
(4 answers)
Closed 4 years ago.
i have Multi array data like "cars name & cars modal"
Car name match with car model. both are different column in database (cars_name,cars_model). I want to store data from this array into database in their fields
Output:
Array
(
[car_name] => Array
(
[0] => Honda
[1] => Ford Mustang
[2] => Volvo
)
[car_modal] => Array
(
[0] => 2015
[1] => 2016
[2] => 2014
)
)
i want to store array values into single column in each row using "mysql". For this purpose i like query like this but it shows error.
$sql = "INSERT INTO cars_data (cars_name,cars_modal)
VALUES ($cars_name,$cars_modal)";
nothing happened. But errors show like this...
Notice: Array to string conversion in
E:\xampp\htdocs\car_records\modal_data.php on line 45 Error: INSERT INTO cars_data (cars_name,cars_model)
VALUES (Array,Array)Unknown column 'Array' in 'field list'
The question is how to fix it. Please help me
To use one statement and mysqli prepared statements (comments in code)...
$cars_name = ["Honda", "Volvo"];
// Create an entry for each name
$params = str_repeat("(?),", count($cars_name));
// Build a bind for a list of strings
$binds = str_repeat("s", count($cars_name));
// Add the params to the insert (remove the last ,)
$sql = "INSERT INTO car_data (cars_name)
VALUES ".rtrim($params, ",");
$insert = $conn->prepare ( $sql );
// Bind the parameters, using ... is the argument unpacking operator
$insert->bind_param($binds, ...$cars_name);
// Execute the SQL
$insert->execute();
Update:
If you had two data items in the array, you would be able to adapt the above to something like...
// Source data - ensure that the two sets of data have the same number of entries
$car_data = [ 'cars_name' => ["Honda", "Volvo"],
'cars_modal' => [ '2015', '2016' ]];
$car_count = count($car_data['cars_name']);
// Create an entry for each name (2 binds per entry)
$params = str_repeat("(?,?),", $car_count);
// Build a bind for a list of strings
$binds = str_repeat("ss", $car_count);
// Reformat data for binding (needs to be a single list of the data
// with cars_name followed by cars_modal for each entry)
$merged_data = [];
foreach ( $car_data['cars_name'] as $key => $name ) {
$merged_data[] = $name;
$merged_data[] = $car_data['cars_modal'][$key];
}
// Add the params to the insert (remove the last ,)
$sql = "INSERT INTO car_data (cars_name,car_model)
VALUES ".rtrim($params, ",");
$insert = $conn->prepare ( $sql );
// Bind the parameters, using ... is the argument unpacking operator
$insert->bind_param($binds, ...$merged_data);
// Execute the SQL
$insert->execute();
you can insert multiple elements in one row, you just need to bring it in the right format :
insert into x (columns) VALUES (x1),(x2),(x3)
$string = "";
foreach($cars_name as $car){
$string .= "(" . $car . "), ";
}
$sql = "INSERT INTO car_data (cars_name) VALUES $string";
Pleas note that you should never accept user input without sanitizing it.
When I want to do so, I first implode this array to get a normal string separated by (,) then when I retrieve data I implode them again.
$cars_name = implode(',', $_POST['cars_name']);
The result will be
Honda,Volvo,Mercedes,Toyota,BMW
Then if you want to get back the array from database again just do as following:
$cars_array = explode(',', $databaseObject['cars']);
The result will be the same as the first array of yours.

Updating DB from grid array

I am using DHTMLX javascript library to present a grid in a form. My users could have hundreds of products in their grid but I am only capturing the the rows they add a price to. I then need to add those rows to a MySQL database along with some other information.
Getting standard $POST information and manipulating it in my PHP scripts is about the limit of my skills so far so where I need help is the array created by the updated rows. An example of what is being captured in my $POST is:
Array
(
[product] => 64
[dept] => 000
[submit] => Submit
[gridbox_1_4] => 422
[gridbox_64_4] => 534
[gridbox_175_4] => 1234
[gridbox_180_4] => 645
)
I currently capture the basic $POST variables with:
$itemcat = filter($_POST['product']);
$dept7 = filter($_POST['dept']);
My question is how do I capture the gridbox variables so I can use them in an INSERT statement? The only number that will change is the middle number which represents a primary key in my database for the products I need to INSERT into another table. I am assuming I need to explode the $POST variable somehow maybe? Then how do I INSERT them using something like:
"INSERT INTO submissions
(product_id, user_id,submission_id, sugg_price)
VALUES ('" . $gridbox_id . "','" . $myid . "','NULL', '" . $sugg . "')";
All of my reading today on arrays has given me a good understanding of how they work from a 100ft view but not how to specifically solve my problem or even how to start. I'm hoping the community can send me down the right path.
You can loop over the $_POST array and perform a regular expression match on each key:
// PDO will make your life so much easier
$dbh = new PDO('mysql:dbname=testdb;host=127.0.0.1', $user, $password);
// prepared statements are great - prepare once, use many
$stmt = $dbh->prepare('
INSERT INTO submissions
( product_id, user_id, submission_id, sugg_price)
VALUES
(:product_id, :user_id, NULL , :sugg_price)
');
// avoid SQL injection by passing your variables in as parameters
// see http://bobby-tables.com to understand why
$stmt->bindValue('user_id', $myid);
foreach ($_POST as $key => $val) {
if (preg_match("/^gridbox_(\d+)_4$/", $key, $matches)) {
$stmt->execute(array(
':product_id' => $matches[1],
':sugg_price' => $val
));
}
}

Insert large amount of array data into DB

I have a multi-dimensional array which contains ten thousands of data. A lot... The array is structured like this:
Array (
[0] => Array ( [0] => city [1] => code [2] => country )
[1] => Array ( [0] => city [1] => code [2] => country )
)
What I am trying to do is to insert the array values city, code and country into a table in a mysql database. I found posts that match exactly what I want to do, but for some reason it is not working with me. When I say it is not working I mean that the php doesn't even start. If I remove the here below code, the file runs normaly. So the problem really comes from that code portion. Hope someone will not mind helping me. Thank you in advance. Cheers. Marc.
//some code to build the array
//db_connect code
$sql = array();
foreach( $myarray as $row )
{
$sql[] = '("'.$row[0].'", "'.$row[1]).'","'.$row[2].'")';
}
mysql_query('INSERT INTO test (t_city, t_code, t_country) VALUES '.implode(',', $sql));
As said before, the error in building the sql array, is a surplus bracket. Change
$sql[] = '("'.$row[0].'", "'.$row[1]).'","'.$row[2].'")';
to
$sql[] = '("'.$row[0].'", "'.$row[1].'","'.$row[2].'")';
As ashein noted in comments, the query length is limited by the "max_allowed_paket" variable. If the query is larger than this, an error is raised and connection gets closed.
There is a bracket after $row[1] :)
Use this (remove the bracket):
$sql[] = '("'.$row[0].'", "'.$row[1].'","'.$row[2].'")';
You can try inserting every array record as separate sql-query.
foreach( $myarray as $row )
{
mysql_query('INSERT INTO test (t_city, t_code, t_country) VALUES ("'.$row[0].'", "'.$row[1]).'","'.$row[2].'");
}
but there would be a lot of queries
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
You want to dynamically build such a query by submitting multiple of the value pairs at once and not running into limits.
So what you do is to build the insert query while adding iteratively one row after the other. If adding a row would trigger the limit, the query is send and the query is reset:
# sample data
$data = array(
array('city1', 'code', 'country'),
array('city2', 'code', 'country'),
array('city3', 'code', 'country'),
array('city4', 'code', 'country'),
array('city5', 'code', 'country'),
array('city6', 'code', 'country'),
array('city7', 'code', 'country'),
);
$max_allowed_packet = 1048576; # mysql default value
$max_allowed_packet = 128; # for demonstration purposes
$sql = new SQLInsertQuery('INSERT INTO test (t_city, t_code, t_country) VALUES ', $max_allowed_packet);
foreach($data as $row) {
$sql->addRow($row);
}
$sql->query(); # manually query any potential left-over query.
This example outputs the following:
Running: INSERT INTO test (t_city, t_code, t_country) VALUES ('city1','code','country'),('city2','code','country');
Running: INSERT INTO test (t_city, t_code, t_country) VALUES ('city3','code','country'),('city4','code','country');
Running: INSERT INTO test (t_city, t_code, t_country) VALUES ('city5','code','country'),('city6','code','country');
Running: INSERT INTO test (t_city, t_code, t_country) VALUES ('city7','code','country');
Demo, Gist
You might want to add a counter for the queries run as well so you can validate after the loop and the final query if at all a query was sent (the limit can be too low so that no query is send at all - depending on your data - so it's worth to have a sanity check for this edge-case).
I hope this example is helpful.

having trouble updating value serialized array in MYSQL

Here what I does
- extract my serialize array from databse
- deserialize it
- change the value
- after changing I serialize it
- then write it back to MYSQL database table.
But Im having trouble writing it back to MYSQL database table
$myarray = Array
(
[dogname] => Array
(
[0] => white
[1] => zeon
[2] => imao
)
[visit] => Array
(
[0] => once
[1] => twice
[2] => twice
)
}
I save it database with this way into my table serialize($myarray)
now I extract it and unserialize
$unserializearray = unserialize($myserialarray);
then search for array value to change and back to table
$keys = array_keys($unserializearray['dogname'], 'imao', true);
foreach($keys as $key)
{
// find dogname imao and change its visit to once
$unserializearray['visit'][$key] = "once";
$update = serialize($unserializearray);
//save the update
mysql_query("UPDATE dogdetails SET checkup = '$update'
WHERE dogid = '1'");
}
but it dont save right into mysql do I miss something in code?
First of all this can't work, if something in your array has a single quote in it, so you need to do
$update=mysql_real_escap_string(serialize($unserializearray),$dbhandle);
or friends.
More importantly, your database design seem heavily flawed to me: As has been said many, many times here, you should not store more than one value in a field, if you want to access them seperately. In your case, you store three dogs with their vists, but you want to access them seperately - this is what your code does.
As Eugen said, there are many things wrong with your code.
Except all that, does this code works any better:
$unserializearray = unserialize($myserialarray);
then search for array value to change and back to table
$keys = array_keys($unserializearray['dogname'], 'imao', true);
foreach($keys as $key)
{
// find dogname imao and change its visit to once
$unserializearray['visit'][$key] = "once";
}
$update = mysql_real_escap_string(serialize($unserializearray));
//save the update
mysql_query("UPDATE dogdetails SET checkup = '$update' WHERE dogid = '1'");

Change the return format on PHP's PDO Select

I am writing my own PDO wrapper to make my life easier and a fair amount safer.
A standard query looks like:
$user = $db->select('users')
->eq('twitter_id', $twitter_id)
->limit(1)
->prepare()
->exec();
Generates this query:
SELECT * FROM users WHERE twitter_id = :twitter_id LIMIT 1
This works perfectly fine as I, currently, want it. Where I am running into a problem is when I have a query to return multiple rows.
My apps stores some dynamic settings that I want to grab and use in one pass and I can do that by running a query like:
$share_datas = $db->select('settings', 'setting, value')
->prepare()
->exec();
Which generates:
SELECT setting, value FROM settings
Which returns:
Array
(
[0] => Array
(
[setting] => since_id
[value] => 17124357332
)
[1] => Array
(
[setting] => last_dm
[value] => 1271237111
)
)
The function prepare() puts the pieces together for the query and the function exec() binds the params and returns the array.
function exec()
{
// echo 'vars: <pre>'.print_r($this->sql_vars, true).'</pre>';
$stmt = $this->dbh->prepare($this->sql_last_query);
foreach($this->sql_vars as $key => $val)
{
if('date_time' === $key) continue;
$bind = $stmt->bindValue($key, $val);
}
$stmt->execute();
$this->sql_vars = array();
return $stmt->fetchAll(PDO::FETCH_ASSOC);
}
Now to the question: Is there a way that I can change exec() or the query itself so that it can return an array that returns like below and avoids any additional loops?
Array
(
[since_id] => 17124357332
[last_dm] => 1271237111
)
No problem with some simple array functions.
$in = $db->exec();
$out = array();
foreach( $in as $row )
$out[ $row['setting'] ] = $row['value'];
If you need a more general function, you'll have to describe the transformation clearer.
The answer is likely going to be either:
Creating multiple versions of your exec method with different return behavior, or
Having exec simply perform the execution and store the statement handle, then have fetching the data be a separate method.
I've found the following convenience methods handy, in addition to your current array of hashes:
Query "one": The first column in the first row as a scalar (for things like SELECT COUNT(*))
Query "list": The first column of all rows as an indexed array (for things like SELECT id FROM ...))
Query "pairs": The first two columns of all rows as a hash (for your current problem)
Query "insert id": The last generated row id as a scalar (autoincrement in MySQL, sequence in Postgres, etc)
These are all occasionally convenient things that PDO (and most other database adapters) simply don't have built-in flags to handle.

Categories