In the old question asked below the top answer describes how to speed up the MySQL insert process. Since this an old question I'm curious on how to do this with prepared statements. I use MySQL with innoDB. I'll paste in an example from the answer here for people who don't want to check the link.
Here I'll describe 3 methods for inserting data, ranging from slow to
fast:
The following is extremely slow if you have many rows to insert:
INSERT INTO mytable (id,name) VALUES (1,'Wouter');
INSERT INTO mytable (id,name) VALUES (2,'Wouter');
INSERT INTO mytable (id,name) VALUES (3,'Wouter');
This is already a lot faster:
INSERT INTO mytable (id, name) VALUES
(1, 'Wouter'),
(2, 'Wouter'),
(3, 'Wouter');
And this is usually the fastest:
Have CSV file that looks like this:
1,Wouter
2,Wouter
3,Wouter
And then run something like
LOAD DATA FROM INFILE 'c:/temp.csv' INTO TABLE mytable
old question
This is obvious an old answer. I want to know how to do this with prepared statements in a way that will insert as fast as possible, but let's go one step at a time. First I want to know how to do this with a prepared statements. Any examples that can be given will be much appreciated. I will reply or edit this question with any extra information that you may need.
I'm using mysqli_ *Edited for Rick James's question.
I'm going to do my best and make an educated guess on how this can be done. Please correct me if I'm wrong.
I think I can convert
INSERT INTO mytable (id, name) VALUES
(1, 'Wouter'),
(2, 'Wouter'),
(3, 'Wouter');
into
$bindParamsToBuild = '($myArray[key], ?,?),'
for (i = 0; i < count($myArray); i++)
{
if (i === count($myArray))
{
$bindParamsToBuild += '($myArray[key], ?,?);';
}
else
{
$bindParamsToBuild += '($myArray[key], ?,?),';
}
}
"INSERT INTO mytable (id, name) VALUES".$bindParamsToBuild;
Are you using PDO or mysqli?
If using mysqli, use real_escape_string() separately on each item in the list. It is a bit tedious but can be made easier by gradually building the insert string. I suggest not putting more than 100 items into a single INSERT.
https://stackoverflow.com/a/780046/1766831 is a pretty good discussion of the details. (But be sure to use mysqli_*, not mysql_*.)
Plan A: Construct the query as discussed in some of those links. Use real_escape_string and PHP string operations instead of "prepare".
Plan B: Build a string with lots of question mark. Simultaneously, build an array with the values to put into those question marks. Then do a single "prepare" to finish the multi-row INSERT statement.
Related
I have a DB table which has approximately 40 columns and the main motive is to insert the records in the database as quickly as possible. I am using PHP for this.
The problems is, to create the insert statement, I have to loop through a for each. I am not sure if I doning this correctly. Please suggest me the best atlernative.. here is the example..
/// to loop through the available data ///
$sqc = "";
for ($i=1; $i<=100; $i++){
if ($sqc == ""){
$sqc = "('".$array_value["col1"]."'.. till .. '".$array_value["col40"]."',)";
} else {
$sqc .= ",('".$array_value["col1"]."'.. till .. '".$array_value["col40"]."',)";
}
}
/// finally the sql query ///
$sql_quyery = "INSERT INTO table_name (`col1`,.. till.. ,`col40`) values ".$sqc;
This concatenation of $sqc is taking a lot of time. and also the insertion in the DB, is there an alternate way of doing this.. i need to find a way to speed this up like 100X.. :(
Thank you
As suggested on MySQL Optimizing INSERT Statements page, there are some ways for this-
If you are inserting many rows from the same client at the same time, use INSERT statements with multiple VALUES lists to insert several rows at a time. This is considerably faster (many times faster in some cases) than using separate single-row INSERT statements. If you are adding data to a nonempty table, you can tune the bulk_insert_buffer_size variable to make data insertion even faster.
When loading a table from a text file, use LOAD DATA INFILE. This is usually 20 times faster than using INSERT statements.
Find the link below-
[MySQL Guide]
[1] https://dev.mysql.com/doc/refman/5.7/en/insert-optimization.html
Is just a small contribute but you can avoid the concat using a binding
$stmt = mysqli_prepare($yuor_conn,
"INSERT INTO table_name (`col1`,.. till.. ,`col40`) VALUES (?, ... till.. ?)");
mysqli_stmt_bind_param($stmt, 'ss.......s',
$array_value["col1"], $array_value["col2"], .. till..,
$array_value["col40"]);
How do I join these two queries and execute them as a single query to insert values in two tables:
$sql = mysql_real_escape_string('INSERT INTO admin_export(datetime, product_name, item_code,quantity,subject,export_no) VALUES').implode(',', $row_data);
$sql2 = "insert into `itflower_exportno` (admin_exportno) values('$exportno1')";
You need to separate your two INSERT statements with a semicolon into the single variable.
This(mysql_*) extension is deprecated as of PHP 5.5.0, and will be removed in the future. Instead, the MySQLi or PDO_MySQL extension should be used. Switching to PreparedStatements is even more better to ward off SQL Injection attacks !
You can however use transactions as following example.
BEGIN;
INSERT INTO users (username, password)
VALUES('test', 'test');
INSERT INTO profiles (userid, bio, homepage)
VALUES(LAST_INSERT_ID(),'Hello world!', 'http://www.stackoverflow.com');
COMMIT;
What you have in the code are two different inserts.
MySQL doesn't allow for multiple inserts to be run in the same query so you will have to do one first and then the other.
However you can insert multiple rows in the same table in just one query, basically you are restricted to do all the insert you want in one table at a time.
You can't do this.
If you want to do it because you are afraid of the integrity of the data, use transaction .
Second, use suitable transaction isolation level, to ensure that data are reading correctly.
When I execute
INSERT INTO `survey`
(`id`,`name`,`date_start`,`date_end`)
values
(:id,:name,NULL,DATE_ADD(NOW(), INTERVAL 1 MINUTE))
on duplicate key UPDATE `name`=:name;
SELECT coalesce(:id,LAST_INSERT_ID()) as 'id'
it inserts a new data fine, but doesn't select the id (which is needed later on in my php code)
I've tried this suggestion
$db->setAttribute(PDO::ATTR_EMULATE_PREPARES, true);
but this SQL throws errors (due to duplicate parameters)
SELECT ASCII(substr(`perm` FROM floor(:i/8)+1))&(1<<(:i%8))>0 as perm FROM `user` WHERE `id`=:id
I'm in a lose-lose situation, re-writing all my SQL code to not have duplicate parameters would be very messy, doing a separate select straight after inserting may not return the id I want. Any suggestions would be great
You cannot run two queries at the same time, only one at the time.
If you want to do the whole thing at once then create a stored procedure.
Same goes for complex queries, when it gets complicated you want to have your logic in the database.
Here is an example:
DELIMITER //
CREATE PROCEDURE sp_insert_survey(IN `p_id`,
IN `p_name`,
IN `p_date_start`,
IN `p_date_end`)
BEGIN
INSERT INTO `survey`(`id`,`name`,`date_start`,`date_end`)
VALUES (p_id, p_name, p_date_start, p_date_end);
SELECT `id`,`name`,`date_start`,`date_end`
FROM survey WHERE `id` =LAST_INSERT_ID();
END //
DELIMITER ;
Call the sp from PDO:
$stmt = $db->prepare('CALL sp_insert_survey(?, ?, ?, ?)');
then fetch the data as a SELECT query.
Upon typing this up, one of the similar questions that came up on the right getting last inserted row id with PDO (not suitable result) gave a suitable answer in the question itself, although I'm a little dubious considering the method is being questioned itself.
$db->lastInsertId();
Seems to work for me, but in the question linked it isn't working as desired for the questioner, so I'm not entirely settled with this answer. It does seem to be a bit of a hack.
I have only just begun learning about joining tables in MySQL. Now, I have a small project where I simply want to let the visitor insert data through a form. The data is then displayed in a HTML table with four rows, joining together two tables in my database. The "problem" is that the data should be submitted into those two different tables in my database.
I tried
$query = "INSERT INTO table1, table2 (col1, col2, col3, col4) VALUES ('value1', 'value2', 'value3', 'value4')";
but that doesn't seem to do it. What is the correct syntax for submitting form data to several database tables? Oh, and I read some similar threads mentioning using transactions. Is this necessary? My tables are run with MyISAM. Thanks!
You can read more about it from the MySQL Manual. In short, you cannot insert into multiple tables at once. This leaves you with three options:
Multiple INSERT statements
Triggers
Stored Procedures
The answer to this question: MySQL Insert into multiple tables? (Database normalization?) suggests using transactions, which will not work with MyISAM, but is a good FYI if you ever switch to InnoDB.
I really recommend you read up on Triggers. They can make your life a lot easier. But if you don't want to use them, look into PHP's mysqli_multi_query, which will allow you to execute two different queries at the same time, for example:
$query = "INSERT INTO table1 (col1,col2) VALUES ('$value1','$value2');";
$query = "INSERT INTO table2 (col3,col4) VALUES ('$value3','$value4');";
$result = mysqli_multi_query($dbcon, $query);
You can perform this by using MySQL Transactions By:
Try:
BEGIN
INSERT INTO table1 (col1, col2...ETC)
VALUES('value1', 'value2'...ETC)
INSERT INTO table2 (col1, col2...ETC)
VALUES('value1', 'value2'...ETC);
COMMIT;
With MyISM you will need to execute the query for each table you want to insert into, I do not believe that in a single query you can add to multiple tables.
In your case you can not use Transactions because they are not supported by your engine.
Your only solution is to use several separate queries, preferably within a transaction. Transactions are necessary if you want to make sure that the data from each query is inserted, in which case you COMMIT the transaction; should one of the queries fail, you can ROLLBACK.
P.S. Use InnoDB. It's better in pretty much any environment where INSERT queries make up at least 5% of all queries (sadly I cannot give the source as I had read it several months ago and no longer remember where).
I may be wrong, but you don't insert into multiple tables at the same time. You split it into two or more commands, each handling the specific insertion, whats the big deal, that one extra line of code (which makes everything clearer) too much of a hassle to type?
Look at it this way, if you write a large script, for instance a routine to process some data, the more you segment the code, the easier it is to debug, and, if necessary, inoculate instructions that are problematic, it will end up saving you time in the long run.
I have this problem before You can use multiple query function
$query = "INSERT INTO table1 (col1,col2) VALUES ('$value1','$value2')";
$query = "INSERT INTO table2 (col3,col4) VALUES ('$value3','$value4')";
$result = mysqli_multi_query($dbcon, $query);
I have 5 text box contaning 5 contact no of a particular person. I want to insert those 5 contact no in tblContact(person_id|contact_no). Can I insert these 5 contact no in one sql statement OR I have to call a loop to insert 5 records?
INSERT INTO `tblContact` (`person_id`, `contact_no`) VALUES
('PERSON_ID_VALUE_1', 'CONTACT_NO_VALUE_1'),
('PERSON_ID_VALUE_2', 'CONTACT_NO_VALUE_2'),
('PERSON_ID_VALUE_3', 'CONTACT_NO_VALUE_3'),
('PERSON_ID_VALUE_4', 'CONTACT_NO_VALUE_4'),
('PERSON_ID_VALUE_5', 'CONTACT_NO_VALUE_5');
Many databases have a multi-row insert capability and MySQL is one of them. You can use (in 5.0+, not sure about earlier versions although a brief look at the 3.23/4.0/4.1 docs seems to indicate yes) something like:
insert into tblContact (person_id,contact_no) values
(1, '555-5555'),
(2, '555-1234');
More details here.
Aside: In genneral, if your database didn't support multi-row insert, you'd probably just use a transaction around the group of individual insert statements. We've found that multi-row inserts actually give us a pretty hefty speed increase (on our DBMS anyway - YMMV).
you can insert your 5 records in 1 insert statements.
like: insert into tblContact(person_id, contact_no) values(1,'145566'),(2,'233366'),(3,'564666')