Bulk insert using PDO and PHP variable containing all database values - php

Question

I am new to PHP and am trying to update a deprecated code from mysql to PDO.
Considering that the variable $insert contains all values to bulk insert such as:
('82817cf5-52be-4ee4-953c-d3f4ed1459b0','1','EM3X001P.1a','04.03.10.42.00.02'),
('82817cf5-52be-4ee4-953c-d3f4ed1459b0','2','EM3X001P.2a','04.03.10.33.00.02'),
...etc 13k lines to insert
here is the deprecated code:
mysql_connect('localhost', 'root', '') or die(mysql_error());
mysql_select_db("IPXTools") or die(mysql_error());
if ($insert != '')
{
$insert = "INSERT INTO IPXTools.MSSWireList (ID,Record,VlookupNode,HostWireLocation) VALUES ".$insert;
$insert .= "ON DUPLICATE KEY UPDATE Record=VALUES(Record),VlookupNode=VALUES(VlookupNode),HostWireLocation=VALUES(HostWireLocation)";
mysql_query($insert) or die(mysql_error());
$insert = '';
}
here is the new code:
try
{
$conn = new PDO("mysql:host=$servername;dbname=$dbname", $username, $password);
$conn->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); //set the PDO error mode to exception
// prepare sql and bind parameters
$stmt = $conn->prepare("INSERT INTO IPXTools.MSSWireList (ID, Record, VlookupNode, HostWireLocation)
VALUES (:ID, :Record, :VlookupNode, :HostWireLocation)");
$stmt->bindParam(':ID', $ID);
$stmt->bindParam(':Record', $Record);
$stmt->bindParam(':VlookupNode', $VlookupNode);
$stmt->bindParam(':HostWireLocation', $HostWireLocation);
// insert a row
// loop through all values inside the $insert variable??????? how?
$stmt->execute();
}
catch(PDOException $e)
{
echo "Error: " . $e->getMessage();
}
$conn = null;
During my research I found an excellent post:
PDO Prepared Inserts multiple rows in single query
One method says I would have to change my $insert variable to include all the field names.
And other method says I dont have to do that. I am looking at Chris M. suggestion:
The Accepted Answer by Herbert Balagtas works well when the $data array is small. With larger $data arrays the array_merge function becomes prohibitively slow. My test file to create the $data array has 28 cols and is about 80,000 lines. The final script took 41s to complete
but I didnt understand what he is doing and I am trying to adapt my code to his. The PHP sintax is new to me so I am strugling with handling the arrays, etc...
I guess the starting point would be the variable $insert which contains all the database values I need.
Do I need to modify my $insert variable to include the field names?
Or I could just use its content and extract the values (how?) and include the values in a loop statement? (that would probably execute my 13k rows one at at time)
Thank you

Answer 1

If you have 13k records to insert, it is good for performance to do not use prepared SQL statement. Just generate SQL query in format like this:
INSERT INTO IPXTools.MSSWireList
(ID, Record, VlookupNode, HostWireLocation)
VALUES
('id1', 'r1', 'node1', 'location1'),
('id2', 'r2', 'node2', 'location2'),
...
('id13000', 'r13000', 'node13000', 'location13000');
What you may do for it - use maner of your legacy code. Your try block will looks loke this:
try
{
$conn = new PDO("mysql:host=$servername;dbname=$dbname", $username, $password);
$conn->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$stmt = $conn->exec($insert);
}