Memory/Time Efficient Database Entry - php

Question

I have a foreach loop that iterates through an array.
In each instance, I organise the array into a query string and use MySQLi to add it to the database.
function storeProperties($data, $db) {
foreach ($data['property'] as $property) {
$query_string = "INSERT INTO table VALUES(..., ..., ...,)"
$db->query($query_string);
echo $db->error;
}
}
Is there a better way I should be doing this?
Obviously, this method uses n database queries one after another so this is memory intensive and time intensive.
Is there a better way to do this?
Should I be concatenating each query into a single string and running it all outside the for loop?

Answer 1

The following method is from my PDO workhorse, used for bulk insertions. It creates a single INSERT statement with multiple VALUES entries.
Use it as
$db->bulkinsert(
'tbl_my_tablename',
array('fieldname_1','fieldname_2', 'fieldname_3'),
array(
/* rec1 */ array('r1f1', 'r1f2', 'r1f3'),
/* rec2 */ array('r2f1', 'r2f2', 'r2f3'),
/* rec3 */ array('r3f1', 'r3f2', 'r3f3')
));
Please note that the method is an snip from a complex class definition, some methods used here are not contained in the code snippet, especially $this->connect() (connects to PDO),$this->begin() (starts transaction), $this->commit()and $this->rollback(), and the static Log class for Logging similar to Apache Commons ;-)
But I'm sure this is what you might need.
/**
* Performs fast bulk insertion
* Parameters:
* $tablename
* $datafields - non-assiciative array of fieldnames
* or propertynames if $data is an array of objects
* $data - array of either non-associative arrays (in the correct order)
* or array of objects with property names matching the $datafields array
*/
const MAX_BULK_DATA = 3000;
public function bulkinsert($tablename, $datafields, &$data) {
$result = 0;
try {
try {
$this->connect();
$datacount = count($data);
// loop until all data has been processed
$start = 0;
$lastbinds = 0;
$this->begin();
while ($start < $datacount) {
$ins = array();
$bindscount = min(self::MAX_BULK_DATA, $datacount - $start);
if ($bindscount != $lastbinds) {
// prepare the binds
$binds = substr(str_repeat(',?', count($datafields)), 1);
$binds = substr(str_repeat(",($binds)", $bindscount), 1);
$lastbinds = $bindscount;
}
for ($go = $start, $last = $start + $bindscount; $go < $last; $go++) {
if (is_object($data[$go])) {
try {
foreach($datafields as $propname) {
$rfl = new ReflectionProperty($data[$go], $propname);
$rfl->setAccessible(true);
$ins[] = $rfl->getValue($data[$go]);
}
}
catch(ReflectionException $e) {
throw new InvalidArgumentException('PDOCONNECT_ERR_SQL_UNKNOWN_PROPERTY', 0, $e);
}
}
else {
foreach($data[$go] as $value) {
$ins[] = $value;
}
}
}
$sql = sprintf('INSERT INTO %s (%s) VALUES %s', $tablename, join(',',$datafields), $binds);
Log::trace($sql);
$stmt = $this->pdo->prepare($sql);
$stmt->execute($ins);
$start = $last;
$result += $bindscount;
}
$this->commit();
}
catch(PDOException $e) {
// do something with the exception if necessary
throw $e;
}
}
catch(Exception $e) {
$this->rollback();
throw $e;
}
return $result;
}
}