PHP Mysql multiple inserts (big dataset) - php

Question

I have been running a relatively simple script, however it tends to lock up the database.
Each time a lookup value is inserted, it is also checked against to make sure it isnt inserted again.
This works very well on small data sets (<50k), however it has issues with large data sets (>2m). Any help would be appreciated.
$insertCounter = 20;
$matchCounter = 200;
$insertIndex = 0;
$sqlInsert = 'INSERT INTO `database` (`lookup_value`, `timestamp`, `source`) VALUES ';
$matchIndex = 0;
$resetCount = 0;
$indexCounter = 0;
foreach ($matches as $lookup) {
$sqlSelect = 'SELECT `id` FROM `database` WHERE `lookup_value` = \'' . $lookup . '\'';
$qExisting = ExecuteSQL($sqlSelect);
if (mysql_num_rows($qExisting) == 0) {
$insertIndex += 1;
$sqlInsert .= '(\'' . strtolower($lookup) . '\', \'' . date('Y-m-d H:i:s') . '\', \'database\'), ';
if ($insertIndex >= $insertCounter) {
$sqlInsert = substr($sqlInsert, 0, strlen($sqlInsert) - 2);
ExecuteSQLNoResult($sqlInsert);
echo '<p><strong>' . date('Y-m-d H:i:s') . '</strong><br />' . $sqlInsert . '</p>';
$sqlInsert = 'INSERT INTO `database` (`lookup_value`, `timestamp`, `source`) VALUES ';
$insertIndex = 0;
}
}
mysql_free_result($qExisting);
$matchIndex += 1;
$indexCounter += 1;
if ($matchIndex > $matchCounter) {
$resetCount += 1;
$matchIndex = 0;
echo '<p>(' . $indexCounter . ', reset no.' . $resetCount . ') Counter Reached, resetting.</p>';
}
}

Answer 1

How about adding a UNIQUE index with lookup_value field and then using INSERT IGNORE statement?