Very slow PHP - MySQL script - php

Question

I am new at using PHP-MySQL. I have two MySQL tables:
Concreteness: A table that contains concreteness scores for 80K words
Brian: A table with 1 million rows, each containing one or two words.
I have a small PHP script that takes each row in "Brian", parses it, looks for the scores in "Concreteness" and records it in "Brian."
I have been running this script with several other tables that had 300-400k rows with each hundreds of words. "Brian" is different because it has 1 million rows with 1 or 2 words per row. For some reason, my script is SUPER slow with Brian.
Here is the actual script:
<?php
include "functions.php";
set_time_limit(0); // NOTE: no time limit
if (!$conn)
die('Not connected : ' . mysql_error());
$remove = array('{J}','{/J}','{N}','{/N}','{V}','{/V}','{RB}','{/RB}'); // tags to remove
$db = 'LCM';
mysql_select_db($db);
$resultconcreteness = mysql_query('SELECT `word`, `score` FROM `concreteness`') or die(mysql_error());
$array = array(); // NOTE: init score cache
while($row = mysql_fetch_assoc($resultconcreteness))
$array[strtolower($row['word'])] = $row['score']; // NOTE: php array as hashmap
mysql_free_result($resultconcreteness);
$data = mysql_query('SELECT `key`, `tagged` FROM `brian`') or die(mysql_error()); // NOTE: single query instead of multiple
while ($row = mysql_fetch_assoc($data)) {
$key = $row['key'];
$tagged = $row['tagged'];
$weight = $count = 0;
$speech = explode(' ', $tagged);
foreach ($speech as $word) {
if (preg_match('/({V}|{J}|{N}|{RB})/', $word, $matches)) {
$weight += $array[strtolower(str_replace($remove, '', $word))]; // NOTE: quick access to word's score
if(empty($array[strtolower(str_replace($remove, '', $word))])){}else{$count++;}
}
}
mysql_query('UPDATE `brian` SET `weight`='.$weight.', `count`='.$count.' WHERE `key`='.$key, $conn) or die(mysql_error());
// Print out the contents of the entry
Print "<b>Key:</b> ".$info['key'] . " <br>";
}
mysql_free_result($data);
?>

Answer 1

I guess the real problem is the 1 million mysql update statements you fire to the database. Consider bundling the update statements (and also remove the print):
$i=0;
while ($row = mysql_fetch_assoc($data)) {
// ... left out the obvious part
$sql .= "'UPDATE `brian` SET `weight`='.$weight.', `count`='.$count.' WHERE `key`='.$key;";
$i++;
if ($i%1000 == 0) {
mysql_query($sql) or die(mysql_error());
$i=0;
$sql = "";
}
}
// remember to save the last few updates
mysql_query($sql) or die(mysql_error());