slow table comparison - php

Question

I have written down this code to compare two tables and find out the difference but it is very slow, normally I have to compare 4k rows. It actually takes 3 min to complete.
$query = $pdo->query("select * from tab1 order by date_time ASC");
$calls = array();
foreach($query as $row){
//check the differences
$from = substr($row['from'],4,15); //remove prefix
$date_time = date('Y-m-d H:i:s'
, strtotime('-2 minute',strtotime($row['date_time'])));
//decrease of 2 min the time to match all time differences
$duration = $pdo->query(
"select duration
, abs(duration - ".$row['duration'].") as duration_diff
, price from tab2
where date_time between '".$date_time."' and '".$row['date_time']."'
and from like '%".$row['from']."'
and duration >0
order by duration_diff"
)->fetch();
//highlight the differences
if ($row['duration'] > $duration['duration'] ):
$color = "#ff0000";
elseif ($row['duration'] < $duration['duration'] ):
$color = "#ff9900";
else:
$color = "#fff";
endif;
$calls[] = array(
"date_time" => $row['date_time'],
"from" => $row['from'],
"to" => $row['to'],
"duration_tab1" => $row['duration'],
"duration_tab2" => $duration['duration'],
"price_tab1" => $row['price'],
"price_tab2" => substr($duration['price'],0,6),
"color" => $color);
}
All the fields in table structure are varchar, there are no indexes.
Which indexes on which fields have to be added to increase the performance?

Answer 1

There is N+1 Mysql Query in your code.
Since 4k rows are not too much, I suggest that you can fetch the whole table1 and table2. And do the comparison in PHP code. It should be faster.
$query = $pdo->query("select * from tab1 order by date_time ASC");
$query1 = $pdo->query("select * from tab2 order by date_time ASC");
....

Answer 2

Pulling a value into php land from a query in a loop then injecting that value into another query executed in a loop is an anti-pattern.
Your database is not normalized - which is causing a lot of complications for you.
Your database is very loosely coupled - which is compounding the complexity.
You've not provided any details of the table structure nor the indexes.
Your problem statement does not address the relative cardinality of the datasets - e.g. what happens if there are no matching rows in tab2.
If you fixed your schema, then it would be trivial to do the join in the database. As it stands, I am extremely dubious as to whether your code will produce reproducible results. However the same results could be obtained by using a function to pull the relevant records out of table 2, something like....
CREATE FUNCTION tab2data(pfrom VARCHAR
, pdate_time DATETIME
, pduration FLOAT)
RETURNS VARCHAR
BEGIN
DECLARE result VARCHAR(200);
SELECT CONCAT(tab2.duration, '#',
ABS(tab2.duration-pduration), '#',
tab2.date_time, '#',
tab2.price, '#',
tab2.`from`, '#',
tab2.`to`, '#')
INTO result
FROM tab2
WHERE `from` LIKE CONCAT('%',pfrom)
AND date_time BETWEEN pdate_time AND pdate_time + INTERVAL 2 MINUTE
AND duration>0
ORDER BY ABS(tab2.duration-pduration)
LIMIT 0,1;
return result;
END;