Select query with LIMIT and UNION - Performance Problem - php

Question

I would like to SELECT certain data out of my mysql DB. I am working with a php loop and a sql statement with a LIMIT and UNION.
Problem: The speed of my query is terrible. One UNION statement tooks 2-4 seconds. Due to the loop the Overall-Query takes 3 Minutes.
Is there a chance to optimize my query?
I tried to separate the "three" statements and merge the results. But this is not really faster. So I think that the UNION is not my problem.
PHP/SQL:
My code is running through two-foreach-loops. The code is working properly. But the performance is the problem.
$sql_country = "SELECT country FROM country_list";
foreach ($db->query($sql_country) as $row_country) { //first loop (150 entries)
$sql_color = "SELECT color FROM color_list";
foreach ($db->query($sql_color) as $row_color) { //second loop (10 entries)
$sql_all = "(SELECT ID, price FROM company
WHERE country = '".$row_country['country']."'
AND color = '".$row_color['color']."'
AND price BETWEEN 2.5 AND 4.5
order by price DESC LIMIT 2)
UNION
(SELECT ID, price FROM company
WHERE country = '".$row_country['country']."'
AND color = '".$row_color['color']."'
AND price BETWEEN 5.5 AND 8.2
order by price DESC LIMIT 2)
UNION
(SELECT ID, price FROM company
WHERE country = '".$row_country['country']."'
AND color = '".$row_color['color']."'
AND price BETWEEN 8.5 AND 10.8
order by price DESC LIMIT 2)";
foreach ($db->query($sql_all) as $row_all) {
$shopID[] = $row_all['ID']; //I just need these IDs
}
}
}
Do you have any idea or hints to get this faster?

Answer 1

An index on (country, color, price, ID) should improve the performance of single queries from seconds to a couple of milliseconds or even less. But you still have the problem of executing 1500 queries. Depending on your system, a single query execution can add an overhead of about 10 ms, which would add up to 15 seconds in your case. You need to find a way to minimize the number of queries - In best case to a single query.
For low limits (like 2 in your case), you can combine multiple LIMIT 1 subqueries with different offsets. I would generate such a query dynamically.
$priceRanges = [
['2.5', '4.5'],
['5.5', '8.2'],
['8.5', '10.8'],
];
$limit = 2;
$offsets = range(0, $limit - 1);
$queryParts = [];
foreach ($priceRanges as $range) {
$rangeFrom = $range[0];
$rangeTo = $range[1];
foreach ($offsets as $offset) {
$queryParts[] = "
select (
select ID
from company cmp
where cmp.country = cnt.country
and cmp.color = clr.color
and cmp.price between {$rangeFrom} AND {$rangeTo}
order by cmp.price desc
limit 1
offset {$offset}
) as ID
from country_list cnt
cross join color_list clr
having ID is not null
";
}
}
$query = implode(' UNION ALL ', $queryParts);
This will generate a quite long UNION query. You can see a PHP demo on rexester.com and SQL demo on db-fiddle.com.
I can't guarantee it will be any faster. But it's worth a try.