SQL Query taking too long - php

I am trying to optimize the SQL query listed below.
It is basically a search engine code that retrieves products based on the products name. It also checks products model number and whether or not it is enabled.
This executes in about 1.6 seconds when I run it directly through the phpMyAdmin tool but takes about 3 seconds in total to load in conjunction with the PHP file it is placed in.
I need to add a category search functionality and now that is crashing the MySQL server, HELP!
Justin
SELECT DISTINCT
p.products_id,
p.products_image,
p.products_price,
s.specials_new_products_price,
p.products_weight,
p.products_unit_quantity,
pd.products_name,
pd.products_img_alt,
pd.products_affiliate_url
FROM
products AS p
LEFT JOIN
vendors v
ON
v.vendors_id = p.vendors_id
LEFT JOIN
specials AS s
ON
s.products_id = p.products_id
AND
s.status = 1,
categories AS c,
products_description AS pd,
products_to_categories AS p2c
WHERE
(
(
pd.products_name LIKE '%cleaning%'
AND
pd.products_name LIKE '%supplies%'
)
OR
(
p.products_model LIKE '%cleaning%'
AND
p.products_model LIKE '%supplies%'
)
OR
p.products_id = 'cleaning supplies'
OR
v.vendors_prefix = 'cleaning supplies'
OR
CONCAT (CAST (v.vendors_prefix AS CHAR), '-', CAST (p.products_id AS CHAR)) = 'cleaning supplies'
)
AND
p.products_status = '1'
AND
c.categories_status = '1'
AND
p.products_id = pd.products_id
AND
p2c.products_id = pd.products_id
AND
p2c.categories_id = c.categories_id
ORDER BY
pd.products_name

left join specials as s on s.products_id = p.products_id and s.status = 1,
there you've a full table scan. That kind of data (tinyint, true/false) are not good handled by InnoDB. In Oracle you have something like a bitmap index, that is good for low selectivity indexes. You could use a subquery, in your where, something like this:
...
where
p.product_id IN (SELECT S.status FROM specials s WHERE s.status = 1)
AND
(
( pd.products_name like '%cleaning%' and pd.products_name like '%supplies%' ) or
...
Then, you pray for the query cache to cache the results from that subquery.
POST YOUR EXPLAIN!

select p.products_id, p.products_image, p.products_price,
p.products_weight, p.products_unit_quantity,
s.specials_new_products_price,
pd.products_name, pd.products_img_alt, pd.products_affiliate_url
from products as p
left join vendors v ON v.vendors_id = p.vendors_id
left join specials as s on s.products_id = p.products_id,
left join products_description as pd on pd.products_id = p.products_id
left join products_to_categories as p2c on p2c.products_id = p.products_id
left join categories as c on c.categories_id = p2c.categories_id
where
(
( pd.products_name like '%cleaning%' and pd.products_name like '%supplies%' ) or
( p.products_model like '%cleaning%' and p.products_model like '%supplies%' ) or
p.products_id = 'cleaning supplies' or
v.vendors_prefix = 'cleaning supplies' or
CONCAT( CAST(v.vendors_prefix as char), '-', CAST(p.products_id as char) ) = 'cleaning supplies'
)
and p.products_status = '1'
and s.status = 1
and c.categories_status = '1'
group by p.products_id
order by pd.products_name
Make sure you have the following indexes set:
create index prod_prodid_status on products(products_id, products_status);
create index vendors_vendorid on vendors(vendors_id);
create index specials_prodid_status on vendors(products_id, `status`);
create index prod_desc_prodid_prodname on products_description(products_id, products_name);
create index prod_cat_prodid_catid on products_to_categories(products_id, categories_id);
create index categories_catid_status on categories(categories_id, categories_status);
And i hope all the fields with "id" in their name are integers

Related

How do I solve this sql syntax error? near (pl.name, ' ', CASE WHEN pal.name is not null THEN GROUP_CONCAT(DISTINCT(pal.nam'

Apparently,I have a syntax error that I don't get, could you help me?
SELECT p.id_product, pa.ean13,pa.upc,p.active, p.isBike, p.isDirect, p.weight,p.height, p.width, p.depth, pa.reference, pl.description_short, pa.upc, p.price, pai.id_image CONCAT(pl.name, \' \', CASE WHEN pal.name is not null THEN GROUP_CONCAT(DISTINCT(pal.name) SEPARATOR ", ") ELSE "" END) as name
FROM ps_product p
LEFT JOIN ps_product_lang pl ON (p.id_product = pl.id_product)
LEFT JOIN ps_product_attribute pa ON (p.id_product = pa.id_product)
LEFT JOIN ps_product_attribute_combination pac ON (pac.id_product_attribute = pa.id_product_attribute)
LEFT JOIN ps_attribute_lang pal ON (pal.id_attribute = pac.id_attribute)
LEFT JOIN ps_stock_available s ON (p.id_product = s.id_product)
WHERE pl.id_lang = 1
AND p.id_shop_default = 1
AND pl.name LIKE "%PRODUCT NAME%"
GROUP BY pac.id_product_attribute
ORDER BY pl.name
LIMIT 60'
Besides a missing comma in the SELECT statement, you were also missing a LEFT JOIN on the ps_product_attribute_image table, here's the correct SQL query:
SELECT p.id_product, pa.ean13, pa.upc, p.active, p.isBike, p.isDirect, p.weight, p.height, p.width, p.depth, pa.reference, pl.description_short, pa.upc, p.price, pai.id_image,
CONCAT(pl.name, ' ', CASE WHEN pal.name is not null THEN GROUP_CONCAT(DISTINCT(pal.name) SEPARATOR ', ') ELSE '' END) as name
FROM ps_product p
LEFT JOIN ps_product_lang pl ON (p.id_product = pl.id_product)
LEFT JOIN ps_product_attribute pa ON (p.id_product = pa.id_product)
LEFT JOIN ps_product_attribute_combination pac ON (pac.id_product_attribute = pa.id_product_attribute)
LEFT JOIN ps_product_attribute_image pai ON (pai.id_product_attribute = pa.id_product_attribute)
LEFT JOIN ps_attribute_lang pal ON (pal.id_attribute = pac.id_attribute)
LEFT JOIN ps_stock_available s ON (p.id_product = s.id_product)
WHERE pl.id_lang = 1
AND p.id_shop_default = 1
AND pl.name LIKE '%PRODUCT NAME%'
GROUP BY pac.id_product_attribute
ORDER BY pl.name
LIMIT 60

MySQL count issue

In my application (osCommerce) I had to modify a query that look like this:
SELECT sql_cache distinct p.products_image,
p.products_subimage1,
pd.products_name,
p.products_quantity,
p.products_model,
p.products_ordered,
p.products_id,
p.products_price,
p.products_date_added,
p.products_weight,
p.products_length,
p.products_width,
p.products_height,
p.products_tax_class_id,
p.products_status,
IF(s.status, s.specials_new_products_price, NULL) AS specials_new_products_price,
IF(s.status, s.specials_new_products_price, p.products_price) AS final_price
FROM products p
LEFT JOIN specials s
ON p.products_id = s.products_id
LEFT JOIN products_to_categories p2c
ON p.products_id=p2c.products_id
LEFT JOIN products_description pd
ON p.products_id=pd.products_id
INNER JOIN filter_association_products fap
ON p.products_id =fap.products_id
LEFT JOIN products_attributes pa
ON p.products_id = pa.products_id
WHERE p.products_status = '1'
AND date_sub(curdate(),INTERVAL 3000 day) <= p.products_date_added
AND find_in_set(fap.filter_id,'126, 130')
ORDER BY p.products_date_added DESC,
pd.products_name
to end like this, for accurate results:
AND fap.filter_id IN (126, 130)
GROUP BY p.products_id
HAVING COUNT(DISTINCT fap.filter_id) = 2;
The issue I have now is that there is the following query that is using this new query and giving me wrong results.
SELECT COUNT( DISTINCT p.products_id ) AS total
FROM products p
LEFT JOIN specials s ON p.products_id = s.products_id
LEFT JOIN products_to_categories p2c
ON p.products_id = p2c.products_id
LEFT JOIN products_description pd
ON p.products_id = pd.products_id
INNER JOIN filter_association_products fap ON p.products_id = fap.products_id
LEFT JOIN products_attributes pa
ON p.products_id = pa.products_id
WHERE p.products_status = '1'
AND DATE_SUB( CURDATE( ) , INTERVAL 3000 DAY ) <= p.products_date_added
AND fap.filter_id IN ( 126, 130 )
GROUP BY p.products_id
HAVING COUNT( DISTINCT fap.filter_id ) =2
ORDER BY p.products_date_added DESC , pd.products_name
Which instead of giving a result of 1 row with the count of all the product_ids in the original query, now gives a result of multiple rows (the same amount of rows as the expected total products count) with the number 1 in each of them.
The main issue seems to be the GROUP BY p.products_id
HAVING COUNT( DISTINCT fap.filter_id ) =2
Is there any way to modify the original query so that the count query will work correctly while still using AND fap.filter_id IN ( 126, 130 )?
You need to use a subquery to get the results that you want:
select count(*)
from (<old query here>) s;
You need to aggregate at the product id level to get the products that meet the original condition. Counting the number of such products requires another aggregation.
As Gordon suggested, using a subquery should generally work.
Since this query looks like a product listing query and since you are using osCommerce, you will bump into another core issue when you modify the query to use a subquery:
The count query of the split page results will fail as it does not support subqueries.
To resolve this also, you will need to modify the core code in catalog/includes/classes/split_page_results.php
Change:
$count_query = tep_db_query("select count(" . $count_string . ") as total " . substr($this->sql_query, $pos_from, ($pos_to - $pos_from)));
To:
$count_query = tep_db_query("select count(*) as total from (" . $this->sql_query . ") AS derivedtable1");
More details here: http://forums.oscommerce.com/topic/290110-class-splitpageresults/

Combining 2 MySQL queries

I have 2 queries:
SELECT sql_cache distinct p.products_image,
p.products_subimage1,
pd.products_name,
p.products_quantity,
p.products_model,
p.products_ordered,
p.products_id,
p.products_price,
p.products_date_added,
p.products_weight,
p.products_length,
p.products_width,
p.products_height,
p.products_tax_class_id,
p.products_status,
IF(s.status, s.specials_new_products_price, NULL) AS specials_new_products_price,
IF(s.status, s.specials_new_products_price, p.products_price) AS final_price
FROM products p
LEFT JOIN specials s
ON p.products_id = s.products_id
LEFT JOIN products_to_categories p2c
ON p.products_id=p2c.products_id
LEFT JOIN products_description pd
ON p.products_id=pd.products_id
INNER JOIN filter_association_products fap
ON p.products_id =fap.products_id
LEFT JOIN products_attributes pa
ON p.products_id = pa.products_id
WHERE p.products_status = '1'
AND date_sub(curdate(),INTERVAL 3000 day) <= p.products_date_added
AND fap.filter_id = 126
ORDER BY p.products_date_added DESC,
pd.products_name
Which gives me a result of 52 rows (products).
And an identical query with only difference:
AND fap.filter_id = 130
Which gives me a result of 4 rows.
One of the common things between these rows/products is that 3 out of the 4 with filter_id 130 also have filter_id 126 and I want to modify the query to give me results of only products with both (or even more, depends on the filter_ids applied) of the indicated filter_id.
I have tried
...
AND FIND_IN_SET(fap.filter_id,'126', '130')
ORDER BY p.products_date_added DESC, pd.products_name
But I get a result of 53 rows/products, meaning it's showing all the products that have either filter, while the result I am looking for in this case is of only the 3 rows that have both filter_id.
What is the best way to rewrite the query to get the correct results?
Use GROUP BY to get the product information that you want. You can join in additional information as well, but this is the basic query:
SELECT p.*
FROM products p JOIN
specials s
ON p.products_id = s.products_id JOIN
products_to_categories p2c
ON p.products_id = p2c.products_id JOIN
products_description pd
ON p.products_id = pd.products_id JOIN
filter_association_products fap
ON p.products_id = fap.products_id JOIN
products_attributes pa
ON p.products_id = pa.products_id
WHERE p.products_status = '1' AND
date_sub(curdate(),INTERVAL 3000 day) <= p.products_date_added AND
fap.filter_id IN (126, 130)
GROUP BY p.products_id
HAVING COUNT(DISTINCT fap.filter_id) = 2; -- make sure both match
I don't see any reason for the LEFT JOINs (at least for most of the tables, because everything up to filter_association_products is being turned into an inner join by the WHERE clause), so I changed them to inner joins.
Make two LEFT JOIN's on filter_association_products and filter in WHERE, like this:
SELECT sql_cache distinct p.products_image,
p.products_subimage1,
pd.products_name,
p.products_quantity,
p.products_model,
p.products_ordered,
p.products_id,
p.products_price,
p.products_date_added,
p.products_weight,
p.products_length,
p.products_width,
p.products_height,
p.products_tax_class_id,
p.products_status,
IF(s.status, s.specials_new_products_price, NULL) AS specials_new_products_price,
IF(s.status, s.specials_new_products_price, p.products_price) AS final_price
FROM products p
LEFT JOIN specials s
ON p.products_id = s.products_id
LEFT JOIN products_to_categories p2c
ON p.products_id=p2c.products_id
LEFT JOIN products_description pd
ON p.products_id=pd.products_id
LEFT JOIN filter_association_products fap1
ON p.products_id =fap1.products_id
LEFT JOIN filter_association_products fap2
ON p.products_id =fap2.products_id
LEFT JOIN products_attributes pa
ON p.products_id = pa.products_id
WHERE p.products_status = '1'
AND date_sub(curdate(),INTERVAL 3000 day) <= p.products_date_added
AND fap1.filter_id = 126
AND fap2.filter_id = 130
ORDER BY p.products_date_added DESC,
pd.products_name
Another approach (sub select) :
SELECT sql_cache distinct p.products_image,
p.products_subimage1,
pd.products_name,
p.products_quantity,
p.products_model,
p.products_ordered,
p.products_id,
p.products_price,
p.products_date_added,
p.products_weight,
p.products_length,
p.products_width,
p.products_height,
p.products_tax_class_id,
p.products_status,
IF(s.status, s.specials_new_products_price, NULL) AS specials_new_products_price,
IF(s.status, s.specials_new_products_price, p.products_price) AS final_price
FROM products p
LEFT JOIN specials s
ON p.products_id = s.products_id
LEFT JOIN products_to_categories p2c
ON p.products_id=p2c.products_id
LEFT JOIN products_description pd
ON p.products_id=pd.products_id
LEFT JOIN products_attributes pa
ON p.products_id = pa.products_id
WHERE p.products_status = '1'
AND date_sub(curdate(),INTERVAL 3000 day) <= p.products_date_added
AND 126 in (select fap1.filter_id from filter_association_products fap1
where p.products_id = fap1.products_id)
AND 130 in (select fap2.filter_id from filter_association_products fap2
where p.products_id = fap2.products_id)
ORDER BY p.products_date_added DESC,
pd.products_name
While both answers by Mikey and Gordon Linoff are good, I preferred Gordon's approach due to its extensibility. However, his answer has some syntax errors and the switch to JOINs didn't produce any results, so I had to revise them back to LEFT JOINs, as suggested by Mikey. (Thank you both).
This is the final query that works:
SELECT sql_cache distinct p.products_image,
p.products_subimage1,
pd.products_name,
p.products_quantity,
p.products_model,
p.products_ordered,
p.products_id,
p.products_price,
p.products_date_added,
p.products_weight,
p.products_length,
p.products_width,
p.products_height,
p.products_tax_class_id,
p.products_status,
IF(s.status, s.specials_new_products_price, NULL) AS specials_new_products_price,
IF(s.status, s.specials_new_products_price, p.products_price) AS final_price
FROM products p
LEFT JOIN specials s
ON p.products_id = s.products_id
LEFT JOIN products_to_categories p2c
ON p.products_id=p2c.products_id
LEFT JOIN products_description pd
ON p.products_id=pd.products_id
INNER JOIN filter_association_products fap
ON p.products_id =fap.products_id
LEFT JOIN products_attributes pa
ON p.products_id = pa.products_id
WHERE p.products_status = '1'
AND date_sub(curdate(),INTERVAL 3000 day) <= p.products_date_added
AND fap.filter_id IN (126, 130)
GROUP BY p.products_id
HAVING COUNT(DISTINCT fap.filter_id) = 2;

How to total results within the query

I have the following query:
SELECT cb.customers_id, cb.products_id, p.products_model, pd.products_name, cb.customers_basket_quantity, p.products_price, (p.products_price * cb.customers_basket_quantity) AS product_total
FROM customers_basket cb, products p, products_description pd
WHERE cb.customers_id =194075
AND cb.products_id = pd.products_id
AND p.products_id = pd.products_id
I am trying to figure out if I can get a "cart_total" without having to do another query, or use PHP to keep a running total.
I wasn't sure if I could use the CASE statement to do something.
Suggestions?
SELECT cb.customers_id, cb.products_id, p.products_model, pd.products_name, cb.customers_basket_quantity, p.products_price, (p.products_price * cb.customers_basket_quantity) AS product_total,
(SELECT sum(p.products_price * cb.customers_basket_quantity)
FROM customers_basket cb, products p
WHERE cb.customers_id =194075
AND cb.products_id = p.products_id
group by cb.customers_id) AS cart_total
FROM customers_basket cb, products p, products_description pd
WHERE cb.customers_id =194075
AND cb.products_id = pd.products_id
AND p.products_id = pd.products_id
A link to Fiddle
You can do a cart total, but you want to drop some of the columns from the group by and select (they are not appropriate at the cart level). You should also use standard ANSI join syntax:
SELECT cb.customers_id, cb.customers_basket_quantity,
sum(p.products_price * cb.customers_basket_quantity) AS product_total
FROM customers_basket cb join
products p
on cb.products_id = p.products_id join
products_description pd
on p.products_id = pd.products_id
WHERE cb.customers_id = 194075
GROUP BY cb.customer_basket_id;
Information at the product level doesn't make sense at the basket level. You could create lists of product names, using, say:
group_concat(p.products_name) as Products_Name
but individual values don't make sense.
Try this, you will get the resulting sum at the last row of the query.
All other columns will be null.
SELECT cb.customers_id,
cb.products_id,
cb.customers_basket_quantity,
p.products_model,
p.products_price,
pd.products_name,
(p.products_price * cb.customers_basket_quantity) AS product_total,
null as cart_total
FROM products p
INNER JOIN customers_basket cb ON p.products_id = cb.products_id
INNER JOIN products_description pd ON pd.products_id = p.products_id
WHERE cb.customers_id = 194075
union all
SELECT null, null, null, null, null, null, null,
SUM(p.products_price * cb.customers_basket_quantity) AS cart_total
FROM products p
INNER JOIN customers_basket cb ON p.products_id = cb.products_id
WHERE cb.customers_id = 194075

Joining one more table to a SQL query that's just too complex for me

I'm not all that well-versed in MySQL querying, and I usually only do very simple JOINs. I am working with an installation of osCommerce, and I want the category page to include products from all subcategories as well.
select
p.products_image,
pd.products_name,
pd.products_description,
p.products_id,
p.manufacturers_id,
p.products_price,
p.products_tax_class_id,
IF(s.status, s.specials_new_products_price, NULL) as specials_new_products_price,
IF(s.status, s.specials_new_products_price, p.products_price) as final_price
from
products_description pd,
products p
left join
manufacturers m on p.manufacturers_id = m.manufacturers_id
left join
specials s on p.products_id = s.products_id,
products_to_categories p2c
where
p.products_status = '1' and
p.products_id = p2c.products_id and
pd.products_id = p2c.products_id and
pd.language_id = '1' and
(p2c.categories_id = '24' or ###.parent_id = '24')
order by pd.products_name asc
Basically, I need to join the categories table to this query as well, pulling the row from the categories table where categories_id = p2c.categories_id. Then, I can reference the parent_id column from the selected row from the categories table (I would replace the "###" above with something like "cat").
However, I'm getting confused with all the left joins as to where I should insert another JOIN clause.
Any help would be much appreciated.
Thanks!
Don't have the tables to test against, but should be something as simple as;
SELECT
p.products_image,
...
FROM products_description pd
JOIN products p
ON pd.products_id = p.products_id
LEFT JOIN manufacturers m
ON p.manufacturers_id = m.manufacturers_id
LEFT JOIN specials s
ON p.products_id = s.products_id
JOIN products_to_categories p2c
ON p2c.products_id = p.products_id
JOIN categories c
ON c.categories_id = p2c.categories_id
WHERE
p.products_status = '1' and
pd.language_id = '1' and
(p2c.categories_id = '24' or c.parent_id = '24')
ORDER BY by pd.products_name ASC

Categories