Improving MySQL query performance - possible index problems - php

I've seen many questions like mine, but after reading them all I got rather confused.
To sum up - I've got a query that select products from a table and adds more information about them from other tables.
Query:
SELECT
p.product_id,
p.product_name,
p.product_seo_url,
p.product_second_name,
p.product_intro_plain,
p.product_price,
p.product_price_promo,
p.product_promo_expire_date,
p.product_views,
p.product_code,
p.product_exquisite,
p.product_rating,
p.product_votes,
p.product_date_added,
p.product_returned,
p.product_price_returned,
( SELECT gal.image_filelocation
FROM 3w_products_gallery gal
WHERE gal.product_id = p.product_id
ORDER BY show_order ASC
LIMIT 1 ) image_filelocation,
m.man_image_location,
m.man_name,
m.man_seo_url
FROM
3w_products p
LEFT JOIN 3w_manufacturers m
ON p.man_id = m.man_id
LEFT JOIN 3w_products_cat_rel pcr
ON p.product_id = pcr.product_id
WHERE
pcr.ctg_id = '19'
AND p.man_id = '190'
ORDER BY
p.product_id DESC
LIMIT
0, 24
The strange things that happen are that the query sometimes executes for 0.001 sec. and sometimes for 30+ seconds.
There is what EXPLAIN shows:
http://i.stack.imgur.com/ZNtBX.png
I assume the problem lies in the indexes of the tables. Can you tell me how to setup them?
Let me know if you need any more information about the tables or whatever!
Best,
Dimitar

IF your "ID" columns are actually numeric, remove the quotes around them implying strings... even though it would to an implied conversion. If numeric, keep it numeric.
As stated by another in the comments, your LEFT JOIN via the "pcr" alias with its criteria in the WHERE clause turns this into an inner join.
FROM
3w_products p
LEFT JOIN 3w_manufacturers m
ON p.man_id = m.man_id
LEFT JOIN 3w_products_cat_rel pcr
ON p.product_id = pcr.product_id
AND pcr.ctg_id = 19
WHERE
AND p.man_id = 190
Field-level queries can kill performance since the select for each field (your image location) is done once for every record. To at least help this performance, table 3w_products_gallery should have an index ON ( product_id, show_order )
Your main 3w_products table should have an index on (man_id, product_id )... The Man_ID to optimize the WHERE clause by manufacturer, but also the product ID to help optimize the ORDER BY criteria.
Your 3w_manufacturers table, I would suspect already has a valid index on (man_id), since it appears to be the primary key to the table.
Additionally, being web-based content, you might be better to DE-NORMALIZE your product table by adding one new column for "GalleryShowOrder". Then, add a trigger to your Gallery table that any insert or update will then push the first "showOrder" value back to the product table. This way, when you query, you can just add another join to that table on the product and KNOWN show order. If your gallery is returning 1000's of records, even though you are only limiting to 24, it still needs to get all record before the order by is applied. Thus 1000's of subqueries for each of the gallery images.
and your field selection would just become
gal.image_filelocation,
and your JOIN would add the following
LEFT JOIN 3w_products_gallery gal
on p.product_id = gal.product_id
AND p.GalleryShowOrder = gal.show_order

Related

MYSQL combining results into single item for display

I'm currently having an issue where I run a query on multiple tables getting the results, but they are all being considered independent. I've tried a couple ways of combining them, but because my SQL knowledge is limited I can't seem to get what I want to happen.
SELECT DISTINCT t.*, s.quantity, s.rrp, ts.thumbnail, ts.bigpic, t.rating
FROM tyres t
INNER JOIN stocklevels s
ON t.stockcode = s.stockcode
LEFT JOIN tyre_treads ts
ON t.treadid = ts.recid
LEFT JOIN reseller r
ON s.city=r.recid
WHERE s.quantity> 0 AND s.rrp > 0
I've tried adding GROUP BY t.recid and a couple other basic solutions but this doesn't seem to work. I've added a couple images which might help.
As you can see the bottom Toyo tyres are the same, just with varying cities and quantities.
Here they are on the website.
I'm wanting to combine they so that they say minimum 6 in stock and shows only once on the site.
As long as there is at least one column in your SQK resukt, which contains different values (like city in your example for "the same" tire, group by won't work. You must adapt your SQL statement in a way, that it only picks columns with the same values. Especially, you should remove the t.* from your sql and name all columns (You then will not need the distinct anymore).
Then, you sum over quantity to get the combined value for this column as wanted.
SELECT r.recid, sum(s.quantity), s.rrp, ts.thumbnail, ts.bigpic, t.rating
FROM tyres t
INNER JOIN stocklevels s
ON t.stockcode = s.stockcode
LEFT JOIN tyre_treads ts
ON t.treadid = ts.recid
LEFT JOIN reseller r
ON s.city=r.recid
WHERE s.quantity> 0 AND s.rrp > 0
GROUP BY recid

mysql query retrieve too slow from 10,000 of data (Query Optimizing)

10,000 of data in memberships, members and payments table. Retreiving the query is too slow, while searching a particular payment status in each members latest payment.
SELECT m.id AS member_id, m.full_name, m.unit, m.street, m.block, m.country, m.postal_code, cat . * , cat.id AS cat_id, mem.membership_num, mem.id AS membership_id
FROM memberships mem
LEFT JOIN category cat ON mem.category_id = cat.id
LEFT JOIN members m ON mem.member_id = m.id
WHERE m.id >0
AND m.status = 'active'
AND (
mem.category_id
BETWEEN 1
AND 11
)
AND (
SELECT p1.payment_status_id
FROM payments p1
WHERE p1.category_id = cat.id
AND p1.member_id = mem.member_id
AND p1.payment_status_id = '1'
LIMIT 1
) != ''
GROUP BY CONCAT( cat.id, '_', m.unit, '_', m.postal_code )
ORDER BY m.full_name ASC
LIMIT 0 , 25
EXPLAIN
Query Runs too slow 21.00sec to 99.00sec
ISSUE
The page is slowing down (3-5 mins instead of seconds) in mysql queries which has multiple joins and sub-queries to retrieve massive amount of data (~10,000) in tables.
SOLUTION - Used Index to the columns
Added indexes and done various search queries, It is retrieving better.
NOTES
Indexes are used to find rows with specific column values quickly. Without an index, MySQL must begin with the first row and then read through the entire table to find the relevant rows. The larger the table, the more this costs. If the table has an index for the columns in question, MySQL can quickly determine the position to seek to in the middle of the data file without having to look at all the data. This is much faster than reading every row sequentially.
The best way to improve the performance of SELECT operations is to create indexes on one or more of the columns that are tested in the query. The index entries act like pointers to the table rows, allowing the query to quickly determine which rows match a condition in the WHERE clause, and retrieve the other column values for those rows. All MySQL data types can be indexed.
eg: ALTER TABLE `memberships` ADD INDEX ( `category_id` ) ;
DrawBack: Indexes are something extra that you can enable on your MySQL tables to increase performance,but they do have some downsides. When you create a new index MySQL builds a separate block of information that needs to be updated every time there are changes made to the table. This means that if you are constantly updating, inserting and removing entries in your table this could have a negative impact on performance.
Tutorials mysql.com, howtoforge.com, tizag.com
THANKS
#venca #Boris #Raja Amer Khan and all
Thanks for all helping me to solve the issue.
First make sure you have created indexes for all columns involved in JOINS and WHERE clause. Also, if you have made composite indexes then make sure they are in same order as in your query. Try following and see if it makes any difference:
SELECT m.id AS member_id,
m.full_name,
m.unit,
m.street,
m.block,
m.country,
m.postal_code,
cat . * ,
cat.id AS cat_id,
mem.membership_num,
mem.id AS membership_id
FROM memberships mem
LEFT JOIN members m ON m.id = mem.member_id
LEFT JOIN category cat ON cat.id = mem.category_id
INNER JOIN payments p1 ON p1.category_id = mem.category_id
AND p1.member_id = mem.member_id
AND p1.payment_status_id = '1'
WHERE m.id > 0
AND mem.category_id BETWEEN 1 AND 11
AND m.status = 'active'
GROUP BY CONCAT( cat.id, '_', m.unit, '_', m.postal_code )
ORDER BY m.full_name ASC
LIMIT 0 , 25

php and sql commands crashing my server

I'm trying to query my database for all my products and when I do that it's crashing the server and I can't get the information
SELECT categories.category_id,
categories.category_name,
categories.category_name_fr,
subcategories.subcategory_id,
subcategories.subcategory_name,
subcategories.subcategory_name_fr,
suppliers.supplier_id,
suppliers.supplier_name,
products.product_id,
products.product_name,
products.product_name_fr,
products.product_url
FROM categories,
subcategories,
suppliers,
products
WHERE products.product_active = 1
ORDER BY categories.category_id ASC
I tried putting a LIMIT 0, 1 and it STILL crashed...
First, you should explicitly specify your joins and join conditions, as right now you are getting a Cartesian product of all the tables, which is likely not what you want.
You should list the products table first amongst the joined tables, as this is the table where your WHERE clause filtering is in effect. So something like:
SELECT categories.category_id, categories.category_name,
categories.category_name_fr, subcategories.subcategory_id,
subcategories.subcategory_name, subcategories.subcategory_name_fr,
suppliers.supplier_id, suppliers.supplier_name, products.product_id,
products.product_name, products.product_name_fr, products.product_url
FROM products
INNER JOIN categories ON products.[some field] = categories.[some field]
INNER JOIN subcategories ON categories.[some field] = subcategories.[some field]
INNER JOIN suppliers ON products.[some field] = suppliers.[some field]
WHERE products.product_active = 1
ORDER BY categories.category_id ASC
Second, you need to make sure you have an index on each of products.product_active and categories.category_id as well as any other columns used in the join conditions.
Third, just as an FYI, a LIMIT condition would do nothing to help the query execution time here as you still need to perform the join, the filter, and the sort before deciding to limit the result set to just a subset of the items. The only place where it would help would be in processing through the result set (which would obviously happen much quicker).

Multiple joins in a MySQL query giving incorrect results

I've got a large mysql query with 5 joins which may not seem efficient but I'm struggling to find a different solution which would work.
The views table is the main table here, because both clicks and conversions table rely on it via the token column(which is indexed and set as a foreign key in all tables).
The query:
SELECT
var.id,
var.disabled,
var.name,
var.updated,
var.cid,
var.outdated,
IF(var.type <> 0,'DL','LP') AS `type`,
COUNT(DISTINCT v.id) AS `views`,
COUNT(DISTINCT c.id) AS `clicks`,
COUNT(DISTINCT co.id) AS `conversions`,
SUM(tc.cost) AS `cost`,
SUM(cp.value) AS `revenue`
FROM variants AS var
LEFT JOIN views AS v ON v.vid = var.id
LEFT JOIN traffic_cost AS tc ON tc.id = v.source
LEFT JOIN clicks AS c ON c.token = v.token
LEFT JOIN conversions AS co ON co.token = v.token
LEFT JOIN c_profiles AS cp ON cp.id = co.profile
WHERE var.cid = 28
GROUP BY var.id
The results I'm getting are:
The problem is the revenue and cost results are too hight, because for views,clicks and impressions only the distinct rows are counted, but for revenue and cost for some reason(I would really appreciate an explanation here) all rows in all tables are taken into the result set.
I know this is a large query, but both clicks and conversions tables rely on the views table which is used for filtering the results e.g. views.country = 'uk'. I've tried doing 3 queries and merging them, but that didn't work(it gave me wrong results).
One more thing that I find weird is that if I remove the joins with clicks, conversions, c_profiles the costs column shows correct results.
Any help would be appreciated.
In the end I had to use 3 different queries and do a merge on them. Seemed like an overhead, but worked for me.

How can I join 3 tables with mysql & php?

I have a page that pulls the users Post,username,xbc/xlk tags etc which is perfect... BUT since I am pulling information from a MyBB bulletin board system, its quite different. When replying, people are are allowed to change the "Thread Subject" by simplying replying and changing it.
I dont want it to SHOW the changed subject title, just the original title of all posts in that thread.
By default it repies with "RE:thread title". They can easily edit this and it will show up in the "Subject" cell & people wont know which thread it was posted in because they changed their thread to when replying to the post.
So I just want to keep the orginial thread title when they are replying.
Make sense~??
Tables:mybb_users
Fields:uid,username
Tables:mybb_userfields
Fields:ufid
Tables:mybb_posts
Fields:pid,tid,replyto,subject,ufid,username,uid,message
Tables:mybb_threads
Fields:tid,fid,subject,uid,username,lastpost,lastposter,lastposteruid
I haev tried multiple queries with no success:
$result = mysql_query("
SELECT * FROM mybb_users
LEFT JOIN (mybb_posts, mybb_userfields, mybb_threads)
ON (
mybb_userfields.ufid=mybb_posts.uid
AND mybb_threads.tid=mybb_posts.tid
AND mybb_users.uid=mybb_userfields.ufid
)
WHERE mybb_posts.fid=42");
$result = mysql_query("
SELECT * FROM mybb_users
LEFT JOIN (mybb_posts, mybb_userfields, mybb_threads)
ON (
mybb_userfields.ufid=mybb_posts.uid
AND mybb_threads.tid=mybb_posts.tid
AND mybb_users.uid=mybb_posts.uid
)
WHERE mybb_threads.fid=42");
$result = mysql_query("
SELECT * FROM mybb_posts
LEFT JOIN (mybb_userfields, mybb_threads)
ON (
mybb_userfields.ufid=mybb_posts.uid
AND mybb_threads.tid=mybb_posts.tid
)
WHERE mybb_posts.fid=42");
Your syntax isn't appropriate for carrying out multiple LEFT JOINs. Each join needs its own ON clause.
SELECT
*
FROM
mybb_users
LEFT JOIN mybb_userfields ON mybb_users.uid = mybb_userfields.ufid
LEFT JOIN mybb_posts ON mybb_userfields.ufid = mybb_posts.uid
LEFT JOIN mybb_threads ON mybb_posts.tid = mybb_threads.tid
WHERE
mybb_posts.fid = 42
This query should give the results you want. But it may not be the most efficient query for getting those results. Check the output of EXPLAIN as part of testing, to make sure it is not using table scans or anything like that.
Do all of these joins need to be LEFT JOINs? LEFT JOIN forces MySQL to join the tables in the indicated order, rather than allowing the query optimiser to determine the best order in which to join them. That's why you might need to be careful about the query execution plan. The main difference between JOIN and LEFT JOIN as far as query output is concerned is that LEFT JOIN resultsets will contain at least one row for each row of the table on the left-hand side of the join, whereas a regular JOIN will not contain a row if there aren't matches on the right-hand side of the join.
Edit: Also, you say that "I don't want it to SHOW the changed subject title, just the original title of all posts in that thread." This suggests that you only want a subset of the columns from these tables, in which case SELECT * is inappropriate.

Categories