How to optimize query with table scans? - php

Question

This is by far the slowest query in my web application.
SELECT prof.user_id AS userId,
prof.first_name AS first,
prof.last_name AS last,
prof.birthdate,
prof.class_string AS classes,
prof.city,
prof.country,
prof.state,
prof.images,
prof.videos,
u.username,
u.avatar,
(SELECT Count(*)
FROM company_member_sponsorship
WHERE member_id = prof.user_id
AND status = 'sponsored') AS sponsor_count,
(SELECT Count(*)
FROM member_schedules
WHERE user_id = prof.user_id) AS sched_count
FROM member_profiles prof
LEFT JOIN users u
ON u.id = prof.user_id
ORDER BY ( prof.images + prof.videos * 5 + (
CASE
WHEN prof.expire_date > :time THEN 50
ELSE 0
end ) + sponsor_count * 20 + sched_count * 4
) DESC,
prof.last_name ASC
LIMIT :start, :records
Everything else on the site takes less than a second to load even with lots of queries happening on all levels. This one takes about 3-4 seconds.
It's obviously the table scans that are causing the slowdown. I can understand why; the first table has 50,000+ rows, the second 160,000+ rows.
Is there any way I can optimize this query to make it go faster?
If worse comes to worst I can always go through my code and maintain a tally for sponsorships and events in the profile table like I do for images and videos though I'd like to avoid it.
EDIT: I added the results of an EXPLAIN on the query.
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY prof ALL NULL NULL NULL NULL 44377 Using temporary; Using filesort
1 PRIMARY u eq_ref PRIMARY PRIMARY 3 mxsponsor.prof.user_id 1
3 DEPENDENT SUBQUERY member_schedules ref user_id user_id 3 mxsponsor.prof.user_id 6 Using index
2 DEPENDENT SUBQUERY company_member_sponsorship ref member_id member_id 3 mxsponsor.prof.user_id 2 Using where; Using index
EDIT2:
I ended up dealing with the problem by maintaining a count in the member profile. Wherever sponsorships/events are added/deleted I just invoke a function that scans the sponsorship/events table and updates the count for that member. There might still be a way to optimize a query like this, but we're publishing this site rather soon so I'm going with the quick and dirty solution for now.

Answer 1

Not guaranteed to work, but try using join and group by rather than inner selects:
SELECT prof.user_id AS userId,
prof.first_name AS first,
prof.last_name AS last,
prof.birthdate,
prof.class_string AS classes,
prof.city,
prof.country,
prof.state,
prof.images,
prof.videos,
u.username,
u.avatar,
Count(cms.id) AS sponsor_count,
Count(ms.id) AS sched_count
FROM member_profiles prof
LEFT JOIN users u
ON u.id = prof.user_id
LEFT JOIN company_member_sponsorship cms
ON cms.member_id = prof.user_id
AND cms.status = 'sponsored'
LEFT JOIN member_schedules ms
ON ms.user_id = prof.user_id
GROUP BY u.id
ORDER BY ( prof.images + prof.videos * 5 + (
CASE
WHEN prof.expire_date > :time THEN 50
ELSE 0
end ) + sponsor_count * 20 + sched_count * 4
) DESC,
prof.last_name ASC
LIMIT :start, :records
If that's not any better, a explain of that query would help.