query with dependent subqueries too slow - php

Question

select mt.from_user, mt.to_user, mt.group_id, g.name, g.created_by as adminuser,
msg.*,
(
SELECT id
from messages
where t.thread_id = thread_id
and id NOT IN (
SELECT message_id from message_deleted
where user_id=275 and status='deleted' )
order by CreatedDate DESC
limit 1
) as msgid,
(
SELECT CreatedDate
from messages
where t.thread_id = thread_id
and id NOT IN (
SELECT message_id from message_deleted
where user_id=275 and status='deleted' )
order by CreatedDate DESC
limit 1
) as msgDate
from user_thread as t
left join message_thread as mt ON t.thread_id = mt.id
left join group_master as g ON mt.group_id = g.id
left join group_member as gm ON gm.group_id = g.id
left join messages as msg ON t.thread_id = msg.thread_id
where ( gm.user_id=275
or msg.from_id=275
or msg.to_id=275
)
and t.status = 'Active'
group by mt.id
order by msgDate DESC
This takes about 50 sec.
In above code, I have try to split above query and note that below subquery take too much time to execute. Can I convert subquery into join. please help me. I am stuck.please note that all tables which are joined are necessary.
(
SELECT id
from messages
where t.thread_id = thread_id
and id NOT IN (
SELECT message_id from message_deleted
where user_id=275 and status='deleted' )
order by CreatedDate DESC
limit 1
) as msgid,
(
SELECT CreatedDate
from messages
where t.thread_id = thread_id
and id NOT IN (
SELECT message_id from message_deleted
where user_id=275 and status='deleted' )
order by CreatedDate DESC
limit 1
) as msgDate

Answer 1

First, You are misusing a notorious MySQL extension to GROUP BY. This will probably cause your results to be unpredictable. Read this. https://dev.mysql.com/doc/refman/5.7/en/group-by-handling.html
Second, You have a couple of nested dependent subqueries. The first of them is this.
(select id
from messages
where t.thread_id = thread_id
and id NOT IN (select message_id
from message_deleted
where user_id=275
and status='deleted')
order by CreatedDate DESC limit 1) as msgid
Such nested dependent subqueries perform notoriously badly. They're even worse when they contain LIMIT clauses. Your route to fixing this is refactoring into an independent query and then JOINing it.
This may work as a replacement for the query to find the most recent undeleted message on the thread.
SELECT MAX(m.id) id, m.thread_id
FROM messages m
LEFT JOIN message_deleted d
ON m.id = d.id
AND d.user_id = 275
AND d.status = 'deleted'
WHERE d.id IS NULL
GROUP BY m.thread_id
This uses the LEFT JOIN .... IS NULL pattern in place of NOT IN. It's faster. It uses the MAX(id) method of finding the most recent row in a table in place of the ORDER BY CreatedDate DESC LIMIT 1 method, which is also much faster. It's good because it's guaranteed to generate either 0 or 1 row per value of thread_id. That means you can use it in a LEFT JOIN ... ON ... thread_id operation and not add any rows to your result set.
You can test this subquery by running it. Then you JOIN it, as if it were a table, to the rest of your query, something like this.
SELECT whatever,
q.id, r.CreatedDate
FROM whatever
LEFT JOIN (
SELECT MAX(m.id) id, m.thread_id
FROM messages m
LEFT JOIN message_deleted d
ON m.id = d.id
AND d.user_id = 275
AND d.status = 'deleted'
WHERE d.id IS NULL
GROUP BY m.thread_id
) q ON q.id = t.id
LEFT JOIN messages r ON r.id = q.id
The second LEFT JOIN operation here is used to retrieve the CreatedDate value of the newest undeleted message from the messages table.