How to speed up this MySQL query? Latest messages query - php

i need speed up this query for list latest messages. This query running too long (eg. 10 seconds ...)
SELECT datas.uid,
datas.message,
datas.date,
CONCAT(conv.first_name, ' ', conv.last_name) AS conversation_name
FROM (SELECT m.message_id,
m.message,
IF (m.from_uid = 1, m.to_uid, m.from_uid) AS uid,
m.readed,
m.sended AS `date`
FROM users u
LEFT JOIN messages m
ON m.from_uid = u.user_id
WHERE m.message_id IN (SELECT MAX(message_id)
FROM messages
WHERE to_uid = 1
OR from_uid = 1
GROUP BY LEAST(from_uid, to_uid),
GREATEST(from_uid, to_uid))) datas
LEFT JOIN users conv
ON conv.user_id = datas.uid
ORDER BY datas.date DESC
LIMIT 5
This query use 2 tables (users and messages).
Table users:
user_id (primary, autoincrement)
login
pass
first_name
last_name
....
Table messages:
message_id (primary, autoincrement)
from_uid (sender message, reference to table users -> user_id)
to_uid (receiver message, reference to table users -> user_id)
sended (timestamp)
message (varchar)
EDIT
I added indexes to messages:
- from_uid
- to_uid
- sended
and this is without efect...

Try creating indexed on the id's you're checking.
In this case you might want to create an index on: conv.user_id, datas.uid, m.message_id, messages.to_uid, messages.from_uid, and datas.date might be a good idea as well, since you're sorting on that.

Add indexes on from_uid and to_uid to speed up the SELECT MAX(message_id) subquery. Otherwise, it has to do a full scan of the table.

I would try and remove the sub query(s)
You can clean it up a bit by joining against users twice when getting the user details from te from and to uid. This way the joins can use indexes effectively. Then just use IF in the SELECT to decide which one to return:-
SELECT IF (m.from_uid = 1, m.to_uid, m.from_uid) AS uid,
m.message,
m.sended AS `date`,
IF (m.from_uid = 1, CONCAT(to_conv.first_name, ' ', to_conv.last_name), CONCAT(from_conv.first_name, ' ', from_conv.last_name)) AS conversation_name
FROM users u
LEFT JOIN messages m
ON m.from_uid = u.user_id
LEFT JOIN users to_conv
ON to_conv.user_id = m.to_uid
LEFT JOIN users from_conv
ON from_conv.user_id = m.from_uid
WHERE m.message_id IN
(
SELECT MAX(message_id)
FROM messages
WHERE to_uid = 1
OR from_uid = 1
GROUP BY LEAST(from_uid, to_uid),
GREATEST(from_uid, to_uid)
)
ORDER BY date DESC
LIMIT 5
The sub query to check the message id is a bit more difficult to remove. Generally IN performs badly. Might be worth changing it to use EXISTS (although this will require the check in the HAVING clause which might not be good)
SELECT IF (m.from_uid = 1, m.to_uid, m.from_uid) AS uid,
m.message,
m.sended AS `date`,
IF (m.from_uid = 1, CONCAT(to_conv.first_name, ' ', to_conv.last_name), CONCAT(from_conv.first_name, ' ', from_conv.last_name)) AS conversation_name
FROM users u
LEFT JOIN messages m
ON m.from_uid = u.user_id
LEFT JOIN users to_conv
ON to_conv.user_id = m.to_uid
LEFT JOIN users from_conv
ON from_conv.user_id = m.from_uid
WHERE EXISTS
(
SELECT MAX(message_id) AS max_message_id
FROM messages
WHERE to_uid = 1
OR from_uid = 1
GROUP BY LEAST(from_uid, to_uid), GREATEST(from_uid, to_uid)
HAVING m.message_id = max_message_id
)
ORDER BY date DESC
LIMIT 5

Related

get unique records between two fields from mysql in php

I have a table where my private message are stored which I done with my friends.
I have table structure as shown in image
Suppose my logged user_id is 1, Now I want show his last five unique conversations with last message of each conversation.
Not very elegant but working:
SELECT m1.*
FROM private_msgs m1 LEFT OUTER JOIN private_msgs m2
ON (m1.message_from_uid=m2.message_from_uid AND m1.message_to_uid=m2.message_to_uid AND m1.message_time<m2.message_time)
WHERE m2.message_id IS NULL AND m1.message_from_uid=$uid
ORDER BY message_time DESC;
Add limit to 5 or whatever you want
EDIT. Sorry I didn't notice that UID can be as sender or receiver. Can you publish table with sample data for example here: http://sqlfiddle.com?
Anyway that should to the trick:
WHERE m2.message_id IS NULL AND (m1.message_from_uid=$uid OR m1.message_to_uid=$uid)
Here it is, long and ugly. You can replace temporary tables as subqueries in one big final query. Explanations are at the end.
CREATE TEMPORARY TABLE IF NOT EXISTS t1 AS
(SELECT message_id, message_from_uid, message_to_uid, message_time
FROM private_msgs
WHERE message_from_uid=$UID)
UNION
(SELECT message_id, message_to_uid, message_from_uid, message_time
FROM private_msgs
WHERE message_to_uid=$UID);
CREATE TEMPORARY TABLE IF NOT EXISTS t2 AS SELECT * FROM t1;
SELECT m.*
FROM (t1 LEFT OUTER JOIN t2
ON (t1.message_from_uid=t2.message_from_uid AND t1.message_to_uid=t2.message_to_uid AND t1.message_time<t2.message_time))
JOIN private_msgs m ON t1.message_id=m.message_id
WHERE t2.message_id IS NULL AND t1.message_from_uid=$UID
ORDER BY t1.message_time DESC;
DROP TABLE IF EXISTS t1;
DROP TABLE IF EXISTS t2;
t1 temporary table takes all messages where $UID is a sender and union them with messages where $UID is receiver, but puts $UID as sender. In t1 doesn't matter who is sender/receiver. What does matter that these are conversations between $UID and someone else.
t2 is copy of t1, because in MySQL you can't join temporary table with itself.
And finally we do the query. We join t1 and t2, take only newest messages in every conversation and sort it from latest to oldest. As in t1 and t2 sender and receiver are mixed we do the last join with private_msg to get original data.
PS. Sorry for my english, I'm not native speaker.
Found among my old code:
select m.*
from private_messages m
join user_table s ON m.sender_id = s.id
join user_table r ON m.receiver_id = r.id
join (select if(sender_id = $UID, receiver_id, sender_id) as user_id_other,
max(posted_at) AS date_time_max from private_messages
where (sender_id = $UID OR receiver_id = $UID)
group by if(sender_id=$UID, receiver_id, sender_id)
) AS t
on if(m.sender_id = $UID, m.receiver_id, m.sender_id) = user_id_other
AND m.posted_at=date_time_max
where m.sender_id = $UID OR m.receiver_id = $UID
order by m.posted_at desc
Not responsible for result, but this can help. By itself variables must be replaced, I did not modify the code very much, as you can see. But this piece of... code... worked.
In your case smths like:
select m.*
from YOUR_MESSAGES_TABLE m
--- OPTIONAL: join YOUR_USER_TABLE s ON m.message_from_uid = s.id
--- OPTIONAL: join YOUR_USER_TABLE r ON m.message_to_uid = r.id
join (select if(message_from_uid = $YOUR_USER_ID, message_to_uid, message_from_uid) as user_id_other,
max(message_time) AS date_time_max from YOUR_MESSAGES_TABLE
where (message_from_uid = $YOUR_USER_ID OR message_to_uid = $YOUR_USER_ID)
group by if(message_from_uid=$YOUR_USER_ID, message_to_uid, message_from_uid)
) AS t
on if(m.message_from_uid = $UID, m.message_to_uid, m.message_from_uid) = user_id_other
AND m.message_time=date_time_max
where m.message_from_uid = $UID OR m.message_to_uid = $UID
order by m.message_time desc limit 5
Test: http://sqlfiddle.com/#!9/f66fd8/4

EXISTS query optimization on mysql query

I have a big data problem with MySQL.
I have:
a users table with 59033 rows, and
a user_notes table with 8753 rows.
But when I search which users have user note in some dates.
My query like this :
SELECT u.*, rep.name as rep_name FROM users as u
LEFT JOIN users as rep on rep.id = u.add_user
LEFT JOIN authorization on authorization.id = u.authorization
LEFT JOIN user_situation_list on user_situation_list.user_situation_id = u.user_situation
WHERE
EXISTS(
select * from user_notes
where user_notes.note_user_id = u.id AND user_notes.create_date
BETWEEN "2017-10-20" AND "2017-10-22"
)
ORDER BY u.lp_modify_date DESC, u.id DESC
Turn it around -- find the ids first; deal with the joins later.
SELECT u.*,
( SELECT rep.name
FROM users AS rep
WHERE rep.id = u.add_user ) AS rep_name
FROM (
SELECT DISTINCT note_user_id
FROM user_notes
WHERE create_date >= "2017-10-20"
AND create_date < "2017-10-20" + INTERVAL 3 DAY
) AS un
JOIN users AS u ON u.id = un.note_user_id
ORDER BY lp_modify_date DESC, id DESC
Notes
No GROUP BY needed;
2 tables seem to be unused; I removed them;
I changed the date range;
User notes needs INDEX(create_date, note_user_id);
Notice how I turned a LEFT JOIN into a subquery in the SELECT list.
If there can be multiple rep_names, then the original query is "wrong" in that the GROUP BY will pick a random name. My Answer can be 'fixed' by changing rep.name to one of these:
MAX(rep.name) -- deliver only one; arbitrarily the max
GROUP_CONCAT(rep.name) -- deliver a commalist of names
Rewriting your query to use a JOIN rather than an EXISTS check in the where should speed it up. If you then group the results by the user.id it should give you the same result:
SELECT u.*, rep.name as rep_name FROM users as u
LEFT JOIN users as rep on rep.id = u.add_user
LEFT JOIN authorization on authorization.id = u.authorization
LEFT JOIN user_situation_list on user_situation_list.user_situation_id = u.user_situation
JOIN user_notes AS un
ON un.note_user_id
AND un.create_date BETWEEN "2017-10-20" AND "2017-10-22"
GROUP BY u.id
ORDER BY u.lp_modify_date DESC, u.id DESC

how to get latest messages from all users

I have two tables one is for users and other is for messages i want the latest messages of every user who sent the message to user 1 my code is below
SELECT u.profile_pic
, u.username
, u.firstname
, u.lastname
, m.message_from
, m.message_body
FROM user u
JOIN messages m
ON m.message_from = u.user_no
WHERE m.message_to = $userno
ORDER
BY m.sent_time DESC
It would be easier to give you an answer if you could include your table structures (is there a unique id on the messages table?). But from the information in your question, this should work:
SELECT
u.profile_pic,
u.username,
u.firstname,
u.llastname,
m.message_from,
m2.message_body
FROM user u
JOIN (SELECT message_from,message_to,max(sent_time)
FROM messages
GROUP BY message_from,message_to) m
ON user.user_no = messages.message_from
AND messages.message_to = '$userno'
JOIN messages m2 ON m.message_from = m2.message_from
AND m.message_to = m2.message_to
AND m.sent_time = m2.sent_time
Basically, the subselect will pull out all of the latest messages from one user to another, which you then join against to filter out all the others.
Use LEFT JOIN to message table on same user and newer sent_time. The latest of each user is the one that doesn't have such message:
SELECT profile_pic,username,firstname,lastname,messages.message_from,messages.message_body
FROM user
JOIN messages ON user.user_no = messages.message_from
LEFT JOIN messages AS newermessages ON user.user_no = newermessages.message_from AND messages.sent_time < newermessages.sent_time
WHERE messages.message_to = '$userno'
AND newermessages.id IS NULL
ORDER BY messages.sent_time DESC
The latest message from each user:
SELECT a.*
FROM messages a
JOIN
( SELECT message_from
, MAX(sent_time) max_sent_time
FROM messages
GROUP
BY message_from
) b
ON b.message_from = a.message_from
AND b.max_sent_time = a.sent_time;
The remainder of this problem has been left as an exercise for the reader.
Try adding the field of the ORDER BY clause into your selection:
SELECT u.profile_pic
, u.username
, u.firstname
, u.lastname
, m.message_from
, m.message_body
, m.sent_time
FROM user u
JOIN messages m
ON m.message_from = u.user_no
WHERE m.message_to = $userno
ORDER
BY m.sent_time DESC

Mysql - how to get data for two foreign key columns of same table

I have two tables in my database
Table: users with columns: id, username, email etcs
Table: users_messages with columns: id, to, from, messages, sent_time
Columns to and from are keys users from users table. Issue is that I need to:
"select all users to and from username and message".
For instance if table row is like below:
From, To, Message
11, 14, Hi, How are you
14, 11, Hello,
and if 11 is john and 14 is mark, then I need output like this:
From, To, Message
john, mark, Hi, How are you
mark, john, Hello,
With my query I can only get username for from column or to column but not for both. How can I perform a query that gives me from and to both usernames?
The following query gives me empty result:
SELECT u.username, m.message, m.from, m.to
FROM `users_messages` m, `users` u
where
m.from = u.id and m.to = u.id
LIMIT 0 , 30
I have tried other queries, but not able to get the desired output. Do I need a self join as well for messages table?
SELECT u1.username AS "sender", u2.username AS "recipient", m.message, m.from, m.to
FROM `users_messages` m
LEFT JOIN `users` u1 ON (u1.id = m.from)
LEFT JOIN `users` u2 ON (u2.id = m.to)
WHERE m.from = XX OR m.to = XX
try this
select u.`name` as `FROM`, u2.`name` as `TO`, `Message`
from users_messages um
inner join users u On um.`From` = u.id
inner join users u2 On um.`To` = u2.id
LIMIT 0 , 30
DEMO HERE
if you want to get specified messages from one user add this
WHERE u.`name` = 'mark' // to get messages FROM mark
or this
WHERE u2.`name` = 'mark' // to get messages TO mark
Yes self join can help you achieve the desired output :
SELECT u1.username, m.message, u2.username
FROM users_messages m, users u1, users u2
where
m.from = u1.id and m.to = u2.id
LIMIT 0 , 30
Assuming : from and to will not have null values

mysql: linking 2 tables with 2 different fields

I have a user table, e.g.
userId
userName
and I have a message table, e.g.
messageId
messageToId
messageFromId
messageContent
I am trying to make a query to pull a message, but also get the user names from the user table based on the messageToId and messageFromId.
I have done this before with only 1 field between tables, e.g.
SELECT message.*, user.userName
FROM message, user
WHERE user.userId = message.messageToId
AND messageId = (whatever)
But I am having trouble with 2 links.
I want the result as follows:
messageId
messageToId
toUserName
messageFromId
fromUserName
messageContent
Any help would be much appreciated, or if someone had another way of attempting a private message system with PHP/MySQL.
You just have to use joins and different table aliases:
SELECT m.*, u1.userName AS toUserName, u2.username AS fromUserName
FROM message m INNER JOIN user u1 ON m.messageToId = u1.userId
INNER JOIN user u2 ON m.messageFromId = u2.userId
WHERE messageId = "XXX";
You need to use a join from to achieve this:
SELECT `m`.*,
`to`.`userName` AS `to`,
`from`.`userName` AS `from`,
FROM `message` `m`
JOIN `user` `to` ON `m`.`messageToId` = `to`.`userId`
JOIN `user` `from` ON `m`.`messageFromId` = `from`.`userId`
WHERE `m`.`messageId` = 1
So you join against the user table twice to get both users for a particular message. To do this you need to use table aliases as I have done with to and from so that you distinguish between them.
I have also used a field alias to get their usernames separately eg:
`to`.`username` AS `from`
Will this work?
SELECT b.userName AS author, c.userName AS reciever, a.messageId, a.messageContent FROM message a JOIN user b ON a.messageFromId = b.userId JOIN user c ON a.messageToId = c.userId

Categories