I have two tables posts and followings
posts (id,userid,post,timestamp) 30 000 rows
and
followings(id_me,userid) 90 000 rows
I want to get lattest 10 posts form posts table based on the people i follow and my posts
SELECT p.*
FROM posts as p INNER JOIN
followings as f
ON (f.id_me=(my user id) AND p.userid=f.userid )
OR
p.userid=(my user id)
ORDER BY id DESC LIMIT 10
But it takes about 10-15 seconds to return. Thanks in advance!
First, remove the filter from the join clause, let the join just correlate the joining tables.
(
SELECT p.*
FROM posts as p
INNER JOIN followings as f ON p.userid=f.userid
where f.id_me=(my user id)
UNION
SELECT p.*
FROM posts as p
where p.userid=(my user id)
)
ORDER BY id DESC LIMIT 10
second, verify your indexes if that ids got no indexes it ill perform a full table scan for each cartesian product of both tables (30k x 90k =~ 3700k pairs being compared)
third, if you don't follow yourself you need a union from post you are following and your posts
Using an OR in SQL is a performance killer, try this:
SELEC p.*
FROM posts as p INNER JOIN
followings as f
ON (f.id_me=(my user id) AND p.userid IN (f.userid,(my user id)))
ORDER BY id DESC LIMIT 10
Do this query using union:
(SELECT p.*
FROM posts p INNER JOIN
followings f
ON (f.id_me=(my user id) AND p.userid=f.userid
)
union
(select p.*
from posts p
where p.userid=(my user id)
)
ORDER BY id DESC
LIMIT 10
If the two conditions never overlap, then use union all instead.
An OR condition like that prevents the query optimizer from making use of indexes. Use a UNION instead:
SELECT *
FROM (SELECT p.*
FROM posts as p
INNER JOIN followings as f
ON f.id_me=(my user id) AND p.userid=f.userid
UNION
SELECT *
FROM posts
WHERE userid = (my user id)) u
ORDER BY id DESC
LIMIT 10
It might be just me but I think your WHERE clause is in an inefficient location:
SELECT
p.*
FROM
posts p
INNER JOIN
followings f
ON p.userid=f.userid
WHERE
MyUserID IN (p.userid, f.id_me)
ORDER BY
id DESC
LIMIT
10
I read in comments that you have the required indexes. The problem is the query. Combining OR with a JOIN confuses the poor and (often) dumb optimizer. The LIMIT 10 should be helpful but the optimizer is not (yet) smart enough to make the best plan.
Try this query:
( SELECT p.*
FROM posts AS p
JOIN followings AS f
ON f.id_me = (my_user_id)
AND p.userid = f.userid
ORDER BY p.id DESC
LIMIT 10
)
UNION ALL
( SELECT p.*
FROM posts AS p
WHERE p.userid = (my_user_id)
ORDER BY p.id DESC
LIMIT 10
) AS x
ORDER BY id DESC
LIMIT 10 ;
Related
I have a big data problem with MySQL.
I have:
a users table with 59033 rows, and
a user_notes table with 8753 rows.
But when I search which users have user note in some dates.
My query like this :
SELECT u.*, rep.name as rep_name FROM users as u
LEFT JOIN users as rep on rep.id = u.add_user
LEFT JOIN authorization on authorization.id = u.authorization
LEFT JOIN user_situation_list on user_situation_list.user_situation_id = u.user_situation
WHERE
EXISTS(
select * from user_notes
where user_notes.note_user_id = u.id AND user_notes.create_date
BETWEEN "2017-10-20" AND "2017-10-22"
)
ORDER BY u.lp_modify_date DESC, u.id DESC
Turn it around -- find the ids first; deal with the joins later.
SELECT u.*,
( SELECT rep.name
FROM users AS rep
WHERE rep.id = u.add_user ) AS rep_name
FROM (
SELECT DISTINCT note_user_id
FROM user_notes
WHERE create_date >= "2017-10-20"
AND create_date < "2017-10-20" + INTERVAL 3 DAY
) AS un
JOIN users AS u ON u.id = un.note_user_id
ORDER BY lp_modify_date DESC, id DESC
Notes
No GROUP BY needed;
2 tables seem to be unused; I removed them;
I changed the date range;
User notes needs INDEX(create_date, note_user_id);
Notice how I turned a LEFT JOIN into a subquery in the SELECT list.
If there can be multiple rep_names, then the original query is "wrong" in that the GROUP BY will pick a random name. My Answer can be 'fixed' by changing rep.name to one of these:
MAX(rep.name) -- deliver only one; arbitrarily the max
GROUP_CONCAT(rep.name) -- deliver a commalist of names
Rewriting your query to use a JOIN rather than an EXISTS check in the where should speed it up. If you then group the results by the user.id it should give you the same result:
SELECT u.*, rep.name as rep_name FROM users as u
LEFT JOIN users as rep on rep.id = u.add_user
LEFT JOIN authorization on authorization.id = u.authorization
LEFT JOIN user_situation_list on user_situation_list.user_situation_id = u.user_situation
JOIN user_notes AS un
ON un.note_user_id
AND un.create_date BETWEEN "2017-10-20" AND "2017-10-22"
GROUP BY u.id
ORDER BY u.lp_modify_date DESC, u.id DESC
I am working on an sql query where I have three tables posts, users and comments. I want to display all posts with its users and number of comments on this. I have following query but it is giving me wrong result:
SELECT
c.userid, count(c.userid), p.postid
FROM comments c, posts p
where c.userid = p.userid group by c.userid
In addition to above query I also require firstname and lastname from users table.
Try something like this,
SELECT
u.userid, u.firstname, u.lastname,p.post, p.postid,
count(c.userid) totalcoments -- may be c.commentid
FROM users u
JOIN posts p ON p.userid=u.userid
LEFT JOIN comments c OM c.postid=p.postid
GROUP BY u.userid, u.firstname, u.lastname,p.post, p.postid
Try something like the following:
SELECT
postid
, p.userid
, COALESCE((
SELECT COUNT( * ) FROM comments WHERE postid = p.id
), 0 ) AS cnt_postid
, COALESCE( ( SELECT CONCAT( firstname, ' ', lastname ) FROM users WHERE userid = p.userid ), 'N/A' ) AS NAME
FROM posts p
LEFT JOIN comments c ON c.postid = p.id
GROUP BY postid
ORDER BY postid
you are probably getting the same amount of count because you not using a group by.
GROUP BY must always be used when using aggregate function. What the group by does is that it will select all unique posts and the the count will return the number of users for that one unique post
I'm wondering about these queries and I got a work that want to list all users in table user and count for their post photo and video. and can choose to view in sort by these count limit by ASC or DESC.
I've tried them both but see that sub-query is fast than join. i want to know the different between these queries. Why sometimes join is slower that a sub-query. is join best for only two tables? Is this both best for my work? or you can suggest another better solution.
SUB-QUERY
select
user.*,
(select count(*) from post where post.userid = user.id) postCount,
(select count(*) from photo where photo.userid = user.id) photoCount,
(select count(*) from video where video.userid = user.id) videoCount
from user order by postCOunt desc limit $starrow 20
JOIN
SELECT u.id,
COUNT(DISTINCT p.id) AS postCount,
COUNT(DISTINCT ph.id) AS photoCount,
COUNT(DISTINCT v.id) AS videoCount
FROM user u
LEFT JOIN post p
ON p.userid = u.id
LEFT JOIN photo ph
ON ph.userid = u.id
LEFT JOIN video v
ON v.userid = u.id
GROUP BY u.id
ORDER BY postCount LIMIT $startrow 20
Example in HTML page that order by postCount DESC and have paging.
userid postCount photoCount videCount
1 34 5 4
2 30 12 2
3 21 5 6
4 15 8 4
5 12 15 9
6 8 3 10
.. .. .. ..
You can try it this way with JOIN
SELECT u.id, postCount, photoCount, videoCount
FROM user u LEFT JOIN
(
SELECT userid, COUNT(*) postCount
FROM post
GROUP BY userid
) p ON p.userid = u.id LEFT JOIN
(
SELECT userid, COUNT(*) photoCount
FROM photo
GROUP BY userid
) ph ON ph.userid = u.id LEFT JOIN
(
SELECT userid, COUNT(*) videoCount
FROM video
GROUP BY userid
) v ON v.userid = u.id
ORDER BY postCount
LIMIT $startrow, 20
Let's take 3 tables that has all tons of rows:
TABLE Posts
PostPID
PostUID
PostText
TABLE Users
UserUID
UserName
TABLE Favorites
FavoriteUID
FavoritePID
Now, in order to get all the recent posts I perform a query such as:
SELECT p.PostPID, p.PostUID, p.PostText, u.UserUID, u.UserName
FROM Posts AS p
JOIN Users AS u
ON p.PostUID = u.UserUID
ORDER BY p.PostPID DESC
LIMIT 0, 30
Which works fine. Now I was wondering, how could I get only the posts a certain UserUID prefers? So only the one with FavoriteUID = UserUID = X?
You could use a subquery.
...
Where p.PostUID in (select f.FavoritePID from Favorite f where f.FavoriteUID = UserUID)
...
second join will do the same
SELECT
p.PostPID, p.PostUID, p.PostText, u.UserUID, u.UserName
FROM
Posts AS p
JOIN
Users AS u ON p.PostUID = u.UserUID
Join
Favorites as f on f.FavoriteUID = u.UserUID and f.FavoritePID=p.PostPID
ORDER
BY p.PostPID DESC
LIMIT 0, 30
I'm not quite sure if this is the right approach, this is my situation:
I'm currently trying to select 15 galleries and then left join it with the user table through the id but I also want to select one random picture from each gallery however from what I know you can't limit the left join (picture) to only pick up one random picture without doing a subquery.
Here is what I got so far but its not working as it should:
SELECT galleries.id, galleries.name, users.username, pictures.url
FROM galleries
LEFT JOIN users ON users.id = galleries.user_id
LEFT JOIN pictures ON (
SELECT pictures.url
FROM pictures
WHERE pictures.gallery_id = galleries.id
ORDER BY RAND()
LIMIT 1)
WHERE active = 1
ORDER BY RAND()
LIMIT 15
I also tried to do this with Active Record but I got stuck after doing two left joins, is it possible to do get a subquery in here:
$this->db->select('galleries.id, galleries.name, users.id as user_id, users.username');
$this->db->from('galleries');
$this->db->join('users', 'users.id = galleries.user_id','left');
$this->db->join('pictures','pictures.gallery_id = galleries.id AND','left');
$this->db->where('active',1);
I hope its not to messy but I'm really starting to get confusing by all the sql queries..
Edit:
Active Record with CodeIgniter
You could fetch a random picture in a subquery:
select
g.name, u.username,
(select url from pictures p where p.gallery_id = g.gallery_id
order by rand() limit 1) as url
from galleries g
left join users u on g.user_id = u.id
where g.active = 1
Based on your comment, you could select a picture for each gallery in a subquery. This is assuming the picture table has an ID column.
select
g.name, u.username, p.url, p.name
from (
select id, user_id, name,
(select id from pictures p
where p.gallery_id = g.gallery_id
order by rand() limit 1) as samplepictureid
from galleries
where g.active = 1
) g
left join users u on g.user_id = u.id
left join pictures p on p.id = g.samplepictureid
SELECT
g.id,
g.name,
u.username,
p.url
FROM
galleries g
INNER JOIN (SELECT DISTINCT
gallery_id,
(SELECT url FROM pictures ss WHERE ss.gallery_id = s.gallery_id
ORDER BY RAND() LIMIT 1) AS url
FROM
pictures s) p ON
g.id = p.gallery_id
LEFT OUTER JOIN users u ON
g.user_id = u.id
WHERE
g.active = 1
This query will go out and select a gallery, then it will find any gallery with a picture (if you want to return galleries without a picture, change INNER JOIN to LEFT OUTER JOIN, and you'll be fine). After that, it joins it up with users. Now, of course, this puppy is going to return every frigging gallery for however many users you have (hoorah!). You may want to limit the user in the WHERE clause (e.g.-WHERE u.id = 123). Otherwise, you're going to get more results than you'd expect. That, or do an INNER JOIN on it.