I have below query, which I believe I have optimized as much as I possible could:
SELECT
c.forum_id as category_id,
c.forum_name as category_name,
t.forum_id as id,
t.forum_name as name,
t.forum_desc as description,
(SELECT COUNT(*) FROM forum_topics WHERE forum_id=t.forum_id AND topic_deleted=0) as topics_count,
(SELECT COUNT(*) FROM forum_posts WHERE forum_id=t.forum_id AND post_deleted=0) as posts_count,
(SELECT COUNT(*) FROM forum_posts WHERE topic_id=lp.topic_id AND post_deleted=0) as last_post_count,
lp.topic_id as last_post_topic_id,
lp.topic_title as last_post_topic_title,
lp.post_time as last_post_time,
lp.username as last_post_username,
lp.avatar
FROM forum_cats as t
JOIN forum_cats as c on c.forum_id = t.forum_type_id
left join (
SELECT
ft.topic_id,
ft.title as topic_title,
tmp.post_time,
u.username,
u.avatar,
fp.forum_id
FROM
forum_posts fp
join forum_topics ft on ft.topic_id = fp.topic_id
join users u on u.id = fp.userid
join (
select forum_id, max(`post_time`) `post_time`
from forum_posts fp
where fp.post_deleted = 0
group by forum_id
) as tmp on (fp.forum_id = tmp.forum_id and fp.post_time = tmp.post_time)
where post_deleted = 0 and ft.topic_deleted = 0
) as lp on lp.forum_id = t.forum_id
where t.forum_active = 1 and c.forum_active = 1
order by category_id, t.forum_id
This is the stats for each forum:
forum_cats has 20 rows
forum_topics has 900 rows
forum_posts has 9000 rows
users has 18000 rows
I have added index on all the columns that is being selected in above query. Yet it still takes more than 2 seconds to execute this.
What am I missing here?
Won't topics_count, posts_count & last_post_count be the same for every row? Wouldn't it be more efficient to pull these values separately, rather than make the same 3 calculations for every row you pull?
Related
SELECT posts, its last 3 comments and count of all comments
My code:
SELECT p.*, u.id, u.username username, u.usersurname usersurname, u.usermainphoto userphoto, GROUP_CONCAT(c.text SEPARATOR 'a!k#h#md%o^v&') commenttext,GROUP_CONCAT(c.likes SEPARATOR '-') commentlikes,GROUP_CONCAT(c.dislikes SEPARATOR '-') commentdislikes, GROUP_CONCAT(c.commentdate) commentdate, GROUP_CONCAT(u2.username) commentauthorname, GROUP_CONCAT(c.anonim) commentanonym, GROUP_CONCAT(c.id) commentid, GROUP_CONCAT(u2.id) commentauthorid, GROUP_CONCAT(u2.usersurname) commentauthorsurname, GROUP_CONCAT(u2.usermainphoto) commentauthorphoto, GROUP_CONCAT(c.commentphotoid) commentphotoid
FROM posts p
LEFT JOIN comments c ON c.post = p.postid AND c.commentdel=0
LEFT JOIN users u ON u.id = p.postauthorid
LEFT JOIN users u2 ON u2.id = c.author
WHERE p.postwallid = :id AND p.postdel=0
GROUP BY postid
ORDER BY postid DESC
it gives me all comments, but I need only 3
You need a timestamp or counter of something to determine which comments are the three you want. Add that column name between the angle brackets below.
SELECT p.*, u.id, u.username username, u.usersurname usersurname, u.usermainphoto userphoto, GROUP_CONCAT(c.text SEPARATOR 'a!k#h#md%o^v&') commenttext,GROUP_CONCAT(c.likes SEPARATOR '-') commentlikes,GROUP_CONCAT(c.dislikes SEPARATOR '-') commentdislikes, GROUP_CONCAT(c.commentdate) commentdate, GROUP_CONCAT(u2.username) commentauthorname, GROUP_CONCAT(c.anonim) commentanonym, GROUP_CONCAT(c.id) commentid, GROUP_CONCAT(u2.id) commentauthorid, GROUP_CONCAT(u2.usersurname) commentauthorsurname, GROUP_CONCAT(u2.usermainphoto) commentauthorphoto, GROUP_CONCAT(c.commentphotoid) commentphotoid
FROM posts p
LEFT JOIN comments c ON c.post = p.postid AND c.commentdel = 0
LEFT JOIN users u ON u.id = p.postauthorid
LEFT JOIN users u2 ON u2.id = c.author
WHERE p.postwallid = :id AND p.postdel = 0
and (
select count(*) from comments as c2
where c2.postid = p.postid and c2.commentdel = 0
and c2.<timestamp> <= c.timestamp
) < 3
GROUP BY postid
ORDER BY postid DESC
Edit: I didn't add the count of all comments. I think you can easily add it with another subquery in the select list but I know MySQL people don't like subqueries very much.
(
select count(*) from comments as c2
where c2.postid = c.postid and c2.commentdel = 0
) as comment_count
I add in SELECT count(postid) as all_comments and at the end LIMIT 0, 3
SELECT count(postid) as all_comments, p.*, u.id, u.username username, u.usersurname usersurname, u.usermainphoto userphoto, GROUP_CONCAT(c.text SEPARATOR 'a!k#h#md%o^v&') commenttext,GROUP_CONCAT(c.likes SEPARATOR '-') commentlikes,GROUP_CONCAT(c.dislikes SEPARATOR '-') commentdislikes, GROUP_CONCAT(c.commentdate) commentdate, GROUP_CONCAT(u2.username) commentauthorname, GROUP_CONCAT(c.anonim) commentanonym, GROUP_CONCAT(c.id) commentid, GROUP_CONCAT(u2.id) commentauthorid, GROUP_CONCAT(u2.usersurname) commentauthorsurname, GROUP_CONCAT(u2.usermainphoto) commentauthorphoto, GROUP_CONCAT(c.commentphotoid) commentphotoid
FROM posts p
LEFT JOIN comments c ON c.post = p.postid AND c.commentdel=0
LEFT JOIN users u ON u.id = p.postauthorid
LEFT JOIN users u2 ON u2.id = c.author
WHERE p.postwallid = :id AND p.postdel=0
GROUP BY postid
ORDER BY postid DESC
LIMIT 0, 3
I have a PDO statement, that selects some data from some different tables. Although, I am unsure on how I can select data from yet another table:
SELECT
c.forum_id as category_id,
c.forum_name as category_name,
t.forum_id as id,
t.forum_name as name,
t.forum_desc as description,
(SELECT COUNT(*) FROM forum_topics WHERE forum_id=t.forum_id AND topic_deleted=0) as topics_count,
(SELECT COUNT(*) FROM forum_posts WHERE forum_id=t.forum_id AND post_deleted=0) as posts_count,
(SELECT COUNT(*) FROM forum_posts WHERE topic_id=lp.topic_id AND post_deleted=0) as last_post_count,
lp.topic_id as last_post_topic_id,
lp.topic_title as last_post_topic_title,
lp.post_time as last_post_time,
lp.username as last_post_username
FROM forum_cats as t
JOIN forum_cats as c on c.forum_id = t.forum_type_id
left join (
SELECT
ft.topic_id,
ft.title as topic_title,
tmp.post_time,
u.username,
fp.forum_id
FROM
forum_posts fp
join forum_topics ft on ft.topic_id = fp.topic_id
join users u on u.id = fp.userid
join (
select forum_id, max(`post_time`) `post_time`
from forum_posts fp
where fp.post_deleted = 0
group by forum_id
) as tmp on (fp.forum_id = tmp.forum_id and fp.post_time = tmp.post_time)
where post_deleted = 0 and ft.topic_deleted = 0
) as lp on lp.forum_id = t.forum_id
where t.forum_active = 1 and c.forum_active = 1
order by category_id, t.forum_id
");
Now, I want to select the column avatar.users where username.users is = last_post_username (avatar column, from the users table).
I have absolute no idea on where to start. Should I put it something like this?
(SELECT avatar FROM users WHERE username=last_post_username)
That doesn't work though.
Any help is appreciated.
You already have "last_post_username" based on a join as "u.id = fp.userid". You just need to retrieve "lp.avatar"
Here is my answer :
SELECT
c.forum_id as category_id,
c.forum_name as category_name,
t.forum_id as id,
t.forum_name as name,
t.forum_desc as description,
(SELECT COUNT(*) FROM forum_topics WHERE forum_id=t.forum_id AND topic_deleted=0) as topics_count,
(SELECT COUNT(*) FROM forum_posts WHERE forum_id=t.forum_id AND post_deleted=0) as posts_count,
(SELECT COUNT(*) FROM forum_posts WHERE topic_id=lp.topic_id AND post_deleted=0) as last_post_count,
lp.topic_id as last_post_topic_id,
lp.topic_title as last_post_topic_title,
lp.post_time as last_post_time,
lp.username as last_post_username,
lp.avatar
FROM forum_cats as t
JOIN forum_cats as c on c.forum_id = t.forum_type_id
left join (
SELECT
ft.topic_id,
ft.title as topic_title,
tmp.post_time,
u.username,
u.avatar,
fp.forum_id
FROM
forum_posts fp
join forum_topics ft on ft.topic_id = fp.topic_id
join users u on u.id = fp.userid
join (
select forum_id, max(`post_time`) `post_time`
from forum_posts fp
where fp.post_deleted = 0
group by forum_id
) as tmp on (fp.forum_id = tmp.forum_id and fp.post_time = tmp.post_time)
where post_deleted = 0 and ft.topic_deleted = 0
) as lp on lp.forum_id = t.forum_id
where t.forum_active = 1 and c.forum_active = 1
order by category_id, t.forum_id
");
i create a web app like facebook by php and mysqli
in my app i have a table for posts , one table for likes , and one table for comments
i want to get the number of comments and likes of each post in one row with his post_id!!!
i try some querys likes this :
select `tblpost`.`post_id`, COALESCE(TCOMM.`comment_num`,0) as `c_num`, COALESCE(TLIKE.`like_num`,0) as `l_num`
from
(select `tblpost`.`post_id`, count(*) as `like_num` from `tblpost` join `tbllikes` on `tbllikes`.`post_id` = `tblpost`.`post_id` group by `tblpost`.`post_id`
) TLIKE
inner join
(select `tblpost`.`post_id`, count(*) as `comment_num` from `tblpost` join `tblcomments` on `tblcomments`.`post_id` = `tblpost`.`post_id` group by `tblpost`.`post_id`) TCOMM
on
TCOMM.`post_id` = TLIKE.`post_id`
but i don't know what's my problem
You can do count distincts with two left joins.
Something like this would work if there are fields like_id and comment_id in the tables tbllikes and tblcomments
SELECT
tblpost.post_id AS post_id,
COUNT(DISTINCT tbllikes.like_id) AS likes,
COUNT(DiSTINCT tblcomments.comment_id) AS comments
FROM tblpost
LEFT JOIN tbllikes ON tbllikes.post_id = tblpost.post_id
LEFT JOIN tblcomments on tblcomments.post_id = tblpost.post_id
GROUP BY tblpost.post_id
First, I think you can greatly simplify your query:
select l.post_id,
COALESCE(c.comment_num, 0) as c_num, COALESCE(l.like_num, 0) as l_num
from (select l.post_id, count(*) as like_num
from tbllikes l
group by l.post_id
) l inner join
(select c.post_id, count(*) as comment_num
from tblcomments c
group by c.post_id
) c
on l.post_id = c.post_id;
This will only get you posts that have both likes and comments. To get what you want, use a left join:
select p.post_id,
COALESCE(c.comment_num, 0) as c_num, COALESCE(l.like_num, 0) as l_num
from tblpost p left join
(select l.post_id, count(*) as like_num
from tbllikes l
group by l.post_id
) l
on l.post_id = p.post_id left join
(select c.post_id, count(*) as comment_num
from tblcomments c
group by c.post_id
) c
on c.post_id = p.post_id;
I have quite a large query, which basically shows the different forums I have stored in my database, to my users:
$stmt = $dbh->prepare("
SELECT
c.forum_id as category_id,
c.forum_name as category_name,
t.forum_id as id,
t.forum_name as name,
t.forum_desc as description,
(SELECT COUNT(*) FROM forum_topics WHERE forum_id=t.forum_id AND topic_deleted=0) as topics_count,
(SELECT COUNT(*) FROM forum_posts WHERE forum_id=t.forum_id AND post_deleted=0) as posts_count,
(SELECT COUNT(*) FROM forum_posts WHERE topic_id=lp.topic_id AND post_deleted=0) as last_post_count,
lp.topic_id as last_post_topic_id,
lp.topic_title as last_post_topic_title,
lp.post_time as last_post_time,
lp.username as last_post_username
FROM forum_cats as t
JOIN forum_cats as c on c.forum_id = t.forum_type_id
left join (
SELECT
ft.topic_id,
ft.title as topic_title,
tmp.post_time,
u.username,
fp.forum_id
FROM
forum_posts fp
join forum_topics ft on ft.topic_id = fp.topic_id
join users u on u.id = fp.userid
join (
select forum_id, max(`post_time`) `post_time`
from forum_posts fp
where fp.post_deleted = 0
group by forum_id
) as tmp on (fp.forum_id = tmp.forum_id and fp.post_time = tmp.post_time)
where post_deleted = 0 and ft.topic_deleted = 0
) as lp on lp.forum_id = t.forum_id
where t.forum_active = 1 and c.forum_active = 1
order by category_id, t.forum_id
");
$stmt->execute();
I then do a while loop to show all the records:
while (($row = $stmt->fetch())) {
echo $row['name'];
}
The problem is, that during the while loop, I get fine results sometimes!
It should look like this:
forum1
forum2
forum3
forum4
forum5
forum6
etc.
But sometimes I get results like this:
forum1
forum1
forum1
forum1
forum2
forum3
forum4
forum5
forum6
Some of the rows (it's quite random which row) are being 'duplicated'.
Can someone help me shed light on this?
Try grouping by adding GROUP BY t.forum_id:
SELECT
c.forum_id as category_id,
c.forum_name as category_name,
t.forum_id as id,
t.forum_name as name,
t.forum_desc as description,
(SELECT COUNT(*) FROM forum_topics WHERE forum_id=t.forum_id AND topic_deleted=0) as topics_count,
(SELECT COUNT(*) FROM forum_posts WHERE forum_id=t.forum_id AND post_deleted=0) as posts_count,
(SELECT COUNT(*) FROM forum_posts WHERE topic_id=lp.topic_id AND post_deleted=0) as last_post_count,
lp.topic_id as last_post_topic_id,
lp.topic_title as last_post_topic_title,
lp.post_time as last_post_time,
lp.username as last_post_username
FROM forum_cats as t
JOIN forum_cats as c on c.forum_id = t.forum_type_id
left join (
SELECT
ft.topic_id,
ft.title as topic_title,
tmp.post_time,
u.username,
fp.forum_id
FROM
forum_posts fp
join forum_topics ft on ft.topic_id = fp.topic_id
join users u on u.id = fp.userid
join (
select forum_id, max(`post_time`) `post_time`
from forum_posts fp
where fp.post_deleted = 0
group by forum_id
) as tmp on (fp.forum_id = tmp.forum_id and fp.post_time = tmp.post_time)
where post_deleted = 0 and ft.topic_deleted = 0
) as lp on lp.forum_id = t.forum_id
where t.forum_active = 1 and c.forum_active = 1
GROUP BY t.forum_id
order by category_id, t.forum_id
My website is experiencing very slow queries like the one below. It is Joomla 2.5.
# Query_time: 19.797091 Lock_time: 0.000561 Rows_sent: 12 Rows_examined: 4127
SET timestamp=1385467501;
SELECT a.fulltext, a.id, a.title, a.alias, a.title_alias, a.introtext, a.state, a.catid, a.created, a.created_by, a.created_by_alias,a.images, a.modified, a.modified_by,a.publish_up, a.publish_down, a.attribs, a.metadata, a.metakey, a.metadesc, a.access, a.hits, a.featured, LENGTH(a.fulltext) AS readmore,a.fulltext, a.id, a.title, a.alias, a.title_alias, a.introtext, a.state, a.catid, a.created, a.created_by, a.created_by_alias,a.images, a.modified, a.modified_by,a.publish_up, a.publish_down, a.attribs, a.metadata, a.metakey, a.metadesc, a.access, a.hits, a.featured, LENGTH(a.fulltext) AS readmore,c.title AS category_title, c.path AS category_route, c.access AS category_access, c.alias AS category_alias,CASE WHEN a.created_by_alias > ' ' THEN a.created_by_alias ELSE ua.name END AS author,ua.email AS author_email,contact.id as contactid,parent.title as parent_title, parent.id as parent_id, parent.path as parent_route, parent.alias as parent_alias,ROUND(v.rating_sum / v.rating_count, 0) AS rating, v.rating_count as rating_count,c.published, CASE WHEN badcats.id is null THEN c.published ELSE 0 END AS parents_published
FROM j25_content AS a
LEFT JOIN j25_content_frontpage AS fp ON fp.content_id = a.id
LEFT JOIN j25_categories AS c ON c.id = a.catid
LEFT JOIN j25_users AS ua ON ua.id = a.created_by
LEFT JOIN j25_users AS uam ON uam.id = a.modified_by
LEFT JOIN (
SELECT contact.user_id, MAX(contact.id) AS id, contact.language
FROM j25_contact_details AS contact
WHERE contact.published = 1
GROUP BY contact.user_id, contact.language) AS contact ON contact.user_id = a.created_by
LEFT JOIN j25_categories as parent ON parent.id = c.parent_id
LEFT JOIN j25_content_rating AS v ON a.id = v.content_id
LEFT OUTER JOIN (SELECT cat.id as id FROM j25_categories AS cat JOIN j25_categories AS parent ON cat.lft BETWEEN parent.lft AND parent.rgt WHERE parent.extension = 'com_content' AND parent.published != 1 GROUP BY cat.id ) AS badcats ON badcats.id = c.id
WHERE a.access IN (1,1) AND c.access IN (1,1) AND CASE WHEN badcats.id is null THEN a.state ELSE 0 END = 1 AND a.featured = 0 AND a.catid IN (181,89,173,174,49,128,124,175,129,133,121,176,135,180,177,178,179,90) AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2013-11-26 12:04:24') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2013-11-26 12:04:24')
GROUP BY a.id, a.title, a.alias, a.title_alias, a.introtext, a.checked_out, a.checked_out_time, a.catid, a.created, a.created_by, a.created_by_alias, a.created, a.modified, a.modified_by, uam.name, a.publish_up, a.attribs, a.metadata, a.metakey, a.metadesc, a.access, a.hits, a.xreference, a.featured, a.fulltext, a.state, a.publish_down, badcats.id, c.title, c.path, c.access, c.alias, uam.id, ua.name, ua.email, contact.id, parent.title, parent.id, parent.path, parent.alias, v.rating_sum, v.rating_count, c.published, c.lft, a.ordering, parent.lft, fp.ordering, c.id, a.images, a.urls
ORDER BY a.created DESC LIMIT 0, 12;
What can I do to speed it up? Would indexing some of these tables help?
Thank you.
Absolutely indexes help in queries. And each of what I consider your secondary lookup reference tables (categories, users, content rating, etc) should have an index at a MINIMUM on what your join condition is between the tables.
table index on
j25_content ( access, featured, catid, publish_up, publish_down, state, id, created_by, modified_by )
j25_content_frontpage ( content_id )
j25_categories ( id, parent_id )
ANOTHER index...
j25_categories ( extension, published, lft, rgt, id )
j25_users ( id )
j25_contact_details ( published, user_id, language, id )
j25_content_rating ( content_id )
I've set these indexes up to be "covering" indexes... meaning they should help the optimization of the query by having all the fields as part of the index so it does not have to go to the raw data pages to see if all the pieces qualify to return a record
TRY that first...
NEXT thing I would try is MySQL has a special keyword "STRAIGHT_JOIN" which tells the
engine to run the query in the order you have stated. You seem to have a good basis
of the tables as the primary and your lookup as secondary. The engine might be trying
to think for you and use one of the smaller tables to run the query.
select STRAIGHT_JOIN (rest of your query exactly as it is)