MySQL - slowly query - php

Question

I have a problem with a search post on my forum.
1) I split search string using explode(" ", $string);
2) I search all tag_id where tag_value contains word - SELECT tag_id WHERE tag_value like '%{word}%'; all results are added to array $tags_array
3) I created INNER JOIN
foreach ($tags_array as $k => $v) {
if (!empty($v)) {
$short_name = "frt_".$i++;
$inner_array[] = " INNER JOIN forum_rel_tags as ".$short_name." ON (".$short_name.".tag_id IN (".implode(", ", $v).")) ";
$more[] = " ".$short_name.".post_id = fp.post_id ";
}
}
4) and finally I have something like:
SELECT COUNT(fp.post_id) AS total
FROM forum_categories c
JOIN forum_thread t USE INDEX (thread_id)
ON t.category_id = c.category_id
AND t.posts_limit <= 1
AND t.contest_limit <= 3
AND (t.register_limit*86400) <= 5841372
JOIN forum_auth a
ON a.auth_id = c.category_id
AND a.auth_group_id = 1
AND a.auth_type = 1
AND a.auth_visible = 1
JOIN forum_posts fp
ON fp.thread_id = t.thread_id
AND fp.post_deleted = 0
JOIN forum_rel_tags frt_0
ON frt_0.post_id = fp.post_id
AND frt_0.tag_id IN (1000 tag_id)
JOIN forum_rel_tags frt_1
ON frt_1.post_id = fp.post_id
AND frt_1.tag_id IN (200 tag_id)
JOIN forum_rel_tags frt_2
ON frt_2.post_id = fp.post_id
AND frt_2.tag_id IN (432 tag_id)
JOIN forum_rel_tags frt_3
ON frt_3.post_id = fp.post_id
AND frt_3.tag_id IN (50 tag_id)
But this query is very slow.
What can I change to make it faster?
EXPLAIN SELECT
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t ref thread_id thread_id 4 const 1 Using where; Using index
1 SIMPLE c eq_ref PRIMARY PRIMARY 4 pionas.t.category_id 1 Using index
1 SIMPLE a eq_ref auth_type auth_type 10 const,pionas.c.category_id,const 1 Using where
1 SIMPLE frt_0 range post_id,tag_id tag_id 4 NULL 372226 Using where; Using join buffer
1 SIMPLE frt_1 range post_id,tag_id tag_id 4 NULL 37787 Using where; Using join buffer
1 SIMPLE fp eq_ref PRIMARY,thread_id,post_deleted PRIMARY 4 pionas.frt_1.post_id 1 Using where
1 SIMPLE u eq_ref PRIMARY PRIMARY 4 pionas.fp.user_id 1 Using index
1 SIMPLE frt_3 range post_id,tag_id tag_id 4 NULL 23608 Using where; Using join buffer
1 SIMPLE frt_2 ref post_id,tag_id post_id 4 pionas.frt_3.post_id 296144 Using where; Using index
Tables
CREATE TABLE `forum_auth` (
`auth_type` smallint(1) unsigned NOT NULL DEFAULT '0',
`auth_id` int(10) unsigned NOT NULL,
`auth_visible` int(10) unsigned NOT NULL,
`auth_group_id` int(10) unsigned NOT NULL,
`auth_last_post_id` int(11) unsigned NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
ALTER TABLE `forum_auth`
ADD UNIQUE KEY `auth_type` (`auth_type`,`auth_id`,`auth_group_id`);
CREATE TABLE `forum_categories` (
`category_id` int(10) unsigned NOT NULL,
`category_name` varchar(200) NOT NULL,
`category_desc` text NOT NULL,
`category_order` tinyint(2) NOT NULL,
`category_parent` int(10) unsigned NOT NULL DEFAULT '0',
`last_post_id` int(11) NOT NULL,
`total_thread` int(11) NOT NULL,
`total_posts` int(11) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
ALTER TABLE `forum_categories`
ADD PRIMARY KEY (`category_id`);
CREATE TABLE `forum_posts` (
`post_id` int(10) unsigned NOT NULL,
`post_subject` varchar(250) NOT NULL,
`post_message` text NOT NULL,
`post_create_date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`post_last_modify` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`post_count_modify` int(11) NOT NULL,
`post_deleted` tinyint(1) NOT NULL DEFAULT '0',
`post_ip` int(11) NOT NULL,
`user_id` int(10) NOT NULL,
`thread_id` int(10) unsigned NOT NULL,
`guest_name` varchar(100) NOT NULL,
`guest_mail` varchar(150) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
ALTER TABLE `forum_posts`
ADD PRIMARY KEY (`post_id`),
ADD KEY `thread_id` (`thread_id`),
ADD KEY `user_id` (`user_id`),
ADD KEY `post_deleted` (`thread_id`,`post_deleted`),
ADD KEY `post_create_date` (`post_create_date`),
ADD FULLTEXT KEY `post_message` (`post_message`);
CREATE TABLE `forum_rel_tags` (
`post_id` int(10) unsigned NOT NULL,
`tag_id` int(10) unsigned NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
ALTER TABLE `forum_rel_tags`
ADD UNIQUE KEY `post_id` (`post_id`,`tag_id`),
ADD KEY `tag_id` (`tag_id`);
CREATE TABLE `forum_tags` (
`tag_id` int(10) unsigned NOT NULL,
`name` varchar(150) NOT NULL,
`tag_url` varchar(150) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
ALTER TABLE `forum_tags`
ADD PRIMARY KEY (`tag_id`),
ADD UNIQUE KEY `tag_url` (`tag_url`);
CREATE TABLE `forum_thread` (
`thread_id` int(10) unsigned NOT NULL,
`thread_subject` varchar(250) CHARACTER SET utf8 NOT NULL,
`thread_desc` text CHARACTER SET utf8 NOT NULL,
`thread_create_date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`thread_view` int(10) unsigned NOT NULL DEFAULT '0',
`thread_reply` int(10) unsigned NOT NULL DEFAULT '0',
`thread_ip` int(10) unsigned NOT NULL,
`last_post_id` int(11) NOT NULL,
`user_id` varchar(100) CHARACTER SET utf8 NOT NULL,
`category_id` int(10) unsigned NOT NULL,
`guest_name` varchar(100) COLLATE utf8_unicode_ci DEFAULT NULL,
`guest_mail` varchar(150) COLLATE utf8_unicode_ci DEFAULT NULL,
`guest_can_reply` smallint(1) NOT NULL DEFAULT '0',
`thread_block` tinyint(1) NOT NULL,
`thread_sticky` tinyint(1) unsigned NOT NULL DEFAULT '0',
`posts_limit` int(2) NOT NULL DEFAULT '0',
`register_limit` int(3) NOT NULL DEFAULT '0',
`contest_limit` int(2) NOT NULL DEFAULT '0'
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
ALTER TABLE `forum_thread`
ADD PRIMARY KEY (`thread_id`),
ADD KEY `category_id` (`category_id`),
ADD KEY `thread_id` (`thread_id`,`category_id`,`posts_limit`,`register_limit`,`contest_limit`);
Second query:
SELECT fp.*, u.username, u.rang, u.post, u.date_register, u.avatar, u.pkt, f.city
FROM forum_categories c
JOIN forum_thread t USE INDEX (thread_id)
ON t.category_id = c.category_id
AND t.posts_limit <= 1
AND t.contest_limit <= 3
AND (t.register_limit*86400) <= 5841372
JOIN forum_auth a
ON a.auth_id = c.category_id
AND a.auth_group_id = 1
AND a.auth_type = 1
AND a.auth_visible = 1
JOIN forum_posts fp
ON fp.thread_id = t.thread_id
AND fp.post_deleted = 0
JOIN forum_rel_tags frt_0
ON frt_0.post_id = fp.post_id
AND frt_0.tag_id IN (1000 tag_id)
JOIN forum_rel_tags frt_1
ON frt_1.post_id = fp.post_id
AND frt_1.tag_id IN (200 tag_id)
JOIN forum_rel_tags frt_2
ON frt_2.post_id = fp.post_id
AND frt_2.tag_id IN (432 tag_id)
JOIN forum_rel_tags frt_3
ON frt_3.post_id = fp.post_id
AND frt_3.tag_id IN (50 tag_id)
LEFT JOIN users u
ON fp.user_id=u.user_id
LEFT JOIN users_field f
ON u.user_id=f.user_id
GROUP BY fp.post_id
ORDER BY fp.post_id DESC
LIMIT 0,30;
[EDIT]
What you think about it:
EXPLAIN SELECT fp.*, u.username, u.rang, u.post, u.date_register, u.avatar, u.pkt, f.city
FROM forum_categories c
JOIN forum_thread t USE INDEX (thread_id)
ON t.category_id = c.category_id
AND t.posts_limit <= 1
AND t.contest_limit <= 3
AND t.register_limit <= (5841372/86400)
JOIN forum_auth a
ON a.auth_id = c.category_id
AND a.auth_group_id = 1
AND a.auth_type = 1
AND a.auth_visible = 1
JOIN forum_posts fp
ON fp.thread_id = t.thread_id
AND fp.post_deleted = 0
JOIN forum_rel_tags frt_0
ON frt_0.post_id = fp.post_id
JOIN forum_tags ft_0
ON ft_0.tag_id = frt_0.tag_id
AND ft_0.tag_url like '%play%'
JOIN forum_rel_tags frt_1
ON frt_0.post_id = frt_1.post_id
JOIN forum_tags ft_1
ON ft_1.tag_id = frt_1.tag_id
AND ft_1.tag_url like '%how%'
JOIN forum_rel_tags frt_2
ON frt_1.post_id = frt_2.post_id
JOIN forum_tags ft_2
ON ft_2.tag_id = frt_2.tag_id
AND ft_2.tag_url like '%win%'
JOIN forum_rel_tags frt_3
ON frt_2.post_id = frt_3.post_id
JOIN forum_tags ft_3
ON ft_3.tag_id = frt_3.tag_id
AND ft_3.tag_url like '%to%'
LEFT JOIN users u
ON fp.user_id=u.user_id
LEFT JOIN users_field f
ON u.user_id=f.user_id
GROUP BY fp.post_id
ORDER BY fp.post_id DESC

Answer 1

Assuming that user_id is unique in the users table, the LEFT JOIN to users table is unnecessary.
There are four joins to the forum_rel_tags table; there seems to be a potential there for multiple rows to be be returned from each join; and each of those rows is going to get "matched" to the rows returned from the other joins... which is a a partial cross product i.e. the same fp.post_id can be "counted" multiple times.
The most likely explanation for poor performance is the absence of suitable indexes.
Some predicates e.g. (t.register_limit*86400)<=5841372 cannot make use of an range scan operation on an index... the expression on t.register_limit*86400 has to be evaluated for every row (that isn't excluded by some other predicate) before the comparison to the literal can be made. We typically prefer to reference a bare column in the predicate, where it's possible to do that to return an equivalent result... e.g.
t.register_limit <= (584137/86400)
We can re-write the query, removing the unnecessary join to the users table, to something like this:
SELECT COUNT(fp.post_id) AS total
FROM forum_categories c
JOIN forum_thread t
ON t.category_id = c.category_id
AND t.posts_limit <= 1
AND t.contest_limit <= 3
AND (t.register_limit*86400) <= 5841372
JOIN forum_auth a
ON a.auth_id = c.category_id
AND a.auth_group_id = 1
AND a.auth_type = 1
AND a.auth_visible = 1
JOIN forum_posts fp
ON fp.thread_id = t.thread_id
AND fp.post_deleted = 0
JOIN forum_rel_tags frt_0
ON frt_0.post_id = fp.post_id
AND frt_0.tag_id IN (1000 tag_id)
JOIN forum_rel_tags frt_1
ON frt_1.post_id = fp.post_id
AND frt_1.tag_id IN (200 tag_id)
JOIN forum_rel_tags frt_2
ON frt_2.post_id = fp.post_id
AND frt_2.tag_id IN (432 tag_id)
JOIN forum_rel_tags frt_3
ON frt_3.post_id = fp.post_id
AND frt_3.tag_id IN (50 tag_id)
This makes it much easier to decipher the query, especially in terms of figuring out what indexes are likely to be the most appropriate for this query.
... ON forum_thread (category_id, posts_limit, contest_limit, register_limit)
... ON forum_auth (auth_id, auth_group, auth_type, auth_visible)
... ON forum_posts (thread_id, post_deleted, post_id)
... ON forum_rel_tags (post_id, tag_id)
Since these are covering indexes for the query, we expect the EXPLAIN output to include "Using index" for these tables. This is just a first cut at some indexes; these may already exist, or there may be more appropriate ordering of the columns in the indexes.
But absent table definitions, including indexes and estimated cardinalities, it's not really possible to give a more definitive answer.
Again...
I'm very suspicious of the partial cross product between the references to forum_rel_tags table. I suspect that this query can return a value for "total" that is higher than is expected. This is just a suspicion, because I don't see any specification (beyond the query) as to what result is supposed to be returned.