I'm trying to convert my SELECT query into a faster version of SELECT including JOIN, that I found online.
Here is my code:
$stmt->prepare("SELECT * FROM books WHERE person_id = :person_id ORDER BY id DESC LIMIT 5, 10");
convert to:
SELECT l.id, value, LENGTH(stuffing) AS len
FROM (
SELECT id
FROM t_limit
ORDER BY
id
LIMIT 150000, 10
) o
JOIN t_limit l
ON l.id = o.id
ORDER BY
l.id
I found that code online (last piece of code). Its supposed to help with performance when doing OFFSET and LIMIT. anyways... here is what i tried:
$stmt->prepare("SELECT * FROM (SELECT id FROM books WHERE person_id = :person_id ORDER BY id DESC LIMIT 5, 10) o WHERE person_id = :person_id JOIN books l ON l.id = o.id ORDER BY l.id");
but it does not seem to work.
any help would be appreciated!
I found that code online. Its supposed to help with performance when doing OFFSET and LIMIT.
Where did you hear that? The statement you've quoted appears to be very specific to a particular database schema, and is not applicable to your situation. Forget about it.
As it stands, the SQL statement you are currently using is already optimal. If it is not performing adequately, you may need to create an index on books (person_id, id).
Here's two articles describing the seek method:
https://blog.jooq.org/2013/10/26/faster-sql-paging-with-jooq-using-the-seek-method/
and
https://use-the-index-luke.com/sql/partial-results/fetch-next-page
It looks like a better fit for what you're doing. I would still venture to guess this is an indexing issue, that an explain would help with.
if anyone is wondering, I fixed it. I put the WHERE clause in the wrong place.
This is the fixed version now:
$stmt->prepare("SELECT * FROM (SELECT id FROM books WHERE person_id = :person_id ORDER BY id DESC LIMIT 5, 10) o JOIN books l ON l.id = o.id WHERE person_id = :person_id ORDER BY l.id");
Related
I want to ask how sql join uses limits on one table
I have searched Google but the query I did was different and that made me confused
$queryAnime = "SELECT *
FROM `anime`
JOIN `episode` ON `anime`.`link` = `episode`.`idanime`
ORDER BY `anime`.`id` DESC
";
$anime = $this->db->query($queryAnime)->result_array();
when I add a limit in the normal way only 1 array appears
I let 'anime' appear all and 'episodes' only show one array
and sorry my English is bad I hope you understand and dabat helps me
If I understand your question correctly, you need to specify the columns you need in the Select statements and not use *, and use a Select Distinct
So for example
SELECT DISTINCT `show_name`,`eposode`
FROM `anime` JOIN `episode`
ON `anime`.`link` = `episode`.`idanime`
ORDER BY `anime`.`id` DESC
I'm currently working on a project that needs a "Similar Series" Panel. I have a Table for the series and one of their genres. Need to select movies with least 4 genres in common with the selected movie.
I came up with this query:
SELECT s_id,stitle,sdesc FROM s_genre
INNER JOIN series ON s_genre.s_id=series.id
WHERE s_id IN (
SELECT s_id FROM s_genre
INNER JOIN (SELECT g_id FROM s_genre WHERE s_id = $mid) a USING (g_id)
GROUP BY s_id HAVING COUNT(s_id)>3
) AND s_id != $mid
ORDER BY RAND()
LIMIT 5
It works, but it takes almost 20 seconds to load :(
Any ideas how I can reduce the loading time? Any suggestions are appreciated :)
Thanks in advance.
Looks like the problem is cause by "ORDER BY RAND()" ... removing it would reduce the loading time to 0.5s
While ORDER BY rand() seems to be the major problem, it might be worth recoding the query to do a join against the sub query:-
SELECT s_genre.s_id,
series.stitle,
series.sdesc
FROM s_genre
INNER JOIN series ON s_genre.s_id = series.id
INNER JOIN
(
SELECT a.s_id
FROM s_genre a
INNER JOIN s_genre b
ON a.g_id = b.g_id
WHERE b.s_id = $mid
GROUP BY a.s_id
HAVING COUNT(a.s_id) > 3
) sub0
ON s_genre.s_id = sub0.s_id
WHERE s_genre.s_id != $mid
ORDER BY RAND()
LIMIT 5
There are techniques to select random records, but doing so with this query could be a bit messy. Most rely on having a fairly evenly distributed unique identifier for the rows, and (while I assume s_genre.s_id is a unique id) the query will have generated holes in the range of these and they will doubtless have holes in the range.
dear php and mysql expertor
i have two table one large for posts artices 200,000records (index colume: sid) , and one small table (index colume topicid ) for topics has 20 record .. have same topicid
curent im using : ( it took round 0.4s)
+do get last 50 record from table:
SELECT sid, aid, title, time, topic, informant, ihome, alanguage, counter, type, images, chainid FROM veryzoo_stories ORDER BY sid DESC LIMIT 0,50
+then do while loop in each records for find the maching name of topic in each post:
while ( .. ) {
SELECT topicname FROM veryzoo_topics WHERE topicid='$topic'"
....
}
+Now
I going to use Inner Join for speed up process but as my test it took much longer from 1.5s up to 3.5s
SELECT a.sid, a.aid, a.title, a.time, a.topic, a.informant, a.ihome, a.alanguage, a.counter, a.type, a.images, a.chainid, t.topicname FROM veryzoo_stories a INNER JOIN veryzoo_topics t ON a.topic = t.topicid ORDER BY sid DESC LIMIT 0,50
It look like the inner join do all joining 200k records from two table fist then limit result at 50 .. that took long time..
Please help to point me right way doing this..
eg take last 50 records from table one.. then join it to table 2 .. ect
Do not use inner join unless the two tables share the same primary key, or you'll get duplicate values (and of course a slower query).
Please try this :
SELECT *
FROM (
SELECT a.sid, a.aid, a.title, a.time, a.topic, a.informant, a.ihome, a.alanguage, a.counter, a.type, a.images, a.chainid
FROM veryzoo_stories a
ORDER BY sid DESC
LIMIT 0 , 50
)b
INNER JOIN veryzoo_topics t ON b.topic = t.topicid
I made a small test and it seems to be faster. It uses a subquery (nested query) to first select the 50 records and then join.
Also make sure that veryzoo_stories.sid, veryzoo_stories.topic and veryzoo_topics.topicid are indexes (and that the relation exists if you use InnoDB). It should improve the performance.
Now it leaves the problem of the ORDER BY LIMIT. It is heavy because it orders the 200,000 records before selecting. I guess it's necessary. The indexes are very important when using ORDER BY.
Here is an article on the problem : ORDER BY … LIMIT Performance Optimization
I'm just give test to nested query + inner join and suprised that performace increase much: it now took only 0.22s . Here is my query:
SELECT a.*, t.topicname
FROM (SELECT sid, aid, title, TIME, topic, informant, ihome, alanguage, counter, TYPE, images, chainid
FROM veryzoo_stories
ORDER BY sid DESC
LIMIT 0, 50) a
INNER JOIN veryzoo_topics t ON a.topic = t.topicid
if no more solution come up , i may use this one .. thanks for anyone look at this post
I have a database schema like this:
My database schema: http://i.stack.imgur.com/vFKRk.png
To explain the context: One user writes one message. He can send it to one or more users.
I succeeded to get the title of message, the author for one user. However Doctrine, which I use for this project, do it with 2 queries. It's a little bit strange for me and I'm looking to understand, why. Normally, we can do it with one SQL query.
My DQL query:
$q = Doctrine_Query::create()->select('id_me, users_id_us, state_me, type_me, mc.title_mc, us.login_us') ->from('messages m')->innerJoin('m.messages_content mc')->innerJoin('mc.Users us') ->where('users_id_us = ?', $user)->limit($opt['limit'])->offset($opt['offset'])->orderBy($opt['order']);return $q->fetchArray();
SQL queries returned by Doctrine:
SELECT DISTINCT m3.id_me FROM messages m3 INNER JOIN messages_content m4 ON m3.messages_content_id_mc = m4.id_mc INNER JOIN users u2 ON m4.users_id_us = u2.id_us WHERE m3.users_id_us = '6' ORDER BY m3.id_me DESC LIMIT 2
SELECT m.id_me AS m__id_me, m.users_id_us AS m__users_id_us, m.state_me AS m__state_me, m.type_me AS m__type_me, m2.id_mc AS m2__id_mc, m2.title_mc AS m2__title_mc, u.id_us AS u__id_us, u.login_us AS u__login_us FROM messages m INNER JOIN messages_content m2 ON m.messages_content_id_mc = m2.id_mc INNER JOIN users u ON m2.users_id_us = u.id_us WHERE m.id_me IN ('11') AND (m.users_id_us = '6') ORDER BY m.id_me DESC
Why my Doctrine query doesn't return the query like this:
SELECT m.id_me, m.users_id_us, m.state_me, m.type_me, mc.title_mc, u.login_us FROM messages m JOIN messages_content mc ON mc.id_mc = m.messages_content_id_mc JOIN users u ON u.id_us = mc.users_id_us WHERE m.users_id_us = 6;
Any idea to transform my DQL query and execute it one time ?
The ORM Limit issue
This has to do with the LIMIT part. :) Doctrine LIMIT works a bit different than MySQL limit does.
MySQL LIMIT just issues the query, and stops searching as soon as n rows are found that matches your SQL query. Since in ORM, this is really unexpected behaviour (it might well be that in a scalar layout the SQL SELECT * FROM myModel LEFT JOIN someOtherModel ON someCondition LIMIT 3 actually only returns one myModel instance rather than three, since a left join can result 3 rows.
What does Doctrine do?
If your DQL query is FROM school s INNER JOIN s.students LIMIT 15, it means: give me 15 instances of school that have at least one student associated (hence INNER JOIN), with ALL of their student associates. To do this, Doctrine first asks for DISTINCT school with the exact same query parameters and a LIMIT part, to figure out which 15 school IDs should be returned. After this is done, these IDs are queried next, without the LIMIT part.
How to solve your issue
If you are not having an actual problem, huzzah, find the explaination of this behaviour above. If your query output is other than you expected, make sure you take these steps into consideraton. If for instance your DQL is FROM school s INNER JOIN s.students LIMIT 15, and you are wondering why you get more than 15 students, try: FROM students s INNER JOIN s.school LIMIT 15. In MySQL this means basically the same (disregarding the order of the result), though in Doctrine, this means you will get 15 students instead of 15 schools.
This bothered me too with some of the more complex queries. The only solution that I found was to bypass the sillyness altogether:
$q = Doctrine_Manager::getInstance()->getCurrentConnection();
$my_result = $q->fetchAssoc(" ... PUT SQL HERE ... ");
The solution which works:
I changed the relation alias and specified the columns participating in ON joint between messages_content and users.
The right Doctrine query is:
$q = Doctrine_Query::create()
->select('id_me, users_id_us, state_me, type_me, mc.title_mc, us.login_us')
->from('messages m')
->innerJoin('m.messages_content mc')
->innerJoin('m.Users us ON mc.users_id_us=us.id_us')
->where('users_id_us = ?', $user)
->limit($opt['limit'])
->offset($opt['offset'])
->orderBy($opt['order']);
It gives a SQL query like this:
SELECT m.id_me AS m__id_me, m.users_id_us AS m__users_id_us, m.state_me AS m__state_me, m.type_me AS m__type_me, m2.id_mc AS m2__id_mc, m2.title_mc AS m2__title_mc, u.id_us AS u__id_us, u.login_us AS u__login_us FROM messages m INNER JOIN messages_content m2 ON m.messages_content_id_mc = m2.id_mc INNER JOIN users u ON (m2.users_id_us = u.id_us) WHERE (m.users_id_us = '7') ORDER BY m.id_me DESC LIMIT 2
Tom and Pelle ten Cate, thanks for your participation.
I have 10k forum topics. I run this query:
SELECT *
FROM `topics`
WHERE category_id = '2'
ORDER BY `last_message` DESC
LIMIT 2990, 10
This checks all rows! All rows! Even after adding index to last_message. last_message is a time() and I want to order all messages by that descending. So, what do you suggest me? Without BETWEEN, because it can't be done with that, unfortunately.
Since you have a filtering condition on category_id, an index on last_message most probably won't help you much.
The query will have to traverse the index and filter the records for category_id = 2. Since traversing an index is more expensive than scanning the table, a filesort can be a more efficient solution, and MySQL may prefer it over the index scan (that's most probably is what is happening in your case).
Also, as #OMG Ponies pointed out, MySQL is not capable of doing late row lookups.
Assuming that your PRIMARY KEY column is called id, you need to create an index on (category_id, last_message, id) and rewrite your query as follows:
SELECT t.*
FROM (
SELECT id
FROM topics
WHERE category_id = 2
ORDER BY
category_id DESC, last_message DESC, id DESC
LIMIT 2990, 10
) q
JOIN topics t
ON t.id = q.id
To get MySQL to use late row lookups, use:
SELECT t.*
FROM TOPICS t
JOIN (SELECT *
FROM `topics`
WHERE category_id = '2'
ORDER BY `last_message` DESC
LIMIT 2990, 10) x ON x.topic_id = t.topic_id
ORDER BY t.topic_id
Without TOPICS column details, I have to assume there is an id column that is auto_increment for the table...
You can read more about the behavior in this article.