Can anyone tell me how to make this query faster?
$session_id = '000000000015';
$start = 0;
$finish = 30;
try {
$stmt = $conn->prepare("SELECT TOPUSERS.ID, TOPUSERS.USERNAME, TOPUSERS.NAME, TOPUSERS.NAME2, TOPUSERS.PHOTO, TOPUSERS.FB_USERID, TOPUSERS.IMAGE_TYPE, TOPUSERS.TW_USERID, TOPUSERS.TW_PHOTO,
COALESCE((SELECT COUNT(USERS_BUCKETS.ID) FROM USERS_BUCKETS WHERE USERS_BUCKETS.USERID=TOPUSERS.ID),0) AS NUM_ALL,
COALESCE((SELECT SUM(CASE WHEN USERS_BUCKETS.STATUS='Completed' THEN 1 ELSE 0 END) FROM USERS_BUCKETS WHERE USERS_BUCKETS.USERID=TOPUSERS.ID),0) AS NUM_DONE,
COALESCE((SELECT COUNT(USERS_LIKES.ID) FROM USERS_LIKES WHERE USERS_LIKES.USERID=TOPUSERS.ID),0) AS NUM_LIKES,
(SELECT USERS_BUCKETS.BUCKETID FROM USERS_BUCKETS WHERE USERS_BUCKETS.USERID=TOPUSERS.ID ORDER BY USERS_BUCKETS.DATE_MODIFIED DESC LIMIT 1) AS RECENT_BUCKET,
(SELECT BUCKETS_NEW.BUCKET_NAME FROM BUCKETS_NEW WHERE BUCKETS_NEW.ID=RECENT_BUCKET) AS REC,
COALESCE((SELECT COUNT(ID) FROM FOLLOW WHERE FOLLOW.USER_ID=TOPUSERS.ID),0) AS FOLLOWING,
COALESCE((SELECT COUNT(ID) FROM FOLLOW WHERE FOLLOW.FOLLOW_ID=TOPUSERS.ID),0) AS FOLLOWERS,
(SELECT IF(TOPUSERS.NAME = '',0,1) + IF(TOPUSERS.BIO = '',0,1) + IF(TOPUSERS.LOCATION = '',0,1) + IF(TOPUSERS.BIRTHDAY = '0000-00-00',0,1) + IF(TOPUSERS.GENDER = '',0,1)) as COMPLETENESS,
CASE WHEN ? IN (SELECT USER_ID FROM FOLLOW WHERE FOLLOW_ID = TOPUSERS.ID) THEN 'Yes' ELSE 'No' END AS DO_I_FOLLOW_HIM
FROM TOPUSERS
LEFT JOIN FOLLOW ON TOPUSERS.ID = FOLLOW.FOLLOW_ID
LEFT JOIN USERS_BUCKETS ON USERS_BUCKETS.USERID=TOPUSERS.ID
LEFT JOIN BUCKETS_NEW ON BUCKETS_NEW.ID=USERS_BUCKETS.BUCKETID
WHERE NOT TOPUSERS.ID = ?
GROUP BY TOPUSERS.ID ORDER BY TOPUSERS.RANDOM, TOPUSERS.USERNAME LIMIT $start, $finish");
When I run this in a browser it takes about 7 seconds to load. Without a few lines (the COALESCE in the middle, the two SELECTS above and the line below them) the time is reduced to 3-4 seconds.
The result of the query is a list of people with names, profile picture and some data.
TL,DR: you need to rewrite the query.
You need to rewrite your query to make it more efficient. I had to rewrite a similar query at work last week and here is what I have done.
The structure of your query should look like this to be efficient:
select ...
...
from ...
join ...
where ...
what you have now is something like:
select ...
inner select
inner select
from ...
join ...
where ...
That's the inner selects that kill your query. You need to find a way to move the inner select into the from section. Especially that you already query the tables.
What you need to understand is that your inner selects run for every records you have. So if you have 10 records, it would be alright (speed wise). But with hundred or thousand of records, it would be very slow.
If you want more information on your query run it with the explain keyword in from of it.
Related
I'm having some problems with a query that finds the next ID of an orders with certain filters on it - Like it should be from a specific city, etc.
Currently it's used for a function, where it'll either spit out the previous or the next ID based on the current order. So it can either be min(id) or max(id), where max(id) is obviously faster, since it has to go through less rows.
The query is working just fine, but it's rather slow, since it's going through 123953 rows to find the ID. Is there any way I could optimize this?
Function example:
SELECT $minmax(orders.orders_id) AS new_id FROM orders LEFT JOIN orders_status ON orders.orders_status = orders_status.orders_status_id $where_join WHERE orders_status.language_id = '$languages_id' AND orders.orders_date_finished != '1900-01-01 00:00:00' AND orders.orders_id $largersmaller $ordersid $where;
Live example
SELECT min(orders.orders_id)
FROM orders
LEFT JOIN orders_status ON orders.orders_status = orders_status.orders_status_id
WHERE orders_status.language_id = '4'
AND orders.orders_date_finished != '1900-01-01 00:00:00'
AND orders.orders_id < 4868771
LIMIT 1
so concluding:
SELECT orders.orders_id
FROM orders
JOIN orders_status ON orders.orders_status = orders_status.orders_status_id
WHERE orders_status.language_id = '4'
AND orders.orders_date_finished != '1900-01-01 00:00:00'
AND orders.orders_id < 4868771
ORDER BY orders.orders_id ASC
LIMIT 1
Extra:
to get the MAX value, use DESC where ASC is now.
And looking at your question: be sure to escape the values like $language_id etcetera. I suppose they could come from some html form?
(or use prepared statements)
I have this nice and neat way of loading posts for my blog website to fit specified page:
$end = $count - ($page * $ppp); //count = select max(id) from art;
$start = $count- ($page * $ppp) - ($ppp-1);
$nxtpage = $page +1; //this is set beforehand in case no posts exists
$prvpage = $page == 0 ? 0 : $page -1;
$sql = "SELECT
a.id AS id,
a.nazwa AS nazwa,
a.data AS data,
a.wstep AS wstep,
a.imgs AS imgs,
a.zdj AS zdj,
GROUP_CONCAT(t.nazwa) all_tags
FROM
art a INNER JOIN tagart ta ON a.id = ta.id INNER JOIN tags t ON t.idt = ta.idt
WHERE a.id BETWEEN $start AND $end
GROUP BY a.id
ORDER BY a.id desc";
This way I can load only a specified numer of posts depended by blogs page (pagination).
There is a pretty big problem with it tho.
Lets say my client make a mistake like writing BLACK PPL somewhere in one article half a year ago, and now he has to delete it.
Or even better, has to delete about 10 posts from it. When middle posts are deleted, the whole alrorithm gets messed up, because it scans posts based it their ID.
So my question here for you is what better way of picking the posts I could use, that would always get the correct order of the posts?
It looks like you're trying to implement your own way of doing LIMIT, which is a MySQL feature that handles pagination. Instead of manually defining your start and end ID's, you should be looking to order your posts and then only fetching the next X posts, no matter what their ID's are. Here's how you would do that
$start = ($page - 1) * $ppp;
$sql = "SELECT
a.id AS id,
a.nazwa AS nazwa,
a.data AS data,
a.wstep AS wstep,
a.imgs AS imgs,
a.zdj AS zdj,
GROUP_CONCAT(t.nazwa) all_tags
FROM
art a INNER JOIN tagart ta ON a.id = ta.id INNER JOIN tags t ON t.idt = ta.idt
GROUP BY a.id
ORDER BY a.id DESC
LIMIT $start,$ppp";
LIMIT is used as either
LIMIT 5 #Fetch first 5 items
or
LIMIT 5,10 #Starting from the 5th item, fetch the next 10 items
Instead of using max(id) to determine the number of posts, use count(id) or count(*) to actually count them. If a post gets deleted, the count can take that into account.
In the select query use limit to select the range of posts to show.
As you already figured out, you should not be relying on MAX(id) to count the number of records.
I would just use a separate count query to get the count. It's simple and relatively inexpensive:
SELECT COUNT (id) FROM art
And as others have already mentioned, use LIMIT to paginate instead of limiting by id.
I have three tables named issue_details, nature_payments, and rci_records. Now I have this query which joins this three tables.
SELECT issue_details.issue_date AS Date,
issue_details.check_no AS Check_No,
payees.payee_name AS Name_payee,
nature_payments.nature_payment AS Nature_of_Payment,
issue_details.issue_amount AS Checks_issued,
issue_details.nca_balance AS Nca_balance
FROM
issue_details
INNER JOIN
nature_payments ON
issue_details.nature_id = nature_payments.nature_id
INNER JOIN
payees ON
issue_details.payee_id = payees.payee_id
ORDER BY Date Asc, Check_no ASC
On my column in Nca_balance, this is a computed differences of every issuances of check. But you may not know what really the process of how I got the difference but to make it simple, let's say that I have another query
that dynamically get also the difference of this nca_balance column. Here is the query:
SELECT r.*,
(#tot := #tot - issue_amount) as bank_balance
FROM (SELECT #tot := SUM(nca_amount) as nca_total FROM nca
WHERE account_type = 'DBP-TRUST' AND
year(issue_date) = year('2015-01-11') AND
month(issue_date) = month('2015-01-11')
)
vars CROSS JOIN issue_details r
WHERE r.account_type = 'DBP-TRUST' AND
r.issue_date = '2015-01-11'
ORDER BY r.issue_date, r.check_no
I know it you may not get my point but I just want to replace the first query of the line
issue_details.nca_balance AS Nca_balance
with my own computation on my second query.
Please help me combine those two query into a single query. Thanks
I'm working with the join plus union plus group by query, and I developed a query something like mentioned below:
SELECT *
FROM (
(SELECT countries_listing.id,
countries_listing.country,
1 AS is_country
FROM countries_listing
LEFT JOIN product_prices ON (product_prices.country_id = countries_listing.id)
WHERE countries_listing.status = 'Yes'
AND product_prices.product_id = '3521')
UNION
(SELECT countries_listing.id,
countries_listing.country,
0 AS is_country
FROM countries_listing
WHERE countries_listing.id NOT IN
(SELECT country_id
FROM product_prices
WHERE product_id='3521')
AND countries_listing.status='Yes')) AS partss
GROUP BY id
ORDER BY country
And I just realised that this query is taking a lot of time to load results, almost 8 seconds.
I was wondering if there is the possibility to optimize this query to the fastest one?
If I understand the logic correctly, you just want to add a flag for the country as to whether or not there is a price for a given product. I think you can use an exists clause to get what you want:
SELECT cl.id, cl.country,
(exists (SELECT 1
FROM product_prices pp
WHERE pp.country_id = cl.id AND
pp.product_id = '3521'
)
) as is_country
FROM countries_listing cl
WHERE cl.status = 'Yes'
ORDER BY country;
For performance, you want two indexes: countries_listing(status, country) and
product_prices(country_id, product_id)`.
Depending on how often it is executed, prepared statements could help. See PDO for more information.
I am making a stats page about golf for the people I play with. I am trying to pull out of the database the number of times out of all our scorecards that we received birdies (which is -1 under par). It does pull out the -1s per hole, however I noticed that you if you had 2 birdies on a scorecard, it still only counts as 1 birdie instead of 2. I want it to keep counting, so if someone gets 9 birdies, those 9 are added to the total.
$query_p321 = "SELECT t1.*,COUNT(t1.player_id),t2.* FROM scorecards t1 LEFT JOIN courses t2 ON t1.course_id=t2.course_id
WHERE t1.hole1<t2.hole1_par AND t1.hole1>t2.hole1_par-2
OR t1.hole2<t2.hole2_par AND t1.hole2>t2.hole2_par-2
OR t1.hole3<t2.hole3_par AND t1.hole3>t2.hole3_par-2
OR t1.hole4<t2.hole4_par AND t1.hole4>t2.hole4_par-2
OR t1.hole5<t2.hole5_par AND t1.hole5>t2.hole5_par-2
OR t1.hole6<t2.hole6_par AND t1.hole6>t2.hole6_par-2
OR t1.hole7<t2.hole7_par AND t1.hole7>t2.hole7_par-2
OR t1.hole8<t2.hole8_par AND t1.hole8>t2.hole8_par-2
OR t1.hole9<t2.hole9_par AND t1.hole9>t2.hole9_par-2
OR t1.hole10<t2.hole10_par AND t1.hole10>t2.hole10_par-2
OR t1.hole11<t2.hole11_par AND t1.hole11>t2.hole11_par-2
OR t1.hole12<t2.hole12_par AND t1.hole12>t2.hole12_par-2
OR t1.hole13<t2.hole13_par AND t1.hole13>t2.hole13_par-2
OR t1.hole14<t2.hole14_par AND t1.hole14>t2.hole14_par-2
OR t1.hole15<t2.hole15_par AND t1.hole15>t2.hole15_par-2
OR t1.hole16<t2.hole16_par AND t1.hole16>t2.hole16_par-2
OR t1.hole17<t2.hole17_par AND t1.hole17>t2.hole17_par-2
OR t1.hole18<t2.hole18_par AND t1.hole18>t2.hole18_par-2
GROUP BY t1.player_id ORDER BY count(t1.player_id) DESC";
$result_p321 = mysql_query($query_p321);
$number = 1;
while ($row_p321 = mysql_fetch_array($result_p321)) {
$player_id2 = $row_p321["player_id"];
}
and so on..
You'll notice the "-2" in there. That is taking the par minus 2, as I don't want to record if the person is 2 strokes under. Just one stroke under. Any help is appreciated. Thank you.
Oh, also, GROUP BY needs to be used as I don't want to list the player name more than once. Just want it to count all the birdies. I guess my big problem is its not counting more than 1 per row. Thanks.
The problem is the where clause. You need to do the comparisons in the select clause in order to count them:
SELECT t1.*,
sum((t1.hole1 = t2.hole1_par - 1) +
(t1.hole2 = t2.hole2_par - 1) +
. . .
(t1.hole18 = t2.hole18_par - 1)
) as birdies
FROM scorecards t1 LEFT JOIN
courses t2 ON t1.course_id=t2.course_id
GROUP BY t1.player_id
ORDER BY birdies DESC
This uses the MySQL convention that true is 1 and false 0 to add the numbers up. An alternative formulation using standard SQL is:
sum((case when t1.hole1 = t2.hole1_par - 1) then 1 else 0 end) +
Try something like that:
SELECT t1.*, SUM( IF(t1.hole1 = t2.hole1_par-1,1,0) +
IF(t1.hole2 = t2.hole2_par-1,1,0) +
IF(t1.hole3 = t2.hole3_par-1,1,0) +
IF(t1.hole4 = t2.hole4_par-1,1,0) +
-- etc.
IF(t1.hole18 = t2.hole18_par-1,1,0) ) AS birdies
FROM scorecards t1
LEFT JOIN courses t2 ON t1.course_id=t2.course_id
GROUP BY t1.player_id
ORDER BY birdies DESC