Can anyone help me optimise this query? I have the following table:
cdu_user_progress:
--------------------------------------------------------------
|id |uid |lesson_id |game_id |date |score |
--------------------------------------------------------------
For each user, I'm trying to obtain the difference between the best and first scores for a particular game_id for a particular lesson_id, and order the results by that difference ('progress' in my query):
SELECT ms.uid AS id, ms.max_score - fs.first_score AS progress
FROM (
SELECT up.uid, MAX(CASE WHEN game_id = 3 THEN score ELSE NULL END) AS max_score
FROM cdu_user_progress up
WHERE (up.uid IN ('1671', '1672', '1673', '1674', '1675', '1676', '1679', '1716', '1725', '1726', '1937', '1964', '1996', '2062', '2065', '2066', '2085', '2086')) AND (up.lesson_id = '65') AND (up.score > '-1')
GROUP BY up.uid
) ms
LEFT JOIN (
SELECT up.uid, up.score AS first_score
FROM cdu_user_progress up
INNER JOIN (
SELECT up.uid, MIN(CASE WHEN game_id = 3 THEN date ELSE NULL END) AS first_date
FROM cdu_user_progress up
WHERE (up.uid IN ('1671', '1672', '1673', '1674', '1675', '1676', '1679', '1716', '1725', '1726', '1937', '1964', '1996', '2062', '2065', '2066', '2085', '2086')) AND (up.lesson_id = '65') AND (up.score > '-1')
GROUP BY up.uid
) fd ON fd.uid = up.uid AND fd.first_date = up.date
) fs ON fs.uid = ms.uid
ORDER BY progress DESC
Any help would be most appreciated!
Absent any EXPLAIN output or index definitions, we can't make any recommendations. (I noted in a comment that it looks like some join predicates are missing, if we don't have guaranteed uniqueness on the (uid,date) tuple in cdu_user_progress... there's potential that we are going to get rows that are for a different lesson_id or a score that isn't greater than '-1'.
In the query text, immediately before ) fs , I'd be adding
AND up.lesson_id = '65'
AND up.score > '-1'
GROUP BY up.uid
I'd also wrap the up.score column (in the SELECT list of the fd view) in an aggregate function, either MIN() or MAX(), for compliance with the ANSI standard (even though it isn't required by MySQL when SQL_MODE doesn't include ONLY_FULL_GROUP_BY)
If I didn't have a suitable index defined, I'd consider adding an index:
... ON cdu_user_progress (lesson_id, uid, score, game_id, date)
There's some overhead for the derived tables (materializing the inline views) and those derived tables aren't going to have indexes on them (in MySQL 5.5 and earlier.) But the GROUP BY in each inline view ensures that we'll have less than 20 rows, so that's not really going to be a problem.
So, if there's a performance issue, it's in the view queries. Again, we'd really need to see the output from EXPLAIN and the index definitions, and some cardinality estimates, in order to make recommendations.
FOLLOWUP
Given that there's not a unique constraint on (uid,date), I'd add those predicates in the fs view query. I'd also use unique table aliases in the query (for each references to cdu_user_progress) to make both the statement and the EXPLAIN output easier to read. Also, adding the GROUP BY clause and the aggregate function in the fd view... I'd write the query like this:
SELECT ms.uid AS id
, ms.max_score - fs.first_score AS progress
FROM ( SELECT up.uid
, MAX(CASE WHEN up.game_id = 3 THEN up.score ELSE NULL END) AS max_score
FROM cdu_user_progress up
WHERE up.uid IN ('1671','1672','1673','1674','1675','1676','1679','1716','1725','1726','1937','1964','1996','2062','2065','2066','2085','2086')
AND up.lesson_id = '65'
AND up.score > '-1'
GROUP BY up.uid
) ms
LEFT
JOIN ( SELECT uo.uid
, MIN(uo.score) AS first_score
FROM ( SELECT un.uid
, MIN(CASE WHEN un.game_id = 3 THEN un.date ELSE NULL END) AS first_date
FROM cdu_user_progress un
WHERE un.uid IN ('1671','1672','1673','1674','1675','1676','1679','1716','1725','1726','1937','1964','1996','2062','2065','2066','2085','2086')
AND un.lesson_id = '65'
AND un.score > '-1'
GROUP BY un.uid
) fd
JOIN cdu_user_progress uo
ON uo.uid = fd.uid
AND uo.date = fd.first_date
AND uo.lesson_id = '65'
AND uo.score > '-1'
GROUP BY uo.uid
) fs
ON fs.uid = ms.uid
ORDER BY progress DESC
And I believe that would make the index I recommended above suitable for all of the references to cdu_user_progress.
Related
I would like to better optimize my code. I'd like to have a single query that allows an alias name to have it's own limit and also include a result with no limit.
Currently I'm using two queries like this:
// ALL TIME //
$mikep = mysqli_query($link, "SELECT tasks.EID, reports.how_did_gig_go FROM tasks INNER JOIN reports ON tasks.EID=reports.eid WHERE `priority` IS NOT NULL AND `partners_name` IS NOT NULL AND mike IS NOT NULL GROUP BY EID ORDER BY tasks.show_date DESC;");
$num_rows_mikep = mysqli_num_rows($mikep);
$rating_sum_mikep = 0;
while ($row = mysqli_fetch_assoc($mikep)) {
$rating_mikep = $row['how_did_gig_go'];
$rating_sum_mikep += $rating_mikep;
}
$average_mikep = $rating_sum_mikep/$num_rows_mikep;
// AND NOW WITH A LIMIT 10 //
$mikep_limit = mysqli_query($link, "SELECT tasks.EID, reports.how_did_gig_go FROM tasks INNER JOIN reports ON tasks.EID=reports.eid WHERE `priority` IS NOT NULL AND `partners_name` IS NOT NULL AND mike IS NOT NULL GROUP BY EID ORDER BY tasks.show_date DESC LIMIT 10;");
$num_rows_mikep_limit = mysqli_num_rows($mikep_limit);
$rating_sum_mikep_limit = 0;
while ($row = mysqli_fetch_assoc($mikep_limit)) {
$rating_mikep_limit = $row['how_did_gig_go'];
$rating_sum_mikep_limit += $rating_mikep_limit;
}
$average_mikep_limit = $rating_sum_mikep_limit/$num_rows_mikep_limit;
This allows me to show an all-time average and also an average over the last 10 reviews. Is it really necessary for me to set up two queries?
Also, I understand I could get the sum in the query, but not all the values are numbers, so I've actually converted them in PHP, but left out that code in order to try and simplify what is displayed in the code.
All-time average and average over the last 10 reviews
In the best case scenario, where your column how_did_gig_go was 100% numeric, a single query like this could work like so:
SELECT
AVG(how_did_gig_go) AS avg_how_did_gig_go
, SUM(CASE
WHEN rn <= 10 THEN how_did_gig_go
ELSE 0
END) / 10 AS latest10_avg
FROM (
SELECT
#num + 1 AS rn
, tasks.show_date
, reports.how_did_gig_go
FROM tasks
INNER JOIN reports ON tasks.EID = reports.eid
CROSS JOIN ( SELECT #num := 0 AS n ) AS v
WHERE priority IS NOT NULL
AND partners_name IS NOT NULL
AND mike IS NOT NULL
ORDER BY tasks.show_date DESC
) AS d
But; Unless all the "numbers" are in fact numeric you are doomed to sending every row back from the server for php to process unless you can clean-up the data in MySQL somehow.
You might avoid sending all that data twice if you establish a way for your php to use only the top 10 from the whole list. There are probably way of doing that in PHP.
If you wanted assistance in SQL to do that, then maybe having 2 columns would help, it would reduce the number of table scans.
SELECT
EID
, how_did_gig_go
, CASE
WHEN rn <= 10 THEN how_did_gig_go
ELSE 0
END AS latest10_how_did_gig_go
FROM (
SELECT
#num + 1 AS rn
, tasks.EID
, reports.how_did_gig_go
FROM tasks
INNER JOIN reports ON tasks.EID = reports.eid
CROSS JOIN ( SELECT #num := 0 AS n ) AS v
WHERE priority IS NOT NULL
AND partners_name IS NOT NULL
AND mike IS NOT NULL
ORDER BY tasks.show_date DESC
) AS d
In future (MySQL 8.x) ROW_NUMBER() OVER(order by tasks.show_date DESC) would be a better method than the "roll your own" row numbering (using #num+1) shown before.
There are two type of questions there 1.Passage and 2.Normal questions.
usally in test i want to pick random questions which consist type_id=0 in that if type=1 question come the the next passage should be relates to that question(Comprehension question should come in sequential). By using the below query i am able to get the questions
SELECT *
FROM tbl_testquestion
ORDER BY
CASE
WHEN type_id=0 THEN RAND()
WHEN type_id=1 THEN qu_id
END ASC
all the passage questions are coming last
and i have limit of 40 questions for test and in the table i have 50 passage questions and 70 Normal questions.
How can i write a query to call passage questions in between normal
questions.
EXAMPLE
1.who is the president of America.?(type_id=0)
2.A,B,C are 3 students Aname is "Arun" B name is "Mike" C name is "Jhon"(type_id=1)
who is C from the above passage
3.A,B,C are 3 students Aname is "Arun" B name is "Mike" C name is "Jhon"(type_id=1)
who is A from the above passage
4.Who is CEO of Facebook.?(type_id=0)
Form the Above 4 question we will pick random if Question 1 comes in that rand() no problem when the question 2 comes in the rand() the next question should be sequential. it means next question should be 3 after that passage questions completed it should switch back to rand() functionality
I think that the design of your database should be improved, but I’m going to answer your question as it stands.
I think I have a rather simple solution, which I can express in portable SQL without CTE’s.
It works this way: let’s assign two numbers to each row, call them major (an integer, just to be safe let’s make it a multiple of ten) and minor (a float between 0 and 1). For type 0 questions, minor is always 0. Each type 1 question relating to the same passage gets the same major (we do this with a join with a grouped subselect). We then order the table by the sum of the two values.
It will be slow, because it joins using a text field. It would be better if each distinct passage_description had an integer id to be used for the join.
I assume that all type 0 questions have empty or null passage_description, while type 1 questions have them non-empty (it would make no sense otherwise.)
I assume you have a RAND() function which yields floating values between 0 and 1.
Here we go:
SELECT u.qu_id, u.type_id,
u.passage_description, u.passage_image,
u.cat_id, u.subcat_id,
u.question, u.q_instruction, u.qu_status
FROM (
SELECT grouped.major, RAND()+0.001 AS minor, t1.*
FROM tbl_testquestion t1
JOIN (SELECT 10*FLOOR(1000*RAND()) major, passage_description
FROM tbl_testquestion WHERE type_id = 1
GROUP BY passage_description) grouped
USING (passage_description)
-- LIMIT 39
UNION
SELECT 10*FLOOR(1000*RAND()) major, 0 minor, t0.*
FROM tbl_testquestion t0 WHERE type_id = 0
) u ORDER BY u.major+u.minor ASC LIMIT 40;
With the above query without modifications, there is still a small probability that you get questions of only one type. If you want to be sure that you have at least one type 0 question, you can uncomment the LIMIT 39 on the first part of the UNION. If you want at least two, then say LIMIT 38, and so on. All type 1 questions related to the same passage will be grouped together in one test; it is not guaranteed that all questions in the database related to that passage will be in the test, but in a comment above you mention that this can be “broke”.
Edited:
I added a small amount to minor, just to bypass the rare but possible case in which RAND() returns exactly zero. Since major goes by tens, the fact that minor might now be greater than one is immaterial.
Use the following, I haven't tested this so, if there are any errors please report back, I will correct them. $r is a random value produced by PHP for this query. You could do $r = rand(); before calling the query
SELECT * FROM (
UNION((
SELECT *, RAND()*(SELECT COUNT(*) FROM tbl_testquestions) as orderid
FROM tbl_testquestion
WHERE type_id=0
ORDER BY orderid
LIMIT 20
),(
SELECT *, MD5(CONCAT('$r', passage_description)) as orderid
FROM tbl_testquestion
WHERE type_id=1
ORDER BY orderid
LIMIT 20
))
) AS t1
ORDER BY orderid
Explanation: orderid will keep type_id=1 entries together as it would produce the same random sequence for the same passage questions.
Warning: Unless you add passage_id to the table, this question will work quite slowly.
Edit: Fixed the ordering (I hope), forgot that MYSQL generates random numbers between 0 and 1.
This is the solution for mysql,
sorry it is not so readable because mysql does not supports CTE like sql-server.
Maybe you can compare with sql-server CTE syntax to the bottom to better understand how it works.
select
d.*
, o.q_ix, rnd_ord -- this is only for your reference
from (
select *, floor(rand()*1000) as rnd_ord -- this is main order for questions and groups
from (
select * from (
select
(#r1 := #r1 - 1) as q_ix, -- this is row_number() (negative so we can keep group separated)
passage_description, 0 qu_id, type_id
from (
select distinct passage_description, type_id
from tbl_testquestion,
(SELECT #r1 := 0) v, -- this is the trick for row_number()
(SELECT #rnd_limit := -floor(rand()*3)) r -- this is the trick for dynamic random limit
where type_id=1
) p
order by passage_description -- order by for row_number()
) op
where q_ix < #rnd_limit
union all
select * from (
select
(#r2 := #r2 + 1) as q_ix, -- again row_number()
'' as passage_description, qu_id, type_id
from tbl_testquestion,
(SELECT #r2 := 0) v -- var for row_number
where type_id=0
order by qu_id -- order by for row_number()
) oq
) q
) o
-- look at double join for questions and groups
join tbl_testquestion d on
((d.passage_description = o.passage_description) and (d.type_id=1))
or
((d.qu_id=o.qu_id) and (d.type_id=0))
order by rnd_ord
limit 40
and this is the more readable sql-server syntax:
;with
p as (
-- select a random number of groups (0-2) and label groups (-1,-2)
select top (abs(checksum(NEWID())) % 3) -ROW_NUMBER() over (order by passage_description) p_id, passage_description
from (
select distinct passage_description
from d
where type_id=1
) x
),
q as (
-- label questions (1..n)
select ROW_NUMBER() over (order by qu_id) q_ix, qu_id
from d
where type_id=0
),
o as (
-- calculate final order
select *, ROW_NUMBER() over (order by newid()) rnd_ord
from (
select p.q_ix, passage_description, 0 qu_id from p
union all
select q.q_ix, '', qu_id from q
) x
)
select top 40
d.*
, o.rnd_ord, o.q_ix
from o
join d on
((d.passage_description = o.passage_description) and (d.type_id=1))
or
((d.qu_id = o.qu_id) and (d.type_id=0))
order by
rnd_ord
that's all
So I am working with a client to implement a similar system as the "badges and privileges system" on StackExchange. Although in her system, she is looking to use points and rewards for her staff. It's the same basic principle. The users are rewarded points for good team work and gain rewards from these points. I thought it would be handy to add the same kind of feature which SE uses to display these in the top nav bar, where it shows your rep and badges in order of the date you have earned either of them. This is my issue, I have found help retrieving the data together from the two separate tables but am not sure how I would display these results in order of date earned? As an example:
User ID #1 has earned 50 points on 18/12/2015 would be in ap_user_points table
User ID #1 has earned 'The Gift Voucher' reward on '17/12/2015'
If I simply:
echo $row8['reward'] . $row8['points_added']
It would echo as:
The Gift Voucher 50
Where I need it in order by date as:
50
The Gift Voucher
If you look at your rep and badge icon in the nav bar you'll see what I'm getting at here, it's a similar system.
<?php
$user_id = $_SESSION['userid'];
$sql8 = "
SELECT r.reward_id,
r.user_id,
r.reward as reward,
r.date_earned as date_earned,
r.badge_desc,
NULL AS points_added,
NULL AS added_for,
NULL AS date_added
FROM ap_user_rewards as r
WHERE r.user_id = '$user_id'
UNION ALL
SELECT NULL,
NULL,
NULL,
NULL,
NULL,
p.points_added AS points_added,
p.added_for AS added_for,
p.date_added AS date_added
FROM ap_user_points as p
WHERE p.user_id = '$user_id' ORDER BY date_earned DESC, date_added DESC;";
$result8 = $conn->query($sql8);
if ($result8->num_rows > 0) {
// output data of each row
while($row8 = $result8->fetch_assoc()) {
////// NOT SURE WHAT TO ECHO HERE?
}
}
?>
Add another column to the result set. In that new column, populate it from both queries... looks like it would be the date_added expression in the first query and the date_earned expression in the second query. When those are in the same column, then ordering is easy. (This also assumes that these expressions are of the same or compatible datatypes, preferably DATE, DATETIME or TIMESTAMP.)
Then you can order by ordinal position, e.g. ORDER BY 2 to order by the second column in the resultset.
SELECT a1
, b1
, NULL
, NULL
, a1 AS sortexpr
FROM ...
UNION ALL
SELECT NULL
, NULL
, x2
, y2
, x2 AS sortexpr
FROM ...
ORDER BY 5 DESC
That's just one possibility. If you can't add an extra column, to line up the expressions from the two queries, then you need a way to discriminate which query is returning the row. I typically include a literal as a discriminator column.
Then you can use implicit-style UNION syntax, wrapping the queries in parens...
( SELECT 'q1' AS `source`
, a1
, b1 AS date_earned
, NULL
, NULL AS date_added
FROM ...
)
UNION ALL
( SELECT 'q2' AS `source`
, NULL
, NULL AS date_earned
, x2
, y2 AS date_added
FROM ...
)
ORDER BY IF(`source`='q1',date_earned,date_added) DESC
Followup
I may have misunderstood the question. I though the question was how to get the rows from a UNION/UNION ALL returned in a particular order.
Personally, I would write the query to include a discriminator column, and then line up the columns as much as I could, so they would be processed the same.
As an example:
SELECT 'reward' AS `source`
, r.date_earned AS `seq`
, r.user_id AS `user_id`
, r.date_earned AS `date_earned`
, r.reward_id
, r.reward
, r.badge_desc
, NULL AS `points_added`
, NULL AS `added_for`
FROM r ...
UNION ALL
SELECT 'points' AS `source`
, p.date_added AS `seq`
, p.user_id AS `user_id`
, p.date_added AS `date_earned`
, NULL
, NULL
, NULL
, p.points_added AS `points_added`
, p.added_for AS `added_for`
FROM p ...
ORDER BY 2 DESC, 1 DESC
(It's probably not really necessary to return user_id, since we already know what the value will be. I've returned it here to demonstrate how the columns from the two resultsets can be "lined up".)
Then, when I fetched the rows...
if ( $row8['source'] == 'points' ) {
# process columns from a row of 'points' type
echo $row8['badge_desc'];
echo $row8['user_id'];
} elsif ( $row8['source'] == 'reward' ) {
# process columns from a row of 'reward' type
echo $row8['added_for'];
echo $row8['user_id'];
}
That's how I would do it.
I'm working with the join plus union plus group by query, and I developed a query something like mentioned below:
SELECT *
FROM (
(SELECT countries_listing.id,
countries_listing.country,
1 AS is_country
FROM countries_listing
LEFT JOIN product_prices ON (product_prices.country_id = countries_listing.id)
WHERE countries_listing.status = 'Yes'
AND product_prices.product_id = '3521')
UNION
(SELECT countries_listing.id,
countries_listing.country,
0 AS is_country
FROM countries_listing
WHERE countries_listing.id NOT IN
(SELECT country_id
FROM product_prices
WHERE product_id='3521')
AND countries_listing.status='Yes')) AS partss
GROUP BY id
ORDER BY country
And I just realised that this query is taking a lot of time to load results, almost 8 seconds.
I was wondering if there is the possibility to optimize this query to the fastest one?
If I understand the logic correctly, you just want to add a flag for the country as to whether or not there is a price for a given product. I think you can use an exists clause to get what you want:
SELECT cl.id, cl.country,
(exists (SELECT 1
FROM product_prices pp
WHERE pp.country_id = cl.id AND
pp.product_id = '3521'
)
) as is_country
FROM countries_listing cl
WHERE cl.status = 'Yes'
ORDER BY country;
For performance, you want two indexes: countries_listing(status, country) and
product_prices(country_id, product_id)`.
Depending on how often it is executed, prepared statements could help. See PDO for more information.
I have a table in a MySQL database (level_records) which has 3 columns (id, date, reading). I want to put the differences between the most recent 20 readings (by date) into an array and then average them, to find the average difference.
I have looked everywhere, but no one seems to have a scenario quite like mine.
I will be very grateful for any help. Thanks.
SELECT AVG(difference)
FROM (
SELECT #next_reading - reading AS difference, #next_reading := reading
FROM (SELECT reading
FROM level_records
ORDER BY date DESC
LIMIT 20) AS recent20
CROSS JOIN (SELECT #next_reading := NULL) AS var
) AS recent_diffs
DEMO
If we consider "differences" to be signed, and if we ignore/exclude any rows that have a NULL values of reading...
If you want to return just the values of the difference between a reading and the immediately preceding reading (to get the latest nineteen differences), then you could do something like this:
SELECT d.diff
FROM ( SELECT e.reading - #prev_reading AS diff
, #prev_reading AS prev_reading
, #prev_reading := e.reading AS reading
FROM ( SELECT r.date
, r.reading
FROM level_records r
CROSS
JOIN (SELECT #prev_reading := NULL) p
ORDER BY r.date DESC
LIMIT 20
) e
ORDER BY e.date ASC
) d
That'll get you the rows returned from MySQL and you can monkey with them in PHP however you want. (The question of how to monkey around with arrays in PHP is a question that doesn't really have anything to do with MySQL.)
If you want to know how to return rows from a SQL resultset into a PHP array, that doesn't really have anything to do with "latest twenty", "difference", or "average" at all. You'd use the same pattern you'd use for returning the result from any query. There's nothing at all unique about that, there are plenty of examples of that, (most of them unfortunately using the deprecated mysql_ interface; for new development, you want to use either PDO or mysqli_.
If you mean by "all 19 sets of differences" that you want to get the difference between a reading and every other reading, and do that for each reading, such that you get a total of 380 rows ( = 20 * (20-1) rows ) then:
SELECT a.reading - b.reading AS diff
, a.id AS a_id
, a.date AS a_date
, a.reading AS a_reading
, b.id AS b_id
, b.date AS b_date
, b.reading AS b_reading
FROM ( SELECT aa.id
, aa.date
, aa.reading
FROM level_record aa
WHERE aa.reading IS NOT NULL
ORDER BY aa.date DESC, aa.id DESC
LIMIT 20
) a
JOIN ( SELECT bb.id
, bb.date
, bb.reading
FROM level_record bb
WHERE bb.reading IS NOT NULL
ORDER BY bb.date DESC, bb.id DESC
LIMIT 20
) b
WHERE a.id <> b.id
ORDER BY a.date DESC, b.date DESC
Sometimes, we only want differences in one direction, that is, if we have the difference between r13 and r15, we essentially already have the inverse, the difference between r15 and f13. And sometimes, it's more convenient to have the inverse copies.
What query you run really depends on what result set you want returned.
If the goal is to get an "average", then rather than monkeying with PHP arrays, we know that the average of the differences between the latest twenty readings will be the same as the difference between the first and last readings (in the latest twenty), divided by nineteen.
If we only want to return a row if there are at least twenty readings available, then something like this:
SELECT (l.reading - f.reading)/19 AS avg_difference
FROM ( SELECT ll.reading
FROM level_records ll
WHERE ll.reading IS NOT NULL
ORDER BY ll.date DESC LIMIT 1
) l
CROSS
JOIN (SELECT ff.reading
FROM level_records ff
WHERE ff.reading IS NOT NULL
ORDER BY ff.date DESC LIMIT 19,1
) f
NOTE: That query will only return a row only if there are at least twenty rows with non-NULL values of reading in the level_records table.
For the more general case, if there are fewer than twenty rows in the table (i.e. fewer than nineteen differences) and we want an average of the differences between the latest available rows, we can do something like this:
SELECT (l.reading - f.reading)/f.cnt AS avg_difference
FROM ( SELECT ll.reading
FROM level_records ll
WHERE ll.reading IS NOT NULL
ORDER BY ll.date DESC
LIMIT 1
) l
CROSS
JOIN (SELECT ee.reading
, ee.cnt
FROM ( SELECT e.date
, e.reading
, (#i := #i + 1) AS cnt
FROM level_records e
CROSS
JOIN (SELECT #i := -1) i
WHERE e.reading IS NOT NULL
ORDER BY e.date DESC
LIMIT 20
) ee
ORDER BY ee.date ASC
LIMIT 1
) f
But, if we need to treat "differences" as unsigned (that is, we are taking the absolute value of the differences between the readings),
then we'd need to get the actual differences between the readings, and then average the absolute values of the differences...
then we could do make use of a MySQL user variable to keep track of the "previous" reading, and have that available when we process the next row, so we can get the difference between them, something like this:
SELECT AVG(d.abs_diff)
FROM ( SELECT ABS(e.reading - #prev_reading) AS abs_diff
, #prev_reading AS prev_reading
, #prev_reading := e.reading AS reading
FROM ( SELECT r.date
, r.reading
FROM level_records r
CROSS
JOIN (SELECT #prev_reading := NULL) p
ORDER BY r.date DESC
LIMIT 20
) e
ORDER BY e.date ASC
) d