Get row_number of a specificatet column MySQL [duplicate] - php

Is there a nice way in MySQL to replicate the SQL Server function ROW_NUMBER()?
For example:
SELECT
col1, col2,
ROW_NUMBER() OVER (PARTITION BY col1, col2 ORDER BY col3 DESC) AS intRow
FROM Table1
Then I could, for example, add a condition to limit intRow to 1 to get a single row with the highest col3 for each (col1, col2) pair.

There is no ranking functionality in MySQL. The closest you can get is to use a variable:
SELECT t.*,
#rownum := #rownum + 1 AS rank
FROM YOUR_TABLE t,
(SELECT #rownum := 0) r
so how would that work in my case? I'd need two variables, one for each of col1 and col2? Col2 would need resetting somehow when col1 changed..?
Yes. If it were Oracle, you could use the LEAD function to peak at the next value. Thankfully, Quassnoi covers the logic for what you need to implement in MySQL.

I want the row with the single highest col3 for each (col1, col2) pair.
That's a groupwise maximum, one of the most commonly-asked SQL questions (since it seems like it should be easy, but actually it kind of isn't).
I often plump for a null-self-join:
SELECT t0.col3
FROM table AS t0
LEFT JOIN table AS t1 ON t0.col1=t1.col1 AND t0.col2=t1.col2 AND t1.col3>t0.col3
WHERE t1.col1 IS NULL;
“Get the rows in the table for which no other row with matching col1,col2 has a higher col3.” (You will notice this and most other groupwise-maximum solutions will return multiple rows if more than one row has the same col1,col2,col3. If that's a problem you may need some post-processing.)

I always end up following this pattern. Given this table:
+------+------+
| i | j |
+------+------+
| 1 | 11 |
| 1 | 12 |
| 1 | 13 |
| 2 | 21 |
| 2 | 22 |
| 2 | 23 |
| 3 | 31 |
| 3 | 32 |
| 3 | 33 |
| 4 | 14 |
+------+------+
You can get this result:
+------+------+------------+
| i | j | row_number |
+------+------+------------+
| 1 | 11 | 1 |
| 1 | 12 | 2 |
| 1 | 13 | 3 |
| 2 | 21 | 1 |
| 2 | 22 | 2 |
| 2 | 23 | 3 |
| 3 | 31 | 1 |
| 3 | 32 | 2 |
| 3 | 33 | 3 |
| 4 | 14 | 1 |
+------+------+------------+
By running this query, which doesn't need any variable defined:
SELECT a.i, a.j, count(*) as row_number FROM test a
JOIN test b ON a.i = b.i AND a.j >= b.j
GROUP BY a.i, a.j

SELECT
#i:=#i+1 AS iterator,
t.*
FROM
tablename AS t,
(SELECT #i:=0) AS foo

From MySQL 8.0.0 and above you could natively use windowed functions.
1.4 What Is New in MySQL 8.0:
Window functions.
MySQL now supports window functions that, for each row from a query, perform a calculation using rows related to that row. These include functions such as RANK(), LAG(), and NTILE(). In addition, several existing aggregate functions now can be used as window functions; for example, SUM() and AVG().
ROW_NUMBER() over_clause :
Returns the number of the current row within its partition. Rows numbers range from 1 to the number of partition rows.
ORDER BY affects the order in which rows are numbered. Without ORDER BY, row numbering is indeterminate.
Demo:
CREATE TABLE Table1(
id INT AUTO_INCREMENT PRIMARY KEY, col1 INT,col2 INT, col3 TEXT);
INSERT INTO Table1(col1, col2, col3)
VALUES (1,1,'a'),(1,1,'b'),(1,1,'c'),
(2,1,'x'),(2,1,'y'),(2,2,'z');
SELECT
col1, col2,col3,
ROW_NUMBER() OVER (PARTITION BY col1, col2 ORDER BY col3 DESC) AS intRow
FROM Table1;
DBFiddle Demo

Check out this Article, it shows how to mimic SQL ROW_NUMBER() with a partition by in MySQL. I ran into this very same scenario in a WordPress Implementation. I needed ROW_NUMBER() and it wasn't there.
http://www.explodybits.com/2011/11/mysql-row-number/
The example in the article is using a single partition by field. To partition by additional fields you could do something like this:
SELECT #row_num := IF(#prev_value=concat_ws('',t.col1,t.col2),#row_num+1,1) AS RowNumber
,t.col1
,t.col2
,t.Col3
,t.col4
,#prev_value := concat_ws('',t.col1,t.col2)
FROM table1 t,
(SELECT #row_num := 1) x,
(SELECT #prev_value := '') y
ORDER BY t.col1,t.col2,t.col3,t.col4
Using concat_ws handles null's. I tested this against 3 fields using an int, date, and varchar. Hope this helps. Check out the article as it breaks this query down and explains it.

I would also vote for Mosty Mostacho's solution with minor modification to his query code:
SELECT a.i, a.j, (
SELECT count(*) from test b where a.j >= b.j AND a.i = b.i
) AS row_number FROM test a
Which will give the same result:
+------+------+------------+
| i | j | row_number |
+------+------+------------+
| 1 | 11 | 1 |
| 1 | 12 | 2 |
| 1 | 13 | 3 |
| 2 | 21 | 1 |
| 2 | 22 | 2 |
| 2 | 23 | 3 |
| 3 | 31 | 1 |
| 3 | 32 | 2 |
| 3 | 33 | 3 |
| 4 | 14 | 1 |
+------+------+------------+
for the table:
+------+------+
| i | j |
+------+------+
| 1 | 11 |
| 1 | 12 |
| 1 | 13 |
| 2 | 21 |
| 2 | 22 |
| 2 | 23 |
| 3 | 31 |
| 3 | 32 |
| 3 | 33 |
| 4 | 14 |
+------+------+
With the only difference that the query doesn't use JOIN and GROUP BY, relying on nested select instead.

I would define a function:
delimiter $$
DROP FUNCTION IF EXISTS `getFakeId`$$
CREATE FUNCTION `getFakeId`() RETURNS int(11)
DETERMINISTIC
begin
return if(#fakeId, #fakeId:=#fakeId+1, #fakeId:=1);
end$$
then I could do:
select getFakeId() as id, t.* from table t, (select #fakeId:=0) as t2;
Now you don't have a subquery, which you can't have in views.

query for row_number in mysql
set #row_number=0;
select (#row_number := #row_number +1) as num,id,name from sbs

There is no funtion like rownum, row_num() in MySQL but the way around is like below:
select
#s:=#s+1 serial_no,
tbl.*
from my_table tbl, (select #s:=0) as s;

Important: Please consider upgrading to MySQL 8+ and use the defined and documented ROW_NUMBER() function, and ditch old hacks tied to a feature limited ancient version of MySQL
Now here's one of those hacks:
The answers here that use in-query variables mostly/all seem to ignore the fact that the documentation says (paraphrase):
Don't rely on items in the SELECT list being evaluated in order from top to bottom. Don't assign variables in one SELECT item and use them in another one
As such, there's a risk they will churn out the wrong answer, because they typically do a
select
(row number variable that uses partition variable),
(assign partition variable)
If these are ever evaluated bottom up, the row number will stop working (no partitions)
So we need to use something with a guaranteed order of execution. Enter CASE WHEN:
SELECT
t.*,
#r := CASE
WHEN col = #prevcol THEN #r + 1
WHEN (#prevcol := col) = null THEN null
ELSE 1 END AS rn
FROM
t,
(SELECT #r := 0, #prevcol := null) x
ORDER BY col
As outline ld, order of assignment of prevcol is important - prevcol has to be compared to the current row's value before we assign it a value from the current row (otherwise it would be the current rows col value, not the previous row's col value).
Here's how this fits together:
The first WHEN is evaluated. If this row's col is the same as the previous row's col then #r is incremented and returned from the CASE. This return led values is stored in #r. It's a feature of MySQL that assignment returns the new value of what is assigned into #r into the result rows.
For the first row on the result set, #prevcol is null (it is initialised to null in the subquery) so this predicate is false. This first predicate also returns false every time col changes (current row is different to previous row). This causes the second WHEN to be evaluated.
The second WHEN predicate is always false, and it exists purely to assign a new value to #prevcol. Because this row's col is different to the previous row's col (we know this because if it were the same, the first WHEN would have been used), we have to assign the new value to keep it for testing next time. Because the assignment is made and then the result of the assignment is compared with null, and anything equated with null is false, this predicate is always false. But at least evaluating it did its job of keeping the value of col from this row, so it can be evaluated against the next row's col value
Because the second WHEN is false, it means in situations where the column we are partitioning by (col) has changed, it is the ELSE that gives a new value for #r, restarting the numbering from 1
We this get to a situation where this:
SELECT
t.*,
ROW_NUMBER() OVER(PARTITION BY pcol1, pcol2, ... pcolX ORDER BY ocol1, ocol2, ... ocolX) rn
FROM
t
Has the general form:
SELECT
t.*,
#r := CASE
WHEN col1 = #pcol1 AND col2 = #pcol2 AND ... AND colX = #pcolX THEN #r + 1
WHEN (#pcol1 := pcol1) = null OR (#pcol2 := col2) = null OR ... OR (#pcolX := colX) = null THEN null
ELSE 1
END AS rn
FROM
t,
(SELECT #r := 0, #pcol1 := null, #pcol2 := null, ..., #pcolX := null) x
ORDER BY pcol1, pcol2, ..., pcolX, ocol1, ocol2, ..., ocolX
Footnotes:
The p in pcol means "partition", the o in ocol means "order" - in the general form I dropped the "prev" from the variable name to reduce visual clutter
The brackets around (#pcolX := colX) = null are important. Without them you'll assign null to #pcolX and things stop working
It's a compromise that the result set has to be ordered by the partition columns too, for the previous column compare to work out. You can't thus have your rownumber ordered according to one column but your result set ordered to another You might be able to resolve this with subqueries but I believe the docs also state that subquery ordering may be ignored unless LIMIT is used and this could impact performance
I haven't delved into it beyond testing that the method works, but if there is a risk that the predicates in the second WHEN will be optimised away (anything compared to null is null/false so why bother running the assignment) and not executed, it also stops. This doesn't seem to happen in my experience but I'll gladly accept comments and propose solution if it could reasonably occur
It may be wise to cast the nulls that create #pcolX to the actual types of your columns, in the subquery that creates the #pcolX variables, viz: select #pcol1 := CAST(null as INT), #pcol2 := CAST(null as DATE)

The solution I found to work the best was using a subquery like this:
SELECT
col1, col2,
(
SELECT COUNT(*)
FROM Table1
WHERE col1 = t1.col1
AND col2 = t1.col2
AND col3 > t1.col3
) AS intRow
FROM Table1 t1
The PARTITION BY columns just get compared with '=' and separated by AND. The ORDER BY columns would be compared with '<' or '>', and separated by OR.
I've found this to be very flexible, even if it is a little bit costly.

The rownumber functionality can't be mimicked. You might get the results you expect, but you'll most likely get disappointed at some stage.
Here's what mysql documentation says:
For other statements, such as SELECT, you might get the results you expect, but this is not guaranteed. In the following statement, you might think that MySQL will evaluate #a first and then do an assignment second:
SELECT #a, #a:=#a+1, ...;
However, the order of evaluation for expressions involving user variables is undefined.
Regards,
Georgi.

MariaDB 10.2 is implementing "Window Functions", including RANK(), ROW_NUMBER() and several other things:
https://mariadb.com/kb/en/mariadb/window-functions/
Based on a talk at Percona Live this month, they are reasonably well optimized.
The syntax is identical to the code in the Question.

MySQL has supported the ROW_NUMBER() since version 8.0+.
If you use MySQL 8.0 or later, check it out ROW_NUMBER() function.
Otherwise, you have emulate ROW_NUMBER() function.
The row_number() is a ranking function that returns a sequential number of a row, starting from 1 for the first row.
for older version,
SELECT t.*,
#rowid := #rowid + 1 AS ROWID
FROM TABLE t,
(SELECT #rowid := 0) dummy;

This allows the same functionality that ROW_NUMBER() AND PARTITION BY provides to be achieved in MySQL
SELECT #row_num := IF(#prev_value=GENDER,#row_num+1,1) AS RowNumber
FirstName,
Age,
Gender,
#prev_value := GENDER
FROM Person,
(SELECT #row_num := 1) x,
(SELECT #prev_value := '') y
ORDER BY Gender, Age DESC

I don't see any simple answer covering the "PARTITION BY" part so here's mine :
SELECT
*
FROM (
select
CASE WHEN #partitionBy_1 = l THEN #row_number:=#row_number+1 ELSE #row_number:=1 END AS i
, #partitionBy_1:=l AS p
, t.*
from (
select #row_number:=0,#partitionBy_1:=null
) as x
cross join (
select 1 as n, 'a' as l
union all
select 1 as n, 'b' as l
union all
select 2 as n, 'b' as l
union all
select 2 as n, 'a' as l
union all
select 3 as n, 'a' as l
union all
select 3 as n, 'b' as l
) as t
ORDER BY l, n
) AS X
where i > 1
The ORDER BY clause must reflect your ROW_NUMBER need. Thus there's already a clear limitation: you can't have several ROW_NUMBER "emulation" of this form at the same time.
The order of the "computed column" matters. If you have mysql compute those column in another order, it might not work.
In this simple example I only put one but you can have several "PARTITION BY" parts
CASE WHEN #partitionBy_1 = part1 AND #partitionBy_2 = part2 [...] THEN #row_number:=#row_number+1 ELSE #row_number:=1 END AS i
, #partitionBy_1:=part1 AS P1
, #partitionBy_2:=part2 AS P2
[...]
FROM (
SELECT #row_number:=0,#partitionBy_1:=null,#partitionBy_2:=null[...]
) as x

This could also be a solution:
SET #row_number = 0;
SELECT
(#row_number:=#row_number + 1) AS num, firstName, lastName
FROM
employees

Solutions with cross join and comma won't work if your query has GROUP BY statement. For such cases you can use subselect:
SELECT (#row_number := #row_number + 1) AS rowNumber, res.*
FROM
(
SELECT SUM(r.amount)
FROM Results r
WHERE username = 1
GROUP BY r.amount
) res
CROSS JOIN (SELECT #row_number := 0) AS dummy

I think you can use DENSE_RANK() function here.
Example:
select `score`, DENSE_RANK() OVER( ORDER BY score desc ) as `rank` from Scores;
https://www.mysqltutorial.org/mysql-window-functions/mysql-dense_rank-function/

A bit late but may also help to someone who looks for answers...
Between rows/row_number example - recursive query that may be used in any SQL:
WITH data(row_num, some_val) AS
(
SELECT 1 row_num, 1 some_val FROM any_table --dual in Oracle
UNION ALL
SELECT row_num+1, some_val+row_num FROM data WHERE row_num < 20 -- any number
)
SELECT * FROM data
WHERE row_num BETWEEN 5 AND 10
/
ROW_NUM SOME_VAL
-------------------
5 11
6 16
7 22
8 29
9 37
10 46

Also a bit late but today I had the same need so I did search on Google and finally a simple general approach found here in Pinal Dave's article http://blog.sqlauthority.com/2014/03/09/mysql-reset-row-number-for-each-group-partition-by-row-number/
I wanted to focus on Paul's original question (that was my problem as well) so I summarize my solution as a working example.
Beacuse we want to partition over two column I would create a SET variable during the iteration to identify if a new group was started.
SELECT col1, col2, col3 FROM (
SELECT col1, col2, col3,
#n := CASE WHEN #v = MAKE_SET(3, col1, col2)
THEN #n + 1 -- if we are in the same group
ELSE 1 -- next group starts so we reset the counter
END AS row_number,
#v := MAKE_SET(3, col1, col2) -- we store the current value for next iteration
FROM Table1, (SELECT #n := 0, #v := NULL) r -- helper table for iteration with startup values
ORDER BY col1, col2, col3 DESC -- because we want the row with maximum value
) x WHERE row_number = 1 -- and here we select exactly the wanted row from each group
The 3 means at the first parameter of MAKE_SET that I want both value in the SET (3=1|2).
Of course if we do not have two or more columns constructing the groups we can eliminate the MAKE_SET operation. The construction is exactly the same. This is working for me as required. Many thanks to Pinal Dave for his clear demonstration.

This is not the most robust solution - but if you're just looking to create a partitioned rank on a field with only a few different values, it may not be unwieldily to use some case when logic with as many variables as you require.
Something like this has worked for me in the past:
SELECT t.*,
CASE WHEN <partition_field> = #rownum1 := #rownum1 + 1
WHEN <partition_field> = #rownum2 := #rownum2 + 1
...
END AS rank
FROM YOUR_TABLE t,
(SELECT #rownum1 := 0) r1, (SELECT #rownum2 := 0) r2
ORDER BY <rank_order_by_field>
;
Hope that makes sense / helps!

This Work perfectly for me to create RowNumber when we have more than one column. In this case two column.
SELECT #row_num := IF(#prev_value= concat(`Fk_Business_Unit_Code`,`NetIQ_Job_Code`), #row_num+1, 1) AS RowNumber,
`Fk_Business_Unit_Code`,
`NetIQ_Job_Code`,
`Supervisor_Name`,
#prev_value := concat(`Fk_Business_Unit_Code`,`NetIQ_Job_Code`)
FROM (SELECT DISTINCT `Fk_Business_Unit_Code`,`NetIQ_Job_Code`,`Supervisor_Name`
FROM Employee
ORDER BY `Fk_Business_Unit_Code`, `NetIQ_Job_Code`, `Supervisor_Name` DESC) z,
(SELECT #row_num := 1) x,
(SELECT #prev_value := '') y
ORDER BY `Fk_Business_Unit_Code`, `NetIQ_Job_Code`,`Supervisor_Name` DESC

MySQL Since version 8, supports ROW_NUMBER(), so you can use the below query as you would use in SQL Server
SELECT
col1, col2,
ROW_NUMBER() OVER (PARTITION BY col1, col2 ORDER BY col3 DESC) AS intRow
FROM Table1
I also tested it in Maria DB 10.4.21. It works there as well.

for the partioning over anothe column one way is that described by #abcdn. However, it has a low performance. I propose use this code which does not require joining a table with itself:
Considee the same table.
you can get paritioning like this:
set #row_num := 0;
set #j:= 0;
select IF(j= #j, #row_num := #row_num + 1, #row_num := 1) as row_num,
i, #j:= j as j
from tbl fh
order by j, i;
the reult would be like this :
The advantage is we do not need to join table with itself

Related

MySQL Select top 10 items for each category [duplicate]

This question already has answers here:
Get top n records for each group of grouped results
(12 answers)
Closed 6 years ago.
Hello I have a mysql database in which has multiple categories. I would like to be able to pull only the first 10 items per category for example:
I have the following table, I would like to pull the first 2 rows for name = a, same for name = b and name = c
-----------------------------------
name | value | description | logo
-----------------------------------
a | 2.00 | its a letter| image
-----------------------------------
a | 5.00 | its a letter| image
-----------------------------------
b | 6.00 | its a letter| image
-----------------------------------
c | 3.00 | its a letter| image
-----------------------------------
c | 1.00 | its a letter| image
------------------------------------
This is what I have so farm post filter is a string of objects that comes in when the call is made. unfortunately it only gives me the first 10 of everything together, if you can point me in the right direction that would be great thank you!
code:
SELECT *
FROM object_list
Where object IN (".$_POST['filter'].")
ORDER BY date DESC, object ASC,id DESC
You can get groups along with element count by the below query:
SELECT name, value,
#count := IF(#value = name, #count + 1, 1) AS count,
#value := name AS some_value
FROM test, (SELECT #count := 1, #value := NULL) a
WHERE test.name in ('a', 'b')
Now, if you need to restrict the rows to 2 per group then you just need to wrap this query into another select and add a criteria, e.g.:
SELECT *
FROM (
SELECT name, value,
#count := IF(#value = name, #count + 1, 1) AS count,
#value := name AS some_value
FROM test, (SELECT #count := 1, #value := NULL) a
WHERE test.name in ('a', 'b')) a
WHERE a.count <= 2;
Here's the SQL Fiddle.
Does this work?
SELECT
yourtable.*
FROM
yourtable
JOIN (
SELECT
t1.name,
t1.value,
COUNT(t2.name) AS theCount
FROM yourtable t1
LEFT JOIN yourtable t2 ON t1.name = t2.name AND t1.value > t2.value
WHERE t1.name in ('a', 'b')
GROUP BY t1.name, t1.value
HAVING theCount < 2
) AS dt USING (name, value);
Source: http://thenoyes.com/littlenoise/?p=36

Count total marks and set rank using mysql query [duplicate]

Is there a nice way in MySQL to replicate the SQL Server function ROW_NUMBER()?
For example:
SELECT
col1, col2,
ROW_NUMBER() OVER (PARTITION BY col1, col2 ORDER BY col3 DESC) AS intRow
FROM Table1
Then I could, for example, add a condition to limit intRow to 1 to get a single row with the highest col3 for each (col1, col2) pair.
There is no ranking functionality in MySQL. The closest you can get is to use a variable:
SELECT t.*,
#rownum := #rownum + 1 AS rank
FROM YOUR_TABLE t,
(SELECT #rownum := 0) r
so how would that work in my case? I'd need two variables, one for each of col1 and col2? Col2 would need resetting somehow when col1 changed..?
Yes. If it were Oracle, you could use the LEAD function to peak at the next value. Thankfully, Quassnoi covers the logic for what you need to implement in MySQL.
I want the row with the single highest col3 for each (col1, col2) pair.
That's a groupwise maximum, one of the most commonly-asked SQL questions (since it seems like it should be easy, but actually it kind of isn't).
I often plump for a null-self-join:
SELECT t0.col3
FROM table AS t0
LEFT JOIN table AS t1 ON t0.col1=t1.col1 AND t0.col2=t1.col2 AND t1.col3>t0.col3
WHERE t1.col1 IS NULL;
“Get the rows in the table for which no other row with matching col1,col2 has a higher col3.” (You will notice this and most other groupwise-maximum solutions will return multiple rows if more than one row has the same col1,col2,col3. If that's a problem you may need some post-processing.)
I always end up following this pattern. Given this table:
+------+------+
| i | j |
+------+------+
| 1 | 11 |
| 1 | 12 |
| 1 | 13 |
| 2 | 21 |
| 2 | 22 |
| 2 | 23 |
| 3 | 31 |
| 3 | 32 |
| 3 | 33 |
| 4 | 14 |
+------+------+
You can get this result:
+------+------+------------+
| i | j | row_number |
+------+------+------------+
| 1 | 11 | 1 |
| 1 | 12 | 2 |
| 1 | 13 | 3 |
| 2 | 21 | 1 |
| 2 | 22 | 2 |
| 2 | 23 | 3 |
| 3 | 31 | 1 |
| 3 | 32 | 2 |
| 3 | 33 | 3 |
| 4 | 14 | 1 |
+------+------+------------+
By running this query, which doesn't need any variable defined:
SELECT a.i, a.j, count(*) as row_number FROM test a
JOIN test b ON a.i = b.i AND a.j >= b.j
GROUP BY a.i, a.j
SELECT
#i:=#i+1 AS iterator,
t.*
FROM
tablename AS t,
(SELECT #i:=0) AS foo
From MySQL 8.0.0 and above you could natively use windowed functions.
1.4 What Is New in MySQL 8.0:
Window functions.
MySQL now supports window functions that, for each row from a query, perform a calculation using rows related to that row. These include functions such as RANK(), LAG(), and NTILE(). In addition, several existing aggregate functions now can be used as window functions; for example, SUM() and AVG().
ROW_NUMBER() over_clause :
Returns the number of the current row within its partition. Rows numbers range from 1 to the number of partition rows.
ORDER BY affects the order in which rows are numbered. Without ORDER BY, row numbering is indeterminate.
Demo:
CREATE TABLE Table1(
id INT AUTO_INCREMENT PRIMARY KEY, col1 INT,col2 INT, col3 TEXT);
INSERT INTO Table1(col1, col2, col3)
VALUES (1,1,'a'),(1,1,'b'),(1,1,'c'),
(2,1,'x'),(2,1,'y'),(2,2,'z');
SELECT
col1, col2,col3,
ROW_NUMBER() OVER (PARTITION BY col1, col2 ORDER BY col3 DESC) AS intRow
FROM Table1;
DBFiddle Demo
Check out this Article, it shows how to mimic SQL ROW_NUMBER() with a partition by in MySQL. I ran into this very same scenario in a WordPress Implementation. I needed ROW_NUMBER() and it wasn't there.
http://www.explodybits.com/2011/11/mysql-row-number/
The example in the article is using a single partition by field. To partition by additional fields you could do something like this:
SELECT #row_num := IF(#prev_value=concat_ws('',t.col1,t.col2),#row_num+1,1) AS RowNumber
,t.col1
,t.col2
,t.Col3
,t.col4
,#prev_value := concat_ws('',t.col1,t.col2)
FROM table1 t,
(SELECT #row_num := 1) x,
(SELECT #prev_value := '') y
ORDER BY t.col1,t.col2,t.col3,t.col4
Using concat_ws handles null's. I tested this against 3 fields using an int, date, and varchar. Hope this helps. Check out the article as it breaks this query down and explains it.
I would also vote for Mosty Mostacho's solution with minor modification to his query code:
SELECT a.i, a.j, (
SELECT count(*) from test b where a.j >= b.j AND a.i = b.i
) AS row_number FROM test a
Which will give the same result:
+------+------+------------+
| i | j | row_number |
+------+------+------------+
| 1 | 11 | 1 |
| 1 | 12 | 2 |
| 1 | 13 | 3 |
| 2 | 21 | 1 |
| 2 | 22 | 2 |
| 2 | 23 | 3 |
| 3 | 31 | 1 |
| 3 | 32 | 2 |
| 3 | 33 | 3 |
| 4 | 14 | 1 |
+------+------+------------+
for the table:
+------+------+
| i | j |
+------+------+
| 1 | 11 |
| 1 | 12 |
| 1 | 13 |
| 2 | 21 |
| 2 | 22 |
| 2 | 23 |
| 3 | 31 |
| 3 | 32 |
| 3 | 33 |
| 4 | 14 |
+------+------+
With the only difference that the query doesn't use JOIN and GROUP BY, relying on nested select instead.
I would define a function:
delimiter $$
DROP FUNCTION IF EXISTS `getFakeId`$$
CREATE FUNCTION `getFakeId`() RETURNS int(11)
DETERMINISTIC
begin
return if(#fakeId, #fakeId:=#fakeId+1, #fakeId:=1);
end$$
then I could do:
select getFakeId() as id, t.* from table t, (select #fakeId:=0) as t2;
Now you don't have a subquery, which you can't have in views.
query for row_number in mysql
set #row_number=0;
select (#row_number := #row_number +1) as num,id,name from sbs
There is no funtion like rownum, row_num() in MySQL but the way around is like below:
select
#s:=#s+1 serial_no,
tbl.*
from my_table tbl, (select #s:=0) as s;
Important: Please consider upgrading to MySQL 8+ and use the defined and documented ROW_NUMBER() function, and ditch old hacks tied to a feature limited ancient version of MySQL
Now here's one of those hacks:
The answers here that use in-query variables mostly/all seem to ignore the fact that the documentation says (paraphrase):
Don't rely on items in the SELECT list being evaluated in order from top to bottom. Don't assign variables in one SELECT item and use them in another one
As such, there's a risk they will churn out the wrong answer, because they typically do a
select
(row number variable that uses partition variable),
(assign partition variable)
If these are ever evaluated bottom up, the row number will stop working (no partitions)
So we need to use something with a guaranteed order of execution. Enter CASE WHEN:
SELECT
t.*,
#r := CASE
WHEN col = #prevcol THEN #r + 1
WHEN (#prevcol := col) = null THEN null
ELSE 1 END AS rn
FROM
t,
(SELECT #r := 0, #prevcol := null) x
ORDER BY col
As outline ld, order of assignment of prevcol is important - prevcol has to be compared to the current row's value before we assign it a value from the current row (otherwise it would be the current rows col value, not the previous row's col value).
Here's how this fits together:
The first WHEN is evaluated. If this row's col is the same as the previous row's col then #r is incremented and returned from the CASE. This return led values is stored in #r. It's a feature of MySQL that assignment returns the new value of what is assigned into #r into the result rows.
For the first row on the result set, #prevcol is null (it is initialised to null in the subquery) so this predicate is false. This first predicate also returns false every time col changes (current row is different to previous row). This causes the second WHEN to be evaluated.
The second WHEN predicate is always false, and it exists purely to assign a new value to #prevcol. Because this row's col is different to the previous row's col (we know this because if it were the same, the first WHEN would have been used), we have to assign the new value to keep it for testing next time. Because the assignment is made and then the result of the assignment is compared with null, and anything equated with null is false, this predicate is always false. But at least evaluating it did its job of keeping the value of col from this row, so it can be evaluated against the next row's col value
Because the second WHEN is false, it means in situations where the column we are partitioning by (col) has changed, it is the ELSE that gives a new value for #r, restarting the numbering from 1
We this get to a situation where this:
SELECT
t.*,
ROW_NUMBER() OVER(PARTITION BY pcol1, pcol2, ... pcolX ORDER BY ocol1, ocol2, ... ocolX) rn
FROM
t
Has the general form:
SELECT
t.*,
#r := CASE
WHEN col1 = #pcol1 AND col2 = #pcol2 AND ... AND colX = #pcolX THEN #r + 1
WHEN (#pcol1 := pcol1) = null OR (#pcol2 := col2) = null OR ... OR (#pcolX := colX) = null THEN null
ELSE 1
END AS rn
FROM
t,
(SELECT #r := 0, #pcol1 := null, #pcol2 := null, ..., #pcolX := null) x
ORDER BY pcol1, pcol2, ..., pcolX, ocol1, ocol2, ..., ocolX
Footnotes:
The p in pcol means "partition", the o in ocol means "order" - in the general form I dropped the "prev" from the variable name to reduce visual clutter
The brackets around (#pcolX := colX) = null are important. Without them you'll assign null to #pcolX and things stop working
It's a compromise that the result set has to be ordered by the partition columns too, for the previous column compare to work out. You can't thus have your rownumber ordered according to one column but your result set ordered to another You might be able to resolve this with subqueries but I believe the docs also state that subquery ordering may be ignored unless LIMIT is used and this could impact performance
I haven't delved into it beyond testing that the method works, but if there is a risk that the predicates in the second WHEN will be optimised away (anything compared to null is null/false so why bother running the assignment) and not executed, it also stops. This doesn't seem to happen in my experience but I'll gladly accept comments and propose solution if it could reasonably occur
It may be wise to cast the nulls that create #pcolX to the actual types of your columns, in the subquery that creates the #pcolX variables, viz: select #pcol1 := CAST(null as INT), #pcol2 := CAST(null as DATE)
The solution I found to work the best was using a subquery like this:
SELECT
col1, col2,
(
SELECT COUNT(*)
FROM Table1
WHERE col1 = t1.col1
AND col2 = t1.col2
AND col3 > t1.col3
) AS intRow
FROM Table1 t1
The PARTITION BY columns just get compared with '=' and separated by AND. The ORDER BY columns would be compared with '<' or '>', and separated by OR.
I've found this to be very flexible, even if it is a little bit costly.
The rownumber functionality can't be mimicked. You might get the results you expect, but you'll most likely get disappointed at some stage.
Here's what mysql documentation says:
For other statements, such as SELECT, you might get the results you expect, but this is not guaranteed. In the following statement, you might think that MySQL will evaluate #a first and then do an assignment second:
SELECT #a, #a:=#a+1, ...;
However, the order of evaluation for expressions involving user variables is undefined.
Regards,
Georgi.
MariaDB 10.2 is implementing "Window Functions", including RANK(), ROW_NUMBER() and several other things:
https://mariadb.com/kb/en/mariadb/window-functions/
Based on a talk at Percona Live this month, they are reasonably well optimized.
The syntax is identical to the code in the Question.
MySQL has supported the ROW_NUMBER() since version 8.0+.
If you use MySQL 8.0 or later, check it out ROW_NUMBER() function.
Otherwise, you have emulate ROW_NUMBER() function.
The row_number() is a ranking function that returns a sequential number of a row, starting from 1 for the first row.
for older version,
SELECT t.*,
#rowid := #rowid + 1 AS ROWID
FROM TABLE t,
(SELECT #rowid := 0) dummy;
This allows the same functionality that ROW_NUMBER() AND PARTITION BY provides to be achieved in MySQL
SELECT #row_num := IF(#prev_value=GENDER,#row_num+1,1) AS RowNumber
FirstName,
Age,
Gender,
#prev_value := GENDER
FROM Person,
(SELECT #row_num := 1) x,
(SELECT #prev_value := '') y
ORDER BY Gender, Age DESC
I don't see any simple answer covering the "PARTITION BY" part so here's mine :
SELECT
*
FROM (
select
CASE WHEN #partitionBy_1 = l THEN #row_number:=#row_number+1 ELSE #row_number:=1 END AS i
, #partitionBy_1:=l AS p
, t.*
from (
select #row_number:=0,#partitionBy_1:=null
) as x
cross join (
select 1 as n, 'a' as l
union all
select 1 as n, 'b' as l
union all
select 2 as n, 'b' as l
union all
select 2 as n, 'a' as l
union all
select 3 as n, 'a' as l
union all
select 3 as n, 'b' as l
) as t
ORDER BY l, n
) AS X
where i > 1
The ORDER BY clause must reflect your ROW_NUMBER need. Thus there's already a clear limitation: you can't have several ROW_NUMBER "emulation" of this form at the same time.
The order of the "computed column" matters. If you have mysql compute those column in another order, it might not work.
In this simple example I only put one but you can have several "PARTITION BY" parts
CASE WHEN #partitionBy_1 = part1 AND #partitionBy_2 = part2 [...] THEN #row_number:=#row_number+1 ELSE #row_number:=1 END AS i
, #partitionBy_1:=part1 AS P1
, #partitionBy_2:=part2 AS P2
[...]
FROM (
SELECT #row_number:=0,#partitionBy_1:=null,#partitionBy_2:=null[...]
) as x
This could also be a solution:
SET #row_number = 0;
SELECT
(#row_number:=#row_number + 1) AS num, firstName, lastName
FROM
employees
Solutions with cross join and comma won't work if your query has GROUP BY statement. For such cases you can use subselect:
SELECT (#row_number := #row_number + 1) AS rowNumber, res.*
FROM
(
SELECT SUM(r.amount)
FROM Results r
WHERE username = 1
GROUP BY r.amount
) res
CROSS JOIN (SELECT #row_number := 0) AS dummy
I think you can use DENSE_RANK() function here.
Example:
select `score`, DENSE_RANK() OVER( ORDER BY score desc ) as `rank` from Scores;
https://www.mysqltutorial.org/mysql-window-functions/mysql-dense_rank-function/
A bit late but may also help to someone who looks for answers...
Between rows/row_number example - recursive query that may be used in any SQL:
WITH data(row_num, some_val) AS
(
SELECT 1 row_num, 1 some_val FROM any_table --dual in Oracle
UNION ALL
SELECT row_num+1, some_val+row_num FROM data WHERE row_num < 20 -- any number
)
SELECT * FROM data
WHERE row_num BETWEEN 5 AND 10
/
ROW_NUM SOME_VAL
-------------------
5 11
6 16
7 22
8 29
9 37
10 46
Also a bit late but today I had the same need so I did search on Google and finally a simple general approach found here in Pinal Dave's article http://blog.sqlauthority.com/2014/03/09/mysql-reset-row-number-for-each-group-partition-by-row-number/
I wanted to focus on Paul's original question (that was my problem as well) so I summarize my solution as a working example.
Beacuse we want to partition over two column I would create a SET variable during the iteration to identify if a new group was started.
SELECT col1, col2, col3 FROM (
SELECT col1, col2, col3,
#n := CASE WHEN #v = MAKE_SET(3, col1, col2)
THEN #n + 1 -- if we are in the same group
ELSE 1 -- next group starts so we reset the counter
END AS row_number,
#v := MAKE_SET(3, col1, col2) -- we store the current value for next iteration
FROM Table1, (SELECT #n := 0, #v := NULL) r -- helper table for iteration with startup values
ORDER BY col1, col2, col3 DESC -- because we want the row with maximum value
) x WHERE row_number = 1 -- and here we select exactly the wanted row from each group
The 3 means at the first parameter of MAKE_SET that I want both value in the SET (3=1|2).
Of course if we do not have two or more columns constructing the groups we can eliminate the MAKE_SET operation. The construction is exactly the same. This is working for me as required. Many thanks to Pinal Dave for his clear demonstration.
This is not the most robust solution - but if you're just looking to create a partitioned rank on a field with only a few different values, it may not be unwieldily to use some case when logic with as many variables as you require.
Something like this has worked for me in the past:
SELECT t.*,
CASE WHEN <partition_field> = #rownum1 := #rownum1 + 1
WHEN <partition_field> = #rownum2 := #rownum2 + 1
...
END AS rank
FROM YOUR_TABLE t,
(SELECT #rownum1 := 0) r1, (SELECT #rownum2 := 0) r2
ORDER BY <rank_order_by_field>
;
Hope that makes sense / helps!
This Work perfectly for me to create RowNumber when we have more than one column. In this case two column.
SELECT #row_num := IF(#prev_value= concat(`Fk_Business_Unit_Code`,`NetIQ_Job_Code`), #row_num+1, 1) AS RowNumber,
`Fk_Business_Unit_Code`,
`NetIQ_Job_Code`,
`Supervisor_Name`,
#prev_value := concat(`Fk_Business_Unit_Code`,`NetIQ_Job_Code`)
FROM (SELECT DISTINCT `Fk_Business_Unit_Code`,`NetIQ_Job_Code`,`Supervisor_Name`
FROM Employee
ORDER BY `Fk_Business_Unit_Code`, `NetIQ_Job_Code`, `Supervisor_Name` DESC) z,
(SELECT #row_num := 1) x,
(SELECT #prev_value := '') y
ORDER BY `Fk_Business_Unit_Code`, `NetIQ_Job_Code`,`Supervisor_Name` DESC
MySQL Since version 8, supports ROW_NUMBER(), so you can use the below query as you would use in SQL Server
SELECT
col1, col2,
ROW_NUMBER() OVER (PARTITION BY col1, col2 ORDER BY col3 DESC) AS intRow
FROM Table1
I also tested it in Maria DB 10.4.21. It works there as well.
for the partioning over anothe column one way is that described by #abcdn. However, it has a low performance. I propose use this code which does not require joining a table with itself:
Considee the same table.
you can get paritioning like this:
set #row_num := 0;
set #j:= 0;
select IF(j= #j, #row_num := #row_num + 1, #row_num := 1) as row_num,
i, #j:= j as j
from tbl fh
order by j, i;
the reult would be like this :
The advantage is we do not need to join table with itself

Which is best way of getting min and max date while comparing keys of 2 table

I am learning mysql, I have 2 tables, I have to compare table1 primary keys with table2 primary key, on successful match I need to get date min and max from table2 for this I have tried these commands, though I got result but it takes long time please let me know if there any good way to handle this case.
date format is like this
mysql> select sdate from table2
| 27-Apr-2000 11:50:00 AM |
| 27-Apr-2000 10:20:00 AM |
| 27-Apr-2000 08:30:00 AM |
| 20-Jan-1999 12:00:00 PM |
Commands I tried
mysql> select min(str_to_date(m.sdate,'%d-%M-%Y')) as date_min, max(str_to_date(m.sdate,'%d-%M-%Y')) as date_max from ( select distinct p.key1 as key1, p.key2 as key2 from table1 as p ) as T inner join table2 as m on T.key1 = m.key1 and T.key2 = m.key2 where m.sdate !='';
+------------+------------+
| date_min | date_max |
+------------+------------+
| 1989-02-24 | 2011-12-30 |
+------------+------------+
1 row in set, 11396 warnings (18.95 sec)
mysql> select min(str_to_date(m.sdate,'%d-%M-%Y')) as date_min, max(str_to_date(m.sdate,'%d-%M-%Y')) as date_max from ( select p.key1 as key1, p.key2 as key2 from table1 as p ) as T inner join table2 as m on T.key1 = m.key1 and T.key2 = m.key2 where m.sdate !='';
+------------+------------+
| date_min | date_max |
+------------+------------+
| 1989-02-24 | 2011-12-30 |
+------------+------------+
1 row in set, 11442 warnings (18.78 sec)
mysql> select min(str_to_date(m.sdate,'%d-%M-%Y')) as date_min, max(str_to_date(m.sdate,'%d-%M-%Y')) as date_max from table2 as m, table1 as p where p.key1 = m.key1 and p.key2 = m.key2 and m.sdate !='';
+------------+------------+
| date_min | date_max |
+------------+------------+
| 1989-02-24 | 2011-12-30 |
+------------+------------+
1 row in set, 11442 warnings (18.86 sec)
mysql>
None of the queries is able to make effective use of an index (ie. range scan operation) on the VARCHAR sdate column, because that column is "wrapped" in a function in the query. For optimum performance of queries of this form, ideally the sdate would be an actual MySQL DATETIME or TIMESTAMP datatype, or even a VARCHAR in canonical format. If that were the case, the optimizer would be able to make effective use of an index to quickly locate the "smallest" and "largest" date values, without a need to evaluate the STR_TO_DATE function for every flipping row in the table, and avoid the need for a sort operation to locate the "smallest" and "largest" values returned from the function.
With that (semi-rant) aside...
In the general case, to get a result equivalent to the first two queries in your question, a query of the form suggested in the answer from Gordon Linoff may be your best bet.
(We note the first two queries include a key2=key1 predicates, the third query has a key2=key2 predicate.)
If there are a large number of rows in table2, and if a large majority of those rows will "match" a row from table1, and there are relatively small number of distinct (key1,key2) values in table2, and if the (key1,key2) tuple is either unique or nearly unique in table1,
there's an outside chance that a query of this form may perform better:
SELECT MIN(q.sdate_min) AS date_min
, MAX(q.sdate_max) AS date_max
FROM ( SELECT m.key1
, m.key2
, MIN(STR_TO_DATE(m.sdate,'%d-%M-%Y')) AS sdate_min
, MAX(STR_TO_DATE(m.sdate,'%d-%M-%Y')) AS sdate_max
FROM table2 m
GROUP
BY m.key1
, m.key2
) q
JOIN table1 t
ON t.key1 = q.key1
AND t.key2 = q.key2
To improve performance of the inline view query, you are going to want an index on table2 with leading columns of key1 and key2 (in either order), and also including the sdate column. For example:
... ON table2 (key1, key2, sdate)
To improve performance of the JOIN operation, you are going to want an index on table1 with key1 and key2 as the leading columns in the index. For example:
... ON table1 (key1,key2)
or
... ON table1 (key2,key1)
(This assumes you will be using predicates of the form in your third query i.e. key1=key1 and key2=key**2**
If you will be using predicates of the form key1=key1 and key2=key**1**, then we'd adjust the query and indexes accordingly.
You can try this approach:
select min(str_to_date(m.sdate,'%d-%M-%Y')) as date_min,
max(str_to_date(m.sdate,'%d-%M-%Y')) as date_max
from table2 m
where exists (select 1
from table1 t
where t.key1 = m.key1 and t.key1 = m.key2
);
Then, create an index on table1(key1, key2) for performance.

PHP function to find the median of a column in MySQL

I have a database, db and in it a table, Table.
It looks somewhat like:
id | val
--------
1 | 45
2 | 35
3 | 23
4 | 49
5 | 67
6 | 12
7 | 0
8 | 87
9 | 46
(This is just an example data set. Actual data set is huge. And I need to work in least time possible.)
I need to find the median of the column val. Actually I need a php function to be used multiple times.
A similar question does exist: Simple way to calculate median with MySQL
I tried a few answers in this question, none of them worked for me. The accepted answer doesn't work since it used to work with an older version of SQL only.
PS: It should also work in the case of many duplicates.
just for fun i thought i try and do it all in MySQL, here's the sqlFiddle
SELECT
CASE
WHEN MOD((select count(*) as count from t),2)=1 THEN
(select val from
(select #row:=#row+1 as row,val
from t,(select #row:=0)r
order by val)t1
where t1.row = CEIL((select count(*) as count from t)/2)
)
ELSE
((select val from
(select #row:=#row+1 as row,val
from t,(select #row:=0)r
order by val)t1
where t1.row = (select count(*) as count from t)/2)+
(select val from
(select #row:=#row+1 as row,val
from t,(select #row:=0)r
order by val)t1
where t1.row = ((select count(*) as count from t)/2)+1))/2
END AS median
Just replace occurences of t with your table name, don't change t1.
Also if the table has no rows, it'll return NULL as median.
This query can be further reduced to the below (sqlFiddle)
SELECT #rowCount:=(select count(*) as count from t) AS rowcount,
(select AVG(val) from
(select #row:=#row+1 as row,val
from t,(select #row:=0)r
order by val)t1
where t1.row IN (FLOOR((#rowCount+1)/2),
CEIL((#rowCount+1)/2)
)
) as Median
It'll return 2 columns, a rowcount column and a median column. I put the rowcount column there because i didn't want to count from t multiple times like previous query.

Mysql: Gap detection query not detecting gaps

It seems as though my gap detection is just picking up whatever I set as the timedifference and doing it every interval based on that.
So here's an explanation of my data structure and what I'm after:
I have a database that's set up like this:
(Schema Name)
Historical
-CID int UQ AI NN
-ID Int PK
-Location Varchar(255)
-Status Varchar(255)
-Time datetime
My data comes in looking something like this (example 5 rows for selected ID)
433275 | 97 | MyLocation | OK | 2013-08-20 13:05:54
433275 | 97 | MyLocation | OK | 2013-08-20 13:00:54
433275 | 97 | MyLocation | OK | 2013-08-20 12:25:54
433275 | 97 | MyLocation | OK | 2013-08-20 12:20:54
433275 | 97 | MyLocation | OK | 2013-08-20 12:15:54
In the case above you'll notice that I'm missing data from 12:25:54 -> 13:00 for ID 97. I am trying to write a report that will tell me the: Start of downtime, Duration of DownTime, and End of Downtime (which I've been handling in php by adding timediff to Time)
Here's my code (php -> mysql) (non-working) as it stands now:
select *from (
SELECT
y.*,
TIMESTAMPDIFF(MINUTE, #prevDT, `Time`) AS timedifference,
#prevDT := `Time`
FROM ddHistorical y,
(SELECT #prevDT:=(SELECT MIN(`Time`) FROM ddHistorical)) vars
Where ID = '". mysql_real_escape_string($ID)."'
HAVING timedifference > 16
) s
order by Time desc
Limit 25";
You need two levels of subquery:
SELECT *
FROM (
SELECT y.*,
TIMESTAMPDIFF(MINUTE, #prevDT, `Time`) AS timedifference,
#prevDT := `Time`
FROM (SELECT *
FROM ddHistorical
WHERE ID = '97'
ORDER BY `Time`) y,
(SELECT #prevDT:=(SELECT MIN(`Time`) FROM ddHistorical)) vars) z
WHERE timedifference > 16
LIMIT 25
I'm actually not sure why the outermost level is needed. I tried without it (i.e. starting with SELECT y.*) and using HAVING timedifference > 16. For some reason, this reported a single row with timedifference = 45. But if I removed that HAVING clause, it showed all the rows, with the gap reported as 35 minutes. Usually, there's no difference between
SELECT ... HAVING <cond>
and
SELECT * FROM (SELECT ...) WHERE <cond>
I'm not sure why this query violates the rule -- I suspect it has something to do with the user-variables.
EDIT:
I think I've figured out why it didn't work with HAVING. MySQL is apparently evaluating that condition as soon as it calculates the timedifference column for each row, and discarding the row. When timedifference <= 16, it doesn't bother calculating the rest of the columns in the SELECT clause, so it never does #prevDT := Time. So until it gets past that condition, it's always comparing with MIN(Time).
Moving the timedifference check into an outer query forces it to calculate all the differences before filtering.
SQLFIDDLE
Please try this:
"
SELECT y.*,
TIMESTAMPDIFF(MINUTE, y.prevDT, y.`Time`) AS timedifference
FROM (SELECT w.*, (SELECT MAX(x.`Time`)
FROM ddHistorical x
WHERE x.ID = w.ID
AND x.`Time` < w.`Time`) AS prevDT
FROM ddHistorical w
WHERE w.ID = '". mysql_real_escape_string($ID)."') y
WHERE TIMESTAMPDIFF(MINUTE, y.prevDT, y.`Time`) > 16
ORDER BY y.`Time` DESC
LIMIT 25";

Categories