Mysql Limit column value repetition N times - php

I have two tables
Customer (idCustomer, ecc.. ecc..)
Comment (idCustomer, idComment, ecc.. ecc..)
obviously the two table are joined together, for example
SELECT * FROM Comment AS co
JOIN Customer AS cu ON cu.idCustomer = co.idCustomer
With this I select all comment from that table associated with is Customer, but now I wanna limit the number of Comment by 2 max Comment per Customer.
The first thing I see is to use GROUP BY cu.idCustomer but it limits only 1 Comment per Customer, but I wanna 2 Comment per Customer.
How can I achieve that?

One option in MySQL is server-side variables. For example:
set #num := 0, #customer := -1;
select *
from (
select idCustomer
, commentText
, #num := if(#customer = idCustomer, #num + 1, 1)
as row_number
, #customer := idCustomer
from Comments
order by
idCustomer, PostDate desc
) as co
join Customer cu
on co.idCustomer = cu.idCustomer
where co.row_number <= 2

This version doesn't require the SET operation:
select *
from (select idCustomer
, commentText
, #num := if(#customer = idCustomer, #num + 1, 1) as row_number
, #customer = idCustomer
from Comments
JOIN(SELECT #num := 0, #customer := 1) r
order by idCustomer, PostDate desc) as co
join Customer cu on co.idCustomer = cu.idCustomer
where co.row_number <= 2

SELECT * FROM Comments AS cm1
LEFT JOIN Comments AS cm2 ON cm1.idCustomer = cm2.idCustomer
LEFT JOIN Customer AS cu ON cm1.idCustomer = cu.idCustomer
WHERE cm1.idComment != cm2.idComment
GROUP BY cm1.idCustomer
However, if you are going to change the number of comments it's better to use Andomar's solution.

There is no need to use cursor, which is very slow. See my answer to Complicated SQL Query About Joining And Limitting. DENSE_RANK will do the trick without all cursor intricacies.

If you are using a scripting language such as PHP to process the results, you could limit the number of results shown per customer after running the query. Set up an array to hold all the results, set up another array to hold the number of results per customer and stop adding the query results to the result set after the count exceeds your limit like so:
$RESULTS = array();
$COUNTS = array();
$limit = 2;
$query = "SELECT customer_id, customer_name, customer_comment FROM customers ORDER BY RAND()";
$request = mysql_query($query);
while ($ROW = mysql_fetch_assoc($request))
{
$c = $ROW['customer_id'];
$n = $COUNTS[$c];
if ($n<$limit)
{
$RESULTS[] = $ROW;
$COUNTS[$c]++;
}
}
This guarantees only two comments per customer will be shown pulled randomly or however you want, the rest gets thrown out. Granted you are pulling ALL the results but this is (probably) faster than doing a complex join.

Related

Single query that allows alias with it's own limit

I would like to better optimize my code. I'd like to have a single query that allows an alias name to have it's own limit and also include a result with no limit.
Currently I'm using two queries like this:
// ALL TIME //
$mikep = mysqli_query($link, "SELECT tasks.EID, reports.how_did_gig_go FROM tasks INNER JOIN reports ON tasks.EID=reports.eid WHERE `priority` IS NOT NULL AND `partners_name` IS NOT NULL AND mike IS NOT NULL GROUP BY EID ORDER BY tasks.show_date DESC;");
$num_rows_mikep = mysqli_num_rows($mikep);
$rating_sum_mikep = 0;
while ($row = mysqli_fetch_assoc($mikep)) {
$rating_mikep = $row['how_did_gig_go'];
$rating_sum_mikep += $rating_mikep;
}
$average_mikep = $rating_sum_mikep/$num_rows_mikep;
// AND NOW WITH A LIMIT 10 //
$mikep_limit = mysqli_query($link, "SELECT tasks.EID, reports.how_did_gig_go FROM tasks INNER JOIN reports ON tasks.EID=reports.eid WHERE `priority` IS NOT NULL AND `partners_name` IS NOT NULL AND mike IS NOT NULL GROUP BY EID ORDER BY tasks.show_date DESC LIMIT 10;");
$num_rows_mikep_limit = mysqli_num_rows($mikep_limit);
$rating_sum_mikep_limit = 0;
while ($row = mysqli_fetch_assoc($mikep_limit)) {
$rating_mikep_limit = $row['how_did_gig_go'];
$rating_sum_mikep_limit += $rating_mikep_limit;
}
$average_mikep_limit = $rating_sum_mikep_limit/$num_rows_mikep_limit;
This allows me to show an all-time average and also an average over the last 10 reviews. Is it really necessary for me to set up two queries?
Also, I understand I could get the sum in the query, but not all the values are numbers, so I've actually converted them in PHP, but left out that code in order to try and simplify what is displayed in the code.
All-time average and average over the last 10 reviews
In the best case scenario, where your column how_did_gig_go was 100% numeric, a single query like this could work like so:
SELECT
AVG(how_did_gig_go) AS avg_how_did_gig_go
, SUM(CASE
WHEN rn <= 10 THEN how_did_gig_go
ELSE 0
END) / 10 AS latest10_avg
FROM (
SELECT
#num + 1 AS rn
, tasks.show_date
, reports.how_did_gig_go
FROM tasks
INNER JOIN reports ON tasks.EID = reports.eid
CROSS JOIN ( SELECT #num := 0 AS n ) AS v
WHERE priority IS NOT NULL
AND partners_name IS NOT NULL
AND mike IS NOT NULL
ORDER BY tasks.show_date DESC
) AS d
But; Unless all the "numbers" are in fact numeric you are doomed to sending every row back from the server for php to process unless you can clean-up the data in MySQL somehow.
You might avoid sending all that data twice if you establish a way for your php to use only the top 10 from the whole list. There are probably way of doing that in PHP.
If you wanted assistance in SQL to do that, then maybe having 2 columns would help, it would reduce the number of table scans.
SELECT
EID
, how_did_gig_go
, CASE
WHEN rn <= 10 THEN how_did_gig_go
ELSE 0
END AS latest10_how_did_gig_go
FROM (
SELECT
#num + 1 AS rn
, tasks.EID
, reports.how_did_gig_go
FROM tasks
INNER JOIN reports ON tasks.EID = reports.eid
CROSS JOIN ( SELECT #num := 0 AS n ) AS v
WHERE priority IS NOT NULL
AND partners_name IS NOT NULL
AND mike IS NOT NULL
ORDER BY tasks.show_date DESC
) AS d
In future (MySQL 8.x) ROW_NUMBER() OVER(order by tasks.show_date DESC) would be a better method than the "roll your own" row numbering (using #num+1) shown before.

Random content definitive method [duplicate]

How can I best write a query that selects 10 rows randomly from a total of 600k?
A great post handling several cases, from simple, to gaps, to non-uniform with gaps.
http://jan.kneschke.de/projects/mysql/order-by-rand/
For most general case, here is how you do it:
SELECT name
FROM random AS r1 JOIN
(SELECT CEIL(RAND() *
(SELECT MAX(id)
FROM random)) AS id)
AS r2
WHERE r1.id >= r2.id
ORDER BY r1.id ASC
LIMIT 1
This supposes that the distribution of ids is equal, and that there can be gaps in the id list. See the article for more advanced examples
SELECT column FROM table
ORDER BY RAND()
LIMIT 10
Not the efficient solution but works
Simple query that has excellent performance and works with gaps:
SELECT * FROM tbl AS t1 JOIN (SELECT id FROM tbl ORDER BY RAND() LIMIT 10) as t2 ON t1.id=t2.id
This query on a 200K table takes 0.08s and the normal version (SELECT * FROM tbl ORDER BY RAND() LIMIT 10) takes 0.35s on my machine.
This is fast because the sort phase only uses the indexed ID column. You can see this behaviour in the explain:
SELECT * FROM tbl ORDER BY RAND() LIMIT 10:
SELECT * FROM tbl AS t1 JOIN (SELECT id FROM tbl ORDER BY RAND() LIMIT 10) as t2 ON t1.id=t2.id
Weighted Version: https://stackoverflow.com/a/41577458/893432
I am getting fast queries (around 0.5 seconds) with a slow cpu, selecting 10 random rows in a 400K registers MySQL database non-cached 2Gb size. See here my code: Fast selection of random rows in MySQL
$time= microtime_float();
$sql='SELECT COUNT(*) FROM pages';
$rquery= BD_Ejecutar($sql);
list($num_records)=mysql_fetch_row($rquery);
mysql_free_result($rquery);
$sql="SELECT id FROM pages WHERE RAND()*$num_records<20
ORDER BY RAND() LIMIT 0,10";
$rquery= BD_Ejecutar($sql);
while(list($id)=mysql_fetch_row($rquery)){
if($id_in) $id_in.=",$id";
else $id_in="$id";
}
mysql_free_result($rquery);
$sql="SELECT id,url FROM pages WHERE id IN($id_in)";
$rquery= BD_Ejecutar($sql);
while(list($id,$url)=mysql_fetch_row($rquery)){
logger("$id, $url",1);
}
mysql_free_result($rquery);
$time= microtime_float()-$time;
logger("num_records=$num_records",1);
logger("$id_in",1);
logger("Time elapsed: <b>$time segundos</b>",1);
From book :
Choose a Random Row Using an Offset
Still another technique that avoids problems found in the preceding
alternatives is to count the rows in the data set and return a random
number between 0 and the count. Then use this number as an offset
when querying the data set
$rand = "SELECT ROUND(RAND() * (SELECT COUNT(*) FROM Bugs))";
$offset = $pdo->query($rand)->fetch(PDO::FETCH_ASSOC);
$sql = "SELECT * FROM Bugs LIMIT 1 OFFSET :offset";
$stmt = $pdo->prepare($sql);
$stmt->execute( $offset );
$rand_bug = $stmt->fetch();
Use this solution when you can’t assume contiguous key values and
you need to make sure each row has an even chance of being selected.
Its very simple and single line query.
SELECT * FROM Table_Name ORDER BY RAND() LIMIT 0,10;
Well if you have no gaps in your keys and they are all numeric you can calculate random numbers and select those lines. but this will probably not be the case.
So one solution would be the following:
SELECT * FROM table WHERE key >= FLOOR(RAND()*MAX(id)) LIMIT 1
which will basically ensure that you get a random number in the range of your keys and then you select the next best which is greater.
you have to do this 10 times.
however this is NOT really random because your keys will most likely not be distributed evenly.
It's really a big problem and not easy to solve fulfilling all the requirements, MySQL's rand() is the best you can get if you really want 10 random rows.
There is however another solution which is fast but also has a trade off when it comes to randomness, but may suit you better. Read about it here: How can i optimize MySQL's ORDER BY RAND() function?
Question is how random do you need it to be.
Can you explain a bit more so I can give you a good solution.
For example a company I worked with had a solution where they needed absolute randomness extremely fast. They ended up with pre-populating the database with random values that were selected descending and set to different random values afterwards again.
If you hardly ever update you could also fill an incrementing id so you have no gaps and just can calculate random keys before selecting... It depends on the use case!
How to select random rows from a table:
From here:
Select random rows in MySQL
A quick improvement over "table scan" is to use the index to pick up random ids.
SELECT *
FROM random, (
SELECT id AS sid
FROM random
ORDER BY RAND( )
LIMIT 10
) tmp
WHERE random.id = tmp.sid;
I improved the answer #Riedsio had. This is the most efficient query I can find on a large, uniformly distributed table with gaps (tested on getting 1000 random rows from a table that has > 2.6B rows).
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max := (SELECT MAX(id) FROM table)) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1)
Let me unpack what's going on.
#max := (SELECT MAX(id) FROM table)
I'm calculating and saving the max. For very large tables, there is a slight overhead for calculating MAX(id) each time you need a row
SELECT FLOOR(rand() * #max) + 1 as rand)
Gets a random id
SELECT id FROM table INNER JOIN (...) on id > rand LIMIT 1
This fills in the gaps. Basically if you randomly select a number in the gaps, it will just pick the next id. Assuming the gaps are uniformly distributed, this shouldn't be a problem.
Doing the union helps you fit everything into 1 query so you can avoid doing multiple queries. It also lets you save the overhead of calculating MAX(id). Depending on your application, this might matter a lot or very little.
Note that this gets only the ids and gets them in random order. If you want to do anything more advanced I recommend you do this:
SELECT t.id, t.name -- etc, etc
FROM table t
INNER JOIN (
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max := (SELECT MAX(id) FROM table)) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1) UNION
(SELECT id FROM table INNER JOIN (SELECT FLOOR(RAND() * #max) + 1 as rand) r on id > rand LIMIT 1)
) x ON x.id = t.id
ORDER BY t.id
All the best answers have been already posted (mainly those referencing the link http://jan.kneschke.de/projects/mysql/order-by-rand/).
I want to pinpoint another speed-up possibility - caching. Think of why you need to get random rows. Probably you want display some random post or random ad on a website. If you are getting 100 req/s, is it really needed that each visitor gets random rows? Usually it is completely fine to cache these X random rows for 1 second (or even 10 seconds). It doesn't matter if 100 unique visitors in the same 1 second get the same random posts, because the next second another 100 visitors will get different set of posts.
When using this caching you can use also some of the slower solution for getting the random data as it will be fetched from MySQL only once per second regardless of your req/s.
I've looked through all of the answers, and I don't think anyone mentions this possibility at all, and I'm not sure why.
If you want utmost simplicity and speed, at a minor cost, then to me it seems to make sense to store a random number against each row in the DB. Just create an extra column, random_number, and set it's default to RAND(). Create an index on this column.
Then when you want to retrieve a row generate a random number in your code (PHP, Perl, whatever) and compare that to the column.
SELECT FROM tbl WHERE random_number >= :random LIMIT 1
I guess although it's very neat for a single row, for ten rows like the OP asked you'd have to call it ten separate times (or come up with a clever tweak that escapes me immediately)
I needed a query to return a large number of random rows from a rather large table. This is what I came up with. First get the maximum record id:
SELECT MAX(id) FROM table_name;
Then substitute that value into:
SELECT * FROM table_name WHERE id > FLOOR(RAND() * max) LIMIT n;
Where max is the maximum record id in the table and n is the number of rows you want in your result set. The assumption is that there are no gaps in the record id's although I doubt it would affect the result if there were (haven't tried it though). I also created this stored procedure to be more generic; pass in the table name and number of rows to be returned. I'm running MySQL 5.5.38 on Windows 2008, 32GB, dual 3GHz E5450, and on a table with 17,361,264 rows it's fairly consistent at ~.03 sec / ~11 sec to return 1,000,000 rows. (times are from MySQL Workbench 6.1; you could also use CEIL instead of FLOOR in the 2nd select statement depending on your preference)
DELIMITER $$
USE [schema name] $$
DROP PROCEDURE IF EXISTS `random_rows` $$
CREATE PROCEDURE `random_rows`(IN tab_name VARCHAR(64), IN num_rows INT)
BEGIN
SET #t = CONCAT('SET #max=(SELECT MAX(id) FROM ',tab_name,')');
PREPARE stmt FROM #t;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SET #t = CONCAT(
'SELECT * FROM ',
tab_name,
' WHERE id>FLOOR(RAND()*#max) LIMIT ',
num_rows);
PREPARE stmt FROM #t;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END
$$
then
CALL [schema name].random_rows([table name], n);
Here is a game changer that may be helpfully for many;
I have a table with 200k rows, with sequential id's, I needed to pick N random rows, so I opt to generate random values based in the biggest ID in the table, I created this script to find out which is the fastest operation:
logTime();
query("SELECT COUNT(id) FROM tbl");
logTime();
query("SELECT MAX(id) FROM tbl");
logTime();
query("SELECT id FROM tbl ORDER BY id DESC LIMIT 1");
logTime();
The results are:
Count: 36.8418693542479 ms
Max: 0.241041183472 ms
Order: 0.216960906982 ms
Based in this results, order desc is the fastest operation to get the max id,
Here is my answer to the question:
SELECT GROUP_CONCAT(n SEPARATOR ',') g FROM (
SELECT FLOOR(RAND() * (
SELECT id FROM tbl ORDER BY id DESC LIMIT 1
)) n FROM tbl LIMIT 10) a
...
SELECT * FROM tbl WHERE id IN ($result);
FYI: To get 10 random rows from a 200k table, it took me 1.78 ms (including all the operations in the php side)
I used this http://jan.kneschke.de/projects/mysql/order-by-rand/ posted by Riedsio (i used the case of a stored procedure that returns one or more random values):
DROP TEMPORARY TABLE IF EXISTS rands;
CREATE TEMPORARY TABLE rands ( rand_id INT );
loop_me: LOOP
IF cnt < 1 THEN
LEAVE loop_me;
END IF;
INSERT INTO rands
SELECT r1.id
FROM random AS r1 JOIN
(SELECT (RAND() *
(SELECT MAX(id)
FROM random)) AS id)
AS r2
WHERE r1.id >= r2.id
ORDER BY r1.id ASC
LIMIT 1;
SET cnt = cnt - 1;
END LOOP loop_me;
In the article he solves the problem of gaps in ids causing not so random results by maintaining a table (using triggers, etc...see the article);
I'm solving the problem by adding another column to the table, populated with contiguous numbers, starting from 1 (edit: this column is added to the temporary table created by the subquery at runtime, doesn't affect your permanent table):
DROP TEMPORARY TABLE IF EXISTS rands;
CREATE TEMPORARY TABLE rands ( rand_id INT );
loop_me: LOOP
IF cnt < 1 THEN
LEAVE loop_me;
END IF;
SET #no_gaps_id := 0;
INSERT INTO rands
SELECT r1.id
FROM (SELECT id, #no_gaps_id := #no_gaps_id + 1 AS no_gaps_id FROM random) AS r1 JOIN
(SELECT (RAND() *
(SELECT COUNT(*)
FROM random)) AS id)
AS r2
WHERE r1.no_gaps_id >= r2.id
ORDER BY r1.no_gaps_id ASC
LIMIT 1;
SET cnt = cnt - 1;
END LOOP loop_me;
In the article i can see he went to great lengths to optimize the code; i have no ideea if/how much my changes impact the performance but works very well for me.
You can easily use a random offset with a limit
PREPARE stm from 'select * from table limit 10 offset ?';
SET #total = (select count(*) from table);
SET #_offset = FLOOR(RAND() * #total);
EXECUTE stm using #_offset;
You can also apply a where clause like so
PREPARE stm from 'select * from table where available=true limit 10 offset ?';
SET #total = (select count(*) from table where available=true);
SET #_offset = FLOOR(RAND() * #total);
EXECUTE stm using #_offset;
Tested on 600,000 rows (700MB) table query execution took ~0.016sec HDD drive.
EDIT: The offset might take a value close to the end of the table, which will result in the select statement returning less rows (or maybe only 1 row), to avoid this we can check the offset again after declaring it, like so
SET #rows_count = 10;
PREPARE stm from "select * from table where available=true limit ? offset ?";
SET #total = (select count(*) from table where available=true);
SET #_offset = FLOOR(RAND() * #total);
SET #_offset = (SELECT IF(#total-#_offset<#rows_count,#_offset-#rows_count,#_offset));
SET #_offset = (SELECT IF(#_offset<0,0,#_offset));
EXECUTE stm using #rows_count,#_offset;
I know it is not what you want, but the answer I will give you is what I use in production in a small website.
Depending on the quantity of times you access the random value, it is not worthy to use MySQL, just because you won't be able to cache the answer. We have a button there to access a random page, and a user could click in there several times per minute if he wants. This will cause a mass amount of MySQL usage and, at least for me, MySQL is the biggest problem to optimize.
I would go another approach, where you can store in cache the answer. Do one call to your MySQL:
SELECT min(id) as min, max(id) as max FROM your_table
With your min and max Id, you can, in your server, calculate a random number. In python:
random.randint(min, max)
Then, with your random number, you can get a random Id in your Table:
SELECT *
FROM your_table
WHERE id >= %s
ORDER BY id ASC
LIMIT 1
In this method you do two calls to your Database, but you can cache them and don't access the Database for a long period of time, enhancing performance. Note that this is not random if you have holes in your table. Having more than 1 row is easy since you can create the Id using python and do one request for each row, but since they are cached, it's ok.
If you have too many holes in your table, you can try the same approach, but now going for the total number of records:
SELECT COUNT(*) as total FROM your_table
Then in python you go:
random.randint(0, total)
And to fetch a random result you use the LIMIT like bellow:
SELECT *
FROM your_table
ORDER BY id ASC
LIMIT %s, 1
Notice it will get 1 value after X random rows. Even if you have holes in your table, it will be completely random, but it will cost more for your database.
If you want one random record (no matter if there are gapes between ids):
PREPARE stmt FROM 'SELECT * FROM `table_name` LIMIT 1 OFFSET ?';
SET #count = (SELECT
FLOOR(RAND() * COUNT(*))
FROM `table_name`);
EXECUTE stmt USING #count;
Source: https://www.warpconduit.net/2011/03/23/selecting-a-random-record-using-mysql-benchmark-results/#comment-1266
This is super fast and is 100% random even if you have gaps.
Count the number x of rows that you have available SELECT COUNT(*) as rows FROM TABLE
Pick 10 distinct random numbers a_1,a_2,...,a_10 between 0 and x
Query your rows like this: SELECT * FROM TABLE LIMIT 1 offset a_i for i=1,...,10
I found this hack in the book SQL Antipatterns from Bill Karwin.
The following should be fast, unbiased and independent of id column. However it does not guarantee that the number of rows returned will match the number of rows requested.
SELECT *
FROM t
WHERE RAND() < (SELECT 10 / COUNT(*) FROM t)
Explanation: assuming you want 10 rows out of 100 then each row has 1/10 probability of getting SELECTed which could be achieved by WHERE RAND() < 0.1. This approach does not guarantee 10 rows; but if the query is run enough times the average number of rows per execution will be around 10 and each row in the table will be selected evenly.
If you have just one Read-Request
Combine the answer of #redsio with a temp-table (600K is not that much):
DROP TEMPORARY TABLE IF EXISTS tmp_randorder;
CREATE TABLE tmp_randorder (id int(11) not null auto_increment primary key, data_id int(11));
INSERT INTO tmp_randorder (data_id) select id from datatable;
And then take a version of #redsios Answer:
SELECT dt.*
FROM
(SELECT (RAND() *
(SELECT MAX(id)
FROM tmp_randorder)) AS id)
AS rnd
INNER JOIN tmp_randorder rndo on rndo.id between rnd.id - 10 and rnd.id + 10
INNER JOIN datatable AS dt on dt.id = rndo.data_id
ORDER BY abs(rndo.id - rnd.id)
LIMIT 1;
If the table is big, you can sieve on the first part:
INSERT INTO tmp_randorder (data_id) select id from datatable where rand() < 0.01;
If you have many read-requests
Version: You could keep the table tmp_randorder persistent, call it datatable_idlist. Recreate that table in certain intervals (day, hour), since it also will get holes. If your table gets really big, you could also refill holes
select l.data_id as whole
from datatable_idlist l
left join datatable dt on dt.id = l.data_id
where dt.id is null;
Version: Give your Dataset a random_sortorder column either directly in datatable or in a persistent extra table datatable_sortorder. Index that column. Generate a Random-Value in your Application (I'll call it $rand).
select l.*
from datatable l
order by abs(random_sortorder - $rand) desc
limit 1;
This solution discriminates the 'edge rows' with the highest and the lowest random_sortorder, so rearrange them in intervals (once a day).
Another simple solution would be ranking the rows and fetch one of them randomly and with this solution you won't need to have any 'Id' based column in the table.
SELECT d.* FROM (
SELECT t.*, #rownum := #rownum + 1 AS rank
FROM mytable AS t,
(SELECT #rownum := 0) AS r,
(SELECT #cnt := (SELECT RAND() * (SELECT COUNT(*) FROM mytable))) AS n
) d WHERE rank >= #cnt LIMIT 10;
You can change the limit value as per your need to access as many rows as you want but that would mostly be consecutive values.
However, if you don't want consecutive random values then you can fetch a bigger sample and select randomly from it. something like ...
SELECT * FROM (
SELECT d.* FROM (
SELECT c.*, #rownum := #rownum + 1 AS rank
FROM buildbrain.`commits` AS c,
(SELECT #rownum := 0) AS r,
(SELECT #cnt := (SELECT RAND() * (SELECT COUNT(*) FROM buildbrain.`commits`))) AS rnd
) d
WHERE rank >= #cnt LIMIT 10000
) t ORDER BY RAND() LIMIT 10;
One way that i find pretty good if there's an autogenerated id is to use the modulo operator '%'. For Example, if you need 10,000 random records out 70,000, you could simplify this by saying you need 1 out of every 7 rows. This can be simplified in this query:
SELECT * FROM
table
WHERE
id %
FLOOR(
(SELECT count(1) FROM table)
/ 10000
) = 0;
If the result of dividing target rows by total available is not an integer, you will have some extra rows than what you asked for, so you should add a LIMIT clause to help you trim the result set like this:
SELECT * FROM
table
WHERE
id %
FLOOR(
(SELECT count(1) FROM table)
/ 10000
) = 0
LIMIT 10000;
This does require a full scan, but it is faster than ORDER BY RAND, and in my opinion simpler to understand than other options mentioned in this thread. Also if the system that writes to the DB creates sets of rows in batches you might not get such a random result as you where expecting.
I think here is a simple and yet faster way, I tested it on the live server in comparison with a few above answer and it was faster.
SELECT * FROM `table_name` WHERE id >= (SELECT FLOOR( MAX(id) * RAND()) FROM `table_name` ) ORDER BY id LIMIT 30;
//Took 0.0014secs against a table of 130 rows
SELECT * FROM `table_name` WHERE 1 ORDER BY RAND() LIMIT 30
//Took 0.0042secs against a table of 130 rows
SELECT name
FROM random AS r1 JOIN
(SELECT CEIL(RAND() *
(SELECT MAX(id)
FROM random)) AS id)
AS r2
WHERE r1.id >= r2.id
ORDER BY r1.id ASC
LIMIT 30
//Took 0.0040secs against a table of 130 rows
SELECT
*
FROM
table_with_600k_rows
WHERE
RAND( )
ORDER BY
id DESC
LIMIT 30;
id is the primary key, sorted by id,
EXPLAIN table_with_600k_rows, find that row does not scan the entire table
I Use this query:
select floor(RAND() * (SELECT MAX(key) FROM table)) from table limit 10
query time:0.016s
This is how I do it:
select *
from table_with_600k_rows
where rand() < 10/600000
limit 10
I like it because does not require other tables, it is simple to write, and it is very fast to execute.
Use the below simple query to get random data from a table.
SELECT user_firstname ,
COUNT(DISTINCT usr_fk_id) cnt
FROM userdetails
GROUP BY usr_fk_id
ORDER BY cnt ASC
LIMIT 10
I guess this is the best possible way..
SELECT id, id * RAND( ) AS random_no, first_name, last_name
FROM user
ORDER BY random_no

Finding a ranking from a rating field on MySQL

I have a MySQL table that looks like this:
id (int primary)
name (text)
rating (float)
I have a page showing rankings which looks like this:
$i = 0;
$q = mysql_query("SELECT * FROM teams ORDER BY rating DESC");
while($r = mysql_fetch_assoc($q)){
$i++;
print("$i: {$r['name']}<br>");
}
This shows teams in order of their rating, with a ranking. And it works.
Now, if I'm given the ID of a team, how do I find their ranking without running through the loop like this? A single MySQL query which returns the team's info + a numeric ranking indicating how far down the list they would be, if I had rendered the whole list.
Thanks!
To get the ranking you can do:
SELECT COUNT(*) as ranking
FROM teams t
WHERE t.rating >= (SELECT rating FROM teams WHERE id=$ID);
To get all the relevant info too, you can do:
SELECT t.*,COUNT(*) as rank
FROM teams t
JOIN teams t2 ON t.rating<=t2.rating
WHERE t.id=4;
This joins teams to itself joining on t.rating <= t2.rating, and so you get one row for every team that has a rating higher than or equal you.
The COUNT just counts how many teams have a rating higher than or equal to you.
Note that if there's a tie this will give you the lower rank. You can change the <= to a < if you want the highest.
You can also do it this way:
select * from (
select t.*, #rank := #rank + 1 as rank
from (select #rank := 0) as r, t
order by rating desc
) as t
where id = 20

how to get the position of sorted rows using mysql and php

I have a table which stores high-scores, along with player ids. I want to be able to extract a record by a players id, and then get the rank, or position of their score in the table. Means, Basically I want to be able to say "you are in Nth" position, purely based on the players score against all other scores. For Example: if i am at 46th position then to me the position message will be like you are at 46th position out of total scores. Can anyone show me small example?
There are two ways of doing it:
Method 1:
SET #i = 0;
SELECT * FROM
scores s1 INNER JOIN (SELECT *, #i := #i + 1 AS rank FROM scores ORDER BY score DESC) AS s2 USING (id);
Method 2:
SELECT *, (SELECT COUNT(1) AS num FROM scores WHERE scores.score > s1.score) + 1 AS rank FROM scores AS s1
ORDER BY rank asc
This will provide duplicate rank values when there are duplicates:
SELECT t.playerid,
t.highscore,
(SELECT COUNT(*)
FROM TABLE x
WHERE x.playerid = t.playerid
AND x.highscore >= t.highscore) AS rank
FROM TABLE t
WHERE t.playerid = ?
IE: If three players have the same score for second place, they'll all have a rank value of two.
This will give a distinct value - three players tied for second place, only one will be ranked as 2nd:
SELECT x.playerid,
x.highscore,
x.rank
FROM (SELECT t.playerid,
t.highscore,
#rownum := #rownum + 1 AS rank
FROM TABLE t
JOIN (SELECT #rownum := 0) r
ORDER BY t.highscore DESC) x
WHERE x.playerid = ?
Here is an example.
You want to store the user's ID when they log in, like so...
$_SESSION['username'] = $usernamefromdb;
$_SESSION['id'] = $userid;
And then you want to open a session on every page on yoru website that you will be pulling dynamic information depending on the $_SESSION['id']
session_start();
Then find the row of data in the datebase according to the userID
$userid = $_SESSION['id'];
$rank_query = "SELECT * FROM table_name WHERE id='$userid'";
$rank_result = mysqli_query($cxn, $rank_query) or die("Couldn't execute query.");
$row = mysqli_fetch_assoc($rank_result)
Then using PHP, declare the nth postiion as a variable. And pull the total amount of rows from the DB
$rank = $row['rank'];
$all = $numrows = mysqli_num_rows($result);
echo out the players rank.
echo $rank . "out of" . $all;

Advanced SQL query. Top 12 from each category (MYSQL)

I have a MYSQL5 database and PHP 5. I need a query for a games websites index page that only selects the first 12 from each category of games. Here is what I have so far.
$db->query("SELECT * FROM `games` WHERE status = 'game_published' AND `featured` = '1' ORDER BY `category`");
The php code then groups games of the same category together and displays them. But yeah it doesn't limit the number of games from each category like I want.
Here is exactly what the structure of the table looks like: i49.tinypic.com/aysoll.png
Here is a blog post which sounds like what I am trying to do: http://www.e-nformation.net/content/view/title/MySQL+Top+N+in+each+group+(group+inner+limit) But I can't make sense of it.
Any help is appreciated.
How about this?
SELECT * FROM (
SELECT
games.*,
#rn := CASE WHEN #category=category THEN #rn + 1 ELSE 1 END AS rn,
#category := category
FROM games, (SELECT #rn := 0, #category := NULL) AS vars
WHERE status = 'game_published' AND featured = '1'
ORDER BY category
) AS T1
WHERE rn <= 12
you could use UNION, if we are not talking about million of types...
pseudoSQL:
(SELECT * FROM table WHERE condition AND category = 'action' ORDER BY id LIMIT 10)
UNION
(SELECT * FROM table WHERE condition AND category = 'action' ORDER BY id LIMIT 10)
UNION
(SELECT * FROM table WHERE condition AND category = 'action' ORDER BY id LIMIT 10)
If you have array of categories in your PHP/ASP, you can generate this union on the fly.
More:
http://dev.mysql.com/doc/refman/5.0/en/union.html
EDIT:
Here's probably most useful resource: http://www.xaprb.com/blog/2006/12/07/how-to-select-the-firstleastmax-row-per-group-in-sql/
Use it well ^^
There may be a more elegant solution, but you can just execute a query for each category. First get a list of categories:
SELECT DISTINCT(category) FROM `games`;
Then take each of the results and query for 12 rows:
SELECT * FROM games WHERE status = 'game_published'
AND `featured` = '1' AND `category` = $category LIMIT 12;
Of course you need to add some kind of ranking row (and order by it) to get the top 12.
Note: There may be a way to do this with a single query, but it escapes me at the moment.
To use the technique from the posts you mention, you need a way to order the games. They're using article date. Then they select the number of older articles for that company, and say there can't be more than three.
If your games table has an auto-increment column called id, you can select the top 10 games per category like:
SELECT *
FROM games g1
WHERE status = 'game_published'
AND featured = '1'
AND 10 >
(
SELECT COUNT(*)
FROM games g2
WHERE g2.status = 'game_published'
AND g2.featured = '1'
AND g1.category = g2.category
AND g2.id > g1.id
)
The where condition says that there can't be more than 10 rows with the same category and a higher ID.

Categories