I have a system that automatically paginates all mysql queries with LIMIT and OFFSET.
Example:
// Page 1
SELECT * FROM tbl_products WHERE category_id = 1 LIMIT 20 OFFSET 0
// Page 2
SELECT * FROM tbl_products WHERE category_id = 1 LIMIT 20 OFFSET 20
// Page 3
SELECT * FROM tbl_products WHERE category_id = 1 LIMIT 20 OFFSET 40
etc.
Now the questions itself.
A) Is this the best way to do it? Are there any alternatives?
B) When I land on a product page (products.php?id=12345) and there will be a list of products on the same page, how can I choose the correct page on the product list? If this product happens to be on the 105th page on the product list?
This is to highlight the "selected product" on the list. Now it only works if the product happens to be on the first page, which is always automatically loaded to the products.php page.
For example if your first page has a get variable in the url www.tst.com?page=1
off = (page - 1)*20 ;
row = 20;
select * from table limit (off, row);
optimally you would use something like this. The code is much faster than two queries where you would have to do a count on the table and then use that count result as your new parameters because of the potential for n amount of rows in your table... The larger the table the less efficient queries will be.
A) Is this the best way to do it? Is there any alternatives?
It probably is the best way as pagination is performed by the server and only the desired rows are returned.
B) [...] how can I choose the correct page on the product list [...]
I can think of two possible solutions:
In your detail pages use a query that selects all ids:
SELECT id
FROM tbl_products
WHERE category_id = 1
ORDER BY id
Note the ORDER BY clause... both queries (the one with LIMIT and the one that selects all ids) must be ordered in exactly the same way. You can then loop through all rows using PHP code and locate the index at which the product id exists. Then divide this number by the page size to determine the page number.
An alternate solution is to se a MySQL query that numbers the rows such as the following:
SELECT #row_number := #row_number + 1 AS row_number, id
FROM tbl_products, (SELECT #row_number := 0) AS temp1
WHERE category_id = 1
ORDER BY id
Nest this query inside another query to determine the row number of the specific id:
SELECT row_number FROM (
SELECT #row_number := #row_number + 1 AS row_number, id
FROM tbl_products, (SELECT #row_number := 0) AS temp1
WHERE category_id = 1
ORDER BY id
) AS temp2 WHERE id = 1234
Divide the row number by page size to get page number.
Related
So I'm working on a script that awards "trophies" to the top 4 performers of a game. The table logs each "grab" attempt, the user that performed it, and whether it was successful. I'd like to create a script that is able to pull the top four off of percentage of successful grabs (attempts / successes)
Is something like this possible within the query itself using mysqli?
I have successfully accomplished the code already by just looping through each table entry, but with thousands of attempts per month it just seems like a clunky way to go about it.
Here is an example of a row in the table, I am attempting to grab the top four based off of monthlyTries/monthlySuccessful
id userId PetId PalId tries successfulGrabs monthlyTries MonthlySuccessful
5 44550 84564 3967 825 268 120 37
Assuming you have a success column that's either 1 or 0 you can sum the success and divide that by count(*) which is the total # of attempts
select user_id, sum(success)/count(*) percentage
from attempts a
group by user_id
order by percentage desc
limit 4
If the success column is not a 1/0 value you can use conditional aggregation
select user_id, sum(case when success = 'yes' then 1 else 0 end)/count(*) as percentage
from attempts a
group by user_id
order by percentage desc
limit 4
In MySQL, you can simplify the logic. If success takes on the values 0 and 1:
select a.user_id, avg(success) as percentage
from attempts a
group by a.user_id
order by percentage desc
limit 4;
Otherwise:
select a.user_id, avg(success = 'yes') as percentage
from attempts a
group by a.user_id
order by percentage desc
limit 4;
I'm trying to build a people directory like LinkedIn has:
http://www.linkedin.com/directory/people-a
I don't want to fetch all the rows with name field starting with a and build the link list like LinkedIn. Is there any way in MySQL so I can only fetch rows in this sequence:
1st, 100th,101st,200th,201st,300th,301st,400th,401st, Last
That means I am trying to get two consecutive rows after a certain gap including the first and last item. The ids are not in nice uniformly increasing order, so I can't use this answser. Any help or hint is appreciated.
Say my query SELECT * FROMbusinesseswhere name like 'a%' order by name returns id like this:
1,3,5,6,8, 9,12,33,45,66,77,88,100,103,120,133,155,166,177,178,198
Above is if I want to get all the rows. But what I want is to get only the items after a certain distance. For example if I want to pick after every 5 items:
1,9,12, 88,100,166,177,198
So skip 4 items and take next two. Is that even possible in mysql?
you can order by whatever you want
create some function that gets a number and return 0,1 (all the numbers you specified will be 0 for example), then you need to pass the rank of the row to your function.
example
select first_name,rank
from (
SELECT first_name,
#curRank := #curRank + 1 AS rank
FROM person p, (SELECT #curRank := 0) r ) x
order by myfunc(x.rank);
myfunc being the function you wrote
if you want to only get those rows, not just sort by it and get them first you can do the following :
select first_name,rank
from (
SELECT first_name,
#curRank := #curRank + 1 AS rank
FROM person p, (SELECT #curRank := 0) r ) x
where rank in (1,100,101,102...)
You can select a range with the limit clause
SELECT ...... LIMIT 0,99
This will give the first 100 records
SELECT ...... LIMIT 100,199
will give the next & so on
Well, this is a very old question never gotten real solution. We want 3 random rows from a table with about 30k records. The table is not so big in point of view MySQL, but if it represents products of a store, it's representative. The random selection is useful when one presents 3 random products in a webpage for example. We would like a single SQL string solution that meets these conditions:
In PHP, the recordset by PDO or MySQLi must have exactly 3 rows.
They have to be obtained by a single MySQL query without Stored Procedure used.
The solution must be quick as for example a busy apache2 server, MySQL query is in many situations the bottleneck. So it has to avoid temporary table creation, etc.
The 3 records must be not contiguous, ie, they must not to be at the vicinity one to another.
The table has the following fields:
CREATE TABLE Products (
ID INT(8) NOT NULL AUTO_INCREMENT,
Name VARCHAR(255) default NULL,
HasImages INT default 0,
...
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
The WHERE constraint is Products.HasImages=1 permitting to fetch only records that have images available to show on the webpage. About one-third of records meet the condition of HasImages=1.
Searching for a Perfection, we first let aside the existent Solutions that have drawbacks:
I. This basic solution using ORDER BY RAND(),
is too slow but guarantees 3 really random records at each query:
SELECT ID, Name FROM Products WHERE HasImages=1 ORDER BY RAND() LIMIT 3;
*CPU about 0.10s, scanning 9690 rows because of WHERE clause, Using where; Using temporary; Using filesort, on Debian Squeeze Double-Core Linux box, not so bad but
not so scalable to a bigger table as temporary table and filesort are used, and takes me 8.52s for the first query on the test Windows7::MySQL system. With such a poor performance, to avoid for a webpage isn't-it ?
II. The bright solution of riedsio using JOIN ... RAND(),
from MySQL select 10 random rows from 600K rows fast, adapted here is only valid for a single random record, as the following query results in an almost always contiguous records. In effect it gets only a random set of 3 continuous records in IDs:
SELECT Products.ID, Products.Name
FROM Products
INNER JOIN (SELECT (RAND() * (SELECT MAX(ID) FROM Products)) AS ID)
AS t ON Products.ID >= t.ID
WHERE (Products.HasImages=1)
ORDER BY Products.ID ASC
LIMIT 3;
*CPU about 0.01 - 0.19s, scanning 3200, 9690, 12000 rows or so randomly, but mostly 9690 records, Using where.
III. The best solution seems the following with WHERE ... RAND(),
seen on MySQL select 10 random rows from 600K rows fast proposed by bernardo-siu:
SELECT Products.ID, Products.Name FROM Products
WHERE ((Products.Hasimages=1) AND RAND() < 16 * 3/30000) LIMIT 3;
*CPU about 0.01 - 0.03s, scanning 9690 rows, Using where.
Here 3 is the number of wished rows, 30000 is the RecordCount of the table Products, 16 is the experimental coefficient to enlarge the selection in order to warrant the 3 records selection. I don't know on what basis the factor 16 is an acceptable approximation.
We so get at the majority of cases 3 random records and it's very quick, but it's not warranted: sometimes the query returns only 2 rows, sometimes even no record at all.
The three above methods scan all records of the table meeting WHERE clause, here 9690 rows.
A better SQL String?
Ugly, but quick and random. Can become very ugly very fast, especially with tuning described below, so make sure you really want it this way.
(SELECT Products.ID, Products.Name
FROM Products
INNER JOIN (SELECT RAND()*(SELECT MAX(ID) FROM Products) AS ID) AS t ON Products.ID >= t.ID
WHERE Products.HasImages=1
ORDER BY Products.ID
LIMIT 1)
UNION ALL
(SELECT Products.ID, Products.Name
FROM Products
INNER JOIN (SELECT RAND()*(SELECT MAX(ID) FROM Products) AS ID) AS t ON Products.ID >= t.ID
WHERE Products.HasImages=1
ORDER BY Products.ID
LIMIT 1)
UNION ALL
(SELECT Products.ID, Products.Name
FROM Products
INNER JOIN (SELECT RAND()*(SELECT MAX(ID) FROM Products) AS ID) AS t ON Products.ID >= t.ID
WHERE Products.HasImages=1
ORDER BY Products.ID
LIMIT 1)
First row appears more often than it should
If you have big gaps between IDs in your table, rows right after such gaps will have bigger chance to be fetched by this query. In some cases, they will appear significatnly more often than they should. This can not be solved in general, but there's a fix for a common particular case: when there's a gap between 0 and the first existing ID in a table.
Instead of subquery (SELECT RAND()*<max_id> AS ID) use something like (SELECT <min_id> + RAND()*(<max_id> - <min_id>) AS ID)
Remove duplicates
The query, if used as is, may return duplicate rows. It is possible to avoid that by using UNION instead of UNION ALL. This way duplicates will be merged, but the query no longer guarantees to return exactly 3 rows. You can work around that too, by fetching more rows than you need and limiting the outer result like this:
(SELECT ... LIMIT 1)
UNION (SELECT ... LIMIT 1)
UNION (SELECT ... LIMIT 1)
...
UNION (SELECT ... LIMIT 1)
LIMIT 3
There's still no guarantee that 3 rows will be fetched, though. It just makes it more likely.
SELECT Products.ID, Products.Name
FROM Products
INNER JOIN (SELECT (RAND() * (SELECT MAX(ID) FROM Products)) AS ID) AS t ON Products.ID >= t.ID
WHERE (Products.HasImages=1)
ORDER BY Products.ID ASC
LIMIT 3;
Of course the above is given "near" contiguous records you are feeding it the same ID every time without much regard to the seed of the rand function.
This should give more "randomness"
SELECT Products.ID, Products.Name
FROM Products
INNER JOIN (SELECT (ROUND((RAND() * (max-min))+min)) AS ID) AS t ON Products.ID >= t.ID
WHERE (Products.HasImages=1)
ORDER BY Products.ID ASC
LIMIT 3;
Where max and min are two values you choose, lets say for example sake:
max = select max(id)
min = 225
This statement executes really fast (19 ms on a 30k records table):
$db = new PDO('mysql:host=localhost;dbname=database;charset=utf8', 'username', 'password');
$stmt = $db->query("SELECT p.ID, p.Name, p.HasImages
FROM (SELECT #count := COUNT(*) + 1, #limit := 3 FROM Products WHERE HasImages = 1) vars
STRAIGHT_JOIN (SELECT t.*, #limit := #limit - 1 FROM Products t WHERE t.HasImages = 1 AND (#count := #count -1) AND RAND() < #limit / #count) p");
$products = $stmt->fetchAll(PDO::FETCH_ASSOC);
The Idea is to "inject" a new column with randomized values, and then sort by this column. The generation of and sorting by this injected column is way faster than the "ORDER BY RAND()" command.
There "might" be one caveat: You have to include the WHERE query twice.
What about creating another table containing only items with image ? This table will be much lighter as it will contain only one-third of the items the original table has !
------------------------------------------
|ID | Item ID (on the original table)|
------------------------------------------
|0 | 0 |
------------------------------------------
|1 | 123 |
------------------------------------------
.
.
.
------------------------------------------
|10 000 | 30 000 |
------------------------------------------
You can then generate three random IDs in the PHP part of the code and just fetch'em the from the database.
I've been testing the following bunch of SQLs on a 10M-record, poorly designed database.
SELECT COUNT(ID)
INTO #count
FROM Products
WHERE HasImages = 1;
PREPARE random_records FROM
'(
SELECT * FROM Products WHERE HasImages = 1 LIMIT ?, 1
) UNION (
SELECT * FROM Products WHERE HasImages = 1 LIMIT ?, 1
) UNION (
SELECT * FROM Products WHERE HasImages = 1 LIMIT ?, 1
)';
SET #l1 = ROUND(RAND() * #count);
SET #l2 = ROUND(RAND() * #count);
SET #l3 = ROUND(RAND() * #count);
EXECUTE random_records USING #l1
, #l2
, #l3;
DEALLOCATE PREPARE random_records;
It took almost 7 minutes to get the three results. But I'm sure its performance will be much better in your case. Yet if you are looking for a better performance I suggest the following ones as they took less than 30 seconds for me to get the job done (on the same database).
SELECT COUNT(ID)
INTO #count
FROM Products
WHERE HasImages = 1;
PREPARE random_records FROM
'SELECT * FROM Products WHERE HasImages = 1 LIMIT ?, 1';
SET #l1 = ROUND(RAND() * #count);
SET #l2 = ROUND(RAND() * #count);
SET #l3 = ROUND(RAND() * #count);
EXECUTE random_records USING #l1;
EXECUTE random_records USING #l2;
EXECUTE random_records USING #l3;
DEALLOCATE PREPARE random_records;
Bear in mind that both these commands require MySQLi driver in PHP if you want to execute them in one go. And their only difference is that the later one requires calling MySQLi's next_result method to retrieve all three results.
My personal belief is that this is the fastest way to do this.
On the off-chance that you're willing to accept an 'outside the box' type of answer, I'm going to repeat what I said in some of the comments.
The best way to approach your problem is to cache your data in advance (be that in an external JSON or XML file, or in a separate database table, possibly even an in-memory table).
This way you can schedule your performance-hit on the products table to times when you know the server will be quiet, and reduce your worry about creating a performance hit at "random" times when the visitor arrives to your site.
I'm not going to suggest an explicit solution, because there are far too many possibilities on how to build a solution. However, the answer suggested by #ahmed is not silly. If you don't want to create a join in your query, then simply load more of the data that you require into the new table instead.
I am creating an online store website that needs the functionality to select a random product from the database.
The idea is that there will be an advert for a random product that is different each time the webpage loads!
Using PHP, how would I go about doing this?
tbl_products
id
code
title
stock
cost
rrp
These are the rows I need to get access to from the database.
Thanks
A most straightforward solution would be this:
SELECT *
FROM tbl_products
ORDER BY
RAND()
LIMIT 1
However, this becomes less efficient as the number of products grows.
This solution:
Selecting random rows
is more efficient, though it still requires a full table scan.
If you product ids are distributes more or less uniformly, use this:
SELECT p.*
FROM (
SELECT
(
(
SELECT MAX(id)
FROM tbl_products
) -
(
SELECT MIN(id)
FROM tbl_products
)
) * RAND() AS rnd
) q
JOIN tbl_products p
ON id >= rnd
ORDER BY
id
LIMIT 1;
If you have gaps between ids, the products after large gaps will tend to be selected more often.
Instead of id, you may use a special unique column for this purpose which you should fill without gaps in a cron job.
ORDER BY RAND() is a well-known solution that has well-known problems.
If the product ids are a consecutive range of integers and there is a non-trivial number of rows, then it will much better to SELECT MAX(id) FROM products, generate a number of random integers between 1 and the result in PHP, and do SELECT * FROM products WHERE id IN (x, y, z) as a second query. If the ids are almost, but not quite, consecutive, you can adapt this solution to generate more random ids than needed to account for the fact that not all of them might be valid (the more fragmentation there is among ids, the more surplus numbers you should generate).
If the above is not an option, then querying like this will still be better than a pure ORDER BY RAND().
Here's a PHP solution
$range_res = mysql_query( " SELECT MAX(id) AS max_id , MIN(id) AS min_id FROM products ");
$range_row = mysql_fetch_object( $range_res );
$random = mt_rand( $range_row->min_id , $range_row->max_id );
$res = mysql_query( " SELECT * FROM products WHERE id >= $random LIMIT 0,1 ");
How can I select a single random entry from a MySQL database using PHP?
I want to select the Author, AuthorText, and Date?
SELECT Author, AuthorText, Date FROM table ORDER BY RAND() LIMIT 1
Take a look to this interesting article:
“Do not use ORDER BY RAND()” or “How to get random rows from table?”
ORDER BY rand() LIMIT 1
will sort all the rows in the table, which can be extremely slow.
Better solution : say your table has the usual primary key auto-increment field, generate a rendom number between min(id) and max(id) and select the closest id.
It will not be as random as a "true" random selection, because a id after a large hole of deleted ids will have a higher probability of being chosen. But it will take 50 µs instead of 2 seconds if your table is large...
SET #t = (SELECT FLOOR(a + (b-a)*rand()) FROM (SELECT min(id) as a, max(id) as b FROM table)
SELECT * FROM table WHERE id>#t ORDER BY id LIMIT 1;
You can order by a random & restrict to 1 row as follows:
select
author, authortext, date
from bookstable
order by rand()
limit 1