I'm letting the user page through lists of data. I implemented it with 2 sql statements one to get the count and one to let them get the next section. I get the count to show the number of pages and let the user click on any page, such as:
<< < 1 2 3 4 5 > >> 245 Items
SELECT count(*) FROM list;
SELECT * FROM list limit "$start $rows";
I was just wondering if there was a better way. It looks inefficient, particularly since my SQL is complicated with lots of joins and conditions that I have to execute twice.
Thanks
SELECT SQL_CALC_FOUND_ROWS name, email FROM users WHERE name LIKE 'a%' LIMIT 10;
SELECT FOUND_ROWS();
Source: http://www.arraystudio.com/as-workshop/mysql-get-total-number-of-rows-when-using-limit.html
Related
As I do not know anything about speed and complexity of php and mysql(i) scripts, I had this question:
I have a database with 3 tables:
'Products' with about 9 fields. Containing data of products, like 'long' content text.
'Categories' with 2 fields. Containing name of categories
'Productcategories' with 2 fields. Containing which product has which categories. Each product is part of 1-3 categories.
In order to set up pagination (I need row_count because I wish to know what the last page is), I was wondering what the most sufficient way to do it is, and or it depends on the amount of products (50, 100, 500?). The results returned depends on a chosen category:
"SELECT * FROM `productcategories`
JOIN products ON products.proID = productcategories.proID
WHERE productcategories.catID =$category";
Idea 1:
1 query which only selects 1 field, instead of all. And then counts the total rows for my pagination with mysqli_num_rows().
A second query which directly selects 5 or 10 (with LIMIT I expect) products to be actually shown.
Idea 2:
Only 1 query (above), on which you use mysqli_nuw_rows() for row count and later on, filter out the rows you want to show.
I do not know which is the best. Idea 1 seems faster as you have to select a lot less data, but I do not know or the 2 queries needed influence the speed a lot? Which is the fastest: collecting 'big' amounts of data or doing queries?
Feel free to correct me if I am completely on the wrong path with my ideas.
It is generally considered best practice to return as little data as possible so the short answer is to use the two queries.
However, MySQL does provide one interesting function that will allow you to return the row count that would have been returned without the limit clause:
FOUND_ROWS()
Just keep in mind not all dbms' implement this, so use with care.
Example:
mysql> SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name
-> WHERE id > 100 LIMIT 10;
mysql> SELECT FOUND_ROWS();
Use select count(1) as count... for the total number of rows. Then select data as needed for pagination with limit 0,10 or something like that.
Also for total count you don't need to join to the products or categories tables as that would only be used for displaying extra info.
"SELECT count(1) as count FROM `productcategories` WHERE catID=$category";
Then for data:
"SELECT * FROM `productcategories`
JOIN categories ON categories.catID = productcategories.catID
JOIN products ON products.proID = productcategories.proID
WHERE productcategories.catID=$category limit 0,10";
Replacing * with actual fields needed would be better though.
I'm new to this, sorry if the title is confusing. I am building a simple php/mysql gallery of sorts. It will show the newest 25 entries when a user first goes to it, and also allows off-site linking to individual items in the list. If the URL contains an ID, javascript will scroll to it. But if there are 25+ entries, it's possible that my query will fetch the newest results, but omit an older entry that happens to be in the URL as an ID.
That means I need to do something like this...
SELECT * FROM `submissions` WHERE uid='$sid'
But after that has successfully found the submission with the special ID, also do
SELECT * FROM `submissions` ORDER BY `id` DESC LIMIT 0, 25`
So that I can populate the rest of the gallery.
I could query that database twice, but I am assuming there's some nifty way to avoid that. MySQL is also ordering everything (based on newest, views, and other vars) and using two queries would break that.
You could limit across a UNION like this:
(SELECT * FROM submissions WHERE uid = '$uid')
UNION
(SELECT * FROM submissions WHERE uid <> '$uid' ORDER BY `id` LIMIT 25)
LIMIT 25
Note LIMIT is listed twice as in the case that the first query returns a result, we would have 26 results in the union set. This will also place the "searched for" item first in the returned sort result set (with the other 24 results displayed in sort order). If this is not desirable, you could place an ORDER BY across the union, but your searched for result would be truncated if it happened to be the 26th record.
If you need 25 rows with all of them being sorted, my guess is that you would need to do the two query approach (limiting second query to either 24 or 25 records depending on whether the first query matched), and then simply insert the uid-matched result into the sorted records in the appropriate place before display.
I think the better solution is:
SELECT *
FROM `submissions`
order by (case when usid = $sid then 0 else 1 end),
id desc
limit 25
I don't think the union is guaranteed to return results in the order of the union (there is no guarantee in the standard or in other databases).
I have been working on try to create php web-based paging for our tables
which have over a million rows.
Based on what I have read, I have 3 options
retrieve all rows in resultset - not possiblefor me coz of the size
retrieve 1000 rows, store in temp table and create an iterator for
it and page through it - too many queries - too many inserts!!
run a query each time if someone opts page forward or backwards
Right now I am trying to get option 3 working.
I have the first page showing up as
"select * from accout order by acct fetch first 10 rows only"
Page next
"select * from account where acct>(last record) order by acct fetch
first 10 only"
page last record
"select * from account where acct=(select max(acct) from account)"
The problem is showing the previous page and i really would appreciate
help in this.
SELECT *
FROM (
SELECT
*,
ROW_NUMBER() OVER (ORDER BY acct) AS RowNum
FROM
account
) AS Data
WHERE
RowNum BETWEEN 100 AND 110;
First, you should get rid of the SELECT *. Select only the fields you need.
Place an index on acct, this will help the ROW_NUMBER() OVER (ORDER BY acct) construct.
Use a SELECT COUNT(*) FROM account to determine how many pages you will have.
Also read Fastest most/efficient way to do pagination with SQL searching DB2
The LIMIT..OFFSET solution is supported in DB2 10+.. For older versions, you have to enable the MySQL compatibility with:
$ db2set DB2_COMPATIBILITY_VECTOR=MYS
$ db2stop
$ db2start
in db2cmd in order to use that syntax.
SELECT * FROM foo LIMIT 10, 1;
try the limit and offset in mysql..
this mostly use in creating pagination
I'm trying to get 4 random results from a table that holds approx 7 million records. Additionally, I also want to get 4 random records from the same table that are filtered by category.
Now, as you would imagine doing random sorting on a table this large causes the queries to take a few seconds, which is not ideal.
One other method I thought of for the non-filtered result set would be to just get PHP to select some random numbers between 1 - 7,000,000 or so and then do an IN(...) with the query to only grab those rows - and yes, I know that this method has a caveat in that you may get less than 4 if a record with that id no longer exists.
However, the above method obviously will not work with the category filtering as PHP doesn't know which record numbers belong to which category and hence cannot select the record numbers to select from.
Are there any better ways I can do this? Only way I can think of would be to store the record id's for each category in another table and then select random results from that and then select only those record ID's from the main table in a secondary query; but I'm sure there is a better way!?
You could of course use the RAND() function on a query using a LIMIT and WHERE (for the category). That however as you pointed out, entails a scan of the database which takes time, especially in your case due to the volume of data.
Your other alternative, again as you pointed out, to store id/category_id in another table might prove a bit faster but again there has to be a LIMIT and WHERE on that table which will also contain the same amount of records as the master table.
A different approach (if applicable) would be to have a table per category and store in that the IDs. If your categories are fixed or do not change that often, then you should be able to use that approach. In that case you will effectively remove the WHERE from the clause and getting a RAND() with a LIMIT on each category table would be faster since each category table will contain a subset of records from your main table.
Some other alternatives would be to use a key/value pair database just for that operation. MongoDb or Google AppEngine can help with that and are really fast.
You could also go towards the approach of a Master/Slave in your MySQL. The slave replicates content in real time but when you need to perform the expensive query you query the slave instead of the master, thus passing the load to a different machine.
Finally you could go with Sphinx which is a lot easier to install and maintain. You can then treat each of those category queries as a document search and let Sphinx randomize the results. This way you offset this expensive operation to a different layer and let MySQL continue with other operations.
Just some issues to consider.
Working off your random number approach
Get the max id in the database.
Create a temp table to store your matches.
Loop n times doing the following
Generate a random number between 1 and maxId
Get the first record with a record Id greater than the random number and insert it into your temp table
Your temp table now contains your random results.
Or you could dynamically generate sql with a union to do the query in one step.
SELECT * FROM myTable WHERE ID > RAND() AND Category = zzz LIMIT 1
UNION
SELECT * FROM myTable WHERE ID > RAND() AND Category = zzz LIMIT 1
UNION
SELECT * FROM myTable WHERE ID > RAND() AND Category = zzz LIMIT 1
UNION
SELECT * FROM myTable WHERE ID > RAND() AND Category = zzz LIMIT 1
Note: my sql may not be valid, as I'm not a mySql guy, but the theory should be sound
First you need to get number of rows ... something like this
select count(1) from tbl where category = ?
then select a random number
$offset = rand(1,$rowsNum);
and select a row with offset
select * FROM tbl LIMIT $offset, 1
in this way you avoid missing ids. The only problem is you need to run second query several times. Union may help in this case.
For MySQl you can use
RAND()
SELECT column FROM table
ORDER BY RAND()
LIMIT 4
I have two column table friends table where it has a recode of users whos friends with eacho other. i want to get a list who's friends with a specific user. to do that i wrote a sql query as such and got a result..
SELECT *
FROM `friends`
WHERE `user_idf` = '2'
OR `user_idff` = '2'
++ user_idf ++++ user_idff+++
2 + 1
3 + 2
4 + 2
2 + 5
+++++++++++++++++++++++++++
how to write a php function to echo user who's a friend of 2 (this i will replace with a POST where it gets user id from an application)
i.e.
1, 3, 4, 5
do you mean something like this?
(SELECT `user_idf` FROM `user` WHERE `user_idff` = 2) UNION (SELECT `user_idff` FROM `user` WHERE `user_idf` = 2)
Im not great with sql to be honest, but I think union is what you are looking for (and I think my syntax is correct although im not 100%).
This should from my understanding take the user ids of anyone who is friends with user 2 and output it. The union of results returned when user 2 is in user_idf and user_idff columns.
EDIT: Actually I still stand by my answer, if you don;t do this sql side then you will have to have a more complex PHP function.
For the PHP itself look into mysqli and PDO as your two main options for prepared statements.