I have to modify an existing PHP script so that it displays the total number of matches in the database before it shows the results for the current page. The query is:
SELECT products.features, products.make_id, products.model, products.price,
products.id, brands.name AS brandname, brands.id AS brandid
FROM products
LEFT JOIN brands ON products.make_id=brands.id
WHERE products.active=1 AND brands.active=1 AND (products.category=22)
ORDER BY id DESC LIMIT 0,12
How do I get the number of results? mysql_num_rows() obviously returns 12 but I don't know how to use count in this query, everything I try returns some error.
I could just execute the query once without the limit (clause?) but that seems a bit inefficient.
Thanks.
Use the MySQL function FOUND_ROWS():
A SELECT statement may include a LIMIT
clause to restrict the number of rows
the server returns to the client. In
some cases, it is desirable to know
how many rows the statement would have
returned without the LIMIT, but
without running the statement again.
To obtain this row count, include a
SQL_CALC_FOUND_ROWS option in the
SELECT statement, and then invoke
FOUND_ROWS() afterward:
so run the two following queries in PHP, the first statement gives the first 12 rows, the second one gives the total of rows:
$sql1 = "SELECT SQL_CALC_FOUND_ROWS
products.features
, products.make_id
, products.model
, products.price
, products.id
, brands.name AS brandname
, brands.id AS brandid
FROM products
LEFT JOIN brands ON products.make_id=brands.id
WHERE (products.active=1) AND (brands.active=1) AND (products.category=22)
ORDER BY id DESC LIMIT 0,12"
$sql2 = "SELECT FOUND_ROWS()"
This would return you the total number of rows the current select matches. The result will be one row, with one field and it will hold the total
SELECT COUNT(*) AS total
FROM products
LEFT JOIN brands ON products.make_id=brands.id
WHERE products.active=1 AND brands.active=1 AND products.category=22
You will have to run this as a separate query.
<?php
$q=mysql_query('above query');
$result=mysql_fetch_array($q);
echo $result['total']; // will print out the desired number of rows
You were probably trying to get the total and the limit in the same select and while that might be possible with a union, but since you were worrying about inefficiency before, this is possible worse.
Since in your OP you hint that this is for pagination. Let me tell you, LIMIT m,n may not be as fast as it sounds. Learn how to improve it and read more about Efficient Pagination Using MySQL
Related
I'm working on a project which have a large set of data. The data are stored in a MySql Db.
I want to fetch records with pagination from few tables. One of the table is having over 2 millions of records which is causing the page to freeze for a very long time. Also sometimes the page is not able to load at all.
The query I'm using to fetch the records :
SELECT
EN.`MRN`,
P.`FNAME`,
P.`LNAME`,
P.`MI`,
P.`SSC`,
sum(EN.`AMOUNT`) AS `TOTAL_AMOUNT`
FROM `table_1` AS EN
INNER JOIN `table_2` AS P ON EN.`MRN` = P.`MRN`
GROUP BY EN.`MRN`,P.`FNAME`, P.`LNAME`,P.`MI`,P.`SSC`
HAVING sum(EN.`AMOUNT`) > 0
ORDER BY P.`LNAME`
By this query I'm getting the total number of records for the pagination to work. Then I again run this query to get the actual records :
SELECT
EN.`MRN`,
P.`FNAME`,
P.`LNAME`,
P.`MI`,
P.`SSC`,
sum(EN.`AMOUNT`) AS `TOTAL_AMOUNT`
FROM `table_1` AS EN
INNER JOIN `table_2` AS P ON EN.`MRN` = P.`MRN`
GROUP BY EN.`MRN`,P.`FNAME`, P.`LNAME`,P.`MI`,P.`SSC`
HAVING sum(EN.`AMOUNT`) > 0
ORDER BY P.`LNAME`
LIMIT 0, 100
How can I make this query to work faster. Because it takes a very long time execute the query for the first time to get total number of records.
It is better to separate total_amount from the query because that is the only value involving all the records. You can call this when you load the page. I assume that all your records in table_1 is valid.
SELECT sum(EN.`AMOUNT`) AS `TOTAL_AMOUNT` FROM `table_1` AS EN
HAVING sum(EN.`AMOUNT`) > 0
Then get the query result every time when you flip the page. This should only return 10 records starting from record 0.
SELECT
EN.`MRN`,
P.`FNAME`,
P.`LNAME`,
P.`MI`,
P.`SSC`
FROM `table_1` AS EN
INNER JOIN `table_2` AS P ON EN.`MRN` = P.`MRN`
GROUP BY EN.`MRN`,P.`FNAME`, P.`LNAME`,P.`MI`,P.`SSC`
HAVING sum(EN.`AMOUNT`) > 0
ORDER BY P.`LNAME`
LIMIT 10 OFFSET 0
Hope this helps.
There are a few things you can do to make this faster:
Use EXPLAIN to make sure your indices are set correctly / set the correct indices.
Only execute the complicated query once using SQL_CALC_FOUND_ROWS.
Use the TOTAL_AMOUNT alias in your HAVING statement to avoid doing the calculation twice.
MySQL will do some optimizations and caching itself so they might not all have the same impact.
I have a problem with an SQL query. This is my first time using advanced SQL operations like this so it could be that I'm missing something basic. I am running this query:
SELECT countries.id,
countries.name,
AVG(users.points) AS average
FROM countries
LEFT JOIN users
ON countries.id = users.country
ORDER BY average DESC
This query is only returning 1 row and it's not following the ORDER BY because the returned value is . My aim with this is to get all the records in the Countries table and get the average of the points awarded to the users from each country. I want it to return those countries which do not have users assigned to them as well. I have done this in 2 queries and it worked but I thought that maybe I could do only one query. What am I missing?
It is only returning one row because it is an aggregation query without a group by. Perhaps you mean:
SELECT c.id, c.name, AVG(u.points) AS average
FROM countries c LEFT JOIN
users u
ON c.id = u.country
GROUP BY c.id, c.name
ORDER BY average DESC;
The AVG() makes this an aggregation query. Without the the group by, SQL interprets it as returning one row summarizing all the rows. MySQL supports an extension to the SQL standard where columns in the select do not have to be in the group by. In most databases, you query would return an error.
I have this SQL query here that grabs the 5 latest news posts. I want to make it so it also grabs the total likes and total news comments in the same query. But the query I made seems to be a little slow when working with large amounts of data so I am trying to see if I can find a better solution. Here it is below:
SELECT *,
`id` as `newscode`,
(SELECT COUNT(*) FROM `likes` WHERE `type`="newspost" AND `code`=`newscode`) as `total_likes`,
(SELECT COUNT(*) FROM `news_comments` WHERE `post_id`=`newscode`) as `total_comments`
FROM `news` ORDER BY `id` DESC LIMIT 5
Here is a SQLFiddle as well: http://sqlfiddle.com/#!2/d3ecbf/1
I would recommend adding a total_likes and total_comments fields to the news table which gets incremented/decremented whenever a like and/or comment is added or removed.
Your likes and news_comments tables should be used for historical purposes only.
This strenuous counting should not be performed every time a page is loaded because that is a complete waste of resources.
You could rewrite this using joins, MySQL has known issues with subqueries, especially when dealing with large data sets:
SELECT n.*,
`id` as `newscode`,
COALESCE(l.TotalLikes, 0) AS `total_likes`,
COALESCE(c.TotalComments, 0) AS `total_comments`
FROM `news` n
LEFT JOIN
( SELECT Code, COUNT(*) AS TotalLikes
FROM `likes`
WHERE `type` = "newspost"
GROUP BY Code
) AS l
ON l.`code` = n.`id`
LEFT JOIN
( SELECT post_id, COUNT(*) AS TotalComments
FROM `news_comments`
GROUP BY post_id
) AS c
ON c.`post_id` = n.`id`
ORDER BY n.`id` DESC LIMIT 5;
The reason is that when you use a join as above, MySQL will materialise the results of the subquery when it is first needed, e.g at the start of this query, mySQL will put the results of:
SELECT post_id, COUNT(*) AS TotalComments
FROM `news_comments`
GROUP BY post_id
into an in memory table and hash post_id for faster lookups. Then for each row in news it only has to look up TotalComments from this hashed table, when you use a correlated subquery it will execute the query once for each row in news, which when news is large will result in a large number of executions. If the initial result set is small you may not see a performance benefit and it may be worse.
Examples on SQL Fiddle
Finally, you may want to index the relevant fields in news_comments and likes. For this particular query I think the following indexes will help:
CREATE INDEX IX_Likes_Code_Type ON Likes (Code, Type);
CREATE INDEX IX_newcomments_post_id ON news_comments (post_id);
Although you may need to split the first index into two:
CREATE INDEX IX_Likes_Code ON Likes (Code);
CREATE INDEX IX_Likes_Type ON Likes (Type);
First check for helping indexes on columns id, post_id and type,code.
I assume this is T-SQL, as that is what I am most familiar with.
First I would check indexes. If that looks good, then I'd check statement. Take a look at your query map to see how it's populating your result.
SQL works backward, so it starts with your last AND statement and goes from there. It'll group them all by code, and then type, and finally give you a count.
Right now, you're grabbing everything with certain codes, regardless of date. When you stated that you want the latest, I assume there is a date column somewhere.
In order to speed things up, add another AND to your WHERE and account for the date. Either last 24 hours, last week, whatever.
I'm doing a pagination feature using Codeigniter but I think this applies to PHP/mySQL coding in general.
I am retrieving directory listings using offset and limit depending on how many results I want per page. However to know the total number of pages required, I need to know (total number of results)/(limit). Right now I am thinking of running the SQL query a second time then count the number of rows required but without using LIMIT. But I think this seems to be a waste of computational resources.
Are there any better ways? Thanks!
EDIT: My SQL query uses WHERE as well to select all rows with a particular 'category_id'
Take a look at SQL_CALC_FOUND_ROWS
SELECT COUNT(*) FROM table_name WHERE column = 'value' will return the total number of records in a table matching that condition very quickly.
Database SELECT operations are usually "cheap" (resource-wise), so don't feel too bad about using them in a reasonable manner.
EDIT: Added WHERE after the OP mentioned that they need that feature.
The SQL_CALC_FOUND_ROWS query modifier and accompanying FOUND_ROWS()
function are deprecated as of MySQL 8.0.17 and will be removed in a
future MySQL version. As a replacement, considering executing your
query with LIMIT, and then a second query with COUNT(*) and without
LIMIT to determine whether there are additional rows. For example,
instead of these queries:
SELECT SQL_CALC_FOUND_ROWS * FROM tbl_name WHERE id > 100 LIMIT 10;
SELECT FOUND_ROWS();
Use these queries instead:
SELECT * FROM tbl_name WHERE id > 100 LIMIT 10;
SELECT COUNT(*) WHERE id > 100;
COUNT(*) is subject to certain optimizations. SQL_CALC_FOUND_ROWS
causes some optimizations to be disabled.
Considering that SQL_CALC_FOUND_ROWS requires invoking FOUND_ROWS() afterwards, if you wanted to fetch the total count with the results returned from your limit without having to invoke a second SELECT, I would use JOIN results derived from a subquery:
SELECT * FROM `table` JOIN (SELECT COUNT(*) FROM `table` WHERE `category_id` = 9) t2 WHERE `category_id` = 9 LIMIT 50
Note: Every derived table must have its own alias, so be sure to name the joined table. In my example I used t2.
you can use 2 queries as below, One to fetch the data with limit and other to get the no of total matched rows.
Ex:
SELECT * FROM tbl_name WHERE id > 1000 LIMIT 10;
SELECT COUNT(*) FROM tbl_name WHERE id > 1000;
As described by Mysql guide , this is the most optimized way, and also SQL_CALC_FOUND_ROWS query modifier and FOUND_ROWS() function are deprecated as of MySQL 8.0.17
as of final of 2021, why not:
SELECT
t1.*,
COUNT(t1.*) OVER (PARTITION BY RowCounter) as TotalRecords
FROM (
SELECT a, b, c, 1 as RowCounter
FROM MyTable
) t1
LIMIT 120,10
using a subquery with a column marking every row with the same value, will give us the possibility to count all of the same values of the the resulted column with PARTITION BY window function's group
SELECT COUNT(id) FROM `table` WHERE `category_id` = 9
Gives you the number of rows for your specific category.
As I am trying to count the number of records in a table, even when the SQL statement has a LIMIT into it, overall it works, however something weird happens, the code:
$sql = "SELECT COUNT(*) AS count FROM posts
ORDER BY post_date DESC
LIMIT 5";
// ... mysql_query, etc
while($row = mysql_fetch_array($result))
{
// ... HTML elements, etc
echo $row['post_title'];
// ... HTML elements, etc
echo $row['count']; // this displays the number of posts (which shows "12").
}
Although, when displaying through the while loop, it displays this:
Notice: Undefined index: post_title in /Applications/MAMP/htdocs/blog/index.php on line 55
If I remove the COUNT(*) AS count, everything will display perfectly... how come it's doing this?
Don't use COUNT(*) to count the number of rows (for a lot of reasons). Write out your full query, and add SQL_CALC_FOUND_ROWS right after SELECT:
SELECT SQL_CALC_FOUND_ROWS id, title FROM foo LIMIT 5;
Then, after that query executed (right after), run:
SELECT FOUND_ROWS();
That will return the number of rows the original SELECT would have returned if you didn't have the LIMIT on the end (accounting for all joins and where clauses).
It's not portable, but it's very efficient (and IMHO the right way of handling this type of problem).
LIMIT does not do anything here because you're selecting a single scalar. The error is shown because you are not selecting the post title, so it is not in the $row hash.
This is happening because COUNT() is an aggregate function. You will have to do two separate queries in order to get both the count of rows in the table and separate records.
I had the same problem and found this article: http://www.mysqldiary.com/limited-select-count/
best way (especially for big tables) is using it like this:
SELECT COUNT(*) AS count FROM (SELECT 1 FROM posts
ORDER BY post_date DESC
LIMIT 5) t
now it returns a number between 0 and 5
You're asking the SELECT statement to return only COUNT(*) AS count. If you want more columns returned, you have to specify them in the SELECT statement.
You're mixing up an aggregate function COUNT() with a standard selection. You can't get an accurate post title without a GROUP BY clause in your aggregate query. The easiest thing you should do is to do this two queries - one for your count, the other for your post information. Also, there's no sense in using LIMIT on an aggregate function with no other columns in your (omitted) GROUP BY - the current query will always return one row.
What everyone is trying to say is that you should opt to use 2 separate queries instead of 1 single one.
1) get row count of table
$sql = "SELECT COUNT(*) AS count FROM posts;
2) get last 5 rows of table
$sql = "SELECT * FROM posts
ORDER BY post_date DESC
LIMIT 5";
By including COUNT(*) AS count you've removed the actual columns that you want to get the information from. Try using
SELECT *, COUNT(*) AS count FROM posts ORDER BY post_date DESC LIMIT 5
That query will accomplish both things you're trying to do in one query, but really, these should be broken out into 2 separate queries. What is the overall purpose for having a COUNT in the query? It should be done a different way.
This works for me:
$sql = "
SELECT COUNT( po.id ) AS count, p.post_title
FROM posts p
JOIN posts po
GROUP BY p.post_title
ORDER BY p.post_date DESC
LIMIT 5
";
Important is to JOIN the same table, but name the table differently. Do not forget GROUP BY, must be used.