Related
I am counting the entries in my SQL database:
$sql = "SELECT * FROM files WHERE id = ?";
$q = $pdo->prepare($sql);
$q->execute([$id]);
$rowCount =$q->rowCount();
The result of $rowCount is 500000.
But to output this single number takes 5 seconds! Is it possible to get this result faster?
Use the COUNT() function https://dev.mysql.com/doc/refman/8.0/en/counting-rows.html:
$sql = "SELECT COUNT(*) FROM files WHERE id = ?";
Also ensure that 'id' is an indexed column:
https://dev.mysql.com/doc/refman/8.0/en/mysql-indexes.html
Replace * with a field(use auto-increment id) - This will reduce the time a bit.
Index that field. - If you use indexed field the query performance will increase.
SELECT * ..., then counting in PHP, requires shoveling all columns of all rows back to PHP. That's a lot of effort for very little gain.
SELECT COUNT(col) ... does the counting in by MySQL, but it must check for whether col is NULL. And it needs to get at the value of col for every row.
SELECT COUNT(*) ... counts the rows by whatever way is most efficient. This involves looking for the 'smallest' index (or the whole table, if no secondary indexes), and counting through it.
You must learn about INDEXes to get anywhere in databases! This is only one minor use for them.
I'm just wondering which method is the most effective if I'm literally just wanting to get the number of rows in a table.
$res = mysql_query("SELECT count(*) as `number` FROM `table1`");
$count = mysql_fetch_result($res,0,'number');
or
$res = mysql_query("SELECT `ID` FROM `table1`");
$count = mysql_num_rows($res);
Anyone done any decent testing on this?
mysql_query() transfers all result records from the MySQL into the php pcrocess before it returns (unlike mysql_unbufferd_query()). That alone would make the mysql_num_rows() version slower.
Furthermore for some engines (like MyISAM) MySQL can serve a Count(*) request from the index of the table without hitting the actual data. A SELECT * FROM foo on the other hand results in a full table scan and MySQL has to read every single dataset.
Test in database with more then 2300000 rows, type:InnoDB, size near 1 GiB, using xhprof
test1:
....SELECT COUNT(id) as cnt FROM $table_name....;
row= mysqli_fetch_assoc($res2);
echo $row['cnt'];
//result1:
1,144,106
1,230,576
1,173,449
1,163,163
1,218,992
test2:
....SELECT COUNT(*) as cnt FROM $table_name....;
row= mysqli_fetch_assoc($res2);
echo $row['cnt'];
//result2:
1,120,253
1,118,243
1,118,852
1,092,419
1,081,316
test3:
....SELECT * FROM $table_name....;
echo mysqli_num_rows($res2);
//result3:
7,212,476
6,530,615
7,014,546
7,169,629
7,295,878
test4:
....SELECT * FROM $table_name....;
echo mysqli_num_rows($res2);
//result4:
1,441,228
1,671,616
1,483,050
1,446,315
1,647,019
conclusion:
The fastest method is in the test2 :
....SELECT COUNT(*) as cnt FROM $table_name....;
row= mysqli_fetch_assoc($res2);
echo $row['cnt'];
Definitely the first. MySQL can usually do this by looking at an index rather than the whole table, and if you use MyISAM (the default), the row count for the table is stored in the table metadata and will be returned instantly.
Your second method will not only read the entire table into memory but also send it to the client through the network before the client counts the rows. Extremely wasteful!
I don't really think any testing is needed.
Doing the COUNT in the SQL query
1) Sends only one row of data back the
to client (instead of every row)
2) Lets SQL do the count
for you which is likely always going
to be faster than PHP.
I guess count(1) will be even faster:
$res = mysql_query("SELECT count(1) as `number` FROM `table1`");
$count = mysql_fetch_result($res,0,'number');
Although haven't tried the proposed methods, the first makes database fetch all the records and count them in the database, the second makes database fetch a separate field for all the records and count the number of results on the server.
As a rule of thumb the less data you fetch for a particular record the less time it will take therefore I'd vote for updated first method (fetching constant for every record and counting the number of constants fetched).
Using Count with index and inodb makes it too much slow, but when use it with mysqli_num_rows it returns without any delay. you can check mysqli_num_rows result at http://ssajalandhar.org/generalinstruction-0-1-0.html it wouldn't take fraction of second to load. For me mysqli works awesome.
I try to avoid doing Count() because of performance issue. (i.e. SELECT COUNT() FROM Users)
If I run the followings in phpMyAdmin, it is ok:
SELECT SQL_CALC_FOUND_ROWS * FROM Users;
SELECT FOUND_ROWS();
It will return # of rows. i.e. # of Users.
However, if I run in in PHP, I cannot do this:
$query = 'SELECT SQL_CALC_FOUND_ROWS * FROM Users;
SELECT FOUND_ROWS(); ';
mysql_query($query);
It seems like PHP doesn't like to have two queries passing in. So, how can I do that?
SQL_CALC_FOUND_ROWS is only useful if you're using a LIMIT clause, but still want to know how many rows would've been found without the LIMIT.
Think of how this works:
SELECT SQL_CALC_FOUND_ROWS * FROM Users;
You're forcing the database to retrieve/parse ALL the data in the table, and then you throw it away. Even if you aren't going to retrieve any of the rows, the DB server will still start pulling actual data from the disk on the assumption that you will want that data.
In human terms, you bought the entire contents of the super grocery store, but threw away everything except the pack of gum from the stand by the cashier.
Whereas, doing:
SELECT count(*) FROM users;
lets the DB engine know that while you want to know how many rows there are, you couldn't care less about the actual data. On most any intelligent DBMS, the engine can retrieve this count from the table's metadata, or a simple run through the table's primary key index, without ever touching the on-disk row data.
Its two queries:
$query = 'SELECT SQL_CALC_FOUND_ROWS * FROM Users';
mysql_query($query);
$query = 'SELECT FOUND_ROWS()';
mysql_query($query);
PHP can only issue a single query per mysql_query call
It's a common misconception, that SQL_CALC_FOUND_ROWS performs better than COUNT(). See this comparison from Percona guys: http://www.mysqlperformanceblog.com/2007/08/28/to-sql_calc_found_rows-or-not-to-sql_calc_found_rows/
To answer you question: Only one query is allowed per one mysql_query call, as described in manual: mysql_query() sends a unique query (multiple queries are not supported)
Multiple queries are supported when using ext/mysqli as your MySQL extension:
http://www.php.net/manual/en/mysqli.multi-query.php
Only this code works for me so i want to share it for you.
$Result=mysqli_query($i_link,"SELECT SQL_CALC_FOUND_ROWS id From users LIMIT 10");
$NORResult=mysqli_query($i_link,"Select FOUND_ROWS()");
$NORRow=mysqli_fetch_array($NORResult);
$NOR=$NORRow["FOUND_ROWS()"];
echo $NOR;
Use 'union' and empty columns:
$sql="(select sql_calc_found_rows tb.*, tb1.title
from orders tb
left join goods tb1 on tb.goods_id=tb1.id
where {$where}
order by created desc
limit {$offset}, {$page_size})
union
(select found_rows(), '', '', '', '', '', '', '', '', '')
";
$rs=$db->query($sql)->result_array();
$total=array_pop($rs);
$total=$total['id'];
This is an easy way & works for me :
$query = "
SELECT SQL_CALC_FOUND_ROWS *
FROM tb1
LIMIT 5";
$result = mysqli_query($link, $query);
$query = "SELECT FOUND_ROWS() AS count";
$result2 = mysqli_query($link, $query);
$row = mysqli_fetch_array($result2);
echo $row['count'];
Do you really think that selecting ALL rows from tables is faster than counting them?
Myisam stores a number of records in table's metadata, so SELECT COUNT(*) FROM table don't have to access data.
If I have this:
$results = mysql_query("SELECT * FROM table_name WHERE id=$id");
is there then any way to check how many rows which have a field-value of "Private" or "Company" ?
I need to show the user how many "Private" and "Company" records where found, without making another query. (There is a column called 'ad_type' which contains either "private" or "company")
I already know the mysql_num_rows for counting all rows!
EDIT:
There are 500thousand records! So maybe an iteration through the result is slow, what do you think?
Thanks for all help :)
The above answers are great and all, but the currently checked answer will work very inefficiently should you be dealing with a large amount of data
Example of the above answer (via Gal)
$results = mysql_query("SELECT *,(SELECT COUNT(*) FROM table_name WHERE column=$value) count FROM table_name WHERE id=$id");
It's good and all, and it returns what you need but the obvious design flaw is that making your SQL server return the results then re-return them and look at just the count is very inefficient for large amounts of data.
Simply do this:
$results = mysql_query("SELECT * FROM table_name WHERE column=$value");
$num_rows = mysql_num_rows($result);
It will yield the same results and be much more efficient in the long run, additionally for larger amounts of data.
You can do something like:
$results = mysql_query("SELECT *,(SELECT COUNT(*) FROM table_name WHERE column=$value) count FROM table_name WHERE id=$id");
in order to fetch the number with sql.
If you don't want to change your query you could do a
$results = mysql_query("SELECT * FROM table_name WHERE id=$id");
$count = mysql_num_rows($results);
steps to get a count():
use mysql_query() to get count,
use mysql_fetch_array() to get the only 1 row
get the only one column of the row, this is the count,
here is an example, which check whether the email is already used:
// check whether email used
$check_email_sql = "select count(*) from users where email='$email'";
$row = mysql_fetch_array(mysql_query($check_email_sql));
$email_count = $row[0];
Iterate through the result set of rows and count the number of occurences of Private and Company in ad_type, respectively?
You can do
SELECT COUNT(*) FROM table_name WHERE id=$id GROUP BY fieldvalue HAVING fieldvalue = "Private"
SELECT COUNT(*) FROM table_name WHERE id=$id GROUP BY fieldvalue HAVING fieldvalue = "Company"
but that would be another query. But if you process the data anyway, you could simply sum up the number of "Private" and "Company" rows after doing the query.
In the case you don't have to get all results, use this.
SELECT ad_type, COUNT(*)
FROM table_name
WHERE (id=$id)
GROUP BY ad_type
HAVING ((ad_type = 'Private') OR (ad_type = 'Company'))
If you still have to fetch all the records where id = $id, it won't work. But executing such a query (once) before fetching the real data should be more efficient than using a subquery.
I guess this query would do the job:
SELECT ad_type, count(*) FROM table_name WHERE id=$id GROUP BY ad_type;
I don't see any reason so far to use HAVING, since you probably want to show the user an overview of all the ad_type's found in DB (at least you didn't mention that there are other values for ad_type then the two given).
I also strongly suggest NOT to use sub-queries; always try to use just one.
If there's one thing that will slow your query down, it's a subquery (or subqueries).
Good luck!
Iterate through the results of the query and keep a count of how many of each show up in local variables.
Whats the most efficient way of selecting total number of records from a large table? Currently, Im simply doing
$result = mysql_query("SELECT id FROM table");
$total = mysql_num_rows($result)
I was told this was not very efficient or fast, if you have a lot of records in the table.
You were told correctly. mysql can do this count for you which is much more efficient.
$result = mysql_query( "select count(id) as num_rows from table" );
$row = mysql_fetch_object( $result );
$total = $row->num_rows;
You should use SQL's built in COUNT function:
$result = mysql_query("SELECT COUNT(id) FROM table");
MyISAM tables already store the row count
SELECT COUNT(*) FROM table
on a MyISAM table simply reads that value. It doesn't scan the table or the index(es). So, it's just as fast or faster than reading the value from a different table.
According to the MySQL documentation this is most efficient if you're using a MyISAM table (which is the most usual type of tables used):
$result = mysql_query("SELECT COUNT(*) FROM table");
Otherwise you should do as Wayne stated and be sure that the counted column is indexed.
Can I just add, that the most "efficient" way of getting the total number of records, particularly in a large table, is to save the total amount as a number in another table.
That way, you don't have to query the entire table everytime you want to get the total.
You will however, have to set up some code or Triggers in the database to increase or decrease that number when a row is added/deleted.
So its not the easiest way, but if your website grows, you should definitely consider doing that.
Even though I agree to use the built-in functions, I don't really see any performance difference between mysql_num_rows and count(id). For 25000 results, same performance (can say exact.) Just for the record.
What about something like this:
$result = mysql_query("SELECT COUNT(id) AS total_things from table");
$row = mysql_fetch_array($result,MYSQL_ASSOC);
$num_results = $row["total_things"];
I had a large table (>50 million rows) and it took a long time to count the primary key, so I use the following:
SELECT TABLE_NAME, TABLE_ROWS
FROM information_schema.tables
WHERE TABLE_SCHEMA = "database";
Replace database with the name of your schema.
Just wanted to note that SHOW TABLE STATUS returns a Rows column, though I can't speak to its efficiency. Some light Googling turns up reports of slowness in MySQL 4 over two years ago. Might make for interesting time trials.
Also note the InnoDB caveat regarding inaccurate counts.
Use aggregate function. Try the below SQL Command
$num= mysql_query("SELECT COUNT(id) FROM $table");
mysqli_query() is deprecated. Better use this:
$result = $dbh->query("SELECT id FROM {table_name}");
$total = $result->num_rows;
Using PDO:
$result = $dbh->query("SELECT id FROM {table_name}");
$total = $result->rowCount();
(where '$dbh' = handle of the db connected to)