I am counting the entries in my SQL database:
$sql = "SELECT * FROM files WHERE id = ?";
$q = $pdo->prepare($sql);
$q->execute([$id]);
$rowCount =$q->rowCount();
The result of $rowCount is 500000.
But to output this single number takes 5 seconds! Is it possible to get this result faster?
Use the COUNT() function https://dev.mysql.com/doc/refman/8.0/en/counting-rows.html:
$sql = "SELECT COUNT(*) FROM files WHERE id = ?";
Also ensure that 'id' is an indexed column:
https://dev.mysql.com/doc/refman/8.0/en/mysql-indexes.html
Replace * with a field(use auto-increment id) - This will reduce the time a bit.
Index that field. - If you use indexed field the query performance will increase.
SELECT * ..., then counting in PHP, requires shoveling all columns of all rows back to PHP. That's a lot of effort for very little gain.
SELECT COUNT(col) ... does the counting in by MySQL, but it must check for whether col is NULL. And it needs to get at the value of col for every row.
SELECT COUNT(*) ... counts the rows by whatever way is most efficient. This involves looking for the 'smallest' index (or the whole table, if no secondary indexes), and counting through it.
You must learn about INDEXes to get anywhere in databases! This is only one minor use for them.
Related
Let's say I have a table with following columns: id-1, id-2, col-1, col-2, col-3
Here, id-1 is the primary key and is auto-incremented. id-2 is a different id and is not necessary to be unique. There are multiple instances of same id in that column. col-1, col-2, col-3 are just necessary columns.
I pass a query to select data from the table.
mysqli_query($connect, SELECT * FROM table WHERE id-2='some_specific_id')
It will return multiple rows. I would like to know how can I target specific rows, say row number 3.
First, use ":
mysqli_query($connect, "SELECT * FROM table WHERE id-2 = 'some_specific_id'");
Target specific row? Do you mean to limit the fetched rows? Or get the 3rd row?
For limiting the fetched rows, you can use LIMIT:
SELECT * FROM table WHERE id-2='some_specific_id' LIMIT 3
For getting the third row:
SELECT * FROM table WHERE id-2='some_specific_id' LIMIT 2, 1
Well although it seems you just rather needed to learn basic SQL to get your answer, there is still the question in the title, that may attract other people whose problem is formulated exactly like that. So goes the answer:
Mysqli is not very convenient for this task, so we would use PDO.
In case your query is intended to return multiple rows and you need to address one of them by number (which is rather makes little sense, but anyway), use fetchAll() method:
$stmt = $connect->prepare("SELECT * FROM table WHERE id2=?");
$stmt->execute(['some specific id']);
$data = $stmt->fetchAll();
and you will be able to address returned rows by number, starting from zero:
echo $data[0]['col1'];
However, it makes more sense to address the returned rows by some unique id. In this case just add this unique field fiset in the field list and then use the special PDO street magic:
$stmt = $connect->prepare("SELECT id1, table.* FROM table WHERE id2=?);
$stmt->execute(['some specific id']);
$data = $stmt->fetchAll(PDO::FETCH_UNIQUE);
and you will be able to address returned rows by that unique field :
echo $data[$id1]['col1'];
Use LIMIT to get what you want like:
SELECT * FROM table WHERE id-2='some_specific_id' LIMIT 2, 1;
Or, if you want to fetch from array, then use the 3rd index of array.
LIMIT Explanation:
The following illustrates the LIMIT clause syntax with two arguments:
SELECT
column1,column2,...
FROM
table
LIMIT offset , count;
Let's examine the LIMIT clause parameters:
The offset specifies the offset of the first row to return. The offset of the first row is 0, not 1.
The count specifies the maximum number of rows to return.
I'm just wondering which method is the most effective if I'm literally just wanting to get the number of rows in a table.
$res = mysql_query("SELECT count(*) as `number` FROM `table1`");
$count = mysql_fetch_result($res,0,'number');
or
$res = mysql_query("SELECT `ID` FROM `table1`");
$count = mysql_num_rows($res);
Anyone done any decent testing on this?
mysql_query() transfers all result records from the MySQL into the php pcrocess before it returns (unlike mysql_unbufferd_query()). That alone would make the mysql_num_rows() version slower.
Furthermore for some engines (like MyISAM) MySQL can serve a Count(*) request from the index of the table without hitting the actual data. A SELECT * FROM foo on the other hand results in a full table scan and MySQL has to read every single dataset.
Test in database with more then 2300000 rows, type:InnoDB, size near 1 GiB, using xhprof
test1:
....SELECT COUNT(id) as cnt FROM $table_name....;
row= mysqli_fetch_assoc($res2);
echo $row['cnt'];
//result1:
1,144,106
1,230,576
1,173,449
1,163,163
1,218,992
test2:
....SELECT COUNT(*) as cnt FROM $table_name....;
row= mysqli_fetch_assoc($res2);
echo $row['cnt'];
//result2:
1,120,253
1,118,243
1,118,852
1,092,419
1,081,316
test3:
....SELECT * FROM $table_name....;
echo mysqli_num_rows($res2);
//result3:
7,212,476
6,530,615
7,014,546
7,169,629
7,295,878
test4:
....SELECT * FROM $table_name....;
echo mysqli_num_rows($res2);
//result4:
1,441,228
1,671,616
1,483,050
1,446,315
1,647,019
conclusion:
The fastest method is in the test2 :
....SELECT COUNT(*) as cnt FROM $table_name....;
row= mysqli_fetch_assoc($res2);
echo $row['cnt'];
Definitely the first. MySQL can usually do this by looking at an index rather than the whole table, and if you use MyISAM (the default), the row count for the table is stored in the table metadata and will be returned instantly.
Your second method will not only read the entire table into memory but also send it to the client through the network before the client counts the rows. Extremely wasteful!
I don't really think any testing is needed.
Doing the COUNT in the SQL query
1) Sends only one row of data back the
to client (instead of every row)
2) Lets SQL do the count
for you which is likely always going
to be faster than PHP.
I guess count(1) will be even faster:
$res = mysql_query("SELECT count(1) as `number` FROM `table1`");
$count = mysql_fetch_result($res,0,'number');
Although haven't tried the proposed methods, the first makes database fetch all the records and count them in the database, the second makes database fetch a separate field for all the records and count the number of results on the server.
As a rule of thumb the less data you fetch for a particular record the less time it will take therefore I'd vote for updated first method (fetching constant for every record and counting the number of constants fetched).
Using Count with index and inodb makes it too much slow, but when use it with mysqli_num_rows it returns without any delay. you can check mysqli_num_rows result at http://ssajalandhar.org/generalinstruction-0-1-0.html it wouldn't take fraction of second to load. For me mysqli works awesome.
It has long been known that PDO does not support COUNT(*) and a query like below would fail as it doesn't return any affected rows,
$q = $dbc -> prepare("SELECT COUNT(*) FROM table WHERE id = ?");
$q -> execute(array($id));
echo $q -> rowCount();
Doing some research I found that you can also get the row count using other methods of count and not using count at all, for example the following query is supposed be the same as above but will return correct for PDO,
$q = $dbc -> prepare("SELECT 1 FROM table WHERE id = ?");
$q -> execute(array($id));
echo $q -> rowCount();
There are various sources on the internet claiming that;
"SELECT COUNT(*)
"SELECT COUNT(col)
"SELECT 1
Are all the same as each other (with a few differences) so how come using mysql which PDO cannot properly return a true count, does
"SELECT 1
work?
Methods of count discussion
Why is Select 1 faster than Select count(*)?
PDO does not support COUNT(*)
WTF? Of course PDO supports COUNT(*), you are using it the wrong way.
$q = $dbc->prepare("SELECT COUNT(id) as records FROM table WHERE id = ?");
$q->execute(array($id));
$records = (int) $q->fetch(PDO::FETCH_OBJ)->records;
If you are using a driver other than MySQL, you might have to test rowCount first, like this.
$records = (int) ($q->rowCount()) ? $q->fetch(PDO::FETCH_OBJ)->records : 0;
Oh. You are confusing everything.
PDO do not interfere with SQL queries. It support EVERYTHING supported by SQL.
When doing COUNT(*) you shouldn't use rowcount at all, as it just makes no sense. You have to retreive the query result instead.
Dunno what "various sources" you are talking about but COUNT(*) and COUNT(col) (and even COUNT(1)) are the same and the only proper way to get count of records when you need no records themselves.
COUNT is an aggregate function, it counts rows for you. So, it returns the result already, no more counting required. Ad it returns just a scalar value in the single row. Thus, using rowcount on this single row makes no sense
SELECT 1 is not the same as above, as it selects just literal 1 for the every row found in the table. So, it will return a thousand 1s if there is a thousands rows in your database. So, rowcount will give you the result but it is going to be an extreme waste of the server resources.
there is a simple rule to follow:
Always request the only data you need.
If you need the count of rows - request count of rows. Not a thousand of 1s to count them later.
Sounds sensible?
Best way I think to test if a line exist in your database is to perform.
SELECT 1 FROM table WHERE condition LIMIT 1
If it find a row it will stop and tell you there is a line. If it don't and you have an index on your where clause column it will also goes very fast to see there are none available.
To randomly select records from one table; do I have to always set a temporary variable in PHP? I need some help with selecting random rows within a CodeIgniter model, and then display three different ones in a view every time my homepage is viewed. Does anyone have any thoughts on how to solve this issue? Thanks in advance!
If you don't have a ton of rows, you can simply:
SELECT * FROM myTable ORDER BY RAND() LIMIT 3;
If you have many rows, this will get slow, but for smaller data sets it will work fine.
As Steve Michel mentions in his answer, this method can get very ugly for large tables. His suggestion is a good place to jump off from. If you know the approximate maximum integer PK on the table, you can do something like generating a random number between one and your max PK value, then grab random rows one at a time like:
$q="SELECT * FROM table WHERE id >= {$myRandomValue}";
$row = $db->fetchOne($q); //or whatever CI's interface to grab a single is like
Of course, if you need 3 random rows, you'll have three queries here, but as they're entirely on the PK, they'll be fast(er than randomizing the whole table).
I would do something like:
SELECT * FROM table ORDER BY RAND() LIMIT 1;
This will put the data in a random order and then return only the first row from that random order.
I have this piece of code in production to get a random quote. Using MySQL's RAND function was super slow. Even with 100 quotes in the database, I was noticing a lag time on the website. With this, there was no lag at all.
$result = mysql_query('SELECT COUNT(*) FROM quotes');
$count = mysql_fetch_row($result);
$id = rand(1, $count[0]);
$result = mysql_query("SELECT author, quote FROM quotes WHERE id=$id");
you need a query like this:
SELECT *
FROM tablename
WHERE somefield='something'
ORDER BY RAND() LIMIT 3
It is taken from the second result of
http://www.google.com/search?q=mysql+random
and it should work ;)
Ordering a big table by rand() can be very expensive if the table is very large. MySQL will need to build a temporary table and sort it. If you have primary key and you know how many rows are in the table, use LIMIT x,1 to grab a random row, where x is the number of the row you want to get.
Whats the most efficient way of selecting total number of records from a large table? Currently, Im simply doing
$result = mysql_query("SELECT id FROM table");
$total = mysql_num_rows($result)
I was told this was not very efficient or fast, if you have a lot of records in the table.
You were told correctly. mysql can do this count for you which is much more efficient.
$result = mysql_query( "select count(id) as num_rows from table" );
$row = mysql_fetch_object( $result );
$total = $row->num_rows;
You should use SQL's built in COUNT function:
$result = mysql_query("SELECT COUNT(id) FROM table");
MyISAM tables already store the row count
SELECT COUNT(*) FROM table
on a MyISAM table simply reads that value. It doesn't scan the table or the index(es). So, it's just as fast or faster than reading the value from a different table.
According to the MySQL documentation this is most efficient if you're using a MyISAM table (which is the most usual type of tables used):
$result = mysql_query("SELECT COUNT(*) FROM table");
Otherwise you should do as Wayne stated and be sure that the counted column is indexed.
Can I just add, that the most "efficient" way of getting the total number of records, particularly in a large table, is to save the total amount as a number in another table.
That way, you don't have to query the entire table everytime you want to get the total.
You will however, have to set up some code or Triggers in the database to increase or decrease that number when a row is added/deleted.
So its not the easiest way, but if your website grows, you should definitely consider doing that.
Even though I agree to use the built-in functions, I don't really see any performance difference between mysql_num_rows and count(id). For 25000 results, same performance (can say exact.) Just for the record.
What about something like this:
$result = mysql_query("SELECT COUNT(id) AS total_things from table");
$row = mysql_fetch_array($result,MYSQL_ASSOC);
$num_results = $row["total_things"];
I had a large table (>50 million rows) and it took a long time to count the primary key, so I use the following:
SELECT TABLE_NAME, TABLE_ROWS
FROM information_schema.tables
WHERE TABLE_SCHEMA = "database";
Replace database with the name of your schema.
Just wanted to note that SHOW TABLE STATUS returns a Rows column, though I can't speak to its efficiency. Some light Googling turns up reports of slowness in MySQL 4 over two years ago. Might make for interesting time trials.
Also note the InnoDB caveat regarding inaccurate counts.
Use aggregate function. Try the below SQL Command
$num= mysql_query("SELECT COUNT(id) FROM $table");
mysqli_query() is deprecated. Better use this:
$result = $dbh->query("SELECT id FROM {table_name}");
$total = $result->num_rows;
Using PDO:
$result = $dbh->query("SELECT id FROM {table_name}");
$total = $result->rowCount();
(where '$dbh' = handle of the db connected to)