I am trying to implement a paging feature using PDO's ability to create cursors.
Currently, my code looks a bit like this (very complicated, I know that):
$pdo = new PDO();
$pdo->setAttribute(PDO::ATTR_CURSOR, PDO::CURSOR_SCROLL);
// prepared select-query omitted
$pdoStatement = $pdo->execute();
$start_index = MAX_THINGS_PER_PAGE * $current_page - MAX_THINGS_PER_PAGE;
$stop_index = MAX_THINGS_PER_PAGE * $current_page;
$row_count = $this->statement->rowCount(); // works for the PgSQL driver
$index = $start_index;
while (($row_count > 0) && ($index < $stop_index))
{
// try-catch block omitted
$values[] = $this->statement->fetch(PDO::FETCH_ASSOC, PDO::FETCH_ORI_ABS, $index);
--$row_count;
++$index;
}
However, seemingly, no matter what $start_index is, the query only fetches the first 10 (which is the value of MAX_THINGS_PER_PAGE) rows of the resultset. Always.
Probably I am doing something wrong, but the art of using cursors for pagination seems to be somewhat arcane and underdocumented...
I just ran into this problem today. PDOStatement::rowCount() does not work for SELECT on some databases. From PDOStatement::rowCount:
If the last SQL statement executed by the associated PDOStatement was a SELECT statement, some databases may return the number of rows returned by that statement. However, this behaviour is not guaranteed for all databases and should not be relied on for portable applications.
It took me quite a bit to realize this, as I thought that it was a problem with using cursors.
This is the approach I took: Replace the use of PDOStatement::rowCount() with the following:
<?php
$row_count = count($stmt->fetchAll());
?>
It is not efficient memory-wise, but this is what many databases do to calculate the total number of rows anyway.
Related
I face a problem with a mysql query. The query runs but sometimes it makes php exceed the maximum memory. I have like 80 or 90.000 rows of coordinates, with engine speeds and other things. I have to create KML files to display routes individually. Where the engine speed is not null, the car is moving, if it is, the car is stopped. Half of the table's engine speeds contains 0s. When I'm iterating through the records, I also delete the records at the same time, after I created the routes array, but it runs very slowly and sometimes it runs out of memory. Can it be because of the high and massive data amount in the database or some logical error in my code? Here is the code:
public function getPositions($device_id) {
$db = connect_database(DB_HOST, DB_USER, DB_PASSWORD, DB_NAME, DB_PORT);
$sql = "SELECT * FROM coordinates_log WHERE imei=:imei ORDER BY device_time ASC";
$statement = $db->prepare($sql);
$statement->execute(array(':imei' => $device_id));
$positions = array();
$delete_sql = "DELETE FROM coordinates_log WHERE id=:id";
$delete_statement = $db->prepare($delete_sql);
$counter = 0;
$flag = 0;
while ($row = $statement->fetch(PDO::FETCH_ASSOC)) {
//here I flag the last started route
if ($row['vehicle_engine_speed'] <= 0) {
$flag = $counter;
}
$positions[] = $row;
$counter++;
}
if (!empty($positions)) {
$last_key = count($positions)-1;
//here I check if the route is completed yet, or he is on his way
if ($positions[$last_key]['vehicle_engine_speed'] != 0) {
for($i = $flag; $i<=$last_key; $i++){
unset($positions[$i]);
}
}
foreach ($positions as $position) {
$delete_statement->execute(array(':id' => $position['id']));
}
return $positions;
} else {
return FALSE;
}
}
The PDO subsystem in PHP offers two kinds of queries: buffered and unbuffered. Buffered queries are what you get if you don't specifically request unbuffered queries. Buffered queries consume more RAM in your PHP engine because PDO fetches the entire result set into RAM, then gives it back to your program a line at a time when you use $statement->fetch().
So, if your result sets are quite large and you can process them a row at a time, you will use less RAM with unbuffered mode. You process each row, then fetch the next one, without trying to hold them all in RAM at once.
Here's a writeup on unbuffered mode.
http://php.net/manual/en/mysqlinfo.concepts.buffering.php
Buffered mode is generally easier to use for programmers, because PDO reads the entire resultset from each query and implicitly closes the statement object. That leaves your connection available for the next sql statement, even if you have not yet processed all the information in your resultset. With unbuffered mode, if you want to run other mysql statements while you're processing your result set, you need another database connection to do that.
You should try unbuffered mode for your SELECT * FROM coordinates... result set.
Pro tip: If you avoid SELECT * and instead use SELECT col, col, col you probably can reduce the overhead of your queries, especially if you don't actually need all the columns.
Questions of the "Look at my code and tell me what's wrong with it" kind are off topic here. It is not only because the code is intended to be run, by the computers, not read by the humans, but because code itself is seldom relevant to the problem.
Before asking a question here, you have to profile your code, determining slowest parts, and memory consumption as well.
I could make some guesses though I hate it.
it could be query itself, if not optimized
it could be buffering issue, making whole resultset burden script's memory.
it could be select * issue, burdening your resulting array with lots of junk data
it could be slow writes due to innodb settings.
But guesswork doesn't make a good answer. You have to work out your question first.
I am editing a tutorial I found on inline editing. It uses mysql but obviously I want to use mysqli as mysql is depreciated. I have changed most of it but one part is causing me difficulty. There is a seperate file that is called to connect to the database and run queries so the main page I will declare at the top at the top of my page I declare
require_once("dbcontroller.php");
$db_handle = new DBController();
$sql = "SELECT * from php_interview_questions";
$faq = $db_handle->runQuery($sql);
The relevant part of the dbcontroller.php is:
function runQuery($query) {
$result = mysqli_query($conn, $query);
while($row=mysqli_fetch_assoc($result)) {
$resultset[] = $row;
}
if(!empty($resultset))
return $resultset;
}
function numRows($query) {
$result = mysqli_query($conn,$query);
$rowcount = mysqli_num_rows($result);
return $rowcount;
}
Am I right in thinking that I need to use prepared statements? If so how would this cope with working with a select query that could involve any number of columns or any number of conditions in the WHERE clause?
numRows() is a strange function since it runs a query only to find out how many rows it returns. I would consider it a questionable practice. If it is necessary to find out how many rows a SELECT would return, then one could change SELECT * to SELECT COUNT(*). MySQL would be able to optimize that for MyISAM tables, plus there is a saving on not having to transfer the result set to the client.
Otherwise there is no need to use prepared statements, since a prepared statement only increases the amount of work a DBMS has to do when a statement is executed only once (it has to handle two requests - prepare, execute, -- instead of one -- execute, plus extra work to clean up the cached statement when connection is destroyed).
What is the preferred method for getting the number of rows that are returned for a SELECT state when using PDO with prepared statements?
I am currently using rowCount() but the docs say I shouldn't be using that since "most databases" don't support it (It is actually working just fine for me, so I'm tempted to keep using it. I can't find any sources that list exactly which databases do not support it, but apparently mine is fine).
Instead they recommend I use fetchColumn() but that requires writing a completely separate SQL statement that includes the COUNT(*) in my sql statement.
This is what they propose (http://php.net/manual/en/pdostatement.rowcount.php#example-1038):
//SQL TO GET ROWS TO OUTPUT
$sql = 'SELECT *
FROM properties
WHERE lister_id = :lister_id
AND lister_type = "landlord"';
$result = $dbh->prepare($sql);
$result->bindParam(':lister_id', $_SESSION['loggedin_lister_id'], PDO::PARAM_INT);
$result->execute();
//SQL TO GET NUMBER OF RETURNED ROWS
$row_num_sql = 'SELECT COUNT(*)
FROM properties
WHERE lister_id = :lister_id
AND lister_type = "landlord"';
$row_num_result = $dbh->prepare($row_num_sql);
$row_num_result->bindParam(':lister_id', $_SESSION['loggedin_lister_id'], PDO::PARAM_INT);
$row_num_result->execute();
$num_rows = $row_num_result->fetchColumn();
if($num_rows > 0) {
while($row = $result->fetch(PDO::FETCH_ASSOC)) {
echo $row['name'];
}
}
I find this method that requires me to write a separate and nearly identical sql statement to be redundant and a serious pain when using prepared statements. I can understand how this approach might be acceptable when using a short SQL statement with a basic query, but not in the case of a prepared statement.
1. Is there any other way I can use the fetchColumn() approach
without having to rewrite what is almost exactly the same code?
2. Where can I find an official list of which databases
rowCount() supports when using a SELECT statement? And since it is
working on the database I am currently using, can I assume it is safe
to use(assuming I am not updating my database anytime soon)?
If you don't want to use rowCount I'm think you should two query, or you can use fetchAll and count(fetchAll) for rowCount
the second way, Use SELECT *,COUNT(*) ...
I am attempting to get the following PDO statement to work and running into issues. When I am trying to get the number of rows, I keep getting 0, yet I know there should be 1 row. When I ran it as a mysqli statement( before trying to change it to PDO) it worked perfectly.
Here is the code:
require_once ('pdo.php');
$isbn = $_POST['isbn'];
// check to see if the isbn is a "problem" isbn or not
$problem = $conn->prepare("select isbn, note from problem where isbn = :isbn");
$problem->bindParam(":isbn", $isbn);
$problem->execute();
print_r($problem);
$num_rows = $problem->rowCount();
print_r($num_rows); die;
EDIT: Here is pdo.php:
<?php
function db_connect()
{
$db = new PDO("mysql:host=localhost; db=bookcell_BCOS_final", "xxxxx", "xxxxx");
return($db);
}
?>
I know that my connection works, but I get 0 for $num_rows. What mistakes am I making here?
Besides a little quirk and a optimalisation your code looks fine to me. The posted value isbn could be the reasong that you are getting no data:
$problem = $conn->prepare("select isbn, note from problem where isbn = :isbn");
$problem->bindParam(":isbn", $_POST['isbn'], PDO::PARAM_STR); // <-- thats what parameter binding is for
$problem->execute();
print_r($problem);
$num_rows = $problem->rowCount(); // <-- gives the number of rows, not columnCOunt
print_r($num_rows); die;
The Syntax for $num_rows = $problem->columnCount(); is totally correct. You may try,
$problem->execute(array("isbn" => $isbn));
instead of bindParam.
for getting the no. of rows, you need to use pdo::rowCount() -- manual here
In PDO to verfiy if your execute statement did work, check the return value (bool):
$success = $problem->execute();
if (!$success) {
$arr = $problem->errorInfo();
print_r($arr);
}
Also you might be looking for rowCount() instead of columnCount() but I think the error handling is your furthermost issue.
Additionally you can make PDO throw an exception each time an error appears, compare:
Switching from PHP's mysql extension to PDO. Extend class to reduce lines of code
How do I raise PDOException?
Depending on the database driver and the mode it's running, PDO may not be able to give you a row count. Look carefully at the documentation for PDOStatement::rowCount():
If the last SQL statement executed by the associated PDOStatement was a SELECT statement, some databases may return the number of rows returned by that statement. However, this behaviour is not guaranteed for all databases and should not be relied on for portable applications.
This is because in many cases the database uses a cursor rather than fetching the full results and buffering them (which is how the old mysql_* functions behave). In this case the database doesn't know how many rows there are until you have looked at all the rows. Think of a cursor as something like a filesystem pointer--you can't know the filesize until you seek to the end of the file.
There are a lot of questions on this and I have done a lot of research. However, I am still wondering if I am doing this right.
Here is my statement (I've simplified it):
try {
$stmt = $pdc_db->prepare("SELECT * FROM table WHERE color = :color");
$stmt->bindValue(':color', $selected_color);
$stmt->execute();
$color_query = $stmt->fetchAll();
} catch(PDOException $e) { catchMySQLerror($e->getMessage()); }
Now, I am using the following to see if this has returned any results:
if (count($color_query) > 0) {
This works, HOWEVER... the SELECT statement will only return one result. So now to access stuff in the results, I am using $color_query[0][colorname]. I know this is because I am using fetchAll(), but I really want to be using fetch()
But if I just use fetch(), I am losing the ability to do a count(), which is pretty simple to me. I know I can do a separate query and check the results of SELECT COUNT(*), but that seems like more work (setting two separate queries up for each)
There must be a way, using PDO in PHP with mySQL, to check if fetch() has returned a result?
$stmt->rowCount() after the execute(), but doesn't work with all databases... try it with MySQL and see what you get.
You can do it with fecth, fecth will return false if no results returns.
if ($row = $stmt->fetch()) {
//get the first row of the result.
//....
}