Do unbuffered queries for one request - php

I'm looking to do unbuffered queries only on some requests.
In MySQL I was doing this:
$req = mysql_unbuffered_query('SELECT * FROM forum_topics
ORDER BY (topic_id/topic_stick) DESC, topic_last_post DESC');
while($data = mysql_fetch_assoc($req)) {
// display results...
}
I looked at PHP doc, and according to it in pdo we must proceed this way to do queries unbuffered:
$pdo = new PDO("mysql:host=localhost;dbname=world", 'my_user', 'my_pass');
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
$uresult = $pdo->query("SELECT Name FROM City");
if ($uresult) {
while ($row = $uresult->fetch(PDO::FETCH_ASSOC)) {
echo $row['Name'] . PHP_EOL;
}
}
But is it possible to do it unbuffered only for the "forum_topics" table results without setting all pdo instance to unbuffered?

Re, this doesn't work, I obtain an error while using your method:
SQLSTATE[IM001]: Driver does not support this function: This driver doesn't support setting attributes
What's wrong?
Edit : I found the solution on php.net doc.
If you use this:
$sth->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
It doesn't work.
But if you set it in an array in prepare(), it works fine.
$sth = $pdo->prepare('SELECT * FROM my_table',
array(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => false));
I hope this will help people who haven't found a way for this problem.

You can set the attribute on the PDO connection:
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
then run this particular query which result needs to be unbuffered,
$uresult = $pdo->query("SELECT Name FROM City");
while ($row = $uresult->fetch(PDO::FETCH_ASSOC)) {
echo $row['Name'] . PHP_EOL;
}
and then set the attribute back
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, true);

The answers on here are all trying to use MYSQL_ATTR_USE_BUFFERED_QUERY on the statment. and MYSQL_ATTR_USE_BUFFERED_QUERY only operates on the entire connection, as you seem to have sussed out.
MYSQL_ATTR_USE_BUFFERED_QUERY also only works if you're using the mysqlnd library - which odds are good you are if you're using PHP 7 or higher.
Your original method of setting the connection as unbuffered was the correct and only actually functional method.
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
But is it possible to do it unbuffered only for the "forum_topics" table results without setting all pdo instance to unbuffered?
Not all instances are set to unbuffered, only that "instance" of that connection. You can either simply immediately turn buffering back on ala:
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, true);
The problem is that you can only iterate a single query per connection when in unbuffered mode, so you cannot run a second query until you have retrieved all data from the first set.
Normally you only want to use unbuffered queries for very large datasets. I would recommend to use a second PDO connection to the database specifically for unbuffered queries that you open only when you need to run an unbuffered query, aka:
$pdo2 = new PDO("mysql:host=localhost;dbname=world", 'my_user', 'my_pass');
$pdo2->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, true);

MySQL does not implement statement-level attribute setting.
Source:
Here is your error message:
https://github.com/php/php-src/blob/master/ext/pdo/pdo_stmt.c#L1630
pdo_raise_impl_error(stmt->dbh, stmt, "IM001", "This driver doesn't support setting attributes");
And the condition above is checked on the stmt->methods->set_attribute above.
The stmt->methods is defined above and the type is declared at:
https://github.com/php/php-src/blob/5d6e923d46a89fe9cd8fb6c3a6da675aa67197b4/ext/pdo/php_pdo_driver.h#L417
struct pdo_stmt_methods
The set_attribute parameter is the 10th struct entry.
Here is the MySQL PDO implementation. And the statements methods are defined here:
https://github.com/php/php-src/blob/2ba4fb126391c578d20d859c841c4d31cd14adef/ext/pdo_mysql/mysql_statement.c#L935
NULL, /* set_attr */
This shows that the MySQL PDO module does implement that feature.
Discussion:
I have reviewed other extensions. And only the Firebird PDO database module supports that feature.

As an workaround if my query is a SELECT, I don't call the fetchAll function.
$query = 'SELECT ...... ';
$arr = explode(' ', $query);
$query_type = strtolower($arr[0]);
if ($query_type == 'select') {
$query_response = $query_prepare->fetchAll(PDO::FETCH_ASSOC);
} else {
$query_response = '';
}
Also you must treat the exception when you accidentaly put a space at the begining of the query.
Hope this is helpful.

Related

Mysql query returns boolean after a stored procedure call [duplicate]

I am trying to run the following.
<?php
$db = mysqli_connect("localhost","user","pw") or die("Database error");
mysqli_select_db($db, "database");
$agtid = $_POST['level'];
$sql = sprintf("call agent_hier(%d)", $agtid);
$result = mysqli_query($db, $sql) or exit(mysqli_error($db));
if ($result) {
echo "<table border='1'>
<tr><th>id</th>
<th>name</th>
<th>parent_id</th>
<th>parent_name</th>
<th>level</th>
<th>email</th></tr>";
while ($row = mysqli_fetch_assoc($result))
{
$aid = $row["id"];
$sql2 = "SELECT * FROM members WHERE MEMNO = '$aid'";
$result2 = mysqli_query($db,$sql2) or exit(mysqli_error($db));
while ($newArray = mysqli_fetch_array($result2)) {
$fname = $newArray['FNAME'];
$lname = $newArray['LNAME'];
$mi = $newArray['MI'];
$address = $newArray['ADDRESS'];
$city = $newArray['CITY'];
$state = $newArray['STATE'];
$zip = $newArray['ZIP'];
$kdate = $newArray['KDATE'];
$date = abs(strtotime(date('m/d/Y')) - strtotime(date($kdate))) / (60 * 60 * 24);
}
echo sprintf("<tr><td>%s</td><td>%s</td><td>%s</td><td>%s</td><td>%s</td><td>%s</td></tr>",
$row["id"],$row["name"],
$row["parent_id"],$row["parent_name"],
$row["level"],$row["email"]);
}
echo "</table>";
}
mysqli_free_result($result);
mysqli_close($db);
?>
If I remove lines from:
$aid = $row["agent_id"];
to....
$date = abs(strtotime(date('m/d/Y')) - strtotime(date($kdate))) / (60 * 60 * 24);
}
everything will work fine. If not, I get the following error:
Commands out of sync; you can't run this command now
In researching, I think it could be due to multiple MySQLi queries run at the same time, in which using mysqli_multi_query but for all the samples and general data in the guide does not seem to be applicable.
Any ideas?
The MySQL client does not allow you to execute a new query where there are still rows to be fetched from an in-progress query. See Commands out of sync in the MySQL doc on common errors.
You can use mysqli_store_result() to pre-fetch all the rows from the outer query. That will buffer them in the MySQL client, so from the server's point of view your app has fetched the full result set. Then you can execute more queries even in a loop of fetching rows from the now-buffered outer result set.
Or you mysqli_result::fetch_all() which returns the full result set as a PHP array, and then you can loop over that array.
Calling stored procedures is a special case, because a stored procedure has the potential for returning multiple result sets, each of which may have its own set of rows. That's why the answer from #a1ex07 mentions using mysqli_multi_query() and looping until mysqli_next_result() has no more result sets. This is necessary to satisfy the MySQL protocol, even if in your case your stored procedure has a single result set.
PS: By the way, I see you are doing the nested queries because you have data representing a hierarchy. You might want to consider storing the data differently, so you can query it more easily. I did a presentation about this titled Models for Hierarchical Data with SQL and PHP. I also cover this topic in a chapter of my book SQL Antipatterns Volume 1: Avoiding the Pitfalls of Database Programming.
Here is how to implement mysqli_next_result() in CodeIgnitor 3.0.3:
On line 262 of system/database/drivers/mysqli/mysqli_driver.php change
protected function _execute($sql)
{
return $this->conn_id->query($this->_prep_query($sql));
}
to this
protected function _execute($sql)
{
$results = $this->conn_id->query($this->_prep_query($sql));
#mysqli_next_result($this->conn_id); // Fix 'command out of sync' error
return $results;
}
This has been an issue since 2.x. I just updated to 3.x and had to copy this hack over to the new version.
Simply,
You have to call mysqli_next_result($db) , after mysqli_free_result is called.
mysqli_free_result($result);
mysqli_next_result($db)
mysqli_close($db);
simply call this function :
$this->free_result();
function free_result() {
while (mysqli_more_results($this->conn) && mysqli_next_result($this->conn)) {
$dummyResult = mysqli_use_result($this->conn);
if ($dummyResult instanceof mysqli_result) {
mysqli_free_result($this->conn);
}
}
}
You have to close previous connection hold by Stored Procedure.
Instead of closing connection each time, you can simply use :
mysqli_next_result($conn);
For new mariaDb check in_predicate_conversion_threshold param. By default you can use up to 1000 attributes for "In" queries
If you are also making a few calls to stored procedures, and facing the aforementioned error, I have a solution for you, Mr. Wayne.
IMHO, Somewhere down the line, the CALL to Stored Procedure actually messes up the connection. So all you have to do is reset the database connection handle.
You could even have a function sitting in your config file doing just that for you, and all you do is call that function once before making any query or CALL to the Stored Procedure
My implementation is (just ignore the $app since I am working on silex framework it is there, YMMV)
function flushhandle() {
global $app;
global $db_host;
global $db_user;
global $db_pass;
global $db_name;
$app['mysqlio']->close();
$app['mysqlio'] = new mysqli($db_host, $db_user, $db_pass, $db_name);
return $app['mysqlio'];
}

Calling mssql stored procedure using pdo sqlsrv drivers with multi queries inside procedure

I'm facing the problem while calling stored procedure by using sqlsrv drivers in laravel. Inside procedure there are multiple queries with combination of select and insert. I unable to share code here as its privacy issue.
So can any one please share the code which can work with laravel to call mssql procedure with multiple result row set.
Thanks in advance!!
MS SQL Server supports stored procedures that can return more than one result set. With PHP and PDO you can retrieve these result sets with PDOStatement::nextRowset() method. It is important to note, that if you have output parameters in your stored procedure, you need to fetch all result sets to get the output values.
If Laravel do not support nextRowset(), you may try to get the underlying PDO instance using the getPdo() method on a connection instance and use this instance for statement execution:
<?php
...
$pdo = DB::connection()->getPdo();
try {
$sql = "{CALL usp_TestProcedure(?)}";
$param = 'ParamValue';
$stmt = $pdo->prepare($sql);
$stmt->bindParam(1, $param);
$stmt->execute();
do {
while ($row = $stmt->fetch(PDO::FETCH_ASSOC)) {
print_r($row, true);
echo '<br>';
}
} while ($stmt->nextRowset());
} catch( PDOException $e ) {
die( "Error executing query" );
}
...
?>

Unable to insert new data into MySQL database after deleting an entry using PHPMyAdmin

I managed to run the following code to insert into my table on first try. Then, I deleted that row in PHPMyAdmin to test my code further. I also noticed that it didn't get deleted on the 1st try. Only after few try. This might be due to I didn't set the $pdoHandle to NULL after I'm done with the query.
Then, unfortunately I couldn't insert new row on subsequent run. I even tried to change the input value and to avail I was unable to insert new row. The following are my PHP codes:
public function CreateNewCustomer($userId,$password,$name,$email)
{
$userId = filter_var($userId,FILTER_SANITIZE_STRING);
$password = filter_var($password,FILTER_SANITIZE_STRING);
$password = sha1($password);
$name = filter_var($name,FILTER_SANITIZE_STRING);
$email = filter_var($email,FILTER_SANITIZE_EMAIL);
do{
$customerId = hexdec(bin2hex(openssl_random_pseudo_bytes(4,$isStrong)));
echo $customerId;
$result = $this->connObject->exec("SELECT COUNT(id) FROM customer_tbl WHERE id=$customerId");
var_dump($result);
}while($result>0);
$statement = $this->connObject->prepare("INSERT INTO customer_tbl (id,name,email) VALUES ($customerId,:name,:email)");
$result = $statement->execute(array(':name'=>$name,':email'=>$email ));
var_dump($result);
$statement = $this->connObject->prepare("INSERT INTO login_tbl (username,password,customer_id) VALUES (:userName,PASSWORD(:password),$customerId)");
$result = $statement->execute(array(':userName'=>$userId,':password'=>$password ));
var_dump($result);
}
I used the following code to access the above method.
function Test($userName,$password,$name,$email)
{
try
{
$dbConnect = new DbConnect();
$pdoHandle = $dbConnect->Connect();
$userAccess = new UserAccess($pdoHandle);
$userAccess->CreateNewCustomer($userName,$password,$name,$email);
}
catch(PDOException $e)
{
$pdoHandle = null;
var_dump($e);
}
$pdoHandle = null;
}
Test('tester','password','TestX','test#example.com');
The var_dump of results is always false.
Is there any problem with my codes or is it something wrong with the database?
UPDATE/SOLUTION:
I just read through the PHP document on PDO::exec() and one of the user contributed notes mentioned that you can't use any SELECT statements (even thou the above only returns the count value) and any statements which might return a rows. The return value of PDO::exec() is the number of affected rows (integer), so the PDOStatement::closeCursor() can't be used to solve it. Even when I set the PDO::MYSQL_ATTR_USE_BUFFERED_QUERY=>true, it still doesn't work.
So, don't use PDO::exec() for any SELECT. I changed my code to PDO::query() instead as below,
do{
$customerId = hexdec(bin2hex(openssl_random_pseudo_bytes(4,$isStrong)));
$statement = $this->connObject->query("SELECT COUNT(id) FROM customer_tbl WHERE id=$customerId");
$statement->execute();
}while($statement->fetchColumn(0)>0);
Hope this would be helpful to anyone looking for a solution with similar problem and always remember to read the PHP document first including the user contributions.
Maybe not the answer but here are some things that you can do if you cannto see an obvious error:
If execute returns false, you can get more information about the error that happened by:
$arr = $statement->errorInfo();
print_r($arr);
or you can set different error reporting modes (e.g. throw an exception instead of the defaultsilent mode):
$dsn = 'mysql:dbname=testdb;host=127.0.0.1';
$user = 'dbuser';
$password = 'dbpass';
$dbh = new PDO($dsn, $user, $password);
$dbh->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
This should help you to find the "real" error.
As it turned out (see comments below question), in this case the real error was:
"Cannot execute queries while other unbuffered queries are active.
Consider using PDOStatement::fetchAll(). Alternatively, if your code
is only ever going to run against mysql, you may enable query
buffering by setting the PDO::MYSQL_ATTR_USE_BUFFERED_QUERY"
In this case you have 2 options:
you can set the option to use buffered queries
$dbh = new PDO(’mysql:host=localhost;dbname=test’, ‘root’, ” ,array(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => true))
or change your code and close an open cursor (may depend on the db driver you are using). You always should read the documentation which covers a lot of default problems.
Hope this helps.
I'm assuming the method is inside the UserAccess class and the connection you pass in is set to the local $this->connObject.
I suspect after you deleted the record, $customerId is being set to null in your interesting do-while loop with the select statement. If the id column in the DB is a non-null primary key field and you try to insert an explicit null it will fail.
Also, no need to keep setting your DB connection to null... this isn't C and connections aren't persistent (unless you explicitly declare them as such).

Outputting a list of MySQL table values in PHP/HTML

I have a MySQL list with a few categories and a lot of rows of data. I want to simply output that in PHP/HTML. How would I do that?
<?php
$query = "SELECT * FROM TABLE";
$res = mysql_query($query,$connection);
while($row = mysql_fetch_array($res)) {
print_r($row);
}
?>
To expand on what was already said: http://www.anyexample.com/programming/php/php_mysql_example__display_table_as_html.xml
This will produce a nice html table out of a query.
Please note that the mysql_* functions, as offered by some answers to this question, have been deprecated for quite some time. It is recommended to use either the mysqli_* functions (MySql Improved, which uses a newer underlying library for accessing mysql), or the PDO (PHP Data Objects, an object-oriented interface for connecting to various databases).
For example:
// Create a new PDO connection to localhost.
$dbh = new PDO("mysql:localhost;dbname=testdb", $user, $password);
// Create a PDO Statement object based on a query.
// PDO::FETCH_ASSOC tells PDO to output the data as an associative array.
$stmt = $dbh->query("SELECT * FROM Table", PDO::FETCH_ASSOC);
// Iterate over the statement, using a simple foreach
foreach($stmt as $row) {
echo $row['column1'].' and '.$row['column2'];
}

Why is mysqli giving a "Commands out of sync" error?

I am trying to run the following.
<?php
$db = mysqli_connect("localhost","user","pw") or die("Database error");
mysqli_select_db($db, "database");
$agtid = $_POST['level'];
$sql = sprintf("call agent_hier(%d)", $agtid);
$result = mysqli_query($db, $sql) or exit(mysqli_error($db));
if ($result) {
echo "<table border='1'>
<tr><th>id</th>
<th>name</th>
<th>parent_id</th>
<th>parent_name</th>
<th>level</th>
<th>email</th></tr>";
while ($row = mysqli_fetch_assoc($result))
{
$aid = $row["id"];
$sql2 = "SELECT * FROM members WHERE MEMNO = '$aid'";
$result2 = mysqli_query($db,$sql2) or exit(mysqli_error($db));
while ($newArray = mysqli_fetch_array($result2)) {
$fname = $newArray['FNAME'];
$lname = $newArray['LNAME'];
$mi = $newArray['MI'];
$address = $newArray['ADDRESS'];
$city = $newArray['CITY'];
$state = $newArray['STATE'];
$zip = $newArray['ZIP'];
$kdate = $newArray['KDATE'];
$date = abs(strtotime(date('m/d/Y')) - strtotime(date($kdate))) / (60 * 60 * 24);
}
echo sprintf("<tr><td>%s</td><td>%s</td><td>%s</td><td>%s</td><td>%s</td><td>%s</td></tr>",
$row["id"],$row["name"],
$row["parent_id"],$row["parent_name"],
$row["level"],$row["email"]);
}
echo "</table>";
}
mysqli_free_result($result);
mysqli_close($db);
?>
If I remove lines from:
$aid = $row["agent_id"];
to....
$date = abs(strtotime(date('m/d/Y')) - strtotime(date($kdate))) / (60 * 60 * 24);
}
everything will work fine. If not, I get the following error:
Commands out of sync; you can't run this command now
In researching, I think it could be due to multiple MySQLi queries run at the same time, in which using mysqli_multi_query but for all the samples and general data in the guide does not seem to be applicable.
Any ideas?
The MySQL client does not allow you to execute a new query where there are still rows to be fetched from an in-progress query. See Commands out of sync in the MySQL doc on common errors.
You can use mysqli_store_result() to pre-fetch all the rows from the outer query. That will buffer them in the MySQL client, so from the server's point of view your app has fetched the full result set. Then you can execute more queries even in a loop of fetching rows from the now-buffered outer result set.
Or you mysqli_result::fetch_all() which returns the full result set as a PHP array, and then you can loop over that array.
Calling stored procedures is a special case, because a stored procedure has the potential for returning multiple result sets, each of which may have its own set of rows. That's why the answer from #a1ex07 mentions using mysqli_multi_query() and looping until mysqli_next_result() has no more result sets. This is necessary to satisfy the MySQL protocol, even if in your case your stored procedure has a single result set.
PS: By the way, I see you are doing the nested queries because you have data representing a hierarchy. You might want to consider storing the data differently, so you can query it more easily. I did a presentation about this titled Models for Hierarchical Data with SQL and PHP. I also cover this topic in a chapter of my book SQL Antipatterns Volume 1: Avoiding the Pitfalls of Database Programming.
Here is how to implement mysqli_next_result() in CodeIgnitor 3.0.3:
On line 262 of system/database/drivers/mysqli/mysqli_driver.php change
protected function _execute($sql)
{
return $this->conn_id->query($this->_prep_query($sql));
}
to this
protected function _execute($sql)
{
$results = $this->conn_id->query($this->_prep_query($sql));
#mysqli_next_result($this->conn_id); // Fix 'command out of sync' error
return $results;
}
This has been an issue since 2.x. I just updated to 3.x and had to copy this hack over to the new version.
Simply,
You have to call mysqli_next_result($db) , after mysqli_free_result is called.
mysqli_free_result($result);
mysqli_next_result($db)
mysqli_close($db);
simply call this function :
$this->free_result();
function free_result() {
while (mysqli_more_results($this->conn) && mysqli_next_result($this->conn)) {
$dummyResult = mysqli_use_result($this->conn);
if ($dummyResult instanceof mysqli_result) {
mysqli_free_result($this->conn);
}
}
}
You have to close previous connection hold by Stored Procedure.
Instead of closing connection each time, you can simply use :
mysqli_next_result($conn);
For new mariaDb check in_predicate_conversion_threshold param. By default you can use up to 1000 attributes for "In" queries
If you are also making a few calls to stored procedures, and facing the aforementioned error, I have a solution for you, Mr. Wayne.
IMHO, Somewhere down the line, the CALL to Stored Procedure actually messes up the connection. So all you have to do is reset the database connection handle.
You could even have a function sitting in your config file doing just that for you, and all you do is call that function once before making any query or CALL to the Stored Procedure
My implementation is (just ignore the $app since I am working on silex framework it is there, YMMV)
function flushhandle() {
global $app;
global $db_host;
global $db_user;
global $db_pass;
global $db_name;
$app['mysqlio']->close();
$app['mysqlio'] = new mysqli($db_host, $db_user, $db_pass, $db_name);
return $app['mysqlio'];
}

Categories