SQL connection error from PHP after many operations - php

I'm currently looping to create a MBTiles map, and add information to my database each time.
Here's how I configured my connection and execute actions during the loop:
if ($pdo_mbtiles == null) {
echo "Opening new database connection".PHP_EOL;
$pdo_mbtiles = new PDO('sqlite:'.$filename,
'',
'',
array(
PDO::ATTR_PERSISTENT => true
)
);
$pdo_mbtiles->setAttribute(PDO::ATTR_DEFAULT_FETCH_MODE, PDO::FETCH_ASSOC);
$pdo_mbtiles->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
}
$q = $pdo_mbtiles->prepare("INSERT INTO tiles (zoom_level, tile_column, tile_row,tile_data) VALUES (:zoom_level, :tile_column, :tile_rowTMS, :tile_data)");
$q->bindParam(':zoom_level', $zoom_level);
$q->bindParam(':tile_column', $tile_column);
$q->bindParam(':tile_rowTMS', $tile_rowTMS);
$q->bindParam(':tile_data', $tile_data, PDO::PARAM_LOB);
$q->execute();
After 1018 times looping (this number doesn't change no matter how many times I try), I get this error message:
SQLSTATE[HY000]: General error: 14 unable to open database file
I checked the solution written here:
How to prevent SQLITE SQLSTATE[HY000] [14]?
but the echoed message only appears at the first time of the loop, so I assume the PDO connection isn't closed.
I didn't find other documentation relevant to this error code.
What could possibly go wrong here?
I tried to move the prepare and bind commands in a condition as follows. The exception isn't raised, but only the first tile is saved (or every tile is saved on top of the first one, not sure):
if ($pdo_mbtiles == null) {
echo "Opening new database connection".PHP_EOL;
$pdo_mbtiles = new PDO('sqlite:'.$filename,
'',
'',
array(
PDO::ATTR_PERSISTENT => true
)
);
$pdo_mbtiles->setAttribute(PDO::ATTR_DEFAULT_FETCH_MODE, PDO::FETCH_ASSOC);
$pdo_mbtiles->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
}
if ($q == null) {
$q = $pdo_mbtiles->prepare("INSERT INTO tiles (zoom_level, tile_column, tile_row,tile_data) VALUES (:zoom_level, :tile_column, :tile_rowTMS, :tile_data)");
$q->bindParam(':zoom_level', $zoom_level);
$q->bindParam(':tile_column', $tile_column);
$q->bindParam(':tile_rowTMS', $tile_rowTMS);
$q->bindParam(':tile_data', $tile_data, PDO::PARAM_LOB);
}
$q->execute();
Here are the files during the generation:
And here after the exception is raised:
Also, when the exception is raised, I do a var_dump of my $pdo_mbtiles, and here's the result (exactly the same as when I do it with a success):
object(PDO)#116 (0) {
}
Edit: Still trying to solve this problem, I simplified the code to create the MBTiles file. No success yet, but here's a sample if anyone want's to reproduce the issue. You can download it from https://www.dropbox.com/s/33vqamc9tn4c3ux/sample_PHP_MBTiles_generation_bug.zip?dl=0

I suggest you to reuse the connection if it is open.
Create a property: private $pdo;
And check if it's null before creating a new object:
function opendatabase(){
try{
if($this->pdo==null){
$this->pdo =new PDO("sqlite:database/database.db","","",array(
PDO::ATTR_PERSISTENT => true
));
}
return $this->pdo;
}catch(PDOException $e){
logerror($e->getMessage(), "opendatabase");
print "Error in openhrsedb ".$e->getMessage();
}
}

The error message was misleading. After many hours of debugging, I found that it was completely unrelated to my database connection.
It's juste that I used fopen() to get tiles data, and didn't fclose() after registration, thus reaching the limit of 1024.
1024 is because I used six require or require_once statements, so 1018 tile requests + 6 require = 1024 opened connections.

Related

Run a database dump (Restore) from an sql file in PHP + PostgreSql

I'm trying to create a system where a user can create a database and have it populated from a UI, pretty much similar to the installation of most CMS's. I can programatically create the database but having the tables created and populated is where I'm facing a challenge.
I have a template sql file with these tables and data which is occasionally updated by dumping and this is what I'm doing:
try {
$dbTablesAndData = file_get_contents('path/to/template.sql');
$db = setDBConnection($clientDb);
$query = $db->prepare("$dbTablesAndData");
$query->execute( array() );
} catch (PDOException $e) {
$app->response()->setStatus(401);
$app->response()->headers->set('Content-Type', 'application/json');
echo json_encode(array('response' => 'error', 'data' => $e->getMessage() ));
}
When this runs I get an error: cannot insert multiple commands into prepared statement
So my question is, Is there a way to execute all the queries in this file successfully?
I read of a kind of a related solution from this question but it will not work for me because it proposes splitting the statements by the semi colons:
$statements = explode(';', $script);
foreach($statements as $sql){
if($sql === '') { continue; }
$query = $pdo->prepare($sql);
$query->execute();
if($query->errorCode() !== '00000'){ die("ERROR: SQL error code: ".$query->errorCode()."\n"); }
}
But my sql file contains cases where the semi colons should not be split like declaration of variables in creation of functions etc. Is there any way I can go about this?
Found a solution to this. Apparently it'S only with PDO where you can't run multiple commands. So something like pg_query() would suffice rather than prepare()

No true result return in insert query

This is the class I have created which I am using for the queries:
<?php
mysqli_report(MYSQLI_REPORT_INDEX | MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);
class DBConnect {
private $dbcon;
private $paramquery;
private $result;
public function __construct() {
try {
$this->dbcon = mysqli_init();
mysqli_real_connect($this->dbcon, '127.0.0.1', '', '', '', 3306, '', MYSQLI_CLIENT_COMPRESS);
$this->paramquery = $this->dbcon->stmt_init();
} catch (mysqli_sql_exception $e) {
exit('Database Connection Failed');
}
}
public function dbquery($querysql, $querydata) {
try {
mysqli_ping($this->dbcon);
$this->paramquery->prepare($querysql);
array_walk($querydata, function(&$escval){$escval = mysqli_real_escape_string($this->dbcon, $escval);}); //Problem
call_user_func_array(array($this->paramquery, 'bind_param'), $querydata); //Problem
$this->paramquery->execute();
} catch (mysqli_sql_exception $e) {
exit('Database Query Failed');
}
$this->result = $this->paramquery->get_result(); // problem
if ($this->result) {
$drs = $this->result->fetch_array();
$this->result->free_result();
return $drs;
}
}
public function __destruct() {
if (($this->dbcon !== null) && ($this->paramquery !== null) && ($this->result !== null)) {
$this->paramquery->close();
$this->dbcon->close();
}
unset($this->result);
unset($this->paramquery);
unset($this->dbcon);
}
}
?>
The index.php file code is this:
<?php
require_once('connection.php');
$DBX = new DBConnect();
$DBX->dbquery('INSERT INTO `xathx_key` (`license`, `client`, `server`, `uniquex`) VALUES (?, ?, ?, ?)', array('ssss', '1', '3', '5', '7'));
var_dump($DBX);
unset($DBX)
?>
I am trying to do an INSERT query in this instance. And I want to get a success result or flag when the query is executed successfully. But in the var_dump of the object I get some irrelevant data and if I use echo I get an error that the object cannot be converted to a string. I just want to get a 0 for query execution failure, corruption or problem and a 1 for completion, success, ok status. When am I going wrong in the code?
EDIT: Can you guys just tell me what are the things that are wrong with this simple script? The main goal of this script is to connect to mysql server and execute all possible queries as fast as possible, as securely as possible.
Full Project Source: https://github.com/FSMySQL/PHP-FSMySQL
The main goal of this script is to connect to mysql server and execute all possible queries as fast as possible, as securely as possible.
The goal is a good one but the implementation could benefit from many improvements.
Disclaimer: there will be a lot of links to my own site because I am helping people with PHP for 20+ years and got an obsession with writing articles about most common issues.
The concept of error reporting
First of all, you need to change the concept of error reporting. Your exit() approach would be a nightmare for a programmer, as error messages are a vital source of information when something goes wrong. A programmer should go at any lengths trying to get the error message in the full detail. In my article, PHP error reporting, I do explain how to make error reporting both programmer- and user-friendly. In short, you shouldn't catch errors on the spot, but have a single dedicated place to report errors and exceptions, and then it could be easily configured depends on the current server's role.
Although, as suggested in the other answer, you could use a global try-catch block in your index.php file to act as such a global error handler, I would prefer a dedicated error handler script, as explained in the article above. It will make your code better organized and make index.php less bloated.
Besides, your idea of having "a true result return in insert query" contradicts with your intention to use exceptions. When one is using exceptions, there is no point to verify the immediate function's result. In case of error it will just bubble up to the error handler or a catch block, so, it will never reach the condition. A quick example:
function test() {
throw new Exception("Test");
return false;
}
$result = test();
if ($result === false) {
echo "false";
}
The code execution in this example will never reach the condition, therefore making your functions return false on error useless. Which, in turn, makes returning true on success superfluous. Just return a meaningful result but don't use it as flag: simply write your code without any conditions, as though everything is fine. Remember that you have your error handling code elsewhere that will be magically invoked in case of error.
Connection
As explained in my other article, How to connect properly using mysqli, there is a slight chance to reveal connection credentials in case of a connection error. To avoid even a possibility but keep the programmer informed we have to throw a brand new exception, however keeping the error information - so the stack trace will begin from the throw line, and thus contain no sensitive information.
Also, the connection code lacks an essential part - setting the correct charset. Although in MySQL 8 the correct charset is set by default, it's better to make it explicit.
Also, making a mysqli statement a class variable is a grave mistake that will lead to race condition errors. The only state that your class should keep is that related to the connection but not a single class variable should be used for a statement.
So let's rewrite your constructor based on the code from the article above:
public function __construct()
{
mysqli_report(MYSQLI_REPORT_INDEX | MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);
try {
$this->dbcon = mysqli_init();
$this->dbcon->real_connect('127.0.0.1', '', '', '', 3306, '', MYSQLI_CLIENT_COMPRESS);
$this->dbcon->set_charset('utf8mb4');
} catch (\mysqli_sql_exception $e) {
throw new \mysqli_sql_exception($e->getMessage(), $e->getCode());
}
}
The dbquery function
The function is, frankly, weird. It's a strange mix between prepared statements and escaping. Let's rewrite it based on my mysqli helper function that actually utilizes mysqli prepared statements
public function dbquery($sql, $data = [], $types = "")
{
$this->dbcon->ping(); // not sure if it's necessary
$stmt = $this->dbcon->prepare($sql);
if ($data) {
$types = $types ?: str_repeat("s", count($data));
$stmt->bind_param($types, ...$data);
}
$stmt->execute();
return $stmt->get_result();
}
Now this function fulfills your desire for secure SQL queries
So finally we can rewrite your index.php
<?php
require_once('connection.php');
$DBX = new DBConnect();
$sql = 'INSERT INTO `xathx_key` (`license`, `client`, `server`, `uniquex`) VALUES (?, ?, ?, ?)';
$DBX->dbquery($sql, ['1', '3', '5', '7']);
Just as you learned above, there is no need for a "flag when the query is executed successfully". Just act as though there is always a success. In case of error it will appear without any conditions (an on a live site will be handled properly if you include an error handler script in your index).
In your DBConnect Class, you have try catch blocks. But your catch blocks are simply terminating the request using exit statement. Your Class should not be doing that.
Imagine you deploy this on production and for some reason the DB Connection Fails. In that case User will simply see a white screen with Message "Database Connection Failed" which would not look professional at all.
Instead your class should pass this information back to the index.php which called the method of this Class and let index.php handle the Error Message or Exception.
So I would make following changes to your code:
DBConnect Class should throw an Exception rather than terminating the execution of the program completely. Below is how the __contruct() should look.
public function __construct() {
try {
$this->dbcon = mysqli_init();
mysqli_real_connect($this->dbcon, '127.0.0.1', '', '', '', 3306, '', MYSQLI_CLIENT_COMPRESS);
$this->paramquery = $this->dbcon->stmt_init();
} catch (mysqli_sql_exception $e) {
//exit('Database Connection Failed'); Commented this out.
//Throw the Exception Here. This will then be passed to the calling code.
throw $e;
}
}
You will need to change the other methods accordingly.
In your index.php File, you should be looking to catch the above exception. So you should move your code in a Try Catch Block to catch that exception.
require_once('connection.php');
try {
$DBX = new DBConnect();
$DBX->dbquery('INSERT INTO `xathx_key` (`license`, `client`, `server`, `uniquex`) VALUES (?, ?, ?, ?)', array('ssss', '1', '3', '5', '7'));
} catch (Exception $e) {
$message = 'Caught exception: ', $e->getMessage() . "\n";
//Display this Message to User in an appropriate way.
//Write to Error Log
}
//var_dump($DBX);
//unset($DBX)
So this will catch the Exception in case the DB Connection Fails as well as when the Insert Query Fails. You can write the exception to the logs so that you can check them later and you can display any appropriate error message to user based on the exception caused.
You could read more on Exceptions in PHP Manual
You have a problem with
$this->result = $this->paramquery->get_result();
because mysqli_stmt::get_result returns a resultset for successful SELECT queries, or FALSE for other DML queries or on failure.
Other DML-queries are INSERT, UPDATE, DELETE. And that's exactly what you have in the example.
To resolve your problem you can modify the class by adding some extra-checks to $mysqli->errno:
$this->result = $this->paramquery->get_result();
if ($this->result) {
...
}
if ($this->paramquery->errno !== 0) { // we have some real error
exit('Database Query Failed');
}
// we have DML-query (INSERT, UPDATE, DELETE)
// and we can return number of affected rows (if it's necessary)
return $this->paramquery->affected_rows;
P.S. I agree with this comment and I think that your class should be used for educational purposes only because it has multiple serious flaws.

php PDO Database hande refuses to return results to queries after, several, succesful calls

On my site, we connect to several databases and we have a database handling class that stores the handles in a static array. 99.9% of the time, everything seems to work fine, but every now and then the handle fails silently with no error, and no results are returned. I've been able to recreate the error, then recreate the connection using the following syntax:
$db = new PDO("dblib:host=" . DB_HOST . ";dbname=" . DB_DEFAULT, DB_USER, DB_PASS);
Then, it returns results correctly. However, I would prefer to use the same handle for all databases, though, to preserve the extra overhead associated with creating a new connection to a database... or, at least, recreate the connection a minimum number of times.
Is there anyway to evaluate the PDO object to figure out why it's failing, so I can, hopefully, figure out what's going on?
EDIT:
I've been looking into this further, and profiled the DB while this error is occuring. PDO stops passing queries to the server after it hits certain queries. (In this case, it was an insert query that tests to see if a record already exists, and if it doesn't then it inserts it).
Make no mistake, this is a horrible and inelligant solution to my problem, but I copied and pasted (and universalized) what resolved it. Basically, when no results are returned, I select 'foo' from the database, if it comes back with no results, I rebuild the connection and attempt to execute the query again. I'm not accepting this as an answer, because it's pretty bad. But this code should better illustrate the problem I am, attempting, to fix.
/**
* Executes a line of sql with PDO.
*
* #param string $sql
* #param array $params
* #param bool
* #return array
*/
function execute($sql, $params = array(), $retry = false) {
//this call gets the database handle that is stored in a static array, so we can reuse the same connection
$db = db::getDB($this->_database);
if (get_class($db) != 'PDO') {
error('database handle wasn\'t created succesfully...');
}
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
try {
$query = $db->prepare($sql);
$query->execute($params);
} catch (Exception $ex) {
var_dump($ex);
return false;
}
$results = $query->fetchALL(PDO::FETCH_ASSOC);
if(($retry == false) && (!$results)){
$newResults = $this->testAndFixResultsIfDBGoneAway($sql, $params);
if($newResults != false){
$results = $newResults;
}
}
return $results;
}
function isConnected(){
$db = db::getDB($this->_database);
$query = $db->prepare("SELECT 'foo' AS test");
$query->execute();
$results = $query->fetchAll(PDO::FETCH_ASSOC);
//we aren't connected if we get no response
if($results == array()){
return false;
}
//if there is a response, return true
return true;
}
function testAndFixResultsIfDBGoneAway($sql, $params){
if($this->isConnected()){
return false;
}
//recreates the db connection in the db connection class
db::connect($this->_database);
$results = $this->execute($sql, $params, true);
return $results;
}
I've noticed that my connection goes away, I can restart it successfully, then the next query can fail, and I can repeat this process many times, until the offending queries are done, then everything starts working normally, again.
I REALLY want to know how queries kill my database connection, so I can stop it from happening so I don't have to do a check after every query that returns no results.
Setting the PDO error-handling mode to use exceptions should make the cause immediately apparent so you don't have to hunt around and use a lot of debugging code.
$db = new PDO("dblib:host=" . DB_HOST . ";dbname=" . DB_DEFAULT, DB_USER, DB_PASS, [PDO::ATTR_ERRMODE=>PDO::ERRMODE_EXCEPTION]);
However, you'll have to update your error-handling code [failed prepares, failed queries] to be aware of the Exceptions.

how to find record insert to mysql using commit()

Sorry for this beginners question and i'm not a PHP developer, but now i'm trying to learn it.
i want to add record in MySQL data base and i'm using transactions lock.
my code is as below.
$SqlQuery="INSERT INTO tab_photo VALUES('$PhotoID','$ProjectId','$Day','$barCode','$photoName','$PhotoXml')";
$waiting = true;
while($waiting) {
try {
// save border data
$stmt = $conn->prepare($SqlQuery);
$conn->beginTransaction();
$stmt->execute();
sleep(1);
$x=$conn->commit();
echo "x value-".$x;
echo "Success";
$waiting = false;
}
catch (PDOException $e){
echo "Failled :".$PhotoID."-".$PhotoID;
if(stripos($e->getMessage(), 'DATABASE IS LOCKED') !== false) {
// This should be specific to SQLite, sleep for 0.25 seconds
// and try again. We do have to commit the open transaction first though
$conn->commit();
usleep(250000);
} else {
$conn->rollBack();
throw $e;
}
}
}
in here as output it gives,
x value-1 Success
but actually this record doesn't add to the database.
My Questions:
Even the commit is successful(output 1) how does it not added to the database?
how can i check whether record is added to database? ( Is there any way to find it without write select statement?
As I understand, you expect that PDOException will be thrown when statement is failed to execute. But as I can see, exception is not thrown by default in such cases.
See how you can change that here
Suppose in your case you should have a code like this:
$conn = new PDO($connection_string);
$conn->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); // this will force PDO to throw exception when SQL statement fails instead of simply setting an error.
Suppose this will work fine for you.
Please note that you should not use
$SqlQuery="INSERT INTO tab_photo VALUES('$PhotoID','$ProjectId','$Day','$barCode','$photoName','$PhotoXml')";
Instead of that, you should use parameters binding:
$SqlQuery="INSERT INTO tab_photo VALUES(:PhotoID,:ProjectId,:Day,:barCode,:photoName,:PhotoXml)";
$stmt = $conn->prepare($SqlQuery);
$conn->beginTransaction();
$stmt->execute(array(':PhotoID' => $PhotoID, ':ProjectId' => $ProjectId, ....));
sleep(1);
See this for more details.

PHP PDO check for mysql error

I have this problem that drives me crazy. All i want is just check a query for an error, if so display error, otherwise run the query.
I have the following almost (since it runs the insert query twice) working
[..]
$dbdata = new mySQLAccessData();
$db = new PDO($dbdata->hostname,$dbdata->username,$dbdata->password);
$defaults = new Defaults();
[..]
if(!$db->exec($sql)){
echo($defaults->throwError('MySql error',implode(":",$db->errorInfo())));
}else{
$db->exec($sql);
$defaults->writeLog($table,$db->lastInsertId(),'add');
}
I tried numerous things (amongst others the try(){}catch(){} method) but nothing worked except for the code above. It shows the error the way i want, and only when an error occurs, but runs the exec() twice...
Can someone bail me out?
Why do you want to execute the query again in the else part? Usually you just try to run the query and if errors happen, react on them.
[..]
$dbdata = new mySQLAccessData();
$db = new PDO($dbdata->hostname,$dbdata->username,$dbdata->password);
$defaults = new Defaults();
[..]
if(!$db->exec($sql)){
echo($defaults->throwError('MySql error',implode(":",$db->errorInfo())));
}else{
$defaults->writeLog($table,$db->lastInsertId(),'add');
}
As far as I know, there is no option to "test" a query before actually executing it.
If you want to see an exception thrown when an error occurs, just set the PDO error-mode (see also: Connections and Connection management):
$db = new PDO($dbdata->hostname,$dbdata->username,$dbdata->password);
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
This for example will make your code throw exceptions automatically. Probably exactly what you're looking for.
The actual issue with your code is doing exec twice. You don't need to:
$success = $db->exec($sql);
if (!$success) {
echo $defaults->throwError('MySql error', implode(":", $db->errorInfo()));
} else {
# do not exec *again* here.
$defaults->writeLog($table, $db->lastInsertId(), 'add');
}

Categories