Lately I've been testing my PHP framework's database wrapper class which is based on PHP Data Objects. I've successfully passed all the tests with Oracle database and started to do the tests with MySQL when I came across to a bug which seems like an ACID nightmare.
In short my database driver wrapper class does the following:
1) It establishes a persistent database connection with the following attributes
self::$connection = new PDO(
$dsn
,DATABASE_USERNAME
,DATABASE_PASSWORD
,[
PDO::ATTR_AUTOCOMMIT => FALSE // Do not autocommit every single statement
,PDO::ATTR_CASE => PDO::CASE_LOWER // Force column names to lower case
,PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC // Return result set as an array indexed by column name
,PDO::ATTR_EMULATE_PREPARES => (DATABASE_DRIVER == 'mysql') // Allow emulation of prepared statements only for MySQL
,PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION // Throw an exception and rollback transaction on error
,PDO::ATTR_ORACLE_NULLS => PDO::NULL_EMPTY_STRING // Convert emtpy strings to NULL
,PDO::ATTR_PERSISTENT => TRUE // Use persistent connection
]
);
2) Wrapper class has a execute() method which is a backbone for running various SQL statements. When executing SQL statement with execute() method it checks if transaction is active using PDO::inTransaction() method. If not, it begins the transaction. Here is how this method looks like (skipping all boring parts):
public static function execute($sql, $bind_values = [], $limit = -1, $offset = 0) {
...
if (!self::$connection->inTransaction()) {
self::$connection->beginTransaction();
}
...
}
3) So far so good. But let's look at the following example which calls DELETE statement followed by a SELECT statement against the same table with the very same where conditions:
database::execute('DELETE FROM dms_test WHERE id = 5');
$data = database::execute('SELECT FROM dms_test WHERE id = 5');
4) Everyone would expect that SELECT statement returns an empty result-set since the previous DELETE statement just wiped out all the data within the same transaction.
5) But as crazy as it may sound, the SELECT statement returns non-empty result-set as though as DELETE statement would never have been issued.
6) It's interesting that the very same example works as intended within Oracle database.
Any ideas what is wrong with MySQL?
Have any of you had similar problems?
I was able to solve the problem by completely removing PDO's transaction mechanism and replacing it with my own. Apparently using PDO transaction mechanism with MySQL RDBMS and auto-commit mode disabled may cause unpredictable behavior. I don't know if this is a PDO or MySQL bug.
If you want to implement out-of-the-box transactional access to database using PDO do not use PDO's built-in PDO::beginTransaction(), PDO::commit() and PDO::rollback() methods.
Instead I suggest you to use the following approach:
When establishing connection, use attribute PDO::ATTR_AUTOCOMMIT =>
FALSE
Keep track of in-transaction state by declaring your own variable
for this purpose (e.g. $in_transaction)
Register shutdown function that calls native database ROLLBACK at
the end of the request (if in-transaction state)
Using this approach I was able to overcome the above mentioned bug.
auto commit is set to false, So none of the changes will be saved unless you commit them with transation, commit or rollback
PDO::ATTR_AUTOCOMMIT => FALSE
Related
I have developed a project under which several sql query have been used. Now I want to monitor some query in order increase security. So I want every query to be passed through a function first. As there are too many queries so I can not go back and edit every file and query. Is there a way that I can trap into queries before they are sent to mysql server?
There are four ways to accomplish this depending on what you are using, the last being the much more reliable.
The General Query Log
MySQL provides a mechanism to log just about everything that the mysqld process is doing, via the general query log. As you described in your question you probably do not have persistent connections, so you will need to either:
Enable the MySQL general query log when the mysqld process is started, with the --log[=file_name]
Set a global/session variable with SET GLOBAL general_log = 'ON'.
Fore more information about the general query log, see the MySQL 5.1 reference manual.
Using sed (or manually!)
This technique involves creating a a new function, and renaming all of the mysqli_* function calls to call another function.
Presuming your newly created function is named proxy_query(), you can use sed to traverse through all files and change them automatically:
sed i '.bck' 's/mysqli_query/proxy_query/'
The -i paramater specifies that the file should be edited in place, and that a copy should be made of the original file and have a .bck extension appended.
The runkit extension
I must admit that I'm being naive here, and that I haven't used this particular extension before - but it is possible to rename functions with this PECL extension.
The requirements for this extension can be found here, and note that it is not bundled with PHP.
As with above, you can create a proxy function where all calls will go through. Let's assume it's also called proxy_query. Usage would go something like this:
// rename the function (a very bad idea, really!)
runkit_function_renam('mysqli_connect', 'proxy_super');
function mysqli_query($query, $resultmode = MYSQLI_STORE_RESULT)
{
// do something with the SQL in $query
// .. and call mysqli_query, now proxy_super
return proxy_super($query, $resultmode);
}
I have to note here that this method is highly discouraged. You shouldn't ever need to set default PHP functions.
Using Pdo/OO-mysqli
This is the simplest technique, and probably the most reliable as well. If you're using Pdo already, you can simply extend the \Pdo class. A similar approach could be used with MySQL Improved(mysqli):
class MyPdo extends \Pdo
{
public function query($query [, ... ])
{
// do something with $query
return parent::query($query [, ... ]);
}
}
Also note here, that this will only work if you are using Pdo, and if you are able to change the instantiation of the Pdo object, to overwrite it to your own class: MyPdo.
For more information about the \Pdo class, and it's children, see the manual.
If you want to monitor incoming queries using SQL profiler can be an excellent way to gather information on what's going on inside SQL without passing it all through a single function or procedure.
I am used to mysql database access using the procedural mysql method. I am a beginner - intermediate programmer.
I am trying to learn the PDO api, however all resources which discuss using PDO show the connection string, username and password.
e.g.
<?php
try {
$db_conn = new PDO('mysql:host=localhost;dbname=databaseName','username', 'password');
}
catch (PDOException $e) {
echo 'Could not connect to database';
}
$sql = 'SELECT * FROM Products';
$stmt = $db_conn->prepare($sql);
...
...
...
?>
What I want, and think would be better programming is to put my PDO connection into a new file. then where I want to run an SQL query, I require_once('PDO.php') or similar.
The problem I have with this is as follows:
How do I close the connection? Simply $db_conn = null; ??
Should I close the connection after each query is run, then re-open the connection?
Should I close the connection or is it automatically destroyed when the user closes the browser?
I am working from a book called PHP Master: Writing Cutting Edge Code. http://www.sitepoint.com/books/phppro1/ and this has completely omitted any reference to closing the connection / destroying the object after it has been used.
Furthermore, I have looked at online tutorials, and they all connect to the database using PDO inline as opposed to having a separate database connector. This I am not happy with for many reasons:
I have to type username & password to connect every time.
If I get a developer to take a look at code / write some code, they will all have access to the database.
If I change the DB username & Password, then each file which connects to the database will need to be updated.
Could anybody recommend a better resource? Could anybody advise on what is the best practice way to do this?
Many thanks
Your question about how to store the database name, username and password have nothing to do with the capabilities of PDO. This is an implementation choice. The way you use to work with procedural functions can also be applied to PDO, the difference is that with PDO you work with objects instead.
So for simplicity, store the PDO creation of an object, either in a function or class, in which you can create the PDO instance anytime, e.g.
function createPDO($cfg) {
try {
return new PDO("mysql:host=".$cfg['host'].",port:".($cfg['port']).";dbname=".($cfg['name']).";",$cfg['username'], $cfg['password']);
} catch(PDOException $e) {
// handle exceptions accordingly
}
}
You can centralise these in whatever PHP file you like to include, just like you were used with the procedural functions.
You have two choices, either put all the relevant database information inside the createPDO, or use something like a config ($cfg) variable to store all this information.
$config = array();
$config['db'] = array(
'host' => 'localhost',
'name' => 'databse',
'username' => 'userx',
'password' => 'passy'
/* .. etc */
)
Using the createPDO function would be as followed
$db_conn = createPDO($config['db']);
For connections closing, each connection made to the database automatically disconnects after PHP exits its execution. You can however, close the connection if you wish, by setting the variable of the PDO object you assigned it to, in this example (and in yours) $db_conn to null
$db_conn = null; // connection closed.
The PDO has a manual http://php.net/manual/en/book.pdo.php here, which is a good start getting to know PDO a bit better.
You do not close the connection after a query, you simply leave it open for the next query. When PHP exists and your page is shown, the connection will be closed automatic.
It is a good idea to put the db stuff in a separate file and include that.
Even better, put all your db stuff in a class in use that.
Have a look at the pdo php page. Although not the best examples, they should get you started.
I am using Zend Framework for my PHP developments and here is a small function I used to execute a query. This is not about an error. The code and everything works fine. But I want to know some concept behind this.
/**
* Get dataset by executing sql statement
*
* #param string $sql - SQL Statement to be executed
*
* #return bool
*/
public function executeQuery($sql)
{
$this->sqlStatement = $sql;
if ($this->isDebug)
{
echo $sql;
exit;
}
$objSQL = $this->objDB->getAdapter()->prepare($sql);
try
{
return $objSQL->execute();
}
catch(Exception $error)
{
$this->logMessage($error->getMessage() . " SQL : " .$sql);
return false;
}
return false;
}
Bellow are unclear areas for me.
How Zend_Db_Table_Abstract Maintain database connections?
Is it creating new connection all the time when I call this function or Does it have some connection pooling?
I didn't write any coding to open or close database connection. So will zend framework automatically close connections?
If this open and close connection works all the time if I execute this function, Is there any performance issue?
Thank you and appreciate your suggestions and opinion on this.
Creating Connection
Creating an instance of an Adapter class does not immediately connect to the RDBMS server. The Adapter saves the connection parameters, and makes the actual connection on demand, the first time you need to execute a query. This ensures that creating an Adapter object is quick and inexpensive. You can create an instance of an Adapter even if you are not certain that you need to run any database queries during the current request your application is serving.
If you need to force the Adapter to connect to the RDBMS, use the getConnection() method. This method returns an object for the connection as represented by the respective PHP database extension. For example, if you use any of the Adapter classes for PDO drivers, then getConnection() returns the PDO object, after initiating it as a live connection to the specific database.
It can be useful to force the connection if you want to catch any exceptions it throws as a result of invalid account credentials, or other failure to connect to the RDBMS server. These exceptions are not thrown until the connection is made, so it can help simplify your application code if you handle the exceptions in one place, instead of at the time of the first query against the database.
Additionally, an adapter can get serialized to store it, for example, in a session variable. This can be very useful not only for the adapter itself, but for other objects that aggregate it, like a Zend_Db_Select object. By default, adapters are allowed to be serialized, if you don't want it, you should consider passing the Zend_Db::ALLOW_SERIALIZATION option with FALSE, see the example above. To respect lazy connections principle, the adapter won't reconnect itself after being unserialized. You must then call getConnection() yourself. You can make the adapter auto-reconnect by passing the Zend_Db::AUTO_RECONNECT_ON_UNSERIALIZE with TRUE as an adapter option.
Closing a Connection
Normally it is not necessary to close a database connection. PHP automatically cleans up all resources and the end of a request. Database extensions are designed to close the connection as the reference to the resource object is cleaned up.
However, if you have a long-duration PHP script that initiates many database connections, you might need to close the connection, to avoid exhausting the capacity of your RDBMS server. You can use the Adapter's closeConnection() method to explicitly close the underlying database connection.
Since release 1.7.2, you could check you are currently connected to the RDBMS server with the method isConnected(). This means that a connection resource has been initiated and wasn't closed. This function is not currently able to test for example a server side closing of the connection. This is internally use to close the connection. It allow you to close the connection multiple times without errors. It was already the case before 1.7.2 for PDO adapters but not for the others.
More information
I'm using PDO to access two SQLite 3 databases in PHP. I want to switch the database files during a query by renaming them but I can't do that while the files are open as it gives an error that the file is being used by another process. I've tried turning off persistent connections and setting the handles to null but neither work.
Is there really no way to close a PDO handle and release the lock on the database file?
I believe unset($var) does that, I use it on my pdo sqlite project and it works like I want it to :)
Set all references to the handle to null (or to anything except the PDO object, really) and the runtime will destruct the object, which will close the connection.
$db = new PDO('...');
// Do some stuff
$db = null;
// Assuming this was the last reference to that PDO
// object, the runtime will destroy the object and
// its connection.
I'm using Zend Framework 1.7.2, MySQL and the MySQLi PDO adapter. I would like to call multiple stored procedures during a given action. I've found that on Windows there is a problem calling multiple stored procedures. If you try it you get the following error message:
SQLSTATE[HY000]: General error: 2014
Cannot execute queries while other
unbuffered queries are active.
Consider using
PDOStatement::fetchAll().
Alternatively, if your code is only
ever going to run against mysql, you
may enable query buffering by setting
the PDO::MYSQL_ATTR_USE_BUFFERED_QUERY
attribute.
I found that to work around this issue I could just close the connection to the database after each call to a stored procedure:
if (strtoupper(substr(PHP_OS, 0, 3)) === 'WIN') {
//If on windows close the connection
$db->closeConnection();
}
This has worked well for me, however, now I want to call multiple stored procedures wrapped in a transaction. Of course, closing the connection isn't an option in this situation, since it causes a rollback of the open transaction. Any ideas, how to fix this problem and/or work around the issue.
More info about the work around
Bug report about the problem
I has same errors when called queries like this(variables to use in next query)
$db->query("SET #curr = 0.0;");
To fix this I've changed my config file to
'database' => array(
'adapter' => 'mysqli',
This pattern of preparing, executing and then closing each $sql statement that calls a stored procedure does work.
public function processTeams($leagueid,$raceid,$gender)
{
$db = Zend_Db_Table::getDefaultAdapter();
$sql = $db->prepare(sprintf("CALL addScoringTeams(%d,%d,'%s')",$leagueid,$raceid,$gender));
$sql->execute();
$sql->closeCursor();
$this->logger->info(sprintf("CALL addScoringTeams(%d,%d,'%s')",$leagueid,$raceid,$gender));
$sql1 = $db->prepare(sprintf("CALL updateScoringTeamTotals(%d)",$raceid));
$sql1->execute();
$sql1->closeCursor();
$this->logger->info(sprintf("CALL updateScoringTeamTotals(%d)",$raceid));
$sql2 = $db->prepare(sprintf("CALL updateScoringTeamClasses(%d,'%s')",$raceid,$gender));
$sql2->execute();
$sql2->closeCursor();
$this->logger->info(sprintf("CALL updateScoringTeamClasses(%d,'%s')",$raceid,$gender));
}
You can use prepare statement . No need to change driver
$sql = "CALL procedure()";
$stmt = $this->db->createStatement();
$stmt->prepare($sql);
$result = $stmt->execute();