I am using Zend_Db to insert some data inside a transaction. My function starts a transaction and then calls another method that also attempts to start a transaction and of course fails(I am using MySQL5). So, the question is - how do I detect that transaction has already been started?
Here is a sample bit of code:
try {
Zend_Registry::get('database')->beginTransaction();
$totals = self::calculateTotals($Cart);
$PaymentInstrument = new PaymentInstrument;
$PaymentInstrument->create();
$PaymentInstrument->validate();
$PaymentInstrument->save();
Zend_Registry::get('database')->commit();
return true;
} catch(Zend_Exception $e) {
Bootstrap::$Log->err($e->getMessage());
Zend_Registry::get('database')->rollBack();
return false;
}
Inside PaymentInstrument::create there is another beginTransaction statement that produces the exception that says that transaction has already been started.
The framework has no way of knowing if you started a transaction. You can even use $db->query('START TRANSACTION') which the framework would not know about because it doesn't parse SQL statements you execute.
The point is that it's an application responsibility to track whether you've started a transaction or not. It's not something the framework can do.
I know some frameworks try to do it, and do cockamamie things like count how many times you've begun a transaction, only resolving it when you've done commit or rollback a matching number of times. But this is totally bogus because none of your functions can know if commit or rollback will actually do it, or if they're in another layer of nesting.
(Can you tell I've had this discussion a few times? :-)
Update 1: Propel is a PHP database access library that supports the concept of the "inner transaction" that doesn't commit when you tell it to. Beginning a transaction only increments a counter, and commit/rollback decrements the counter. Below is an excerpt from a mailing list thread where I describe a few scenarios where it fails.
Update 2: Doctrine DBAL also has this feature. They call it Transaction Nesting.
Like it or not, transactions are "global" and they do not obey object-oriented encapsulation.
Problem scenario #1
I call commit(), are my changes committed? If I'm running inside an "inner transaction" they are not. The code that manages the outer transaction could choose to roll back, and my changes would be discarded without my knowledge or control.
For example:
Model A: begin transaction
Model A: execute some changes
Model B: begin transaction (silent no-op)
Model B: execute some changes
Model B: commit (silent no-op)
Model A: rollback (discards both model A changes and model B changes)
Model B: WTF!? What happened to my changes?
Problem scenario #2
An inner transaction rolls back, it could discard legitimate changes made by an outer transaction. When control is returned to the outer code, it believes its transaction is still active and available to be committed. With your patch, they could call commit(), and since the transDepth is now 0, it would silently set $transDepth to -1 and return true, after not committing anything.
Problem scenario #3
If I call commit() or rollback() when there is no transaction active, it sets the $transDepth to -1. The next beginTransaction() increments the level to 0, which means the transaction can neither be rolled back nor committed. Subsequent calls to commit() will just decrement the transaction to -1 or further, and you'll never be able to commit until you do another superfluous beginTransaction() to increment the level again.
Basically, trying to manage transactions in application logic without allowing the database to do the bookkeeping is a doomed idea. If you have a requirement for two models to use explicit transaction control in one application request, then you must open two DB connections, one for each model. Then each model can have its own active transaction, which can be committed or rolled back independently from one another.
Do a try/catch: if the exception is that a transaction has already started (based on error code or the message of the string, whatever), carry on. Otherwise, throw the exception again.
Store the return value of beginTransaction() in Zend_Registry, and check it later.
Looking at the Zend_Db as well as the adapters (both mysqli and PDO versions) I'm not really seeing any nice way to check transaction state. There appears to be a ZF issue regarding this - fortunately with a patch slated to come out soon.
For the time being, if you'd rather not run unofficial ZF code, the mysqli documentation says you can SELECT ##autocommit to find out if you're currently in a transaction (err... not in autocommit mode).
For innoDB you should be able to use
SELECT * FROM INFORMATION_SCHEMA.INNODB_TRX WHERE TRX_MYSQL_THREAD_ID = CONNECTION_ID();
This discussion is fairly old. As some have pointed out, you can do it in your application. PHP has a method since version 5 >= 5.3.3 to know if you are in the middle of a transaction. PDP::inTransaction() returns true or false. Link http://php.net/manual/en/pdo.intransaction.php
You can also write your code as per following:
try {
Zend_Registry::get('database')->beginTransaction();
}
catch (Exception $e) { }
try {
$totals = self::calculateTotals($Cart);
$PaymentInstrument = new PaymentInstrument;
$PaymentInstrument->create();
$PaymentInstrument->validate();
$PaymentInstrument->save();
Zend_Registry::get('database')->commit();
return true;
}
catch (Zend_Exception $e) {
Bootstrap::$Log->err($e->getMessage());
Zend_Registry::get('database')->rollBack();
return false;
}
In web-facing PHP, scripts are almost always invoked during a single web request. What you would really like to do in that case is start a transaction and commit it right before the script ends. If anything goes wrong, throw an exception and roll back the entire thing. Like this:
wrapper.php:
try {
// start transaction
include("your_script.php");
// commit transaction
} catch (RollbackException $e) {
// roll back transaction
}
The situation gets a little more complex with sharding, where you may be opening several connections. You have to add them to a list of connections where the transactions should be committed or rolled back at the end of the script. However, realize that in the case of sharding, unless you have a global mutex on transactions, you will not be easily able to achieve true isolation or atomicity of concurrent transactions because another script might be committing their transactions to the shards while you're committing yours. However, you might want to check out MySQL's distributed transactions.
Use zend profiler to see begin as query text and Zend_Db_Prfiler::TRANSACTION as query type with out commit or rollback as query text afterwards. (By assuming there is no ->query("START TRANSACTION") and zend profiler enabled in your application)
I disagree with Bill Karwin's assessment that keeping track of transactions started is cockamamie, although I do like that word.
I have a situation where I have event handler functions that might get called by a module not written by me. My event handlers create a lot of records in the db. I definitely need to roll back if something wasn't passed correctly or is missing or something goes, well, cockamamie. I cannot know whether the outside module's code triggering the event handler is handling db transactions, because the code is written by other people. I have not found a way to query the database to see if a transaction is in progress.
So I DO keep count. I'm using CodeIgniter, which seems to do strange things if I ask it to start using nested db transactions (e.g. calling it's trans_start() method more than once). In other words, I can't just include trans_start() in my event handler, because if an outside function is also using trans_start(), rollbacks and commits don't occur correctly. There is always the possibility that I haven't yet figured out to manage those functions correctly, but I've run many tests.
All my event handlers need to know is, has a db transaction already been initiated by another module calling in? If so, it does not start another new transaction and does not honor any rollbacks or commits either. It does trust that if some outside function has initiated a db transaction then it will also be handling rollbacks/commits.
I have wrapper functions for CodeIgniter's transaction methods and these functions increment/decrement a counter.
function transBegin(){
//increment our number of levels
$this->_transBegin += 1;
//if we are only one level deep, we can create transaction
if($this->_transBegin ==1) {
$this->db->trans_begin();
}
}
function transCommit(){
if($this->_transBegin == 1) {
//if we are only one level deep, we can commit transaction
$this->db->trans_commit();
}
//decrement our number of levels
$this->_transBegin -= 1;
}
function transRollback(){
if($this->_transBegin == 1) {
//if we are only one level deep, we can roll back transaction
$this->db->trans_rollback();
}
//decrement our number of levels
$this->_transBegin -= 1;
}
In my situation, this is the only way to check for an existing db transaction. And it works. I wouldn't say that "The Application is managing db transactions". That's really untrue in this situation. It is simply checking whether some other part of the application has started any db transactions, so that it can avoid creating nested db transactions.
Maybe you can try PDO::inTransaction...returns TRUE if a transaction is currently active, and FALSE if not.
I have not tested myself but it seems not bad!
Related
This seems like a simple enough question, yet I couldn't find any definitive answers specific for MySQL. Look at this:
$mysqli->autocommit(false); //Start the transaction
$success = true;
/* do a bunch of inserts here, which will be rolled back and
set $success to false if they fail */
if ($success) {
if ($mysqli->commit()) {
/* display success message, possibly redirect to another page */
}
else {
/* display error message */
$mysqli->rollback(); //<----------- Do I need this?
}
}
$mysqli->autocommit(true); //Turns autocommit back on (will be turned off again if needed)
//Keep running regardless, possibly executing more inserts
The thing is, most examples I have seen just end the script if commiting failed, either by letting it finish or by explicitly calling exit(), which apparently auto rolls back the transaction(?), but what if I need to keep running and possibly execute more database-altering operations later? If this commit failed and I didn't have that rollback there, would turning autocommit back on (which according to this comment on the PHP manual's entry on autocommit does commit pending operations) or even explicitly calling another $mysqli->commit() later on, attempt to commit the previous inserts again, since it failed before and they weren't rolled back?
I hope I've been clear enough and that I can get a definitive answer for this, which has been bugging me quite a lot.
Edit: OK, maybe I phrased my question wrong in that line comment. It's not really a matter of whether or not I need the rollback, which, as it was pointed out, would depend on my use case, but really what is the effect of having or not having a rollback in that line. Perhaps a simpler question would be: does a failed commit call discards pending operations or does it just leaves them in their pending state, waiting for a rollback or another commit?
If you are NOT re-using the connection and it is closed immediately after the transaction fails, closing the connection would cause an implicit rollback anyway.
If you are re-using the connection, you should definitely do a rollback to avoid inconsistency with any follow-up statements.
And if you are not really re-using it but it is still in a blocking state (e.g. being left open for a couple of seconds or even minutes, depending e.g. whether you're on a website or a cronjob), please keep in mind that there can be many concurrent connections going on. So if you have a very large transaction, the server needs to hold it in a temporary state which might consume lots of memory (e.g. if you're doing a major database migration that affects lots of columns or tables) you should definitely do an explicit rollback or close the connection for an implicit rollback after it fails.
Another factor is => if you have lots of concurrent connections in different processes, they may or may not already see parts of the transaction, even if it's not committed yet, depending on the transaction isolation level that you are using. See also:
https://dev.mysql.com/doc/refman/8.0/en/innodb-transaction-isolation-levels.html
I've got a PHP app with a custom model layer built on top of PDO. Each of my models is a separate class (with a common base class) that manages interactions with a given table (all SELECTs, INSERTs, and UPDATEs for the given table are instance methods of the given class). However, all the classes share a single PDO object, with a single DB connection, which is passed into the constructor of the model class, in a very simplistic form of dependency injection.
This is a bit of a simplification, but it works something like this:
<?php
$pdo = new \PDO(/*connection details*/);
$modelA = new ModelA($pdo);
$modelB = new ModelB($pdo);
$thingA = $modelA->find(1234); //where 1234 is the primary key
$thingB = $modelB->create([
'modelb_id' => $thingA->id,
'notes' => 'This call generates an runs an insert statement based on the array of fields and values passed in, applying specialized business logic, and other fun stuff.'
]);
print_r($thingB); // Create returns an object containing the data from the newly created row, so this would dump it to the screen.
At any rate, currently, everything just runs through PDO's prepared statements and gets committed immediately, because we're not using transactions. For the most part that's fine, but we're running into some issues.
First, if the database interaction is more complicated than what you see above, you can have a request that generates a large number of SQL queries, including several INSERTs and UPDATEs, that are all interrelated. If you hit an error or exception midway through the execution, the first half of the database stuff is already committed to the database, but everything after that never runs. So you get orphaned records, and other bad data. I know that is essentially the exact problem transaction were invented to solve, so it seems like a compelling case to use them.
The other thing we're running into is intermittent MySQL deadlocks when running at high traffic volumes. I'm still trying to get to the bottom of these, but I suspect transactions would help here, too, from what I'm reading.
Here's the thing though. My database logic is abstracted (as it should be), which ends up scattering it a bit. What I'm thinking I'd like to do is create a transaction right when I create my PDO object, and then all the queries run with the PDO object would be part of that transaction. Then as part of my app's tear down sequence for an HTTP request, commit the transaction, after everything has been done. And in my error handling logic, I'd probably want to call rollback.
I'm going to do some experimentation before I try to implement this fully, but I was hoping to get some feedback from someone who might have gone down with road, about whether I'm on the right track, or this is actually a terrible idea, or some tips for a more successful implementation. I also have some specific questions concerns:
Will wrapping everything in transactions cause a performance hit? It seems like if you've got multiple simultaneous transactions, they have to be queued, so the first transaction can complete before the second can run. Or else you'd have to cache the state for your database, so that all the events in one transaction can follow their logical progression? Maybe not. Maybe it's actually faster. I have no idea.
We use MySQL replication. One environment uses MySQL's Master-Slave support, with the bin log format set to STATEMENT. The other environment uses Percona Cluster, with Master-Master replication. Are there any replication implications of this model?
Is there a timeout for the transaction? And when it times out, does it commit or rollback? I expect my normal application logic will either commit or rollback, depending on whether there was an error. But if the whole thing crashes, what happens to the transaction I've been building?
I mentioned we're running into MySQL deadlocks. I've added some logic to my models, where when PDO throws a PDOException, if it's a 1213, I catch it, sleep for up to 3 seconds, then try again. What happens to my transaction if there's a deadlock, or any PDOException for that matter? Does it automatically rollback? Or does it leave it alone, so I can attempt to recover?
What other risks am I completely overlooking that I need to account for? I can try out my idea and do some tests, but I'm worried about the case where everything I know to look for works, but what about the things I don't know to look for that come up when I push it out into the wild?
Right now, I don't use a connection pooling, I just create a connection for each request, shared between all the model classes. But if I want to do connection pooling later, how does that interact with transactions?
I have a pretty clear picture of how and where to start the transaction, with when I create my shared PDO object. But is there a best practice for the commit part? I have a script that does other teardown stuff for each request. I could put it there. I could maybe subclass PDO, and add it to the destructor, but I've never done much with constructors in PHP.
Oh yeah, I need to get clear about how nested transactions work. In my preliminary tests, it looks like you can call BEGIN several times, and the first time you call COMMIT, it commits everything back to the first BEGIN. And any subsequent COMMITs don't do much. Is that right, or did I misinterpret my tests?
When a deadlock situation occurs in MySQL/InnoDB, it returns this familiar error:
'Deadlock found when trying to get lock; try restarting transaction'
So what i did was record all queries that go into a transaction so that they can simply be reissued if a statement in the transaction fails. Simple.
Problem: When you have queries that depend on the results of previous queries, this doesn't work so well.
For Example:
START TRANSACTION;
INSERT INTO some_table ...;
-- Application here gets ID of thing inserted: $id = $database->LastInsertedID()
INSERT INTO some_other_table (id,data) VALUES ($id,'foo');
COMMIT;
In this situation, I can't simply reissue the transaction as it was originally created. The ID acquired by the first SQL statement is no longer valid after the transaction fails but is used by the second statement. Meanwhile, many objects have been populated with data from the transaction which then become obsolete when the transaction gets rolled back. The application code itself does not "roll back" with the database of course.
Question is: How can i handle these situations in the application code? (PHP)
I'm assuming two things. Please tell me if you think I'm on the right track:
1) Since the database can't just reissue a transaction verbatim in all situations, my original solution doesn't work and should not be used.
2) The only good way to do this is to wrap any and all transaction-issuing code in it's own try/catch block and attempt to reissue the code itself, not just the SQL.
Thanks for your input. You rock.
A transaction can fail. Deadlock is a case of fail, you could have more fails in serializable levels as well. Transaction isolation problems is a nightmare. Trying to avoid fails is the bad way I think.
I think any well written transaction code should effectively be prepared for failing transactions.
As you have seen recording queries and replaying them is not a solution, as when you restart your transaction the database has moved. If it were a valid solution the SQL engine would certainly do that for you. For me the rules are:
redo all your reads inside the transactions (any data you have read outside may have been altered)
throw everything from previous attempt, if you have written things outside of the transaction (logs, LDAP, anything outside the SGBD) it should be cancelled because of the rollback
redo everything in fact :-)
This mean a retry loop.
So you have your try/catch block with the transaction inside. You need to add a while loop with maybe 3 attempts, you leave the while loop if the commit part of the code succeed. If after 3 retry the transaction is still failing then launch an Exception to the user -- so that you do not try an inifinite retry loop, you may have a really big problem in fact --. Note that you should handle SQL error and lock or serializable exception in different ways. 3 is an arbitrary number, you may try a bigger number of attempts.
This may give something like that:
$retry=0;
$notdone=TRUE;
while( $notdone && $retry<3 ) {
try {
$transaction->begin();
do_all_the_transaction_stuff();
$transaction->commit();
$notdone=FALSE;
} catch( Exception $e ) {
// here we could differentiate basic SQL errors and deadlock/serializable errors
$transaction->rollback();
undo_all_non_datatbase_stuff();
$retry++;
}
}
if( 3 == $retry ) {
throw new Exception("Try later, sorry, too much guys other there, or it's not your day.");
}
And that means all the stuff (read, writes, fonctionnal things) must be enclosed in the $do_all_the_transaction_stuff();. Implying the transaction managing code is in the controllers, the high-level-application-functional-main code, not split upon several low-level-database-access-models objects.
In PHP I am using PDO to interact with databases. One procedure that commonly takes place consists of multiple queries (several SELECT and UPDATE). This works most of the time, but occasionally the data becomes corrupt where two (or more) instances of the procedure run concurrently.
What is the best way to work around this issue? Ideally I would like a solution which works with the majority of PDO drivers.
Assuming your database back end supports transactions (mysql with InnoDB, Postgres, etc), then simply wrapping the operation in question in a transaction will solve the problem. If one instance of the script is in the middle of the transaction when the second scripts attempts to start it, then the second script's database changes will be queued up and not be attempted until the first transaction completes. This means the database will always be in a valid state provided the transaction starting and committing logic is implemented correctly.
if ($inTransaction = $pdo -> beginTransaction ())
{
// Do your selects and updates here. Try to keep this section as short as possible though, as you don't want to keep other pending transactions waiting
if ($condition_for_success_met)
{
$pdo -> commit ();
}
else
{
$pdo -> rollback ();
}
}
else
{
// Couldn't start a transaction. Handle error here
}
I have code that looks like this:
function foobar(array& $objects, $con = null)
{
if (is_null($con))
$con = DbSingleton::getConnectio();
$con->beginTransaction(); // <- question 1
try
{
foreach($objects as $object)
{
// allocate memory for new object
$new_obj = new MyShiningNewObject();
// do something to the new object ...
$new_obj->setParentId($object->getId());
$new_obj->save($con);
// mark for garbage collection
unset($new_obj); // <- question 2
}
$con->commit();
}
catch(Exception $e){ $con->rollBack(); }
}
My questions are:
I am begining a transaction, this could well be a nested transaction. In the case of a nested transaction, if an exception is thrown and I rollback, how far back does the rollback go (to the outermost transaction) - common sense suggest that this should be the case, but one never knows.
I am freeing memory (ok, marking as 'freeable' by the Zend GC). Since I am commiting the transaction AFTER the loop, (the variable is marked as frreable IN the loop), is this safe - I mean will the data be safely stored in the db even though I have unset the variable that the value came from?
Since only the results of the outermost transaction as visible from outside, nesting transactions have little sense as such, and of the major systems only SQL Server supports them.
In SQL Server, an intermediate rollback rolls back to the beginning of the intermediate transaction, an intermediate commit does nothing.
The outermost rollback rolls back the whole outermost transaction (even if there were commits in between), the outermost commit commits the whole outermost transaction.
The other systems have only one transaction level with possible SAVEPOINTS in between. You can rollback to each of the previous savepoints (by providing their name), but issuing a COMMIT or ROLLBACK without the savename point always commits or rolls back the outermost transaction.
To answer question 2 :
once the query has been sent to the database, what is done on PHP variables doesn't matter anymore
the only thing you'll do that will have an impact is when you'll commit or rollback.
which means that unsetting the PHP variable will have no impact on the transactions/data on the DB side.
About question 1 :
If your database system supports nested transactions, commiting/rollbacking on an "inner" transaction should, logically, only act on that "inner" transaction.
But note that nested transactions are not supported by all DBMS -- MySQL, for instance, doesn't support them, as far as I remember.
For reference, here are three questions about nested reference (there are probably many more) :
Nested transactions in postgresql 8.2?
Nested transactions in Sql Server
Are nested transactions allowed in MySQL?