I'm working on a Magento module and need to know if it's possible to roll back a series of model saves. Basically, I have five models plus several from my module that I need to save one after the other:
admin/role
admin/user
core/website
core/store_group
core/store
mymodule/model1
mymodule/model2
My problem is that whenever any of these models throw an exception, I need to go into MySQL and manually delete all the rows that were saved. This is very unproductive.
I'm pretty sure that Magento doesn't have a rollback procedure that I can access in my context. For example, I looked in Mage_Core_Model_Abstract and in the save method, the rollback mechanisms are all protected.
So, my question is, is there some best practice for doing database transactions in Magento that I should be aware of?
I've seen the following used in core code, and it looks like its just what you ordered.
$transactionSave = Mage::getModel('core/resource_transaction');
$transactionSave->addObject($model_one)
$transactionSave->addObject($model_two)
$transactionSave->save();
The core/resource_transaction object allows you to add Magento objects, and perform a mass save on them. Give that a try, and I'd love to hear how this does, or doesn't, work for you in the comments.
Related
In my obsolate procedural code (which I'd like now to translate into OOP) I have simple database transaction code like this:
mysql_query("BEGIN");
mysql_query("INSERT INTO customers SET cid=$cid,cname='$cname'");
mysql_query("INSERT INTO departments SET did=$did,dname='$dname'");
mysql_query("COMMIT");
If I build OOP classes Customer and Department for mapping customers and departments database tables I can insert table records like:
$customer=new Customer();
$customer->setId($cid);
$customer->setName($cname);
$customer->save();
$department=new Department();
$department->setId($did);
$department->setName($dname);
$department->save();
My Customer and Department classes internally use other DB class for querying database.
But how to make $customer.save() and $department.save() parts of a database transaction?
Should I have one outer class starting/ending transaction with Customer and Department classes instantiated in it or transaction should be started somehow in Customer (like Customer.startTransaction()) and ended in Department (like Department.endTransaction())? Or...
Additional object is the way to go. Something like this:
$customer=new Customer();
$customer->setId($cid);
$customer->setName($cname);
$department=new Department();
$department->setId($did);
$department->setName($dname);
$transaction = new Transaction();
$transaction->add($customer);
$transaction->add($department);
$transaction->commit();
You can see that there is no call to save() method on $customer and $department anymore. $transaction object takes care of that.
Implementation can be as simple as this:
class Transaction
{
private $stack;
public function __construct()
{
$this->stack = array();
}
public function add($entity)
{
$this->stack[] = $entity;
}
public function commit()
{
mysql_query("BEGIN");
foreach ($this->stack as $entity) {
$entity->save();
}
mysql_query("COMMIT");
}
}
How to make $customer.save() and $department.save() parts of a database transaction?
You don't have to do anything besides start the transaction.
In most DBMS interfaces, the transaction is "global" to the database connection. If you start a transaction, then all subsequent work is automatically done within the scope of that transaction. If you commit, you have committed all changes since the last transaction BEGIN. If you rollback, you discard all changes since the last BEGIN (there's also an option to rollback to the last transaction savepoint).
I've only used one database API that allowed multiple independent transactions to be active per database connection simultaneously (that was InterBase / Firebird). But this is so uncommon, that standard database interfaces like ODBC, JDBC, PDO, Perl DBI just assume that you only get one active transaction per db connection, and all changes happen within the scope of the one active transaction.
Should I have one outer class starting/ending transaction with Customer and Department classes instantiated in it or transaction should be started somehow in Customer (like Customer.startTransaction()) and ended in Department (like Department.endTransaction())? Or...
You should start a transaction, then invoke domain model classes like Customer and Department, then afterwards, either commit or rollback the transaction in the calling code.
The reason for this is that domain model methods can call other domain model methods. You never know how many levels deep these calls go, so it's really difficult for the domain model to know when it's time to commit or rollback.
For some pitfalls of doing this, see How do detect that transaction has already been started?
But they don't have to know that. Customer and Department should just do their work, inserting and deleting and updating as needed. Once they are done, the calling code decides if it wants to commit or rollback the whole set of work.
In a typical PHP application, a transaction is usually the same amount of work as one PHP request. It's possible, though uncommon, to do more than one transaction during a given PHP request, and it's not possible for a transaction to span across multiple PHP requests.
So the simple answer is that your PHP script should start a transaction near the beginning of the script, before invoking any domain model classes, then commit or rollback at the end of the script, or once the domain model classes have finished their work.
You are migrating to OOP, and thats great, but soon you will find yourself migrating to an arquitecture with a well diferenciated Data Access Layer, including a more complex way of separating data from control. Now, i guess you are using some kind of Data access object, that is a great first approach pattern, but for sure you can go further. Some of the answer here already lead you in that direction. You shouldent think in your objects as the basis of your arquitecture, and use some helper objects to query database. Instead, you should think about a fully featured layer, with all required generic classes that takes care of the comunication with the database, that you will use in all your projects, and then have the business-level-objects, like customer or department, than know as litle as possible about database implementations.
For this, for sure you will have an outer class handling transactions, but probably also other taking care of security, other for building queries providing a unique api regardless or the database engine, and even more, a class that reads objects in order to put them in the database, so the object itself doesn't even know that it is meant to end in a database.
Achieve this, would be a hard and long work, but after that, you could have a custom and widely reusable layer that will make your projects more escalable, more stable, and more trustable. And that will be great and you will learn a lot and after that you would fill quite good. You will have some kind of DBAL or ORM.
But that wouldnt also be the best solution, since there are people that already have been years doing that, and it will be hard to achieve what the already have.
So, what i recommend, for any medium size project, is that you take data base abstraction as serious as you can, and any opensource ORM, that happens to be easy to use, and finally you will save time and get a system much better.
for example, doctrine has a very nice way of handling transactions and concurrency, in two ways: implicit, taking automatically care of the normal operations, or implicit, when you need to take over and control transaction demarcation yourself. check it out here. Also, there are some other complex posibilities like transaction nesting, and others.
The most famous and reliable ORM are
Doctrine, and
Propel
I use doctrine mostly, since it has a module to integrate with Zend Framework 2 that i like, but propel has some aspects that i like a lot.
Probably you would have to refactor somethings, and you dont feel like doing it at this point, but i can say for my experience, that this is one of those things you dont even want to think about, and years after you start using it and realize how you wasted time :-)recommend you to consider this if not know, in your very next project.
UPDATE
Some thoughts after Tomas' comment.
It's true that for not so big projects (especially if you are not very familiar with orms, or your model is very complex) it can be a big effort to integrate a vendor orm.
But what i can say after years developing projects of any size, is that for any medium size one, i would use at least a custom, less serious and more flexible home-made orm, with a sort of generic classes, and as few as possible business oriented repositories, where an entity knows its table, and probably other related tables, and where you can encapsulate some sql or custom query function calls, but all around that entity (for example the main table of the entity, the table of pictures associated to that entity, and so) in order to provide to the controller a single interface to the data, so at any range the database engine is independent of the API of the model, and as much important as that, the controller doesn't have to be aware of any DBMS aspects, like the use of a transactions, something that is meant just to ensure a behavior that is purely model-related, and in a scandalous low level: related pretty much to DBMS technical needs. i mean, your controller could know that it is storing stuff in a database, but for sure it doesn't have to even know what a transaction is.
For sure this is a philosophical discussion, and it could be many equally valid points of view.
For any custom ORM, i would recommend to start looking for some DAO/DTO generator that can help you to create the main classes from your database, so you only need to adapt them to your needs at the points where you find exceptions to the normal behavior of a normal create-read-update-delete. This reminds me that you can also look for PHP CRUD and find some useful and fun tools.
This is a long running question that gets me every time I am developing.
I suppose it is not specific to CodeIgniter, but as I am using it, I will consider it in my example.
Firstly, which is better:
function add_entry($data_array)
{
//code to add entry
}
function edit_entry($data_array)
{
//code to update entry
}
OR
function save_changes($what,$data_array)
{
//if what == update update
//otherwise insert
}
Both produce the same action, but does it really matter which one you use?
Getting onto more complicated things.
I have a page where I need to get ONE entry from the database.
I also have a page where I need to get all the entries from the same database ordered by a user specified column.
My resultant method is a function similar to
function($data_array,$order_by='',$limit='')
{
//get where $data_array
//if order_by!='' add order by
//if limit !='' add limit
}
As I develop my application and realise new places where I need 'similar' database functionality I am what feels like hacking previous methods so they work with all my case scenarios. The methods end up containing lots of conditional statements, and getting quite complex with in some cases 4 or 5 input parameters.
Have I missed the point? I don't want duplicate code, and when for the most part the functions are very similar I feel like this 'hacking' methodology works best.
Could someone advise?
Finally my admin functionality is part of the same application in an admin controller. I have an admin model which contains specific methods for admin db interaction. I however use some model functionality from 'user' models.
FOr example if on an admin page I need to get details of a db entry I may load the user model to access this function. There is nothing wrong/insecure about this..? right?
In addition to that within my admin model itself I need to get data about a user database entry so I call my user model directly from my admin model. This is strictly OK, but why? If i need data and there is already a method in my user model which gets it.. it seems a little pointless to rewrite the code in the admin model BUT each time that function is called does it load the whole user model again?
Thanks a lot all.
In order, add edit in the model vs save. Personally I have a save built in MY_Model that chooses whether it is a save or an edit depending on the existence of a primary key in the data being passed, so obviously I prefer that method it means a lot less duplication of code since I can use the save for any table without having functions in the model at all.
As to the second question I think it depends on situation. I also have a number of functions that have a ton of conditionals on them depending on where they're used and for what. In some cases I'm finding this makes the legibility of the code a little rough. If you're running them all through if statements it also could be impacting performance. DRY is a concept, not a rule and like other design concepts there are times when they just don't make sense, it's like database normalization, it's my personal opinion it's VERY easy to over normalize a database and destroy performance and usability.
Finally, using user functions in the admin code. I don't see an issue here at all, the reverse probably isn't true, but rewriting a function just because it's an "admin" function, when it's identical to a user function is utterly pointless. So you're correct there, it's a waste of time and space.
I have created a crud system. Here I have a main model and then dependent details model. So their are multiple rows (some times > 100 ) entered into details model related to parent model. All operations handled through grid ( inline edit ). So create, update, delete operations are performed through single request (logical checking via DB).
Now I am considering to use DB transaction to whole operations. But I am confused, how I can implement this using same structure. I already have suggestions to move my all code to one model, So transaction can be applied there. But I am thinking, if any other approach can be used for retain separation of main and details model code.
Are you using AJAX for the changes or does it rely on manual form submissions?
You can use what's called a UnitOfWork pattern.
Save any changes that the user makes to each line of the grid but don't actually commit them to the DB. Then, place a geneic save button on the page that will cause your server to actually commit all the changes through a transaction.
You can keep a list server side of each row that the user changes. You don't need to track all the rows because if they don't change anything you don't need to save anything.
What sort of approach are you using in your Models? Do your models know about their own persistence (do you do things like $user->save()), or are you doing a more data-mapper approach ($userManager->save($userEntity))? The latter makes transaction handling a lot easier.
If you're doing the active-record type of patter ($user->save()), your best bet is probably just to manually grab your db connection and manage the transaction in your controller.
If you're doing data-mapper stuff, you have more options, up to and including doing a whole unit-of-work implementation.
As for my last comment, Moving code to parent model is my solution for now. So I think I should mark this thread answered.
The whole "when to throw exception or return value" questions has been asked a lot (see the following to see just one example):
Should a retrieval method return 'null' or throw an exception when it can't produce the return value?
and I completely agree with the answers in main.
Now my question arises from adding a little more context to the above when applying this to a more complex system. Ill try and keep this as brief and simple as possible.
Right we have an example MVC PHP application:
Model A: has a function get_car($id) which returns a car object.
Controller A has a simple function for say showing a car to the user
Controller B however has a complex function that say gets the car, modify it (say through one of model A's set functions) and also updates other tables based on some of these new values through other models and libraries throughout the system - very complex ay lol
we now get to the main part of my question:
For data integrity I want to use MySQL transactions. This is where I run into a "what's best / what's best practice" scenario...
We write Model A to return FALSE if the car is not found or there is an SQL error.
This is fine for Controller A as it just wants to know if there was a error and bom out, so we just check the return value an bom - fine.
We now get to Controller B. Controller B say does some database updating before the Model A function is called which we need to roll back on error so we need to use a transactions. now this is where I get to my problem. do I leave Model A as a return value and just check it or do I change it to throw exception with the knock on effect of then having to also re-write Controller A as we now need to catch the exception... then (not done yet ;o)) do I roll back in the catch of the model (but how do we know if a transaction has been used or not?) or do we catch and re-throw or allow to bubble up to the controller catch and do the roll back there?
what I'm trying to say is that if I have lots of models and controllers with database interaction should I just make them throw exceptions and then wrap all my other code eg controller functions in try catches encase the model or library functions ever throw, or, do I make the models "self contained" to tidy and handle there own problems but then what do I do about rolling back a transaction if (for this "call") one was open (as per my example above not every time is a transaction opened...)? if this was the case I would have to make all my functions return something and then check this in the controller, as this is the only place that knows if there is an open transaction or not...
So to clarify I can use a try catch to catch and roll back in a controller, that's ok, but how to I do this from "further down" eg in a model or library function... that could be called both during and transaction or just as an auto commit normal MySQL call?
An explained answer would be great (as I like to understand why I am doing something) but if not a some vote for the favourite of the follow solutions (well the solutions I can see):
1) make all model and library functions always return a value and then the controller can either just bom or do a try catch to roll back where necessary - but meaning that I would have to check the return value of every model and library function everywhere they are used.
2) make all model and library functions throw exceptions (say on SQL error) and wrap every controller (which would call the model and library functions) in a try catch where the catch would either just bom or roll back if necessary...
also please note "bom" is push user somewhere or show a pretty error (before someone says "its bad practice to just allow your application to die..." lol)
I hope you get where Im coming from here and sorry for the long loooooong question.
Thanks in advance
Ben
[There's a theoretical problem implicit in the "For data integrity I want to use MySQL transactions"... since MySQL historically hasn't been very ACID - PostgreSQL and Oracle both provide stronger support for ACID. However, back to the real question...]
Both your (1) and (2) focus on exceptions versus failure-return values, but my impression is that this isn't the key part of detangling exceptions, error returns, and open transactions (and some databases support SQL exceptions as well). Instead, I'd focus on keeping the transaction state tied to the nesting of the functions manipulating the model. Here are some thoughts along this line:
You will probably always have error returns from some library functions anyway, so having Model A return FALSE isn't really breaking the paradigm, nor is there anything particularly troublesome about a mix of error returns versus exceptions. However, error returns MUST bubble up correctly - or be converted to exceptions if they go beyond what can be locally address.
Nested transactions are the most obvious way to have one controller start a database manipulation and still call other stuff in the app that also uses transactions. This allows a failed sub-sub-function to abort just its own part transaction and take either the error return or exception approach to bubbling the error up on the non-SQL side while the closed sub-transactions still maintain reasonable matching state. This usually needs to be simulated in code outside of the database (this is essentially what Django does).
Your code could start a new (potentially large) transaction, and keep track of the fact that it's already open to keep the sub-sub-functions in your code from trying to reopen it.
In some databases, code can detect whether a transaction is already open based on the database session state, allowing you to check the DB session state instead of tracking it in code.
Both of the above allow one to use savepoints to simulate truly nested transactions.
One must be very careful to avoid calling SQL calls with implicit commits (CREATE TABLE, for example). MySQL probably deserves a lot more caution around this issue than, say, PostgreSQL.
One way to implement the one big transaction approach is to have high-level function that initiates the transaction and then itself calls the top of whatever Controller B needs to do. This makes either bubbling up errors or having a special abort-transaction exception pretty straightforward. Only the top function would call commit rather than abort if no subfunction failed and no exception was caught. Subfunctions wouldn't call commit.
Conversely, you could have all of your functions pay attention to transactional depth implemented in the non-SQL side (your code), although this is harder to set up in some languages than others (it's pretty easy using decorators in Python, for example). This would allow any of them to call commit if they were done and the transactional depth at zero.
Hope this helps someone :-)
i am developing a groupon-like system, and i came into this project when the system is already around 70% built, and it was built using cakePHP, to be honest, i know nothing about cakePHP. and i came across this:
a member bought a deal
if(has_enough_account_balance){
if((parameters validated)){
insert into 'deal_user' table
log transaction
update 'deal' table by:
user_count = current user_count + bought deal //to determine whether this deal is tipped or not
if(this deal is tipped){
issue coupon
}
}
}else{
this_user_owed
}
the admin confirmed that a particular user has paid his/her owed deal payment
confirm has_paid
update into 'deal_user' table
log transaction
update 'deal' table by:
user_count = current user_count + bought deal //to determine whether this deal is tipped or not
if(this deal is tipped){
issue coupon
}
now seeing that the two of those has something in common, i am trying to do this:
a member bought a deal
if(has_enough_account_balance){
if((parameters validated)){
process_deal(parameters)
}
}else{
this_user_owed
}
on admin confirmation:
confirm has_paid
process_deal(parameters)
and process_deal would be:
function process_deal(parameters){
if(isset(deal_id)){
update into 'deal_user' table
}else{
insert into 'deal_user' table
}
log transaction
update 'deal' table by:
user_count = current user_count + bought deal //to determine whether this deal is tipped or not
if(this deal is tipped){
issue coupon
}
}
is it possible to do things like this ? and where is the best place should i put this process_deal method, i have tried to put this inside the app_controller class, but it seems that it wont update the table, i am not sure why can't it update (i am using the updateAll method), thank you very much
Looks like you can have this function implemented into /models/deal_user.php or /models/deal.php as model classes, so you can share the process_deal across needy controllers.
When a controller need it, simply include the ModelClass. Fatter Models.
and furthermore, you should not include this function into your app_controller as it might not make sense sharing this method across all other controllers, or instantiating the DealUser and Deal models across all controllers, and some might not need it at all.
If you tried debug($this) inside a controller, you know how horrible the array is. The more Model you include, the messier it will be.
Update
(Base on personal experience), put your code into following files when..
/app/bootstrap.php
Is when you have shared codes among everywhere in your application, like debug, json_encode (when PHP version < 5.2) etc
/app/controllers/components/*.php
Is when you have shared code/logic among controllers, with very minimal DB interaction.
/app/models/*.php
(Fat Models) Is when you have shared functionalities among a few controllers, that manipulate the data before saving into the DB, example such as your question above.
/app/libs/*.php
Is when you have shared external codes that doesn't fit into MVC, like TwitterOauth, or other generic classes/packages, which does not make sense converting it into Components, or too complex to do so. Import them into cake by using App::import('Lib', <name>) is pretty sufficient and neat.
/app/views/helpers/*.php
Is when you have shared html codes to render, that required some logic before converting it from data into html codes.
/app/app_controller.php
Is only when you need to do some hack on beforeFilter, beforeRender etc, that need to have common functionalities among controllers. The reason is because, your *_controller is extending app_controller, and if you are tempted and then add shared components, uses, helpers in the app_controller with the hope of having them in all controllers, it turns out that you have heavy controllers in every requests. Make it a habit try make your app_controller as slim as possible. Write more codes in each controllers.
/app/app_model.php
Same idea goes for app_controller
/app/plugins/*/
Is when you include external cake-like framework/features, or you can wrap certain framework into a plugin if you want.
/app/webroot/*/
And never never put your code under webroot! It's not neat, break Cake's structure, and it's not recommended! It's evil. Considering moving your code out of Cake if they couldn't be fit into Cake.
I think that is all? Again, they are my personal experience base on coding experience. Do comment/edit if you think they are not correct.
I'd go with Lionel Chan's answer above.
Also, it would probably help you to read over the CakePHP documentation. What you're trying to do looks like it shouldn't be difficult, but you'll have much more luck if you learn the framework and work within it rather than trying to work around it just using what you already know.
Any code that deals with a certain model's database table should go in that particular model class (the fat models referred to by Lionel).
So you could put the process_deal method in the Deal model.
The DealsController can then access the method like this:
$this->Deal->process_deal().
Then, if there's a relationship between two models, for example: DealUser hasMany Deal, you can access the method from within an action in the DealUsersController like this:
$this->DealUser->Deal->process_deal();
If there isn't a relationship between the models, you could still call the process_deal() method from within an action in the DealUsersController like this:
$this->loadModel('Deal');
$this->Deal->process_deal();
I hope this helps point you in the right direction, but I'd still recommend spending an hour or two perusing the CakePHP documentation linked to above because you're likely to find the answers to most of your questions there...
P.S. Kudos for going for the modular approach. It's always refreshing to see...