Let's say I have following dummy code that actually copy-pastes company (client) information with all related objects:
class Company extends BaseModel{
public function companyCopyPaste($existingCompanyId)
{
$this->db->transaction->start();
try{
$newCompanyId = $this->createNewCompany($existingCompanyId);
$this->copyPasteClientObjects($companyId, $newCompanyId)
$this->db->transaction->commit();
} catch(Exception $e){
this->db->transaction->rollback();
}
}
...
}
Method copyPasteClientObjects contains a lot of logic inside, like selecting/updating existing data, aggregating it and saving it.
Also whole process may take up to 10 seconds to complete (due to loads of information to process)
Easiest way is to start transaction in the begging of the method and commit it when its done. But I guess this is not the
right way to do it, but still I want to keep everything integral, also to avoid deadlocks as well. So if one of the steps fail, I want previous steps to be rolled back.
Any good advice how to handle such situations properly?
This is not an answer but some opinion.
If I get you right, you want to implement create-new-from-existing kind operation.
There is nothing really dangerous yet happen while you create new records.
I would suggest you to transform code this way:
try{
$newCompanyId = $this->createNewCompany($existingCompanyId);
$this->copyPasteClientObjects($companyId, $newCompanyId)
} catch(Exception $e){
this->deleteNewCompany($newCompanyId);
}
This way you don't need any transaction, but your deleteNewCompany should revert everything that was done but not finished. Yes it is more work to create that functionality, but to me it makes more sense then to block DB for 10 sec.
And } catch(Exception $e){ imho is not the best practice, you need to define some custom case specific Exception type. Like CopyPasteException or whatever.
Related
I have multiple PDO objects, referring to different databases on different servers. I need to be able to rollback all transactions in case of error at any step.
In fact, I want to have something like this:
try {
$db1Connector->beginTransaction();
$db2Connector->beginTransaction();
//some functionality
$db1Connector->commit();
$db2Connector->commit();
} catch (\Exception $exception) {
$db1Connector->rollback();
$db2Connector->rollback();
}
I have this functionality in my code, but it seems it does not work correctly. I noticed some cases that only one of the transactions is rollbacked. I guess this problem occurs when $db1Connector->commit() or $db2Connector->commit() fails.
Yes, it is possible. You need to use a distributed transaction. See https://stackoverflow.com/questions/17772363/distributed-transaction-on-mysql#:~:text=so%20server%20A%20sends%20a,server%20A%20and%20server%20B.
I have a mobile application and server based on Symfony which gives API for the mobile app.
I have a situation, where users can like Post. When users like Post I add an entry in ManyToMany table that this particular user liked this particular Post (step 1). Then in Post table I increase likesCounter (step 2). Then in User table I increase gamification points for user (because he liked the Post) (step 3).
So there is a situation where many users likes particular Post at the same time and deadlock occurs (on Post table or on User table).
How to handle this? In Doctrine Docs I can see solution like this:
<?php
try {
// process stuff
} catch (\Doctrine\DBAL\Exception\RetryableException $e) {
// retry the processing
}
but what should I do in catch part? Retry the whole process of liking (steps 1 to 3) for instance 3 times and if failed return BadRequest to the mobile application? Or something else?
I don't know if this is a good example cause maybe I could try to rebuild the process so the deadlock won't happen but I would like to know what should I do if they actually happen?
I disagree with Stefan, deadlocks are normal as the MySQL documentation says:
Normally, you must write your applications so that they are always prepared to re-issue a transaction if it gets rolled back because of a deadlock.
See: MySQL documentation
However, the loop suggested by Stefan is the right solution. Except that it lacks an important point: after Doctrine has thrown an Exception, the EntityManager becomes unusable and you must create a new one in the catch clause with resetManager() from the ManagerRegistry instance.
When I had exactly the same concern as you, I searched the web but couldn't find any completely satisfactory answer. So I got my hands dirty and came back with an article where you'll find an implementation exemple of what I said above:
Thread-safe business logic with Doctrine
What I'd do is post all likes on a queue and consume them using a batch consumer so that you can group the updates on a single post.
If you insist on keeping you current implementation you could go down the road you yourself suggested like this:
<?php
for ($i = 0; $i < $retryCount; $i++) {
try {
// try updating
break;
} catch (\Doctrine\DBAL\Exception\RetryableException $e) {
// you could also add a delay here
continue;
}
}
if ($i === $retryCount) {
// throw BadRequest
}
This is an ugly solution and I wouldn't suggest it. Deadlocks shouldn't be "avoided" by retrying or using delays. Also have a look at named locks and use the same retry system, but don't wait for the deadlock to happen.
The problem is that after Symfony Entity Manager fails - it closes db connection and you can't continue you work with db even if you catch the ORMException.
First good solution is to process your 'likes' async, with rabbitmq or other queue implementation.
Step-by-step:
Create message like {type: 'like', user:123, post: 456}
Publish it in queue
Consume it and update 'likes' count.
You can have several consumers that try to obtain lock on based on postId. If two consumers try to update same post - one of them will fail obtaining the lock. But it's ok, you can consume failed message after.
Second solution is to have special table e.g. post_likes (userId, postId, timestamp). Your endpoint could create new rows in this table synchronously. And you can count 'likes' on some post with this table. Or you can write some cron script, which will update post likes count by this table.
I've made a special class to retry on deadlock (I'm on Symfony 4.4).
Here it is :
class AntiDeadlockService
{
/**
* #var EntityManagerInterface
*/
private $em;
public function __construct(EntityManagerInterface $em)
{
$this->em = $em;
}
public function safePush(): void
{
// to retry on deadlocks or other retryable exceptions
$connection = $this->em->getConnection();
$retry = 0;
$maxRetries = 3;
while ($retry < $maxRetries) {
try {
if (!$this->em->isOpen()) {
$this->em = $this->em->create(
$connection = $this->em->getConnection(),
$this->em->getConfiguration()
);
}
$connection->beginTransaction(); // suspend auto-commit
$this->em->flush();
$connection->commit();
break;
} catch (RetryableException $exception) {
$connection->rollBack();
$retry++;
if ($retry === $maxRetries) {
throw $exception;
}
}
}
}
}
Use this safePush() method instead of the $entityManager->push() one ;)
In the Magento Ecommerce System, there are three events that fire before the system is fully bootstrapped
resource_get_tablename
core_collection_abstract_load_before
core_collection_abstract_load_after
These events also fire after Magento has bootstrapped.
What's a safe and elegant (and maybe event Mage core team blessed) way to detect when Magento has fully bootstrapped so you may safely use these events?
If you attempt to use certain features in the pre-bootstrapped state, the entire request will 404. The best I've come up with (self-link for context) so far is something like this
class Packagename_Modulename_Model_Observer
{
public function observerMethod($observer)
{
$is_safe = true;
try
{
$store = Mage::app()->getSafeStore();
}
catch(Exception $e)
{
$is_safe = false;
}
if(!$is_safe)
{
return;
}
//if we're still here, we could initialize store object
//and should be well into router initialization
}
}
but that's a little unwieldy.
I don't think there is any event tailored for that.
You could add you own and file a pull request / Magento ticket to include a good one.
Until then I think the only way is to use one of the events you found and do some checks on how far Magento is initialized.
Did you try to get Mage::app()->getStores()? This might save you from the Exception catching.
I was having a memory leak problem in a Doctrine2 script that was aparently caused by a piece of code that was supposed to eliminate memory problems.
Before I knew you could (and should) clear the Entity Manager, every 20 iterations i did the following:
if ($this->usersCalculated % 20 == 0) {
$this->em->close();
$this->em = \Bootstrap::createEm();
$this->loadRepositories();
}
And the Bootstrap::createEm looks like this:
public static function createEm() {
$em = EntityManager::create(Bootstrap::$connectionOptions, Bootstrap::$config);
$em->getConnection()->setCharset('utf8');
return $em;
}
The reason that I recreated the Entity Manager in the first place was because my UnitOfWork was growing wild and I didn't know about the $em->clear() method.
So, even if my current memory leak seems solved at the moment (or at least reduced), i still have to create a new Entity Manager whenever I need to do a separate insert/update query without relying that someone else do the flush. For example, whenever I send an email, I insert a row in the database to indicate so, and the code looks like this:
$emailSent = new \model\EmailSent();
$emailSent->setStuff();
// I do it in a new em to not affect whatever currentunit was doing.
$newEm = \Bootstrap::createEm();
$newEm->persist($emailSent);
$newEm->flush();
$newEm->close();
From what I've learned from before, that leaves some memory leaked behind.
So my question is, what am I doing wrong here? why is this leaking memory and how should I really close/recreate an entity manager?
Have you tried:
$this->em->getConnection()->getConfiguration()->setSQLLogger(null);
I've read that this turns off the SQL Logger which is not cleared and sometimes produces memory leaks like you are experiencing.
Have you tried actually using the clear method instead of close?
I hope this helps you---> Batch Processing
How should I write error reporting modules in PHP?
Say, I want to write a function in PHP: 'bool isDuplicateEmail($email)'.
In that function, I want to check if the $email is already present in the database.
It will return 'true', if exists. Else 'false'.
Now, the query execution can also fail, In that time I want to report 'Internal Error' to the user.
The function should not die with typical mysql error: die(mysql_error(). My web app has two interfaces: browser and email(You can perform certain actions by sending an email).
In both cases it should report error in good aesthetic.
Do I really have to use exception handling for this?
Can anyone point me to some good PHP project where I can learn how to design robust PHP web-app?
In my PHP projects, I have tried several different tacts. I've come to the following solution which seems to work well for me:
First, any major PHP application I write has some sort of central singleton that manages application-level data and behaviors. The "Application" object. I mention that here because I use this object to collect generated feedback from every other module. The rendering module can query the application object for the feedback it deems should be displayed to the user.
On a lower-level, every class is derived from some base class that contains error management methods. For example an "AddError(code,string,global)" and "GetErrors()" and "ClearErrors". The "AddError" method does two things: stores a local copy of that error in an instance-specific array for that object and (optionally) notifies the application object of this error ("global" is a boolean) which then stores that error for future use in rendering.
So now here's how it works in practice:
Note that 'Object' defines the following methods: AddError ClearErrors GetErrorCodes GetErrorsAsStrings GetErrorCount and maybe HasError for convenience
// $GLOBALS['app'] = new Application();
class MyObject extends Object
{
/**
* #return bool Returns false if failed
*/
public function DoThing()
{
$this->ClearErrors();
if ([something succeeded])
{
return true;
}
else
{
$this->AddError(ERR_OP_FAILED,"Thing could not be done");
return false;
}
}
}
$ob = new MyObject();
if ($ob->DoThing())
{
echo 'Success.';
}
else
{
// Right now, i may not really care *why* it didn't work (the user
// may want to know about the problem, though (see below).
$ob->TrySomethingElse();
}
// ...LATER ON IN THE RENDERING MODULE
echo implode('<br/>',$GLOBALS['app']->GetErrorsAsStrings());
The reason I like this is because:
I hate exceptions because I personally believe they make code more convoluted that it needs to be
Sometimes you just need to know that a function succeeded or failed and not exactly what went wrong
A lot of times you don't need a specific error code but you need a specific error string and you don't want to create an error code for every single possible error condition. Sometimes you really just want to use an "opfailed" code but go into some detail for the user's sake in the string itself. This allows for that flexibility
Having two error collection locations (the local level for use by the calling algorithm and global level for use by rendering modules for telling the user about them) has really worked for me to give each functional area exactly what it needs to get things done.
Using MVC, i always use some sort of default error/exception handler, where actions with exceptions (and no own error-/exceptionhandling) will be caught.
There you could decide to answer via email or browser-response, and it will always have the same look :)
I'd use a framework like Zend Framework that has a thorough exception handling mechanism built all through it.
Look into exception handling and error handling in the php manual. Also read the comments at the bottom, very useful.
There's aslo a method explained in those page how to convert PHP errors into exceptions, so you only deal with exceptions (for the most part).