I am developing a bidding application in laravel. In this, I have to run cronjob after each minute and user can also bid from front end. So to avoid collision I am using lockForUpdate() of laravel. I have placed this in two different functions where bid ids are being processed. And In one place I am using it three times and at other its is being used one time. I have placed my code inside DB::transaction. But due to some reason I am getting deadlock error whenever same raw is processed by two users at same time.
I am using this as given below:
At One Place
DB::beginTransaction();
try
{
-----Some Code ----
SecondGameBids::where('id', $big)->lockForUpdate()->get();
SecondGameBids::where('id', $big)->update(['final_value' =>0, 'deal_status' => 1]);
-----Some Code ------
SecondGameBids::where('id', $small)->lockForUpdate()->get();
SecondGameBids::where('id', $small)->update(['final_value' =>0, 'deal_status' => 1]);
------Some Code ------
DB::commit();
}
catch (\Exception $e) {
DB::rollback();
}
At Other Place
DB::beginTransaction();
try
{
-----Some Code ----
SecondGameBids::where('id', $big)->lockForUpdate()->get();
SecondGameBids::where('id', $big)->update(['status' => 3]);
------Some Code ------
DB::commit();
}
catch (\Exception $e) {
DB::rollback();
}
Can anyone give me some idea about how to overcome this error?
Related
I have a Symfony app which exposes a collection of JSON web services used by a mobile app.
On the last few days we are having many concurrent users using the app (~5000 accesses per day) and a Doctrine error started to "randomly" appear in my logs. It appears about 2-3 times per day and this is the error:
Uncaught PHP Exception Doctrine\DBAL\Exception\DriverException: "An exception occurred while executing 'UPDATE fos_user_user SET current_crystals = ?, max_crystals = ?, updated_at = ? WHERE id = ?' with params [31, 34, "2017-12-19 09:31:18", 807]:
SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction" at /var/www/html/rollinz_cms/releases/98/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php line 115
It seems it cannot get the lock while updating the users table. The controller code is the following:
/**
* #Rest\Post("/api/badges/{id}/achieve", name="api_achieve_badge")
*/
public function achieveAction(Badge $badge = null)
{
if (!$badge) {
throw new NotFoundHttpException('Badge not found.');
}
$user = $this->getUser();
$em = $this->getDoctrine()->getManager();
$userBadge = $em->getRepository('AppBundle:UserBadge')->findBy(array(
'user' => $user,
'badge' => $badge,
));
if ($userBadge) {
throw new BadRequestHttpException('Badge already achieved.');
}
$userBadge = new UserBadge();
$userBadge
->setUser($user)
->setBadge($badge)
->setAchievedAt(new \DateTime())
;
$em->persist($userBadge);
// sets the rewards
$user->addCrystals($badge->getCrystals());
$em->flush();
return new ApiResponse(ApiResponse::STATUS_SUCCESS, array(
'current_crystals' => $user->getCurrentCrystals(),
'max_crystals' => $user->getMaxCrystals(),
));
}
I looked into MySQL and Doctrine documentation but I couldn't find a reliable solution. Doctrine suggests retrying the transaction but it doesn't show an actual example:
https://dev.mysql.com/doc/refman/5.7/en/innodb-deadlock-example.html
try {
// process stuff
} catch (\Doctrine\DBAL\Exception\RetryableException $e) {
// retry the processing
}
This posts suggests retrying the transaction. How can I do it?
Could it be a server problem (too many accesses) and I must boost the server or the code is wrong and I must explicitly handle the deadlock in my code?
This is a MySQL issue. Multiple simultaneous transactions blocking the same resources.
Check if you have cronjobs that may block the records for long times.
Otherwise is just concurrent requests updating the same data, you may have better knowledge where this data gets updated.
Dirty attempt for a retry in php:
$retry=0;
while (true) {
try {
// some more code
$em->flush();
return new ApiResponse(ApiResponse::STATUS_SUCCESS, array(
'current_crystals' => $user->getCurrentCrystals(),
'max_crystals' => $user->getMaxCrystals(),
));
} catch (DriverException $e) {
$retry++;
if($retry>3) { throw $e; }
sleep(1); //optional
}
}
Albert's solution is the right one but you also must recreate a new EntityManager in the catch clause using resetManager() of your ManagerRegistry. You'll get exceptions if you continue to use the old EntityManager and its behavior will be unpredictable. Beware of the references to the old EntityManager too.
This issue will be hopefully corrected in Doctrine 3: See issue
Until then, here is my suggestion to handle the problem nicely: Custom EntityManager
Given I have a path to Doctrine migration classes. How could I perform the migration programmatically in Doctrine 2?
I assume there should be a clean way to perform the migration over the API as it could have be done with earlier versions of Doctrine as described here:
http://docs.doctrine-project.org/projects/doctrine1/en/latest/en/manual/migrations.html
As I don't see any answers, I'll try to provide my solution for performing migration programmatically (with code) in doctrine.
use Doctrine\DBAL\DriverManager;
use Doctrine\DBAL\Schema\Schema;
use Doctrine\Migrations\Configuration\Configuration;
use Doctrine\Migrations\Configuration\Connection\ExistingConnection;
use Doctrine\Migrations\Configuration\Migration\ExistingConfiguration;
use Doctrine\Migrations\DependencyFactory;
use Doctrine\Migrations\Metadata\Storage\TableMetadataStorageConfiguration;
use Doctrine\Migrations\MigratorConfiguration;
use Doctrine\Migrations\Provider\SchemaProvider;
use Doctrine\Migrations\Version\Direction;
// connection
$dbParams = [
'dbname' => 'database-name',
'user' => 'database-username',
'password' => 'database-password',
'host' => 'hostname',
'port' => 'port',
'driver' => 'pdo_mysql',
];
try {
$connection = DriverManager::getConnection($dbParams);
} catch (\Doctrine\DBAL\Exception $e) {
echo 'Problem connecting to DB. '.$e->getMessage();
die();
}
// configuration - Be careful what namespace you use
$configuration = new Configuration($connection);
$configuration->addMigrationsDirectory('MyAppNamespace\Migrations', __DIR__ . '/../migrations');
// we want the execution of the migration to make changes to the table doctrine_migration_versions - so the system is aware that executed the migration
$storageConfiguration = new TableMetadataStorageConfiguration();
$storageConfiguration->setTableName('doctrine_migration_versions');
$configuration->setMetadataStorageConfiguration($storageConfiguration);
$dependencyFactory = DependencyFactory::fromConnection(
new ExistingConfiguration($configuration),
new ExistingConnection($connection))
);
$planCalculator = $dependencyFactory->getMigrationPlanCalculator();
// which migration to execute / I assume latest /
$latestMigrationVersion = $dependencyFactory->getVersionAliasResolver()->resolveVersionAlias('latest');
// check if we have at least one migration version to execute
if(!$latestMigrationVersion->equals(new Version(0))){
try {
// so we will execute only latest ONE migration, if you need more, just find a way to list them in the first parameter of method getPlanForVersions()
$planUp = $planCalculator->getPlanForVersions(
[$latestMigrationVersion],
Direction::UP
);
$dependencyFactory->getMetadataStorage()->ensureInitialized();
// do the migration
$dependencyFactory->getMigrator()->migrate($planUp, (new MigratorConfiguration())->setAllOrNothing(false));
}catch (Exception $e){
echo 'There were problems during db-migration.'."\n".$e->getMessage()."\n\n";
}
}
Hope it helps another developer to quick start his prototype.
I tried to be detailed about the code, so people do not waste time into figuring out every single dependency.
Asuming you are using Symfony's DoctrineMigrationsBundle
To migrate to the latest available version use:doctrine:migrations:migrate command.
Here are more available commands.
I am creating a backup system but I got an issue with the import step. Impossible to rollback if there is any error.
I'm using this code on the CLI mode (via artisan myowncommand)
I have a file, with all my instructions:
<?php
// #generated 2016-12-11 01:05:25
use Illuminate\Support\Facades\DB;
use App\Category;
// ...
DB::beginTransaction();
Category::truncate();
// ...
// LOAD MODEL `Category` (CLASS `App\Category`)
Category::unguard();
Category::create(array (
'id' => 1,
'name' => 'Holidays',
'descr' => 'Holidays & sick & off',
'color' => 'default',
'created_at' => '2016-12-11 01:05:21',
'updated_at' => '2016-12-11 01:05:21',
));
// ...
Category::reguard();
// ...
DB::rollback();
?>
If I execute this code, it's like if no rollback happened :( If I clean my table and execute the script ... data are inserted.
According the laravel documentation, rollback instruction should be compatible:
Using the DB facade's transaction methods also controls transactions
for the query builder and Eloquent ORM (https://laravel.com/docs/5.3/database).
I also tried to use the DB::transaction() (with manual error (changed id by ida) to trigger rollback). No difference.
Do you have any explanation ? Or any way to execute correctly the rollback instruction ?
Thanks :)
Solutions I tested
Test 1
try {
DB::beginTransaction();
include $file;
/*** File is:
* use Illuminate\Support\Facades\DB;
* use App\Category;
*
* Category::truncate();
* Category::unguard();
* Category::create(array ('id' => 1, 'name' => 'Holidays'));
* // create error by changing 'id' by 'ida'
* Category::reguard();
***/
DB::commit();
echo sprintf('> Load Backup file `%s` COMPLETED', $file);
}
catch( \Exception $e ) {
DB::rollback();
echo sprintf('> Load file `%s` failled - ROLLBACK', $file);
throw $e;
}
Test 2
DB::transaction(function() {
include $file;
/*** File is:
* use Illuminate\Support\Facades\DB;
* use App\Category;
*
* Category::truncate();
* Category::unguard();
* Category::create(array ('id' => 1, 'name' => 'Holidays'));
* // create error by changing 'id' by 'ida'
* Category::reguard();
***/
});
Test 3 & 4
Same as test 1, include $file and all DB::transaction, DB::beginTransaction & DB::commit & DB:rollback directly in this $file file
Test 5
I tried to:
move the DB::beginTransaction outside the try/catch
move the DB::commit outside the try/catch
Test case
php artian migrate:refresh --seed Clear tables & populate with some masterdata
On phpmyadmin, clean (truncable) table (here: category).
There is no result on the table
Execute the code. "Load Backup failled - ROLLBACK" is executed
On phpmyadmin, check data: there are records. Model:truncable from my code executed, but not rollback
I think it should be
DB::beginTransaction();
try
{
//code for processing multiple related transactions
include $file;
}
catch(\Exception $e)
{
DB::rollBack();
//echo error message
}
DB::commit();
//echo success message
DB::beginransaction and DB::commit should be outside the try-catch block. It works for me.
And your $file should have only the instructions which you want within the try block - no DB::beginTransaction(), DB::commit() or DB::rollBack() statements.
UPDATE
The general logical steps to follow would be
Begin transaction - DB::beginTransaction()
Run database queries - crud operations - try{//run queries}
If there's an error/exception - Rollback the transaction - catch(Exception $e){ DB::rollBack()
Send/display the error/exception message - //display/send error/exception message }
If the queries run smoothly - Commit the transaction - DB::commit()
Hope this helps.
I was with the same problem, I start reading the documentation carefully and it was mysql version, must be 5.7+.
some reason a rollback often requires you to run composer dump-autoload. if your migration work.
I have a slug column in my database, it's unique.
If I try and store another row with a non unique slug I get a QueryException error.
I catch the error, and hope to return an error message something along the lines that "slug exists".
try {
User::create($data);
} catch (\Illuminate\Database\QueryException $e) {
//return the error
}
The above is fine, but I'm just wondering, what if another QueryException is thrown, not to do with the duplicate slug and i return a duplicate slug error message incorrectly.
Is there a way to find out what the query exception was and return an error message based on this? i know the exception provides its own message but I was hoping for something a little more user friendly.
To check if you got a QueryException because of a duplicate, check if $e->getCode() equals to 23000.
When a duplicate occurs due to a constraint, MySQL will issue signal SQLSTATE 23000 which you can use to determine what exactly went wrong. You can implement your own signal in MySQL to signal about various errors (via triggers etc)
Example:
try {
User::create($data);
} catch (\Illuminate\Database\QueryException $e) {
if($e->getCode() === '23000') {
// you got the duplicate
}
}
You can check errorInfo of the exception object
It looks like this..
+errorInfo: array:3 [▼
0 => "23000"
1 => 1062
2 => "Duplicate entry 'abc#gmail.com' for key 'users_email_unique'"
You can write following code in catch..
try{}
catch(QueryException $qe){
If ($qe->errorInfo[0] == "23000" && $qe->errorInfo[1] == "1062"){
return "Your message"
}else{
return "General Message"
}
}
I have a custom Drupal 7 module with an 'edit' page.
The form fields reference a couple of database tables, so to process the form, we attempt to update the first table, and we try to set a '$error' to 'true' and check against $error before we attempt to update the next table. For example:
HTML:
<input name="field1" />
<input name="field2" />
PHP:
$error = false;
$update_table_1 = db_update('table1')
->fields(array(
'field1' => $_POST['field1'],
))
->condition('id', $id)
->execute();
if(!update_table_1) {
$error = true;
}
if(!$error) {
$update_table_2 = db_update('table2')
->fields(array(
'field2' => $_POST['field2'],
))
->condition('id', $id)
->execute();
if(!$update_table_2) {
$error = true;
}
}
Problem: If only updating something in table 2, it will throw an error before it event gets to update table 2 because db_query says that it is not true since the field was the same as what was in the database (no change). Really, I only want to stop it if there was a database / code error.
Does the Drupal 7 db_update API have some kind of error reporting function like mysql_error()? Other suggestions?
The safest way you can do it is with a transaction and proper PHP error checking:
$transaction = db_transaction();
try {
// Query 1
db_update(...);
// Query 2
db_update(...);
}
catch (Exception $e) {
// Rollback the transaction
$transaction->rollback();
// Do something with the exception (inform user, etc)
}
I should mention the transaction is only necessary if you don't want the changes from the first query to persist if the second query fails. It's quite a common requirement but might not fit your use case.