I am creating a backup system but I got an issue with the import step. Impossible to rollback if there is any error.
I'm using this code on the CLI mode (via artisan myowncommand)
I have a file, with all my instructions:
<?php
// #generated 2016-12-11 01:05:25
use Illuminate\Support\Facades\DB;
use App\Category;
// ...
DB::beginTransaction();
Category::truncate();
// ...
// LOAD MODEL `Category` (CLASS `App\Category`)
Category::unguard();
Category::create(array (
'id' => 1,
'name' => 'Holidays',
'descr' => 'Holidays & sick & off',
'color' => 'default',
'created_at' => '2016-12-11 01:05:21',
'updated_at' => '2016-12-11 01:05:21',
));
// ...
Category::reguard();
// ...
DB::rollback();
?>
If I execute this code, it's like if no rollback happened :( If I clean my table and execute the script ... data are inserted.
According the laravel documentation, rollback instruction should be compatible:
Using the DB facade's transaction methods also controls transactions
for the query builder and Eloquent ORM (https://laravel.com/docs/5.3/database).
I also tried to use the DB::transaction() (with manual error (changed id by ida) to trigger rollback). No difference.
Do you have any explanation ? Or any way to execute correctly the rollback instruction ?
Thanks :)
Solutions I tested
Test 1
try {
DB::beginTransaction();
include $file;
/*** File is:
* use Illuminate\Support\Facades\DB;
* use App\Category;
*
* Category::truncate();
* Category::unguard();
* Category::create(array ('id' => 1, 'name' => 'Holidays'));
* // create error by changing 'id' by 'ida'
* Category::reguard();
***/
DB::commit();
echo sprintf('> Load Backup file `%s` COMPLETED', $file);
}
catch( \Exception $e ) {
DB::rollback();
echo sprintf('> Load file `%s` failled - ROLLBACK', $file);
throw $e;
}
Test 2
DB::transaction(function() {
include $file;
/*** File is:
* use Illuminate\Support\Facades\DB;
* use App\Category;
*
* Category::truncate();
* Category::unguard();
* Category::create(array ('id' => 1, 'name' => 'Holidays'));
* // create error by changing 'id' by 'ida'
* Category::reguard();
***/
});
Test 3 & 4
Same as test 1, include $file and all DB::transaction, DB::beginTransaction & DB::commit & DB:rollback directly in this $file file
Test 5
I tried to:
move the DB::beginTransaction outside the try/catch
move the DB::commit outside the try/catch
Test case
php artian migrate:refresh --seed Clear tables & populate with some masterdata
On phpmyadmin, clean (truncable) table (here: category).
There is no result on the table
Execute the code. "Load Backup failled - ROLLBACK" is executed
On phpmyadmin, check data: there are records. Model:truncable from my code executed, but not rollback
I think it should be
DB::beginTransaction();
try
{
//code for processing multiple related transactions
include $file;
}
catch(\Exception $e)
{
DB::rollBack();
//echo error message
}
DB::commit();
//echo success message
DB::beginransaction and DB::commit should be outside the try-catch block. It works for me.
And your $file should have only the instructions which you want within the try block - no DB::beginTransaction(), DB::commit() or DB::rollBack() statements.
UPDATE
The general logical steps to follow would be
Begin transaction - DB::beginTransaction()
Run database queries - crud operations - try{//run queries}
If there's an error/exception - Rollback the transaction - catch(Exception $e){ DB::rollBack()
Send/display the error/exception message - //display/send error/exception message }
If the queries run smoothly - Commit the transaction - DB::commit()
Hope this helps.
I was with the same problem, I start reading the documentation carefully and it was mysql version, must be 5.7+.
some reason a rollback often requires you to run composer dump-autoload. if your migration work.
Related
I am developing a bidding application in laravel. In this, I have to run cronjob after each minute and user can also bid from front end. So to avoid collision I am using lockForUpdate() of laravel. I have placed this in two different functions where bid ids are being processed. And In one place I am using it three times and at other its is being used one time. I have placed my code inside DB::transaction. But due to some reason I am getting deadlock error whenever same raw is processed by two users at same time.
I am using this as given below:
At One Place
DB::beginTransaction();
try
{
-----Some Code ----
SecondGameBids::where('id', $big)->lockForUpdate()->get();
SecondGameBids::where('id', $big)->update(['final_value' =>0, 'deal_status' => 1]);
-----Some Code ------
SecondGameBids::where('id', $small)->lockForUpdate()->get();
SecondGameBids::where('id', $small)->update(['final_value' =>0, 'deal_status' => 1]);
------Some Code ------
DB::commit();
}
catch (\Exception $e) {
DB::rollback();
}
At Other Place
DB::beginTransaction();
try
{
-----Some Code ----
SecondGameBids::where('id', $big)->lockForUpdate()->get();
SecondGameBids::where('id', $big)->update(['status' => 3]);
------Some Code ------
DB::commit();
}
catch (\Exception $e) {
DB::rollback();
}
Can anyone give me some idea about how to overcome this error?
I have a Symfony app which exposes a collection of JSON web services used by a mobile app.
On the last few days we are having many concurrent users using the app (~5000 accesses per day) and a Doctrine error started to "randomly" appear in my logs. It appears about 2-3 times per day and this is the error:
Uncaught PHP Exception Doctrine\DBAL\Exception\DriverException: "An exception occurred while executing 'UPDATE fos_user_user SET current_crystals = ?, max_crystals = ?, updated_at = ? WHERE id = ?' with params [31, 34, "2017-12-19 09:31:18", 807]:
SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction" at /var/www/html/rollinz_cms/releases/98/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php line 115
It seems it cannot get the lock while updating the users table. The controller code is the following:
/**
* #Rest\Post("/api/badges/{id}/achieve", name="api_achieve_badge")
*/
public function achieveAction(Badge $badge = null)
{
if (!$badge) {
throw new NotFoundHttpException('Badge not found.');
}
$user = $this->getUser();
$em = $this->getDoctrine()->getManager();
$userBadge = $em->getRepository('AppBundle:UserBadge')->findBy(array(
'user' => $user,
'badge' => $badge,
));
if ($userBadge) {
throw new BadRequestHttpException('Badge already achieved.');
}
$userBadge = new UserBadge();
$userBadge
->setUser($user)
->setBadge($badge)
->setAchievedAt(new \DateTime())
;
$em->persist($userBadge);
// sets the rewards
$user->addCrystals($badge->getCrystals());
$em->flush();
return new ApiResponse(ApiResponse::STATUS_SUCCESS, array(
'current_crystals' => $user->getCurrentCrystals(),
'max_crystals' => $user->getMaxCrystals(),
));
}
I looked into MySQL and Doctrine documentation but I couldn't find a reliable solution. Doctrine suggests retrying the transaction but it doesn't show an actual example:
https://dev.mysql.com/doc/refman/5.7/en/innodb-deadlock-example.html
try {
// process stuff
} catch (\Doctrine\DBAL\Exception\RetryableException $e) {
// retry the processing
}
This posts suggests retrying the transaction. How can I do it?
Could it be a server problem (too many accesses) and I must boost the server or the code is wrong and I must explicitly handle the deadlock in my code?
This is a MySQL issue. Multiple simultaneous transactions blocking the same resources.
Check if you have cronjobs that may block the records for long times.
Otherwise is just concurrent requests updating the same data, you may have better knowledge where this data gets updated.
Dirty attempt for a retry in php:
$retry=0;
while (true) {
try {
// some more code
$em->flush();
return new ApiResponse(ApiResponse::STATUS_SUCCESS, array(
'current_crystals' => $user->getCurrentCrystals(),
'max_crystals' => $user->getMaxCrystals(),
));
} catch (DriverException $e) {
$retry++;
if($retry>3) { throw $e; }
sleep(1); //optional
}
}
Albert's solution is the right one but you also must recreate a new EntityManager in the catch clause using resetManager() of your ManagerRegistry. You'll get exceptions if you continue to use the old EntityManager and its behavior will be unpredictable. Beware of the references to the old EntityManager too.
This issue will be hopefully corrected in Doctrine 3: See issue
Until then, here is my suggestion to handle the problem nicely: Custom EntityManager
Given I have a path to Doctrine migration classes. How could I perform the migration programmatically in Doctrine 2?
I assume there should be a clean way to perform the migration over the API as it could have be done with earlier versions of Doctrine as described here:
http://docs.doctrine-project.org/projects/doctrine1/en/latest/en/manual/migrations.html
As I don't see any answers, I'll try to provide my solution for performing migration programmatically (with code) in doctrine.
use Doctrine\DBAL\DriverManager;
use Doctrine\DBAL\Schema\Schema;
use Doctrine\Migrations\Configuration\Configuration;
use Doctrine\Migrations\Configuration\Connection\ExistingConnection;
use Doctrine\Migrations\Configuration\Migration\ExistingConfiguration;
use Doctrine\Migrations\DependencyFactory;
use Doctrine\Migrations\Metadata\Storage\TableMetadataStorageConfiguration;
use Doctrine\Migrations\MigratorConfiguration;
use Doctrine\Migrations\Provider\SchemaProvider;
use Doctrine\Migrations\Version\Direction;
// connection
$dbParams = [
'dbname' => 'database-name',
'user' => 'database-username',
'password' => 'database-password',
'host' => 'hostname',
'port' => 'port',
'driver' => 'pdo_mysql',
];
try {
$connection = DriverManager::getConnection($dbParams);
} catch (\Doctrine\DBAL\Exception $e) {
echo 'Problem connecting to DB. '.$e->getMessage();
die();
}
// configuration - Be careful what namespace you use
$configuration = new Configuration($connection);
$configuration->addMigrationsDirectory('MyAppNamespace\Migrations', __DIR__ . '/../migrations');
// we want the execution of the migration to make changes to the table doctrine_migration_versions - so the system is aware that executed the migration
$storageConfiguration = new TableMetadataStorageConfiguration();
$storageConfiguration->setTableName('doctrine_migration_versions');
$configuration->setMetadataStorageConfiguration($storageConfiguration);
$dependencyFactory = DependencyFactory::fromConnection(
new ExistingConfiguration($configuration),
new ExistingConnection($connection))
);
$planCalculator = $dependencyFactory->getMigrationPlanCalculator();
// which migration to execute / I assume latest /
$latestMigrationVersion = $dependencyFactory->getVersionAliasResolver()->resolveVersionAlias('latest');
// check if we have at least one migration version to execute
if(!$latestMigrationVersion->equals(new Version(0))){
try {
// so we will execute only latest ONE migration, if you need more, just find a way to list them in the first parameter of method getPlanForVersions()
$planUp = $planCalculator->getPlanForVersions(
[$latestMigrationVersion],
Direction::UP
);
$dependencyFactory->getMetadataStorage()->ensureInitialized();
// do the migration
$dependencyFactory->getMigrator()->migrate($planUp, (new MigratorConfiguration())->setAllOrNothing(false));
}catch (Exception $e){
echo 'There were problems during db-migration.'."\n".$e->getMessage()."\n\n";
}
}
Hope it helps another developer to quick start his prototype.
I tried to be detailed about the code, so people do not waste time into figuring out every single dependency.
Asuming you are using Symfony's DoctrineMigrationsBundle
To migrate to the latest available version use:doctrine:migrations:migrate command.
Here are more available commands.
In order to manage concurrency - that is ensuring that data being saved to the database is not stale or already edited by some other user - in my CakePHP application I am using the modified attribute in my edit functions. Below is a snippet of the code that is in my controller.
$this->MyModel->recursive = -1;
$event = $this->MyModel->findById($id);
$requestTimeStamp = new DateTime($this->request->data['MyModel']['modified']);
$dbTimeStamp = new DateTime($event['MyModel']['modified']);
if ($requestTimeStamp < $dbTimeStamp) {
$response = array(
'success' => false,
'id' => $id,
'message' => 'A concurrency error occurred while trying to save. Please try again');
echo json_encode($response);
exit;
} else {
//... continue processing
}
This code works fine - but as I try to optimize it across my application I am trying to figure out where best to place it. Is it best placed in my AppModel class or is it better to create a Behavior for the same or is it just best left in the controller? I suppose that an ideal option would consider performance and minimize the amount of class loading overhead as well as database access overhead.
Has anyone come across / solved this problem before? Thoughts / suggestions appreciated.
So I solved this by making concurrency check a part of my AppModel->beforeSave() method. Below is the code for reference of others
/*
* Incorporated concurrency check in the beforeSave callback method to ensure that data is not stale before user saves.
* The function checks if the model has a `modified` field, before it proceeds. If the model does not have such a method
* then concurrency does not apply to this data structure. Upon proceeding, the method checks to see if the value of modified
* data is the same in the database as well as the request that invokes this method. If they are not same then the save is
* aborted
* This method requires the view or controller to pass a variable called data[ModelName][modified].
* This variable must contain the value of the modified field when the record was read and it must be passed back as such.
* I usually set a hidden form field in my view like below -
* <input type="hidden" name="data[Model][modified]" value="<?php echo $model['modifed']; ?>" />
*/
public function beforeSave($options = array()) {
if ($this->hasField('modified') && isset($this->data[$this->name]['id']) && isset($this->data[$this->name]['modified'])) {
CakeLog::debug('AppModel: beforeSave - inside concurrency check');
CakeLog::debug($this->data);
$this->recursive = -1;
// run a select statement to ensure the modified date in the database has not changed. If it has changed then
// the below find query will return 0 rows
$row = $this->find('first', array(
'fields' => array(
'id', 'modified'
),
'conditions' => array(
'id' => $this->data[$this->name]['id'],
'modified' => $this->data[$this->name]['modified']
)
));
// if no row is returned then error out and return - basically a concurrency error has occurred
if (!$row) {
CakeLog::error($this->name.':Concurrency error - [row-id:'.$this->data[$this->name]['id'].']');
return false;
}
// if a row was retrned then there is no concurrency error, so proceed but change the modified date
// to current timestamp to reflect accuracy
$this->data[$this->name]['modified'] = date('Y-m-d H:i:s');
return true;
}
}
I am trying to run some server-side JS in Mongo. Operations I am trying to perform are:
db.dropDatabase(); // removing current database
db.copyDatabase('db_dump', 'db', 'localhost'); // substituting it with a dump
Everything works perfectly nice. When I am storing this as a function:
function () {
db.dropDatabase();
return db.copyDatabase('db_dump', 'db', 'localhost');
}
and executing it, everything is also nice and returns me {"ok" : 1}
But when I try to execute this using php driver:
$db->execute("function(){ db.dropDatabase(); return db.copyDatabase('db_dump', 'db', 'localhost'); }");
I get
{
"retval": {
"errmsg":"exception: can't temprelease nested lock",
"code":10298,
"ok":0
},
"ok":1
}
My first though was that I just need to get out of the lock, so I tried this
$db->command(
array(
'$eval' => "function() { db.dropDatabase(); return db.copyDatabase('db_dump', 'db', 'localhost');}"
),
array(
'nolock'=> true
)
);
Nothing else is using database at this point.
Any ideas how to get rid of this error?
I am using Mongo 2.4.4, PHP 5.3.13 and driver 1.2.10
P.S. Tried this on PHP 5.4.16 and the situation is the same
This command cannot be invoked through the eval command (see this thread for some additional discussion on the matter). Rather than use JS methods, you can invoke the copydb command directly with MongoDB::command(). The following script will copy between two databases on the same server, since I omitted the fromhost option:
$m = new MongoClient();
$m->a->drop();
$m->admin->command([
'copydb' => 1,
'fromdb' => 'b',
'todb' => 'a',
]);