I'm trying to test a class MyConnection I've created that extends Doctrine\DBAL\Connection and handles deadlock cases (or lock waits) by releasing any deadlocks and resolving them.
Therefore I need to create a deadlock on my test, in order to confirm that it works as it should.
The idea is to create a temporary table in the database, and try to access it in two connections at once.
The challenge is that I need to do this without using multiple threads. (threads in php are created with pthreads that needs PHP7.2+ and I have 7.1, and it's not an option to upgrade for now)
What I've already tried is using multiple connections on the database. However this created a lock wait instead. The problems with the lock wait were that:
1. it makes the integration test too slow
2. I can't delete the temporary table I create on my test. Even though the test runs on a test database, it's good practice to clean up after your tests
I have also tried forking my process. The problems with that were
1. it still creates a lock wait instead
2. I had to exit the child process, and have the parent handling everything and making assertions. But since the deadlock happens in the child, I can't catch the errors thrown from the child through the parent
The idea in the code below is that when my executeQuery fails to handle the dead lock, it won't be able to write 'I am handling the deadlock or the lock wait' in the logger, therefore I put that shouldReceive('critical') that will throw an error when things don't work out. Otherwise the test should pass
Class extending Doctrine\DBAL\Connection
class MyConnection extends Connection
{
public function executeQuery($query, array $params = [], $types = [], QueryCacheProfile $qcp = null)
{
try {
$result = parent::executeQuery($query, $params, $types, $qcp);
} catch (\Doctrine\DBAL\DBALException $exception ) {
$this->logger->critical('I am handling the deadlock or the lock wait');
}
}
}
Integration Test
class DeadlockCreationTest extends Test
{
const SERIALIZE_TRANSACTION = 'SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;';
const START_TRANSACTION = 'START TRANSACTION;';
const SELECT_QUERY = 'SELECT * FROM %s.innodb_deadlock_maker WHERE a = %d;';
const UPDATE_QUERY = 'UPDATE %s.innodb_deadlock_maker SET a=%d WHERE a <> %d';
public function setUp()
{
parent::setUp();
$this->currentDbConnection = $this->getConnection();
$this->newDbConnection = $this->getNewIdenticalDBConnection($this->currentDbConnection);
$this->logger = Mockery::mock(Logger::class);
$this->currentDbConnection->executeQuery(
'CREATE TABLE IF NOT EXISTS ' . $this->dbName .
'.innodb_deadlock_maker(b INT PRIMARY KEY AUTO_INCREMENT, a INT NOT NULL) engine=innodb;'
);
$this->currentDbConnection->executeQuery(
'insert into ' . $this->dbName . '.innodb_deadlock_maker(a) values(0), (1);'
);
}
public function testCreateDeadlock()
{
$pid = pcntl_fork();
if ($pid === -1) {
$this->fail('Could not fork child process');
}
if ($pid === 0) {
$this->process = 'child';
// In child process
try {
$this->logger->shouldReceive('critical')
->with(
'I am handling the deadlock or the lock wait'
);
$this->newDbConnection->setLogger($this->logger);
$this->newDbConnection->executeQuery(self::SERIALIZE_TRANSACTION);
$this->newDbConnection->executeQuery(self::START_TRANSACTION);
$this->newDbConnection->executeQuery(sprintf(self::SELECT_QUERY, $this->dbName, 1));
$this->newDbConnection->executeUpdate(sprintf(self::UPDATE_QUERY, $this->dbName, 1, 1));
} catch (\Doctrine\DBAL\DBALException $exception) {
if (stripos($exception->getMessage(), 'try restarting transaction') !== false) {
exit(123);
}
exit(100); // Our exit code to indicate a different error occurred
}
exit(0); // Our exit code to indicate no error occurred
}
$this->process = 'parent';
// Otherwise in parent process:
$this->currentDbConnection->executeQuery(self::SERIALIZE_TRANSACTION);
$this->currentDbConnection->executeQuery(self::START_TRANSACTION);
$this->currentDbConnection->executeQuery(sprintf(self::SELECT_QUERY, $this->dbName, 0));
$this->currentDbConnection->executeUpdate(sprintf(self::UPDATE_QUERY, $this->dbName, 0, 0));
pcntl_waitpid($pid, $status); // Wait for forked child to exit
$childExitCode = pcntl_wexitstatus($status);
switch ($childExitCode) {
case 0:
echo "\nNo error occurred, or deadlock occurred in child but was handled correctly - success!\n";
break;
case 100:
$this->fail('A different error occurred');
break;
case 123:
$this->fail('Deadlock occurred in child and was not handled correctly');
break;
default:
$this->fail('Child exited with unexpected exit code');
}
}
public function tearDown()
{
$this->newDbConnection->executeQuery('DROP TABLE ' . $this->dbName . '.innodb_deadlock_maker;');
$this->newDbConnection->close();
parent::tearDown();
}
}
What I need is ideally to create a deadlock instead of a lock wait.
Also I need to somehow confirm that my Connection class worked as it should.
Another issue I have with the code above is that I get the error message PDO::exec(): MySQL server has gone away and I don't know if and how can I stop that.
PS: The idea of the test is based on this article
Related
#PHP, #Doctrine, #Symfony
I had a problem with the transaction and would be happy if I found a solution how it makes it run properly.
I had a transaction which send data to API and if an exception was thrown It rollback created data from DB and flushed exception message after rollback to DB. If everything was OK I did commit.
It was called in 2 locations
in cron module (it was always ran first)
in controller (if error was thrown, admin could resend it)
In the cron module everything worked fine but if I wanted to rerun the transaction because it failed in the cron I got error with
"Transaction commit failed because the transaction has been marked for rollback only."
I debug it and found that after rollback
I had $this->transactionNestingLevel set to 1
but $this->isRollbackOnly was set on TRUE
so that's why that exception was thrown. Is something it needs to be added above?
$this->em->beginTransaction();
$this->em->commit();
$this->em->rollback();
access to $this->isRollbackOnly is private. Need to be false I guess but don't know how I could make it :(
/**
* Cancels any database changes done during the current transaction.
*
* #return bool
*
* #throws ConnectionException If the rollback operation failed.
*/
public function rollBack()
{
if ($this->transactionNestingLevel === 0) {
throw ConnectionException::noActiveTransaction();
}
$connection = $this->getWrappedConnection();
$logger = $this->_config->getSQLLogger();
if ($this->transactionNestingLevel === 1) {
if ($logger) {
$logger->startQuery('"ROLLBACK"');
}
$this->transactionNestingLevel = 0;
$connection->rollBack();
$this->isRollbackOnly = false;
if ($logger) {
$logger->stopQuery();
}
if ($this->autoCommit === false) {
$this->beginTransaction();
}
} elseif ($this->nestTransactionsWithSavepoints) {
if ($logger) {
$logger->startQuery('"ROLLBACK TO SAVEPOINT"');
}
$this->rollbackSavepoint($this->_getNestedTransactionSavePointName());
--$this->transactionNestingLevel;
if ($logger) {
$logger->stopQuery();
}
} else {
$this->isRollbackOnly = true;
--$this->transactionNestingLevel;
}
return true;
}
doctrine rollback code
I am using PHP 7.2 on a website hosted on Amazon. I have a code similar to this one that writes a record in the MongoDB:
Database connection class:
class Database {
private static $instance;
private $managerMongoDB;
private function __construct() {
#Singleton private constructor
}
public static function getInstance() {
if (!self::$instance) {
self::$instance = new Database();
}
return self::$instance;
}
function writeMongo($collection, $record) {
if (empty($this->managerMongoDB)) {
$this->managerMongoDB = new MongoDB\Driver\Manager(DB_MONGO_HOST ? DB_MONGO_HOST : null);
}
$writeConcern = new MongoDB\Driver\WriteConcern(MongoDB\Driver\WriteConcern::MAJORITY, 1000);
$bulk = new MongoDB\Driver\BulkWrite();
$bulk->insert($record);
try {
$result = $this->managerMongoDB->executeBulkWrite(
DB_MONGO_NAME . '.' . $collection, $bulk, $writeConcern
);
} catch (MongoDB\Driver\Exception\BulkWriteException $e) {
// Not important
} catch (MongoDB\Driver\Exception\Exception $e) {
// Not important
}
return $result->getInsertedCount() > 0;
}
}
Execution:
Database::getInstance()->writeMongo($tableName, $dataForMongo);
The script is working as intended and the records are added in MongoDB.
The problem is that connections are not being closed at all and once there are 500 inserts (500 is the limit of connections in MongoDB on our server) it stops working. If we restart php-fpm the connections are also reset and we can insert 500 more records.
The connection is reused during the request, but we have requests coming from 100s of actual customers.
As far as I can see there is no way to manually close the connections. Is there something I'm doing wrong? Is there some configuration that needs to be done on the driver? I tried setting socketTimeoutMS=1000&wTimeoutMS=1000&connectTimeoutMS=1000 in the connection string but the connections keep staying alive.
You are creating a client instance every time the function is invoked, and never closing it, which would produce the behavior you are seeing.
If you want to create the client instance in the function, close it in the same function.
Alternatively create the client instance once for the entire script and use the same instance in all of the operations done by that script.
So the following code is in its infantile stages, and has a long way to go - but I am working on a database platform that is aware of multiple databases ( slaves, etc ), that is meant to attempt to open a connection to one, and upon failure, continue on to another. The relevant methods in the 'DatabaseConnection' class are as follows:
class DatabaseConnection extends mysqli
{
public function __construct($ConnectAutomatically=false,$Name="database",$Host="127.0.0.1",$Username="root",$Password="",$Catalog="",$Port=3306)
{
$this->CurrentHost = $_SERVER['SERVER_NAME'];
$this->Name = $Name;
$this->Host = $Host;
$this->Username = $Username;
$this->Password = $Password;
$this->Port = $Port;
$this->Catalog = $Catalog;
if( $ConnectAutomatically )
parent::__construct($this->Host, $this->Username, $this->Password, $this->Catalog, $this->Port);
else
parent::init();
return;
}
public function Open()
{
try
{
if(!parent::real_connect($this->Host,$this->Username,$this->Password,$this->Catalog,$this->Port))
{
return false;
}
else
return true;
}
catch( Exception $e )
{
return false;
}
}
}
The idea is that the database could be queried at any time, without having been initialized - therefore, the Query method checks for connectivity, and upon a lack thereof, runs the 'Open' method.
I have intentionally put a bad character in the password to force it to fail to connect, to see if it would connect to the second server in the list - but on the 'parent::real_connect' line, I am getting the obvious response that I cannot connect due to an invalid password.
I am wondering if there is a different approach I can take in this 'Open' method to NOT stop execution upon failure in the real_connect operation. I've taken a few swings at this and come up short, so I had to ask. It seems from my perspective that mysqli won't allow program execution to continue if the connect fails...
I have custom error catching already in place, which seems to catch the fault - but without interpreting the error and re-executing the DatabaseHandler connect method with the next in the list from Within my error handler, I don't see many other approaches at the moment.
I'll be glad to share more if need be.
I recently upgraded from php 5.4.26 to 5.4.28 after the upgrade I am getting this error
Notice: Unknown: send of 6 bytes failed with errno=32 Broken pipe in Unknown on line 0
When ever I run the following code:
<?php
$tasks = array(
'1' => array(),
'2' => array(),
);
ini_set('display_errors', true);
class RedisClass {
private $redis;
public function __construct()
{
$this->redis = new Redis();
$this->redis->connect('localhost', 6379);
}
}
$redis = new RedisClass();
foreach ($tasks as $index => $task)
{
$pid = pcntl_fork();
// This is a child
if($pid == 0)
{
echo "Running ".$index." child in ". getmypid() ."\n";
break;
}
}
switch($pid)
{
case -1 :
die('could not fork');
break;
case 0:
// do the child code
break;
default:
while (pcntl_waitpid(0, $status) != -1)
{
$status = pcntl_wexitstatus($status);
echo "Child completed with status $status\n";
}
echo "Child Done (says: ". getmypid() .")";
exit;
}
If I only fork one child then I do not get the PHP Notice. If I run any more than 1 child I get the PHP Notice for every child except the first child.
Does anyone have any clues as to what is going on here?
I am assuming it is trying to close the Redis connection multiple times but this is code I have been running for at least 4 months with out any issues.
It only starting displaying these notices after the upgrade to 5.4.28.
I have looked at the PHP change logs but I cannot see anything that I think may explain this issue.
Should I report this to PHP as a bug?
UPDATE:
Looks like it "may" be a Redis issue, I am using phpredis I tested the same code with a mysql connection instead of loading Redis and I do not get the error.
class MysqlClass {
private $mysqli;
public function __construct()
{
$this->mysqli = mysqli_init(); //This is not the droid you are looking for
$this->mysqli->real_connect('IP_ADDRESS',
'USER_NAME',
'PASSWORD');
}
}
$mysql = new MysqlClass();
The problem here is that you do not reconnect Redis in the child process. Like Michael had said, you do not have an active connection from the second child onwards. That mysql example should not work if you also make some queries.
I have had the exact problematic behaviour with the "MySQL server has gone away" error and also with Redis.
The solution is to create a new connection to MySQL and to Redis in the child. Make sure if you have a singletone that handles the MySQL/Redis connection to reset the instances ( that was also a problem for me ).
I can't get my head around WHEN to throw and catch exceptions for when I call functions from classes.
Please imagine that my QuizMaker class looks like this:
// Define exceptions for use in class
private class CreateQuizException extends Exception {}
private class InsertQuizException extends Exception {}
class QuizMaker()
{
// Define the items in my quiz object
$quiz_id = null;
$quiz_name = null;
// Function to create a quiz record in the database
function CreateQuizInDatabase()
{
try
{
$create_quiz = // Code to insert quiz
if (!$create_quiz)
{
// There was an error creating record in the database
throw new CreateQuizException("Failed inserting record");
}
else
{
// Return true, the quiz was created successfully
return true;
}
}
catch (CreateQuizException $create_quiz_exception)
{
// There was an error creating the quiz in the database
return false;
}
}
function InsertQuestions()
{
try
{
$insert_quiz = // Code to insert quiz
if (!$insert_quiz)
{
// There was an error creating record in the database
throw new CreateQuizException("Failed inserting quiz in to database");
}
else
{
// Success inserting quiz, return true
return true;
}
}
catch (InsertQuizException $insert_exception)
{
// Error inserting question
return false;
}
}
}
... and using this code, I use the class to create a new quiz in the database
class QuizMakerException extends Exception {}
try
{
// Create a blank new quiz maker object
$quiz_object = new QuizMaker();
// Set the quiz non-question variables
$quiz_object->quiz_name = $_POST['quiz_name'];
$quiz_object->quiz_intro = $_POST['quiz_intro'];
//... and so on ...
// Create the quiz record in the database if it is not already set
$quiz_object->CreateQuizRecord();
// Insert the quiz in to the database
$quiz_object->InsertQuestions();
}
catch (QuizMakerException $quiz_maker_error)
{
// I want to handle any errors from these functions here
}
For this piece of code, I want to call a QuizMakerException if any of the functions don't perform what I want them to (at the moment they return TRUE or FALSE).
What is the correct way to go about catching when any of the functions in this code does not perform what I want them to? At the moment they simply return TRUE or FALSE.
Do I really have to put lots of if/else statements between calling each function, I thought that was the whole point in exceptions, they simply halt the execution of further statements within the try/catch?
Do I throw a QuizMakerException from within the catch of my functions?
What is the right thing to do?
Help!
Well typically in the function which throws the exception, say your InsertQuestions method, you don't want to catch any exceptions, you want to throw them or let ones occurring to "bubble up". Then your "controller" code can make the determination of how to handle the exception.
If your goal here is to halt if CreateQuizRecord fails I would wrap CreateQuizRecord and InsertQuestions each in their own try block.
One advantage of exceptions is they can tell you more than a simple bool pass/fail. Either extending your base exception into things like "Invalid_Parameter" and testing for specific exceptions or -less ideally- inferring from properties of the exception. You can nest your catch blocks to handle exceptions individually.
Do I throw a QuizMakerException from within the catch of my functions?
Yes. Typically your code under // Code to insert quiz would itself return an exception. Say if the Model failed to insert it might be raising a database exception. In which case you can let that database exception bubble up, or do what you sort of doing now and catch it simply to in turn throw another exception (kinda dumbs down your exceptions in a way, doing that though).
Do I really have to put lots of if/else statements between calling each function, I thought that was the whole point in exceptions, they simply halt the execution of further statements within the try/catch?
I look at it like this, each call which throws an exception and is followed by a subsequent call which depends on this one not throwing any exceptions, should be wrapped in a try block. Assuming you want to handle it gracefully, if you simply want it to error out and halt just don't handle the exception. You'll get an error and stack trace. Sometimes that is desirable.
You might want to change the structure a bit. Your class QuizMaker can become:
<?php
// Define exceptions for use in class
public class CreateQuizException extends Exception {}
public class InsertQuizException extends Exception {}
class QuizMaker()
{
// Define the items in my quiz object
$quiz_id = null;
$quiz_name = null;
// Function to create a quiz record in the database
function CreateQuizInDatabase()
{
try
{
$create_quiz = // Code to insert quiz
if (!$create_quiz)
{
// There was an error creating record in the database
throw new CreateQuizException("Failed inserting record");
}
else
{
// Return true, the quiz was created successfully
return true;
}
}
catch (Exception $create_quiz_exception)
{
// There was an error creating the quiz in the database
throw new CreateQuizException(
"Failed inserting record " .
$create_quiz_exception->getMessage()
);
}
}
function InsertQuestions()
{
try
{
$insert_quiz = // Code to insert quiz
if (!$insert_quiz)
{
// There was an error creating record in the database
throw new InsertQuizException("Failed inserting quiz in to database");
}
else
{
// Success inserting quiz, return true
return true;
}
}
catch (Exception $insert_exception)
{
// Error inserting question
throw new InsertQuizException(
"Failed inserting quiz in to database " .
$create_quiz_exception->getMessage()
);
}
}
}
Effectively, if you cannot insert the record correctly, you throw an Insert exception. If anything goes wrong with that block of code, you catch it and throw again an Insert exception. Same goes with the Create function (or any other ones you might have).
In the main block of code you can have:
try
{
// Create a blank new quiz maker object
$quiz_object = new QuizMaker();
// Set the quiz non-question variables
$quiz_object->quiz_name = $_POST['quiz_name'];
$quiz_object->quiz_intro = $_POST['quiz_intro'];
//... and so on ...
// Create the quiz record in the database if it is not already set
$quiz_object->CreateQuizRecord();
// Insert the quiz in to the database
$quiz_object->InsertQuestions();
}
catch (InsertQuizException $insert_exception)
{
// Insert error
}
catch (CreateQuizException $create_quiz_exception)
{
// Create error
}
catch (Exception $quiz_maker_error)
{
// Any other error
}
If you don't want to have the multiple catch block there, just keep the catch(Exception) one and then check in it the type of each exception thrown to specify the actions taken from there.
HTH
The best thing to do is to not have to thhrow exceptions in the first place.
Exceptions are thrown when the program crashes and they are not made to handle wrong output.
If your function returns anything (even its the wrong thing) it doesn't need an exception.
To answer your question, if it's necessaryto use a lot of ifs/elses then you must use them.