I've seen many posts explaining the usage of Select FOR UPDATE and how to lock a row, however I haven't been able to find any that explain what occurs when the code tries to read a row that's locked.
For instance. Say I use the following:
$con->autocommit(FALSE);
$ps = $con->prepare( "SELECT 1 FROM event WHERE row_id = 100 FOR UPDATE");
$ps->execute();
...
//do something if lock successful
...
$mysqli->commit();
In this case, how do I determine if my lock was successful? What is the best way to handle a scenario when the row is locked already?
Sorry if this is described somewhere, but all I seem to find are the 'happy path' explanations out there.
In this case, how do I determine if my lock was successful? What is the best way to handle a scenario when the row is locked already?
If the row you are trying to lock is already locked - the mysql server will not return any response for this row. It will wait², until the locking transaction is either commited or rolled back.
(Obviously: if the row has been deleted already, your SELECT will return an empty result set and not lock anything)
After that, it will return the latest value, commited by the transaction that was holding the lock.
Regular Select Statements will not care about the lock and return the current value, ignoring that there is a uncommited change.
So, in other words: your code will only be executed WHEN the lock is successfull. (Otherwhise waiting² until the prior lock is released)
Note, that using FOR UPDATE will also block any transactional SELECTS for the time beeing locked - If you do not want this, you should use LOCK IN SHARE MODE instead. This would allow transactional selects to proceed with the current value, while just blocking any update or delete statement.
² the query will return an error, after the time defined with innodb_lock_wait_timeout http://dev.mysql.com/doc/refman/5.5/en/innodb-parameters.html#sysvar_innodb_lock_wait_timeout
It then will return ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
In other words: That's the point where your attempt to acquire a lock fails.
Sidenode: This kind of locking is just suitable to ensure data-integrity. (I.e. that no referenced row is deleted while you are inserting something that references this row).
Once the lock is released any blocked (or better call it delayed) delete statement will be executed, maybe deleting the row you just inserted due to Cascading on the row on which you just held the lock to ensure integrity.
If you want to create a system to avoid 2 users modifying the same data at the same time, you should do this at an application level and look at pessimistic vs optimistic locking approches, because it is no good idea to keep transactions running for a long period of time. (I think in PHP your database connections are automatically closed after each request anyway, causing an implicit commit on any running transaction)
Related
Im trying to find a solution to a MariaDB race condition.
There are some cases where multiple process get executed almost at the same time (with the same timestamp that comes from some units in the field) and they need to read the same row each one (but only the first one needs to perform the action).
I was thinking to lock the row (only the row, locking the entire table is not an option), so the first process will read it and check the timestamp of the latest update, perform the task needed and once is unlocked the other processes will be able to read it, and if it has the same timestamp just ignore it.
It will affect performance, but only for those few cases where this happens.
I've been trying to do this in MariaDB, but I can't find the how... starting a transaction and executing FOR UPDATE seems to lock the entire table, because I'm not able to fetch another rows meanwhile.
And reading MariaDB documentation I see this
When LOCK IN SHARE MODE is specified in a SELECT statement, MariaDB will wait until all transactions that have modified the rows are committed. Then, a write lock is acquired. All transactions can read the rows, but if they want to modify them, they have to wait until your transaction is committed.
So basically, what I want to do can't be done in MariaDB?
Any ideas?
Thank you,
We want to prevent some concurrency issues in a database (we use event sourcing and want to insert an event to the event log in case the event is a valid one).
The problem is that we need to check if a certain operation is allowed (requiring a SELECT query and some checks in php) and then run a INSERT query to actually perform the operation.
Now, we can simply LOCK the entire table, do our checks and if they succeed, then insert (and remove the lock).
The problem with this is that it locks the entire table, which is overkill (there will be lots of queries on this table). What I would like to do is to lock all queries that want to do this select-insert operation for a specific object_id, but allow queries for all other object_id's to continue as if there is no lock.
I searched a bit but couldn't find a lock attribute command. There seems to be a lock row command in innoDB, but it's not really what we want (I think I'm not 100% sure what it does).
We can of course try to manually handle the locks (check if there exists some column with object_id in some seperate lock table and wait untill there is none), but that feels a bit fishy and error prone.
So, here's the actual question: is it possible to lock a table for a specific value of a column (object_id)?
It would be awesome if the lock only held for the specific SELECT-INSERT queries, and not for standalone SELECT's, but that doesn't matter that much for now.
Consider manual arbitrary locks with GET_LOCK();
Choose a name specific to the rows you want locking. e.g. 'xxx_event_id_y'. Where 'xxx' is a string specific to the procedure and table and 'y' is the event id.
Call SELECT GET_LOCK('xxx_event_id_y',30) to lock the name 'xxx_event_id_y'.. it will return 1 and set the lock if the name becomes available, or return 0 if the lock is not available after 30 seconds (the second parameter is the timeout).
Use DO RELEASE_LOCK('xxx_event_id_y') when you are finished.
Be aware; You will have to use the same names in each transaction that you want to wait and calling GET_LOCK() again in a transaction will release the previously set lock.
GET_LOCK() docs
I actually use this method to lock our application cache too (even when it doesn't use the DB), so it has scope outside the database as well.
Migrate tables to innodb if not already done, and use transactions.
I've locked entire tables before in MySQL but what I need to do is lock a specific row. More specifically, when my PHP script executes, I need it to Lock a specific row so it cannot be read from (or written to), run the rest of the code, then unlock the row when it's finished.
I know I need to use InnoDB for this, but I've been unable to find how to do this. I'm not sure if it requires transactions or not, and if so, how to use them for row level locking.
Update
Perhaps I'm thinking of this problem the wrong way. From the reading I've done, and been provided with, InnoDB tables auto lock rows when read/written, which I get. My concern though is that I want to introduce a delay in my PHP code via a sleep. I want any and all read attempts on the row that was previously read from to be locked until that sleep finishes, and the script finishes the rest of it's functions.
If I just run:
$result = $mysqli->prepare('...');
$result->bind_param('...', $...);
$result->execute();
$result->bind_result($...);
$result->fetch();
sleep('15');
//More script execution
$result->close();
Will that prevent other MySQL queries from accessing that selected row until I close the connection?
MySQL uses only table-level locking from MyISAM tables.For row-leve locking you would need to switch to innoDB.
Please see the documentation which tells about the lock functioning in mysql.
I have an application, running on php + mysql plattform, using Doctrine2 framework. I need to execute 3 db queries during one http request: first INSERT, second SELECT, third UPDATE. UPDATE is dependent on result of SELECT query. There is a high probability of concurrent http requests. If such situation occurs, and DB queries get mixed up (eg. INS1, INS2, SEL1, SEL2, UPD1, UPD2), it will result in data inconsistency. How do I assure atomicity of INS-SEL-UPD operation? Do I need to use some kind of locks, or transactions are sufficient?
The answer from #YaK is actually a good answer. You should know how to deal with locks in general.
Addressing Doctrine2 specifically, your code should look like:
$em->getConnection()->beginTransaction();
try {
$toUpdate = $em->find('Entity\WhichWillBeUpdated', $id, \Doctrine\DBAL\LockMode::PESSIMISTIC_WRITE);
// this will append FOR UPDATE http://docs.doctrine-project.org/en/2.0.x/reference/transactions-and-concurrency.html
$em->persist($anInsertedOne);
// you can flush here as well, to obtain the ID after insert if needed
$toUpdate->changeValue('new value');
$em->persist($toUpdate);
$em->flush();
$em->getConnection()->commit();
} catch (\Exception $e) {
$em->getConnection()->rollback();
throw $e;
}
The every subsequent request to fetch for update, will wait until this transaction finishes for one process which has acquired the lock. Mysql will release the lock automatically after transaction is finished successfully or failed. By default, innodb lock timeout is 50 seconds. So if your process does not finish transaction in 50 seconds it will rollback and release the lock automatically. You do not need any additional fields on your entity.
A table-wide LOCK is guaranteed to work in all situations. But they are quite bad because they kind of prevent concurrency, rather than deal with it.
However, if your script holds the locks for a very short time frame, it might be an acceptable solution.
If your table uses InnoDB engine (no support for transactions with MyISAM), transaction is the most efficient solution, but also the most complex.
For your very specific need (in the same table, first INSERT, second SELECT, third UPDATE dependending on result of SELECT query):
Start a transaction
INSERT your records. Other transactions will not see these new rows until your own transaction is committed (unless you use a non-standard isolation level)
SELECT your record(s) with SELECT...LOCK IN SHARE MODE. You now have a READ lock on these rows, no one else may change these rows. (*)
Compute whatever you need to compute to determine whether or not you need to UPDATE something.
UPDATE the rows if required.
Commit
Expect errors at any time. If a dead-lock is detected, MySQL may decide to ROLLBACK you transaction to escape the dead-lock. If another transaction is updating the rows you are trying to read from, your transaction may be locked for some time, or even time-out.
The atomicity of your transaction is guaranteed if you proceed this way.
(*) in general, rows not returned by this SELECT may still be inserted in a concurrent transaction, that is, the non-existence is not guaranteed throughout the course of the transaction unless proper precautions are taken
Transactions won't prevent thread B to read the values thread A has not locked
So you must use locks to prevent concurrency access.
#Gediminas explained how you can use locks with Doctrine.
But using locks can result in dead locks or lock timeouts.
Doctrine renders these SQL errors as RetryableExceptions.
These exceptions are often normal if you are in a high concurrency environment.
They can happen very often and your application should handle them properly.
Each time a RetryableException is thrown by Doctrine, the proper way to handle this is to retry the whole transaction.
As easy as it seems, there is a trap. The Doctrine 2 EntityManager becomes unusable after a RetryableException and you must recreate a new one to replay your whole transaction.
I wrote this article illustrated with a full example.
Stack:
If I understand SELECT ... LOCK IN SHARE MODE correctly, you can place it into a mysql transaction to select the rows you will be working with during that transaction in order to "lock out" those selected rows from other session's writing/deleting actions (but other sessions can still read the rows) until your transaction completes, at which point the rows that were locked with the SELECT LOCK IN SHARE MODE statement are released so other sessions can access them for writing/deleting etc.
This is exactly what I want for my comments table. Whenever a comment is added to a post on my site, I need to lock all the comment rows tied to that post while I update some meta data on all the locked rows. And if two comments get submitted at the same time, I don't want them to both have access to the relevant comment rows at the same time because they will basically screw each other (and the meta data) up. So I want to incorporate SELECT LOCK IN SHARE MODE into the comment upload script so the first session to run the lock in query gets complete control over the comment rows until it finishes the entire transaction (and the script that was slightly slower has to wait until the entire transaction from the first script executes).
I am a but concerned about creating deadlocks where script A locks data that script B needs, and script B locks data that script A needs. How can I get around this in my application?
Also, I am using only innodb in my website database, so I don't need to worry about table locks correct?
In the MySQL documentation, the page 14.6.8.1. InnoDB Lock Modes discusses (near the bottom of the page) a deadlock situation caused by the first client requesting a read lock with "LOCK IN SHARE MODE" and the second client requesting a write lock because of a delete. The second client is blocked by the first client's read lock, so its write lock is queued up. But when the first client then tries to do a delete, the following happens:
Deadlock occurs here because client A needs an X (exclusive) lock to delete the
row. However, that lock request cannot be granted because client B
already has a request for an X lock and is waiting for client A to
release its S (shared) lock. Nor can the S lock held by A be upgraded to an X
lock because of the prior request by B for an X lock. As a result,
InnoDB generates an error for client A and releases its locks. At that
point, the lock request for client B can be granted and B deletes the
row from the table.
If I understand this correctly, I think the problem can be solved by the first client using FOR UPDATE instead of "LOCK IN SHARE MODE". This causes the first client to get a write lock from the beginning, and then it never has to wait on the second client.
Use a transaction around the query and update statements, and the lock will be held until you commit the transaction. No other table locks should be necessary.