I have a table named and spot and reservation. spot contains column spot_id and spot_status. For reservation process i start a transaction and then acquire lock on specific row using this query. I am using php and mysql.
//start transaction
SELECT * FROM spot WHERE spot_id = $id FOR UPDATE ;
//if this query is successful then
1. set spot status to 1
2. insert corresponding values in reservation table.
and then commit else rollback
//transactions ends
lets say there are 2 concurrent transactions T1 and T2 which tries to reserve the same spot. From what i learnt from other's questions and answers in this site, if the transactions are not concurrent there would not be any problem, but in concurrent operation the processor can change from schedules of T1 to T2 anytime . After acquiring the locks on row by T1, lets say processor switch to transaction T2. T2 then tries to acquire locks on that same row but it cannot as it is locked by T1.
my questions are theoritical :
When is the lock removed by mysql? or is there any explicit way of
removing lock myself?
Since T2 transaction cannot lock the row which is the first query , does it rollback? or does processor keeps T2 waiting until it can lock the row?
what is the possibility of deadlock occuring in this problem?
Your strategy for lock management is correct.
If T1 first obtains the lock on spot/spot_id=$id, then T2 will wait until T1 either commits or rolls back the transaction. If T1 crashes or times out the rollback will be implicit.
If you want a deadlock, try this.
Get T1 to lock row 1 ("fork") and then row 2 ("knife").
Get T2 to lock row 2 ("knife") and then row 1 ("fork").
Run them concurrently. Eventually you'll get T2 holding only a knife, and T1 holding only a fork. They'll be staring at each other, going hungry, each waiting for the other to set down an implement.
Lock is removed when transaction which obtained the lock is ended that is committed or rolled back. There is no way in mysql to release row level lock until end of transaction because:
InnoDB stores row locks in a format such that it cannot know afterward which lock was set by which statement
However you can use user defined locks (aka advisory locks aka cooperative locks) which can be released at any moment.
T2 transaction will wait till lock can be acquired but no more than inno_lock_wait_timeout seconds. In the later case error will happen that lock cannot be acquired.
If you lock the row by id and then modify only this row then deadlock cannot happen. For deadlock to happen you need at least two resources which are acquired by transactions in different order.
Related
I did a lot of research and I found a lot of information about all the relevant topics. However, I am not confident that I do now understand, how to put all this information together properly.
This application is written in PHP.
For queries I use PDO.
The MySQL database is configured as InnoDB.
What I need
SELECT ... FROM tableA;
// PHP looks at what comes back and does some logic.
INSERT INTO tableA ...;
INSERT INTO tableB ...;
Conditions:
The INSERTs need to be atomic. If one of them fails I want to roll back.
No reads and writes from/to tableA are allowed to happen between the SELECT and the INSERT from/into tableA.
This to me looks like a very simple problem. Yet I am not able to figure out, how to do this properly. So my question is:
What is the best approach?
This is an outline for my current plan, heavily simplified:
try {
SET autocommit = 0;
BLOCK TABLES tableA WRITE, tableB WRITE;
SELECT ... FROM tableA;
INSERT INTO tableA ...;
INSERT INTO tableB ...;
COMMIT;
UNLOCK TABLES;
SET autocommit = 1;
}
catch {
ROLLBACK;
UNLOCK TABLES;
SET autocommit = 1;
}
I feel like there is a lot that could be done better, but I don't know how :/
Why do it like this?
I need some kind of transaction to be able to do a rollback if INSERTs fail.
I need to lock tableA to make sure that no other INSERTs or UPDATEs take place.
Transactions and table locks don't work well together
(https://dev.mysql.com/doc/refman/8.0/en/lock-tables-and-transactions.html)
I want to use autocommit as a standard throughout the rest of my application, which is why I set it back to "1" at the end.
I am really not sure about this: But I somewhere picked up, that after locking a table, I can (from within the current connection) only query to this table until I unlock it (this does not make sense to me). This is why I locked tableB too, altough otherwise I wouldn't need to.
I am open for completely different approaches
I am open for any suggestion within the framework conditions PHP, MySQL, PDO and InnoDB.
Thank You!
Edit 1 (2018-06-01)
I feel like my problem/question needs some more clarification.
Starting point:
If have two tables, t1 and t2.
t1 has multiple columns of non-unique values.
The specifics of t2 are irrelevant for this problem.
What I want to do:
Step by step:
Select multiple columns and rows from t1.
In PHP analyse the retrieved data. Based on the results of this analysis put together a dataset.
INSERT parts of the dataset into t1 and parts of it into t2.
Additional information:
The INSERTs into the 2 tables must be atomic. This can be achieved using transactions.
No INSERTs from a different connection are allowed to occur between steps 1 and 3. This is very important, because every single INSERT into t1 has to occur with full awareness of the current state of the table. I'll best describe this in more detail. I will leave t2 out of this for now, to make things easier to understand.
Imagine this sequence of events (connections con1 and con2):
con1: SELECT ... FROM t1 WHERE xyz;
con1: PHP processes the information.
con2: SELECT ... FROM t1 WHERE uvw;
con2: PHP processes the information.
con1: INSERT INTO t1 ...;
con2: INSERT INTO t1 ...;
So both connections see t1 in the same state. However, they select different information. Con1 takes the information gathered, does some logic with it and then INSERTs data into a new row in t1. Con2 does the same, but using different information.
The problem is this: Both connections INSERTed data based on calculations that did not take into account whatever the other connection inserted into t1, because this information wasn't there when they read from t1.
Con2 might have inserted a row into t1 that would have met the WHERE-conditions of con1's SELECT-statement. In other words: Had con2 inserted its row earlier, con1 might have created completely different data to insert into t1. This is to say: The two INSERTs might have completely invalidated each others inserts.
This is why I want to make sure, that only one connection can work with the data in t1 at a time. No other connection is allowed to write, but also no other connection is allowed to read until the current connection is done.
I hope this clarifies things a bit... :/
Thoughts:
My thoughts were:
I need to make the INSERTs into the 2 tables atomic. --> I will use a transaction for this. Something like this:
try {
$pdo->beginTransaction();
// INSERT INTO t1 ...
// INSERT INTO t2 ...
$pdo->commit();
}
catch (Exception $e) {
$pdo->rollBack();
throw $e;
}
I need to make sure, no other connection writes to or reads from t1. This is where I decided that I need LOCK TABLES.
Assuming I had to use LOCK TABLES, I was confronted with the problem that LOCK TABLES is not transaction aware. Which is why I decided to go with the solution proposed here (https://dev.mysql.com/doc/refman/8.0/en/lock-tables-and-transactions.html) and also in multiple answers on stackoverflow.
But I wasn't happy with how the code looked like, which is why I came here to ask this (meanwhile rather lengthy) question.
Edit 2 (2018-06-01)
This process will not run often. So there is no significant need for high performance and effiency. This, of course, also means that the chances of two of those processes infering with eachother are rather minute. Stil, I'd like to make sure nothing can happen.
Case 1:
BEGIN;
INSERT ..
INSERT ..
COMMIT;
Other connections will not see the inserted rows until after the commit. That is, BEGIN...COMMIT made the two inserts "atomic".
If anything fails, you still need the try/catch to deal with it.
Do not use LOCK TABLES on InnoDB tables.
Don't bother with autocommit; BEGIN..COMMIT overrides it.
My statements apply to (probably) all frameworks. (Except that some do not have "try" and "catch".)
Case 2: Lock a row in anticipation of possibly modifying it:
BEGIN;
SELECT ... FROM t1 FOR UPDATE;
... work with the values SELECTed
UPDATE t1 ...;
COMMIT;
This keeps others away from the rows SELECTed until after the COMMIT.
Case 3: Sometimes IODKU is useful to do two things in a single atomic statement:
INSERT ...
ON DUPLICATE KEY UPDATE ...
instead of
BEGIN;
SELECT ... FOR UPDATE;
if no row found
INSERT ...;
else
UPDATE ...;
COMMIT;
Class 4: Classic banking example:
BEGIN;
UPDATE accounts SET balance = balance - 1000.00 WHERE id='me';
... What if crash occurs here? ...
UPDATE accounts SET balance = balance + 1000.00 WHERE id='you';
COMMIT;
If the system crashes between the two UPDATEs, the first update will be undone. This keeps the system from losing track of the funds transfer.
Case 5: Perhaps close to what the OP wants. It is mostly a combination of Cases 2 and 1.
BEGIN;
SELECT ... FROM t1 FOR UPDATE; -- see note below
... work with the values SELECTed
INSERT INTO t1 ...;
COMMIT;
Notes on Case 5: The SELECT..FOR UPDATE must include any rows that you don't want the other connection to see. This has the effect of delaying the other connection until this connection COMMITs. (Yes, this feels a lot like LOCK TABLES t1 WRITE.)
Case 6: The "processing" that needs to be inside the BEGIN..COMMIT will take too long. (Example: the typical online shopping cart.)
This needs a locking mechanism outside of InnoDB's transactions. One way (useful for shopping cart) is to use a row in some extra table, and have everyone check it. Another way (more practical within a single connection) is to use GET_LOCK('foo') and it's friends.
General Discussion
All of the above examples lock only the row(s) involved, not the entire table(s). This makes the action much less invasive, and allows for the system to handle much more activity.
Also, read about MVCC. This is a general technique used under the cover to let one connection see the values of the table(s) at some instant in time, even while other connections are modifying the table(s).
"Prevent inserts" -- With MVCC, if you start a SELECT, it is like getting a snapshot in time of everything you are looking at. You won't see the INSERTs until after you complete the transaction that the SELECT is in. You can have your cake and eat it, too. That is, it appears as if the inserts were blocked, but you get the performance benefit of them happening in parallel. Magic.
I have a mysql database (InnoDB engine) where I lock a few tables before doing some work. According to the documentation
"The correct way to use LOCK TABLES and UNLOCK TABLES with transactional tables, such as InnoDB tables, is to begin a transaction with SET autocommit = 0 (not START TRANSACTION) followed by LOCK TABLES, and to not call UNLOCK TABLES until you commit the transaction explicitly."
So I'm doing (pseudocode):
mysqli_query("SET autocommit=0");
mysqli_query("LOCK TABLES table1 WRITE, table2 READ ...");
mysqli_query("SOME SELECTS AND INSERTS HERE");
mysqli_query("COMMIT");
mysqli_query("UNLOCK TABLES");
Now, should I also do this:
mysqli_query("SET autocommit=1");
According to the documentation again,
"When you call LOCK TABLES, InnoDB internally takes its own table lock, and MySQL takes its own table lock. InnoDB releases its internal table lock at the next commit, but for MySQL to release its table lock, you have to call UNLOCK TABLES. You should not have autocommit = 1, because then InnoDB releases its internal table lock immediately after the call of LOCK TABLES, and deadlocks can very easily happen. InnoDB does not acquire the internal table lock at all if autocommit = 1, to help old applications avoid unnecessary deadlocks."
I think the documentation is a bit ambigous at this point. As I'm interpreting it, you shouldn't use SET autocommit=1 in place of UNLOCK TABLES.
However, there shouldn't be any harm in doing it AFTER the tables have been unlocked?
But still I'm unsure if it's necessary. I have a single select running in the same script after the COMMIT, and it appears to autocommit even if I don't SET autocommit=1. Why?
I have one table that is read at the same time by different threads.
Each thread must select 100 rows, execute some tasks on each row (unrelated to the database) then they must delete the selected row from the table.
rows are selected using this query:
SELECT id FROM table_name FOR UPDATE;
My question is: How can I ignore (or skip) rows that were previously locked using a select statement in MySQL ?
I typically create a process_id column that is default NULL and then have each thread use a unique identifier to do the following:
UPDATE table_name SET process_id = #{process.id} WHERE process_id IS NULL LIMIT 100;
SELECT id FROM table_name WHERE process_id = #{process.id} FOR UPDATE;
That ensures that each thread selects a unique set of rows from the table.
Hope this helps.
Even though it is not the best solution, as there is no way that I know to ignore locked rows, I select a random one and try to obtain a lock.
START TRANSACTION;
SET #v1 =(SELECT myId FROM tests.table WHERE status is NULL LIMIT 1);
SELECT * FROM tests.table WHERE myId=#v1 FOR UPDATE; #<- lock
Setting a small timeout for the transaction, if that row is locked the transaction is aborted and I try another one. If I obtain the lock, I process it. If (bad luck) that row was locked, it is processed and the lock is released before my timeout, I then select a row that has already been 'processed'! However, I check a field that my processes set (e.g. status): if the other process transaction ended OK, that field tells me that work has already been done and I do not process that row again.
Every other possible solution without transactions (e.g. setting another field if the row has no status and ... etc.) can easily provide race conditions and missed processes (e.g. one thread abruptly dies, the allocated data is still tagged, while a transaction expires; ref. comment here
Hope it helps
If a database transaction is occurring on one thread is there a way for other threads to check to see if this transaction is already occurring before attempting the transaction? I know innoDB has row-level locking but I want the transaction to not be attempted if its already occurring on another thread, instead of waiting for the lock to be released and then attempting it.
To make my question clearer, an explanation of what I am trying to do may help:
I am creating a simple raffle using php and a innoDB table with MySQL. When a user loads the page to view the raffle it checks the raffle's database row to see if its scheduled end time has passed and if its "processed" column in the database is true or false.
If the raffle needs to be processed it will begin a database transaction which takes about 5 seconds before being committed and marked as "processed" in the database.
If multiple users load the page at around the same time I feel that it will process the raffle more than once which is not what I want. Ideally it would only attempt to process the raffle if no other threads are processing it, otherwise it would do nothing.
How would I go about doing this? Thanks.
You could implement table level locking and handle any subsequent connections to either be run in a queue or fail quietly:
https://dev.mysql.com/doc/refman/5.0/en/lock-tables.html
From the MySQL docs:
SET autocommit=0;
LOCK TABLES t1 WRITE, t2 READ, ...;
... do something with tables t1 and t2 here ...
COMMIT;
UNLOCK TABLES;
I am writing a MySQL query that locks a table:
"LOCK TABLE table_1 WRITE"
After that i am executing some functions, and in one of those functions, I am executing another query, on another table that I haven't locked:
"SELECT * FROM completely_different_table_2"
Then i get the following error message as result:
Table 'completely_different_table_2' was not locked with LOCKED TABLES
Indeed, MySql is right to tell me that the table is not locked. But why does it throws an error? Anyone any ideas how I could solve this?
Thanks in advance.
You have to lock every table, that you want to use until the LOCK is released. You can give completely_different_table_2 only a READ LOCK, which allows other processes to read this table while it is locked:
LOCK TABLES table_1 WRITE, completely_different_table_2 READ;
PS: MySQL has a reason to do so. If you request a LOCK, you want to freeze a consistent state of your data. If you read data from completely_different_table_2 inside your LOCK, your data written to table_1 will in some way depend on this other table. Therefore you don’t want anyone to change this table during your LOCK and request a READ LOCK for this second table as well. If your data written to table_1 doesn’t depend on the other table, simply don’t query it until the LOCK is released.