I have a transaction like this (innoDB):
START TRANSACTION;
SELECT 1 FROM test WHERE id > 5; // Let's assume this returns 0 rows
// Some very long operation here
//If the previous SELECT contained 0 results, this Insert will be executed
INSERT INTO test VALUES...;
COMMIT;
Now the problem is that if more sessions execute at the same time, then they will all end up executing the INSERT, because by the time the long task in those sessions has finished, all of the sessions had plenty of time to do the SELECT, and it will return 0 row result for all of them, since the INSERT haven't been executed quite yet due to the long task running.
So basically, I need to somehow lock the whole table test (so it can't be read by other sessions and they will be forced to wait) after I execute START TRANSACTION, but I am not sure how, because I can't use the LOCK TABLES test query, because that COMMITs the transaction I have started.
I also cannot use SELECT .. FOR UPDATE, because that only prevents existing rows from being modified, but it won't prevent new rows from being inserted.
If you've got some long-running task which only needs to be run once, set a flag in a separate table to say that the task has started, and check that instead of the number of rows written to the target table, so that another instance of the service does not kick off the job twice.
This also has the advantage that you're not relying on the specifics of the task in order to know the status of it (and therefore if the nature of the task changes, the integrity of your other code doesn't collapse), and you're not trying to work round the transaction by doing something horrible like locking an entire table. One of the points of using transactions is that it's not necessary to lock everything (although of course different isolation levels can be used, but that's not the subject of this discussion).
Then set the flag back to false when the last bit of the task has finished.
Related
I am building a PHP RESTful-API for remote "worker" machines to self-assign tasks. The MySQL InnoDB table on the API host holds pending records that the workers can pick up from the API whenever they are ready to work on a record. How do I prevent concurrently requesting worker system from ever getting the same record?
My initial plan to prevent this is to UPDATE a single record with a uniquely generated ID in a default NULL field, and then poll for the details of the record where the unique ID field matches.
For example:
UPDATE mytable SET status = 'Assigned', uniqueidfield = '3kj29slsad'
WHERE uniqueidfield IS NULL LIMIT 1
And in the same PHP instance, the next query:
SELECT id, status, etc FROM mytable WHERE uniqueidfield = '3kj29slsad'
The resulting record from the SELECT statement above is then given to the worker. Would this prevent simultaneously requesting workers from getting the same records shown to them? I am not exactly sure on how MySQL handles the lookups within an UPDATE query, and if two UPDATES could "find" the same record, and then update it sequentially. If this works, is there a more elegant or standardized way of doing this (not sure if FOR UPDATE would need to be applied to this)? Thanks!
Nevermind my previous answer. I believe I understand what you are asking. I'll reword it so maybe it is clearer to others.
"If I issue two of the above update statements at the same time, what would happen?"
According to http://dev.mysql.com/doc/refman/5.0/en/lock-tables-restrictions.html, the second statement would not interfere with the first one.
Normally, you do not need to lock tables, because all single UPDATE
statements are atomic; no other session can interfere with any other
currently executing SQL statement.
A more elegant way is probably opinion based, but I don't see anything wrong with what you're doing.
If a database transaction is occurring on one thread is there a way for other threads to check to see if this transaction is already occurring before attempting the transaction? I know innoDB has row-level locking but I want the transaction to not be attempted if its already occurring on another thread, instead of waiting for the lock to be released and then attempting it.
To make my question clearer, an explanation of what I am trying to do may help:
I am creating a simple raffle using php and a innoDB table with MySQL. When a user loads the page to view the raffle it checks the raffle's database row to see if its scheduled end time has passed and if its "processed" column in the database is true or false.
If the raffle needs to be processed it will begin a database transaction which takes about 5 seconds before being committed and marked as "processed" in the database.
If multiple users load the page at around the same time I feel that it will process the raffle more than once which is not what I want. Ideally it would only attempt to process the raffle if no other threads are processing it, otherwise it would do nothing.
How would I go about doing this? Thanks.
You could implement table level locking and handle any subsequent connections to either be run in a queue or fail quietly:
https://dev.mysql.com/doc/refman/5.0/en/lock-tables.html
From the MySQL docs:
SET autocommit=0;
LOCK TABLES t1 WRITE, t2 READ, ...;
... do something with tables t1 and t2 here ...
COMMIT;
UNLOCK TABLES;
We have an automatic car plate reader which records plates of the cars enter to firm. My colleague asked me if we can instantly get the plate number of the car coming. The software uses MySQL and I have only database access. Cannot reach/edit PHP codes.
My offer is to check using a query periodically. For example for 10 seconds. But in this way it is possible to miss the cars coming in 5 seconds. Then decreasing interval increases request/response count which means extra load for the server. I do not want the script to run always. It should run only a new db row added. It shows the plate and exits.
How can I get last recorded row from the db right after inserting? I mean there should be trigger which runs my PHP script after insertion. But I do not know.
What I want is MySQL could run my PHP script after a new record.
If your table is MyISAM, I would stick to your initial idea. Getting the row count from a MyISAM table is instant. It only takes the reading of one single value as MyISAM maintains the row count at all times.
With InnoDB, this approach can still be acceptable. Assuming car_table.id is primary key, SELECT COUNT(id) FROM car_table only requires an index scan, which is very fast. You can improve on this idea by adding another indexed boolean column to your table:
ALTER car_table ADD COLUMN checked BOOLEAN NOT NULL DEFAULT 0, ADD INDEX (checked);
The default value ensures new cars will be inserted with this flag set to 0 without modifying the inserting statement. Then:
BEGIN TRANSACTION; -- make sure nobody interferes
SELECT COUNT(checked) FROM car_table WHERE checked = FALSE FOR UPDATE; -- this gets you the number of new, unchecked cars
UPDATE car_table SET checked = TRUE WHERE checked = FALSE; -- mark these cars as checked
COMMIT;
This way, you only scan a very small number of index entries at each polling.
A more advanced approach consists in adding newly created cars ID's into a side table, through a trigger. This side table is scanned every now and then, without locking the main table, and without altering its structure. Simply TRUNCATE this side table after each polling.
Finally, there is the option of triggering a UDF, as suggested by Panagiotis, but this seems to be an overkill in most situations.
Although this is not the greatest of designs and I have not implemented it, there is way to call an external script through sys_exec() UDF using a trigger as mentioned here:
B.5.11: Can triggers call an external application through a UDF?
Yes. For example, a trigger could invoke the sys_exec() UDF.
http://dev.mysql.com/doc/refman/5.1/en/faqs-triggers.html#qandaitem-B-5-1-11
Also have a look on this thread which is similar to your needs.
Invoking a PHP script from a MySQL trigger
Is it possible that a php script gets terminated between two mysql queries.
For example user registration:
1st INSERT : I will enter the basic first-name,last-name,address in one table
and
2nd INSERT : i will enter the user's hashed password and salt in another table
This operation probably requires two queries and either of them independently are useless records.
What if the php script terminates after executing the first query?
User will just get a server error message but one useless record will be generated.
Any Solutions??
EDIT ------
My Web host does not provide a InnoDB engine.
Only MyISAM supported
Use a transaction:
START TRANSACTION;
INSERT INTO foo ...
INSERT INTO bar ...
COMMIT
If either INSERT fails, you ROLLBACK the transaction and you won't be left with "useless" records.
if you use a transaction such as:
BEGIN;
INSERT INTO TBL(...) VALUES(...);
INSERT INTO TBL(...) VALUES(...);
COMMIT;
Everything is sent to the mysql server in one go, and then ran as a batch, meaning your script can terminate as soon as the transfer is complete, rather than waiting for each individual query. It also has the added bonus of using less ram and being much faster.
I have no knowledge of locking whatsoever. I have been looking through some MySQL documentation and can't fully understand how this whole process goes about. What I need, is for the following events in my script to happen:
step 1) table user gets locked
step 2) my script selects two rows from table user
step 3) my script makes an update to table user
step 4) table user gets unlocked because the script is done
How do I go about this exactly? And what happens when another user runs this same script while the table is locked? Is there a way for the script to know when to proceed (when the table becomes unlocked?). I have looked into start transaction and select for update but the documentation is very unclear. Any help is appreciated. And yes, the table is innodb.
I believe what you are look for is the SELECT ... FOR UPDATE syntax available for InnoDB tables. This will lock only the records you want to update. You do need to wrap it in a transaction.
http://dev.mysql.com/doc/refman/5.0/en/innodb-locking-reads.html
For example, run your queries like this:
START TRANSACTION
SELECT ... FOR UPDATE
UPDATE ...
COMMIT
Eliminate step 2 by performing your select query as part of your update call. Then MySQL takes care of the rest. Only one write query can run at the same time, others will be queued behind.