I am using the following query on MySQL using PHP
$sql = SELECT MAX(SrNo) FROM cart;
$result = mysql_query($sql);
The structure of table CART is
CART (SrNo int(10));
Now I am using the result to do some kind of processing and inserting the maximum value into this table by incrementing one. My problem is that if user1 has got the maximum value of SrNo and is in-between the processing. During this time user2 also requests the server got the same maximum value of SrNo as user1 got and starts processing.
Now when both are done with the processing + insertion into the table, I will have two duplicates in the table CART. How can I prevent this from happening?
In other words, I want no one else to get the maximum value of SrNo until unless one user is finished doing its processing.
NOt a trivial thing with a web application that creates a new connection on each request.
You'd need to add a lockedBy and lockedTime columns to this table, and put into them an ID of user that requested the lock as well as timestamp of when the lock was requested. You need the timestamp, so that you can ignore locks that are longer than certain amount of time.
wouldn't you be fine with the AUTO_INCREMENT feature for PRIMARY KEY?
create table cart ( SrNo int(10) AUTO_INCREMENT PRIMARY KEY ) ENGINE = InnoDB;
then just simply insert new lines and it will automatically increment the new values. That would probably very easily do the trick you are (maybe?) trying to do.
But if you need to lock the maxmium, you can do this:
start transaction;
select max(SrNo) from cart for update;
/* do some other stuff, insert the max value + 1 etc... */
commit;
Remember: You should use transaction for any operation which is not 1 single query!
if you set SrNo as the primary key on the table, then the secound time you try to add the row going to fail, and if it fails, you can request a new number.
ALTER TABLE cart ADD PRIMARY KEY (SrNo);
Related
I'm using MySQL's AUTO_INCREMENT field and InnoDB to support transactions. I noticed when I rollback the transaction, the AUTO_INCREMENT field is not rollbacked? I found out that it was designed this way but are there any workarounds to this?
It can't work that way. Consider:
program one, you open a transaction and insert into a table FOO which has an autoinc primary key (arbitrarily, we say it gets 557 for its key value).
Program two starts, it opens a transaction and inserts into table FOO getting 558.
Program two inserts into table BAR which has a column which is a foreign key to FOO. So now the 558 is located in both FOO and BAR.
Program two now commits.
Program three starts and generates a report from table FOO. The 558 record is printed.
After that, program one rolls back.
How does the database reclaim the 557 value? Does it go into FOO and decrement all the other primary keys greater than 557? How does it fix BAR? How does it erase the 558 printed on the report program three output?
Oracle's sequence numbers are also independent of transactions for the same reason.
If you can solve this problem in constant time, I'm sure you can make a lot of money in the database field.
Now, if you have a requirement that your auto increment field never have gaps (for auditing purposes, say). Then you cannot rollback your transactions. Instead you need to have a status flag on your records. On first insert, the record's status is "Incomplete" then you start the transaction, do your work and update the status to "compete" (or whatever you need). Then when you commit, the record is live. If the transaction rollsback, the incomplete record is still there for auditing. This will cause you many other headaches but is one way to deal with audit trails.
Let me point out something very important:
You should never depend on the numeric features of autogenerated keys.
That is, other than comparing them for equality (=) or unequality (<>), you should not do anything else. No relational operators (<, >), no sorting by indexes, etc. If you need to sort by "date added", have a "date added" column.
Treat them as apples and oranges: Does it make sense to ask if an apple is the same as an orange? Yes. Does it make sense to ask if an apple is larger than an orange? No. (Actually, it does, but you get my point.)
If you stick to this rule, gaps in the continuity of autogenerated indexes will not cause problems.
I had a client needed the ID to rollback on a table of invoices, where the order must be consecutive
My solution in MySQL was to remove the AUTO-INCREMENT and pull the latest Id from the table, add one (+1) and then insert it manually.
If the table is named "TableA" and the Auto-increment column is "Id"
INSERT INTO TableA (Id, Col2, Col3, Col4, ...)
VALUES (
(SELECT Id FROM TableA t ORDER BY t.Id DESC LIMIT 1)+1,
Col2_Val, Col3_Val, Col4_Val, ...)
Why do you care if it is rolled back? AUTO_INCREMENT key fields are not supposed to have any meaning so you really shouldn't care what value is used.
If you have information you're trying to preserve, perhaps another non-key column is needed.
I do not know of any way to do that. According to the MySQL Documentation, this is expected behavior and will happen with all innodb_autoinc_lock_mode lock modes. The specific text is:
In all lock modes (0, 1, and 2), if a
transaction that generated
auto-increment values rolls back,
those auto-increment values are
“lost.” Once a value is generated for
an auto-increment column, it cannot be
rolled back, whether or not the
“INSERT-like” statement is completed,
and whether or not the containing
transaction is rolled back. Such lost
values are not reused. Thus, there may
be gaps in the values stored in an
AUTO_INCREMENT column of a table.
If you set auto_increment to 1 after a rollback or deletion, on the next insert, MySQL will see that 1 is already used and will instead get the MAX() value and add 1 to it.
This will ensure that if the row with the last value is deleted (or the insert is rolled back), it will be reused.
To set the auto_increment to 1, do something like this:
ALTER TABLE tbl auto_increment = 1
This is not as efficient as simply continuing on with the next number because MAX() can be expensive, but if you delete/rollback infrequently and are obsessed with reusing the highest value, then this is a realistic approach.
Be aware that this does not prevent gaps from records deleted in the middle or if another insert should occur prior to you setting auto_increment back to 1.
INSERT INTO prueba(id)
VALUES (
(SELECT IFNULL( MAX( id ) , 0 )+1 FROM prueba target))
If the table doesn't contain values or zero rows
add target for error mysql type update FROM on SELECT
If you need to have the ids assigned in numerical order with no gaps, then you can't use an autoincrement column. You'll need to define a standard integer column and use a stored procedure that calculates the next number in the insert sequence and inserts the record within a transaction. If the insert fails, then the next time the procedure is called it will recalculate the next id.
Having said that, it is a bad idea to rely on ids being in some particular order with no gaps. If you need to preserve ordering, you should probably timestamp the row on insert (and potentially on update).
Concrete answer to this specific dilemma (which I also had) is the following:
1) Create a table that holds different counters for different documents (invoices, receipts, RMA's, etc..); Insert a record for each of your documents and add the initial counter to 0.
2) Before creating a new document, do the following (for invoices, for example):
UPDATE document_counters SET counter = LAST_INSERT_ID(counter + 1) where type = 'invoice'
3) Get the last value that you just updated to, like so:
SELECT LAST_INSERT_ID()
or just use your PHP (or whatever) mysql_insert_id() function to get the same thing
4) Insert your new record along with the primary ID that you just got back from the DB. This will override the current auto increment index, and make sure you have no ID gaps between you records.
This whole thing needs to be wrapped inside a transaction, of course. The beauty of this method is that, when you rollback a transaction, your UPDATE statement from Step 2 will be rolled back, and the counter will not change anymore. Other concurrent transactions will block until the first transaction is either committed or rolled back so they will not have access to either the old counter OR a new one, until all other transactions are finished first.
SOLUTION:
Let's use 'tbl_test' as an example table, and suppose the field 'Id' has AUTO_INCREMENT attribute
CREATE TABLE tbl_test (
Id int NOT NULL AUTO_INCREMENT ,
Name varchar(255) NULL ,
PRIMARY KEY (`Id`)
)
;
Let's suppose that table has houndred or thousand rows already inserted and you don't want to use AUTO_INCREMENT anymore; because when you rollback a transaction the field 'Id' is always adding +1 to AUTO_INCREMENT value.
So to avoid that you might make this:
Let's remove AUTO_INCREMENT value from column 'Id' (this won't delete your inserted rows):
ALTER TABLE tbl_test MODIFY COLUMN Id int(11) NOT NULL FIRST;
Finally, we create a BEFORE INSERT Trigger to generate an 'Id' value automatically. But using this way won't affect your Id value even if you rollback any transaction.
CREATE TRIGGER trg_tbl_test_1
BEFORE INSERT ON tbl_test
FOR EACH ROW
BEGIN
SET NEW.Id= COALESCE((SELECT MAX(Id) FROM tbl_test),0) + 1;
END;
That's it! You're done!
You're welcome.
$masterConn = mysql_connect("localhost", "root", '');
mysql_select_db("sample", $masterConn);
for($i=1; $i<=10; $i++) {
mysql_query("START TRANSACTION",$masterConn);
$qry_insert = "INSERT INTO `customer` (id, `a`, `b`) VALUES (NULL, '$i', 'a')";
mysql_query($qry_insert,$masterConn);
if($i%2==1) mysql_query("COMMIT",$masterConn);
else mysql_query("ROLLBACK",$masterConn);
mysql_query("ALTER TABLE customer auto_increment = 1",$masterConn);
}
echo "Done";
I'm using MySQL's AUTO_INCREMENT field and InnoDB to support transactions. I noticed when I rollback the transaction, the AUTO_INCREMENT field is not rollbacked? I found out that it was designed this way but are there any workarounds to this?
It can't work that way. Consider:
program one, you open a transaction and insert into a table FOO which has an autoinc primary key (arbitrarily, we say it gets 557 for its key value).
Program two starts, it opens a transaction and inserts into table FOO getting 558.
Program two inserts into table BAR which has a column which is a foreign key to FOO. So now the 558 is located in both FOO and BAR.
Program two now commits.
Program three starts and generates a report from table FOO. The 558 record is printed.
After that, program one rolls back.
How does the database reclaim the 557 value? Does it go into FOO and decrement all the other primary keys greater than 557? How does it fix BAR? How does it erase the 558 printed on the report program three output?
Oracle's sequence numbers are also independent of transactions for the same reason.
If you can solve this problem in constant time, I'm sure you can make a lot of money in the database field.
Now, if you have a requirement that your auto increment field never have gaps (for auditing purposes, say). Then you cannot rollback your transactions. Instead you need to have a status flag on your records. On first insert, the record's status is "Incomplete" then you start the transaction, do your work and update the status to "compete" (or whatever you need). Then when you commit, the record is live. If the transaction rollsback, the incomplete record is still there for auditing. This will cause you many other headaches but is one way to deal with audit trails.
Let me point out something very important:
You should never depend on the numeric features of autogenerated keys.
That is, other than comparing them for equality (=) or unequality (<>), you should not do anything else. No relational operators (<, >), no sorting by indexes, etc. If you need to sort by "date added", have a "date added" column.
Treat them as apples and oranges: Does it make sense to ask if an apple is the same as an orange? Yes. Does it make sense to ask if an apple is larger than an orange? No. (Actually, it does, but you get my point.)
If you stick to this rule, gaps in the continuity of autogenerated indexes will not cause problems.
I had a client needed the ID to rollback on a table of invoices, where the order must be consecutive
My solution in MySQL was to remove the AUTO-INCREMENT and pull the latest Id from the table, add one (+1) and then insert it manually.
If the table is named "TableA" and the Auto-increment column is "Id"
INSERT INTO TableA (Id, Col2, Col3, Col4, ...)
VALUES (
(SELECT Id FROM TableA t ORDER BY t.Id DESC LIMIT 1)+1,
Col2_Val, Col3_Val, Col4_Val, ...)
Why do you care if it is rolled back? AUTO_INCREMENT key fields are not supposed to have any meaning so you really shouldn't care what value is used.
If you have information you're trying to preserve, perhaps another non-key column is needed.
I do not know of any way to do that. According to the MySQL Documentation, this is expected behavior and will happen with all innodb_autoinc_lock_mode lock modes. The specific text is:
In all lock modes (0, 1, and 2), if a
transaction that generated
auto-increment values rolls back,
those auto-increment values are
“lost.” Once a value is generated for
an auto-increment column, it cannot be
rolled back, whether or not the
“INSERT-like” statement is completed,
and whether or not the containing
transaction is rolled back. Such lost
values are not reused. Thus, there may
be gaps in the values stored in an
AUTO_INCREMENT column of a table.
If you set auto_increment to 1 after a rollback or deletion, on the next insert, MySQL will see that 1 is already used and will instead get the MAX() value and add 1 to it.
This will ensure that if the row with the last value is deleted (or the insert is rolled back), it will be reused.
To set the auto_increment to 1, do something like this:
ALTER TABLE tbl auto_increment = 1
This is not as efficient as simply continuing on with the next number because MAX() can be expensive, but if you delete/rollback infrequently and are obsessed with reusing the highest value, then this is a realistic approach.
Be aware that this does not prevent gaps from records deleted in the middle or if another insert should occur prior to you setting auto_increment back to 1.
INSERT INTO prueba(id)
VALUES (
(SELECT IFNULL( MAX( id ) , 0 )+1 FROM prueba target))
If the table doesn't contain values or zero rows
add target for error mysql type update FROM on SELECT
If you need to have the ids assigned in numerical order with no gaps, then you can't use an autoincrement column. You'll need to define a standard integer column and use a stored procedure that calculates the next number in the insert sequence and inserts the record within a transaction. If the insert fails, then the next time the procedure is called it will recalculate the next id.
Having said that, it is a bad idea to rely on ids being in some particular order with no gaps. If you need to preserve ordering, you should probably timestamp the row on insert (and potentially on update).
Concrete answer to this specific dilemma (which I also had) is the following:
1) Create a table that holds different counters for different documents (invoices, receipts, RMA's, etc..); Insert a record for each of your documents and add the initial counter to 0.
2) Before creating a new document, do the following (for invoices, for example):
UPDATE document_counters SET counter = LAST_INSERT_ID(counter + 1) where type = 'invoice'
3) Get the last value that you just updated to, like so:
SELECT LAST_INSERT_ID()
or just use your PHP (or whatever) mysql_insert_id() function to get the same thing
4) Insert your new record along with the primary ID that you just got back from the DB. This will override the current auto increment index, and make sure you have no ID gaps between you records.
This whole thing needs to be wrapped inside a transaction, of course. The beauty of this method is that, when you rollback a transaction, your UPDATE statement from Step 2 will be rolled back, and the counter will not change anymore. Other concurrent transactions will block until the first transaction is either committed or rolled back so they will not have access to either the old counter OR a new one, until all other transactions are finished first.
SOLUTION:
Let's use 'tbl_test' as an example table, and suppose the field 'Id' has AUTO_INCREMENT attribute
CREATE TABLE tbl_test (
Id int NOT NULL AUTO_INCREMENT ,
Name varchar(255) NULL ,
PRIMARY KEY (`Id`)
)
;
Let's suppose that table has houndred or thousand rows already inserted and you don't want to use AUTO_INCREMENT anymore; because when you rollback a transaction the field 'Id' is always adding +1 to AUTO_INCREMENT value.
So to avoid that you might make this:
Let's remove AUTO_INCREMENT value from column 'Id' (this won't delete your inserted rows):
ALTER TABLE tbl_test MODIFY COLUMN Id int(11) NOT NULL FIRST;
Finally, we create a BEFORE INSERT Trigger to generate an 'Id' value automatically. But using this way won't affect your Id value even if you rollback any transaction.
CREATE TRIGGER trg_tbl_test_1
BEFORE INSERT ON tbl_test
FOR EACH ROW
BEGIN
SET NEW.Id= COALESCE((SELECT MAX(Id) FROM tbl_test),0) + 1;
END;
That's it! You're done!
You're welcome.
$masterConn = mysql_connect("localhost", "root", '');
mysql_select_db("sample", $masterConn);
for($i=1; $i<=10; $i++) {
mysql_query("START TRANSACTION",$masterConn);
$qry_insert = "INSERT INTO `customer` (id, `a`, `b`) VALUES (NULL, '$i', 'a')";
mysql_query($qry_insert,$masterConn);
if($i%2==1) mysql_query("COMMIT",$masterConn);
else mysql_query("ROLLBACK",$masterConn);
mysql_query("ALTER TABLE customer auto_increment = 1",$masterConn);
}
echo "Done";
I have a MySQL table with over 16 million rows and there is no primary key. Whenever I try to add one, my connection crashes. I have tried adding one as an auto increment in PHPMyAdmin and in command line but the connection is always lost after about 10 minutes.
What I would like to do is loop through the table's rows in PHP so I can limit the number of results and with each returned row add an auto-incremented ID number. Since the number of impacted rows would be reduced by reducing the load on the MySQL query, I won't lose my connection.
I want to do something like
SELECT * FROM MYTABLE LIMIT 1000001, 2000000;
Then, in the loop, update the current row
UPDATE (current row) SET ID='$i++'
How do I do this?
I am open to a MySQL solution as well. I just need a process that will not cause me to lose my MySQL connection
Note: the original data was given to me as a txt file. I don't know if there are duplicates but I cannot eliminate any rows. Also, no rows will be added. This table is going to be used only for querying purposes. When I have added indexes, however, there were no problems.
It is easier
ALTER TABLE table_name MODIFY COLUMN id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
In case you want to remove the primary key after the modification
ALTER TABLE table_name MODIFY COLUMN id INT
I would avoid setting primary keys manually. Much safer to let MySQL do it. You can temporarily increase the timeout in the config file:
wait_timeout = 28800
interactive_timeout = 28800
I'll try and make this question as brief as possible. I was wondering if there is a way to do the following via PHP & MySQL coding:
Delete an auto-incremented id entry (say entry 12 of 20), and at the same time lower the rest of the entries by 1. (Example: Delete entry 12 == Entry 13 becomes 12, 14 becomes 13, etc. etc.)
Manually (i.e via form) insert an entry with an ID so that the rest are pushed up if specified by the user (and defaults to normal behaviour if nothing is specified. (Example: User specifies that he wants his entry to be ID 14, but there is already an ID 14 in the database. What should happen would be that the user input pushes the already existing entry up to 15, & the rest of the existing entries follow a similar fashion. If the user does not specify an ID, default to normal behaviour.)
I'm still learning MySQL & PHP and the code I need is for an image gallery system that I've been building for a while now.
Is it at all possible?
That is really not an efficient way to use auto-increment columns (databases are not designed to do that). You'd wind up doing effectively a table scan any time you inserted or removed an item.
If you want to have an ordering that you can insert at will, it'd be better to maintain a separate mapping (e.g. have a separate entry that stores a list of IDs that you can manipulate at will).
For task 1, you would need to manually set the AUTO_INCREMENT value for that table after modifying all of the IDs so there's no gap when the next insert happens. A far better way is to not use an auto-increment ID at all - have an ID column, and use MAX(id)+1 to find the required ID at the time of record insertion.
here's my psuedocode:
for a deletion:
$n = 12;
DELETE from items where id='$n';
UPDATE items SET id=id-1 WHERE id > '$n'
for a specified insertion:
$n = 15;
(if there is a record with id = 15)
UPDATE items SET id=id+1 WHERE id >= '$n'
INSERT into items (id,..) values ('$n',...)
for an unspecified insertion
$n = SELECT MAX(id)+1 FROM items
INSERT into items (id,..) values ('$n',...)
For #1 try this SQL commands:
DELETE FROM table WHERE id=12;
UPDATE table set id=id-1 WHERE id > 12;
ALTER TABLE table AUTO_INCREMENT=AUTO_INCREMENT-1;
But it is a bad idea to use ID field like this. You shouldn't modify it's value after row was inserted into table.
You can use this for your 1) question. Run the whole block together. #id should be set to the row you want to delete.
SET #id = 12;
DELETE FROM tbl WHERE id = #id;
UPDATE tbl set id = id-1 WHERE id > #id;
ALTER TABLE tbl AUTO_INCREMENT = 0;
MySQL disclaimer: You cannot reset the counter to a value less than or equal to any that have already been used. For MyISAM, if the value is less than or equal to the maximum value currently in the AUTO_INCREMENT column, the value is reset to the current maximum plus one. For InnoDB, you can use ALTER TABLE ... AUTO_INCREMENT = value as of MySQL 5.0.3, but if the value is less than the current maximum value in the column, no error occurs and the current sequence value is not changed.
I have a table called 'messages' (INNODB) where the user can insert their own posts.
I want to put a limitation. In my php script when the table gets to 10 records, you can not add more. The logic of the program is more or less as follows.
Step 1. I run a query to count the lines that are in the table.
Step 2. Recovered that value, I decide whether to insert a new post.
My difficulty is in properly managing the possibility of two user who do the same thing simultaneously. If the user A is in step 1 while user B has just finished entering the tenth post, user A will include the eleventh.
How to avoid it?
You can create CHAR(1) NOT NULL field and cover it with UNIQUE INDEX. This will prevent of inserting more than 10 rows.
Other solution that could work would be to create BEFORE INSERT trigger that checks number of rows and raises error if there are more than 10 (look here for sample) (but in this case you can fail with condition races).
In order to allow you to change your threshold value for the table, you can use a trigger. Because MySQL triggers don't have a "prevent INSERT" option, you need a value in your table set to NOT NULL. The trigger can then set the inserted value for that column to NULL which will prevent the INSERT if your condition check fails.
A trigger like this:
CREATE TRIGGER block_insert
BEFORE INSERT ON table_name
FOR EACH ROW
BEGIN
DECLARE count INT;
SELECT COUNT(*)
FROM table_name INTO count;
IF count >= 10
THEN
SET NEW.non_nullable_value = NULL;
END IF;
END;
would fail if you inserted an 11th row, like this:
ERROR 1048 (23000): Column 'non_nullable_value' cannot be null
You may wish to set the non-nullable column's name to something that represents its use. You could improve this by having the trigger pull the limit value from a configuration table.
Update
To avoid having to use the non-nullable columns, you could alternatively create an error procedure, and CALL it from your trigger - similar to the example in the "Emulating Check Constraints" section of this page - they're referencing Oracle databases, where a check constraint achieves what you want, but MySQL doesn't support them.
The "error procedure" in the example performs an INSERT of a duplicate row into an error table, causing a unique key error and stops the parent transaction also.
Update 2
As pointed out in the comment below, multiple simultaneous transactions may get round the checks - you'll have to use LOCK TABLES <name> WRITE in order to ensure that they can't.
2/ You can also lock the MySQL table.
execute : LOCK TABLES my_table
Then do your business rules.
execute : UNLOCK TABLES
This also ensure that each action is sequentially executed. (but you have to deal with performance overhead)
Hope this could be useful.
Updated since the comments below : transaction don't work in this case
1/ You are using InnoDB, you can also use database transaction
Open transaction
Then do your business rules.
Commit or rollback your transaction
This will ensure that each action are executed one after another.