My question is pretty simple but answer might be tricky.
I'm in PHP and I want to manage manually a unique ID for my objects.
What is tricky is to manage atomicity. I dont want that 2 elements get the same ID.
"Elements" are grouped in "Groups". In each group I want elements ID starting from 1 and grow incrementally for each insert in that group.
My first solution is to have a "lastID" column in the table "Groups" :
CREATE TABLE groups ( id INT AUTO_INCREMENT, lastId INT )
CREATE TABLE elements ( myId INT, multiple values ...)
In order to avoid many elements with the same ID, I have to update lastId and select it in an atomic SQL Query.
After that, one retrieved, I have a unique ID that can't be picked again and I can insert my element.
My question is how to solve the bold part ? My database is MySQL with MyISAM engine so there is no transaction support.
UPDATE groups
SET lastId = lastId + 1
WHERE id = 42
SELECT lastId
FROM groups
WHERE id = 42
Is there something more atomic than these 2 requests ?
Thanks
UPDATE groups SET lastId = last_insert_id(lastId + 1)
and then you can get your new id with
SELECT last_insert_id()
Using last_insert_id with a parameter will store the value and return it when you call it later.
This method of generating autonumbers works best with MyISAM tables having only a few rows (MyISAM always locks the entire table). It also has the benefit of not locking the table for the duration of the transaction (which will happen if it is an InnoDB table).
This is from the MySQL manual:
If expr is given as an argument to LAST_INSERT_ID(), the value of the
argument is returned by the function and is remembered as the next
value to be returned by LAST_INSERT_ID(). This can be used to simulate
sequences:
Create a table to hold the sequence counter and initialize it:
CREATE TABLE sequence (id INT NOT NULL);
INSERT INTO sequence VALUES (0);
Use the table to generate sequence numbers like this:
UPDATE sequence SET id=LAST_INSERT_ID(id+1);
SELECT LAST_INSERT_ID();
The UPDATE statement increments the sequence counter
and causes the next call to LAST_INSERT_ID() to return the updated
value. The SELECT statement retrieves that value. The
mysql_insert_id() C API function can also be used to get the value.
See Section 21.8.3.37, “mysql_insert_id()”.
You can generate sequences without calling LAST_INSERT_ID(), but the
utility of using the function this way is that the ID value is
maintained in the server as the last automatically generated value. It
is multi-user safe because multiple clients can issue the UPDATE
statement and get their own sequence value with the SELECT statement
(or mysql_insert_id()), without affecting or being affected by other
clients that generate their own sequence values.
One option is for you to use the nifty MyISAM feature that let's auto_increment values be incremented for each group.
CREATE UNIQUE INDEX elements_ix1 ON elements (groupId, myID)
myID INT NOT NULL AUTO_INCREMENT
That's more "atomic" than anything that involves updating a separate table. Note that this only works for MyISAM, not InnoDB.
excerpt from http://dev.mysql.com/doc/refman/5.1/en/example-auto-increment.html
MyISAM Notes
For MyISAM tables, you can specify AUTO_INCREMENT on a secondary column in a multiple-column index. In this case, the generated value for the AUTO_INCREMENT column is calculated as MAX(auto_increment_column) + 1 WHERE prefix=given-prefix. This is useful when you want to put data into ordered groups.
I would assume your MySQL installation also has InnoDB engine which does support transactions. You just need to change the engine type of you tables.
Related
So, there is this situation when you have a table in which you want to insert rows in pairs with a reference to each other. Just like in double-entry accounting, every item has it's opposite as pair of it. There is this table:
CREATE SEQUENCE tbl_item_id_seq
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;
CREATE TABLE tbl_item (
id integer NOT NULL PRIMARY KEY DEFAULT nextval('tbl_item_id_seq'),
pair_id integer,
label character varying(50) NOT NULL,
FOREIGN KEY (pair_id) REFERENCES tbl_item (id)
);
ALTER SEQUENCE tbl_item_id_seq OWNED BY tbl_item.id;
The items are generated procedurally. Usually, there are multiple pairs generated at once, and the ultimate goal would be to insert all the pairs with one query. I have solved this with PHP where I inserted a row, returned it's id, inserted the other row with pair_id filled and updated the first row with the id of the second. This means 3 db query started from PHP, and since there are multiple pairs generated, it means number_of_pairs * 3 queries. When I have about 100 pairs, it means 300 queries and gives a nice overhead in processing time what I would like to minimize.
So, the question is given, what's the fastest way to insert pairs of rows with a reference to each other's id into a single table in PSQL?
You could reserve some ids :
select nextval('tbl_item_id_seq') from generate_series(1,200)
then manually assign the id/pair_id. This way, the inserts could even be a single COPY statement(if your Postgres driver supports it).
I'm using PHP to insert groups of records into a MySQL DB.
Whenever I insert a group of records, I want to give that group a unique set ID that is incremented by 1 for each group of records in the DB.
Currently, I'm checking the latest set ID in the DB and incrementing it by 1 for each new set of records.
The thing that scares me though is what happens if I query the DB to get the latest set ID, and before I can insert a new set of records with that set ID + 1, another insert occurs on the table thus taking the set ID I was about to use?
While fairly unlikely, something like that could greatly sacrifice the integrity of the data.
What can I do to prevent such a thing from happening? Is there any way to temporarily lock the DB table so that no other inserts can occur until I have performed a SELECT/INSERT combo?
Locking the table is one option, but that approach impacts concurrency.
The approach I would recommend is that you use a separate table with AUTO_INCREMENT column, and use a separate INSERT into that table, and a SELECT LAST_INSERT_ID() to retrieve the auto_increment value.
And then use that value as the group identifier for the group of rows you insert into your original table.
The basic approach is:
LOCK TABLE foo WRITE;
SELECT MAX(id) + 1 FROM foo
INSERT ...
INSERT ...
UNLOCK TABLES;
Locking the table prevents any other process from changing the table until you explicitly unlock it.
Having said that, seriously consider just using a table with an AUTO_INCREMENT column. MySQL will do the work of maintaining unique keys wholly automatically, and then you can simply refer to those keys from your existing table.
I am having problem to update the list of id number again starting from 1,2,3,4,5. Since I have deleted few records as I was testing the sql commands. Can you please help on how to make this id column again starting from 1.
I could just the name of the id number however if I do that then when I input new record, it will again start from the previous number which was 66.
ID Name
1 A
32 B
34 C
35 D
55 E
66 F
Truncate your table first and then execute this
ALTER TABLE tablename AUTO_INCREMENT = 1
You should truncate the table to reseed it properly and not just use alter table
(tldr; it's usually better not to worry about the density or sequential order an auto-increment column.)
It is not possible1 to use an AUTO_INCREMENT to automatically fill in values less than MAX(ID).
However, the auto increment ID can be reset if existing IDs are updated. The compacting phase is required because MySQL does not allow "filling in gaps" via an auto-increment column.
Compact the existing IDs, like so:
SET #i := 0;
UPDATE t id = #i := (#i+1)
Important: Make sure that all relational usage is identified in the form of Foreign Key relations with CASCADE ON UPDATE before this is done or the data may become irreversibly corrupted.
Assign the auto-ID see to the maximum1 ID value after compacting:
ALTER TABLE t AUTO_INCREMENT = (SELECT MAX(id) FROM t)
1 Per the AUTO_INCREMENT documentation in ALTER TABLE:
You cannot reset the counter to a value less than or equal to the value that is currently in use .. if the value is less than or equal to the maximum value currently in the AUTO_INCREMENT column, the value is reset to the current maximum AUTO_INCREMENT column value plus one.
The rule means that it is not possible to set the increment ID lower than an already used ID; in addition, manually assigning a value higher will automatically raise the AUTO_INCREMENT value.
The easiest (and sometimest fastest) way is to remove column and add it back. Updating column may screw up indexes or make a mess with values. Droping whole table got no sense. But remember that if other columns refer to that ids you can damage your app.
I want to do an auto increment id at my mySQL as my id. And this id will saved in mySQL. Sorry for my poor English; maybe most of you will not understand what im talking about. Let me show an example.
like this:
abc/def/001(001)until abc/def/001(100)
The auto increment id will auto change to
abc/def/002(001).
Is this possible to do it?
Can anyone help me?
Thanks a lot.
(i would like to update my question)
can this possible write with php code or Generator with php code and save into mysql?
You could use 2 columns to achieve that. one of them a normal auto increment integer value, the second one, which is updated whenever a new row is inserted either with a trigger or manually using CONCAT():
$query = "INSERT INTO table1 (column1,colum2,column3) VALUES (1,2,3)";
mysqli_query($link,$query);
$last_insert_id = mysqli_insert_id($link);
$query = "UPDATE table1 SET my_auto_increment = CONCAT('abc/def/', id) WHERE id = '$last_inserted_id'" // where id is an integer auto increment field
mysqli_query($link,$query);
This cannot be done using the MySQL AUTO_INCREMENT. At least, not directly as you describe. But it's possible to emulate that behavior.
But first, as an aside, I question the need for this type of concatenated identifier in the database. It's problematic, and is going to cause more problems and grief down the road, more than you anticipate right now. If I had to include a column like this, I wouldn't make it a primary key.
I strongly advocate surrogate integer keys, because they meet all the desirable aspects of the ideal primary key: simple, unique, anonymous, immutable, et al. There are two schools of thought on surrogate keys: (1) those that think surrogate keys are a really good thing, and (2) those who haven't been burned by choosing a natural key, yet.
I don't know what problem you are trying to solve, or know what your requirements are. But if you do have an option that avoids storing this type of identifier in the database, I strongly recommend you seriously consider availing yourself of that option.
There's good reason that MySQL doesn't natively support generating identifiers like yours. They are trouble prone, cause grief, and are a bad idea.
With that advertisement aside...
In order for the database to "automatically" generate the values for that column (that is, if the value for that column is not being provided as a value in a SQL insert statement), then you could define a BEFORE INSERT trigger to generate it for you.
The trigger could make use of an integer AUTO_INCREMENT column to generate unique values, and then use that integer value as part of an expression to generate the character string you need.
This demonstration query shows an expression that will generate the string in the specified format... i is the unique integer value, a is an expression to generate the first integer, demonstrates how to get the first integer component, b is the second integer component, and val concatenates that together in the specified format.
SELECT t.i
, (t.i DIV 100 - IF(t.i MOD 100,0,1)) a
, (t.i MOD 100 + IF(t.i MOD 100,0,100)) b
, CONCAT('abc/def/'
,IF(t.i>100000
,(t.i DIV 100 - IF(t.i MOD 100,0,1))
,RIGHT(CONCAT('00',(t.i DIV 100 - IF(t.i MOD 100,0,1))),3)
)
,'('
,RIGHT(CONCAT('00',(t.i MOD 100 + IF(t.i MOD 100,0,100))),3)
,')')
AS val
FROM ( SELECT 101 AS i
UNION ALL SELECT 199
UNION ALL SELECT 200
UNION ALL SELECT 201
UNION ALL SELECT 299
UNION ALL SELECT 300
UNION ALL SELECT 301
UNION ALL SELECT 100000
UNION ALL SELECT 100001
) t
Note: to avoid that first integer component from doing a "wrap" for values of i greater than 99900, that first component will need to be more than three places.
To put that expression into a trigger to automatically generate the bizarre identifier value, you could do something like this:
CREATE TABLE mytable
( id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY
, val VARCHAR(80)
);
Note that the AUTO_INCREMENT column in the table is defined to be UNSIGNED, so we won't allow negative values. (Our expression above isn't designed to handle values less than 100, since the starting point was specified as "001(001)", we're going to have our algorithm have that correspond to an integer value of 1.
DELIMITER $$
CREATE TRIGGER trg_mytable_bi
BEFORE INSERT ON mytable
BEGIN
DECLARE i BIGINT;
SET i = 100 + NEW.id;
SET NEW.val = CONCAT('abc/def/'
,IF(i > 100000
,((i DIV 100) - IF(i MOD 100,0,1))
,RIGHT(CONCAT('00',((i DIV 100) - IF((i MOD 100),0,1))),3)
)
,'('
,RIGHT(CONCAT('00',(i MOD 100 + IF(i MOD 100,0,100))),3)
,')');
END;
$$
DELIMITER ;
It's not clear where the 'abc/def' component is coming from. So, I've just hard coded it in the trigger.
If you want to run separate AUTO_INCREMENT integer values for different values of the 'abc/def/', such that you could have 'abc/def/001(001)' and 'ccc/ddd/001(001)'...
the MyISAM engine has a peculiar feature: if the AUTO_INCREMENT column is a secondary column in an index and is not the leading column in any other index, then MySQL will generate auto increment sequences for each distinct value in the leading portion of the index. (I've tested that, but never really used it, because that feature isn't available in the InnoDB engine.)
Just working with a database and some tests were done recently which checked the integrity of the setup.
As a result, a lot of test entries were added which were then deleted. However, when new entries are added, the ID number value continues from after the entries added.
What I want:
ID increases by one from where it left off before the additional rows were added:
4203, 4204, 4205, 4206 etc.
What is happening:
ID increases by one from after the additional rows ID:
4203, 4204, 6207, 6208 6209 etc.
Not sure where to fix this...whether in phpmyadmin or in the PHP code. Any help would be appreciated. Thanks!
I have ran into this before and I solve it easily with phpMyAdmin. Select the database, select the table, open the operations tab, and in the Table Options set the AUTO_INCREMENT to 1 then click GO. This will force mysql to look for the last auto incremented value and then set it to the value directly after that. I do this on a manually basis that way I know that when a row is skipped that it was not from testing but a deletion because when I test and delete the rows I fix the AI value.
I don't think there's a way to do this with an auto-incrementing ID key.
You could probably do it by assigning the ID to (select max(id) + 1 from the_table)
You could drop the primary key then recreate it, but this would reassign all the existing primary keys so could cause issues with relationships (although if you don't have any gaps in your primary key you may get away with it).
I would however say that you should accept (and your app should reflect) the possibility of missing IDs. For example in a web app if someone links to a missing ID you would want a 404 returned not a different record.
There should be no need to "reset" the id values; I concur with the other comments concerning this issue.
The behavior you observe with AUTO_INCREMENT is by design; it is described in the MySQL documentation.
With all that said, I will describe an approach you can use to change the id values of those rows "downwards", and make them all contiguous:
As a "stepping stone" first step, we will create a query that gets a list of the id values that we need changed, along with a proposed new id value we are going to change it to. This query makes use of a MySQL user variable.
Assuming that 4203 is the id value you want to leave as is, and you want the next higher id value to be reset to 4204, the next higher id to be reset to 4205, etc.
SELECT s.id
, #i := #i + 1 AS new_id
FROM mytable s
JOIN (SELECT #i := 4203) i
WHERE s.id > 4203
ORDER BY s.id
(Note: the constant value 4203 appears twice in the query above.)
Once we're satisfied that this query is working, and returning the old and new id values, we can use this query as an inline view (MySQL calls it a derived table), in a multi-table UPDATE statement. We just wrap that query in a set of parentheses, and give assign it an alias, so we can reference it like a regular table. (In an inline view, MySQL actually materializes the resultset returned by the query into a MyISAM table, which probably explains why MySQL refers to it as a "derived table".)
Here's an example UPDATE statement that references the derived table:
UPDATE ( SELECT s.id
, #i := #i + 1 AS new_id
FROM mytable s
JOIN (SELECT #i := 4203) i
WHERE s.id > 4203
ORDER BY s.id
) n
JOIN mytable t
ON t.id = n.id
SET t.id = n.new_id
ORDER BY t.id
Note that the old id value from the inline view is matched to the id value in the existing table (the ON clause), and the "new_id" value generated by the inline view is assigned to the id column (the SET clause.)
Once the id values are assigned, we can reset the AUTO_INCREMENT value on the table:
ALTER TABLE mytable AUTO_INCREMENT = 1;
NOTE: this is just an example, and is provided with the caveat that this should not be necessary to reassign id values. Ideally, primary key values should be IMMUTABLE i.e. they should not change once they have been assigned.