I have to insert data into two tables, Items and Class_Items. (A third table, Classes is related here, but is not being inserted into).
The primary key of Items is Item_ID, and it's an auto-incrementing integer. Aside from this primary key, there are no unique fields in Items. I need to know what the Item_ID is to match it to Classes in Class_Items.
This is all being done through a PHP interface. I'm wondering what the best way is to insert Items, and then match their Item_ID's into Class_Items. Here are the two main options I see:
INSERT each Item, then use mysql_insert_id() to get its Item_ID for the Class_Items INSERT query. This means one query for every Item (thousands of queries in total).
Get the next Autoincrement ID, then LOCK the Class_Items table so that I can just keep adding to an $item_id variable. This would mean just two queries (one for the Items, one for the Class_Items)
Which way is best and why? Also, if you have an unlisted alternative I'm open to whatever is most efficient.
The most efficient is probably going to be to use parameterized queries. That would require using the mysqli functions, but if you're to the point of needing to optimize this kind of query you should think about being there anyway.
No matter how you cut it, you've got two inserts to make. Doing the first, grabbing the new ID value as you've described (which imposes insignificant overhead, because the value is on hand to mysql already,) and using it in the second insert is pretty minimal.
I would investigate using stored procedures and/or transactions to make sure nothing bad happens.
I'm working on a project with mysql and what I did is the following (without using autoincrement fields):
1- I created a table called SEQUENCE with one field of type BIGINT called VALUE with an initial value of 1. This table will store the id value that will be incremented each time you insert a new record.
2- Create a store procedure and handle the id increment inside it within a transaction.
Here is an example.
CREATE PROCEDURE `SP_registerUser`(
IN _username VARCHAR(40),
IN _password VARCHAR(40),
)
BEGIN
DECLARE seq_user BIGINT;
START TRANSACTION;
#Validate that user does not exist etc..........
#Register the user
SELECT value FROM SEQUENCE INTO seq_user;
UPDATE SECUENCE SET value = value + 1;
INSERT INTO users VALUES(seq_user, _username, SHA1(_password));
INSERT INTO user_info VALUES(seq_user, UTC_TIMESTAMP());
COMMIT;
END //
In my case I want to store the user id in two different tables (users and user_info)
Related
In my database (MySQL) I have a table (MyISAM) containing a field called number. Each value of this field is either 0 or a positive number. The non zero values must be unique. And the last thing is that the value of the field is being generated in my php code according to value of another field (called isNew) in this table. The code folows.
$maxNumber = $db->selectField('select max(number)+1 m from confirmed where isNew = ?', array($isNew), 'm');
$db->query('update confirmed set number = ? where dataid = ?', array($maxNumber, $id));
The first line of code select the maximum value of the number field and increments it. The second line updates the record by setting it freshly generated number.
This code is being used concurrently by hundreds of clients so I noticed that sometimes duplicates of the number field occur. As I understand this is happening when two clients read value of the number field almost simultaneously and this fact leads to the duplicate.
I have read about the SELECT ... FOR UPDATE statement but I'm not quite sure it is applicable in my case.
So the question is should I just append FOR UPDATE to my SELECT statement? Or create a stored procedure to do the job? Or maybe completely change the way the numbers are being generated?
This is definitely possible to do. MyISAM doesn't offer transaction locking so forget about stuff like FOR UPDATE. There's definitely room for a race condition between the two statements in your example code. The way you've implemented it, this one is like the talking burro. It's amazing it works at all, not that it works badly! :-)
I don't understand what you're doing with this SQL:
select max(number)+1 m from confirmed where isNew = ?
Are the values of number unique throughout the table, or only within sets where isNew has a certain value? Would it work if the values of number were unique throughout the table? That would be easier to create, debug, and maintain.
You need a multi-connection-safe way of getting a number.
You could try this SQL. It will do the setting of the max number in one statement.
UPDATE confirmed
SET number = (SELECT 1+ MAX(number) FROM confirmed WHERE isNew = ?)
WHERE dataid = ?
This will perform badly. Without a compound index on (isNew, number), and without both those columns declared NOT NULL it will perform very very badly.
If you can use numbers that are unique throughout the table I suggest you create for yourself a sequence setup, which will return a unique number each time you use it. You need to use a series of consecutive SQL statements to do that. Here's how it goes.
First, when you create your tables create yourself a table to use called sequence (or whatever name you like). This is a one-column table.
CREATE TABLE sequence (
sequence_id INT NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`sequence_id`)
) AUTO_INCREMENT = 990000
This will make the sequence table start issuing numbers at 990,000.
Second, when you need a unique number in your application, do the following things.
INSERT INTO sequence () VALUES ();
DELETE FROM sequence WHERE sequence_id < LAST_INSERT_ID();
UPDATE confirmed
SET number = LAST_INSERT_ID()
WHERE dataid = ?
What's going on here? The MySQL function LAST_INSERT_ID() returns the value of the most recent autoincrement-generated ID number. Because you inserted a row into that sequence table, it gives you back that generated ID number. The DELETE FROM command keeps that table from snarfing up disk space; we don't care about old ID numbers.
LAST_INSERT_ID() is connection-safe. If software on different connections to your database uses it, they all get their own values.
If you need to know the last inserted ID number, you can issue this SQL:
SELECT LAST_INSERT_ID() AS sequence_id
and you'll get it returned.
If you were using Oracle or PostgreSQL, instead of MySQL, you'd find they provide SEQUENCE objects that basically do this.
Here's the answer to another similar question.
Fastest way to generate 11,000,000 unique ids
Just looking for some tips and pointers for a small project I am doing. I have some ideas but I am not sure if they are the best practice. I am using mysql and php.
I have a table called nomsing in the database.
It has a primary key called row id which is an integer.
Then I have about 8 other tables referencing this table.
That are called nomplu, accsing,accplu, datsing, datplu for instance.
Each has a column that references the primary key of nomsing.
Withing my php code I have all the information to insert into the tables except one thing , the row id primary key of the nomsing table. So that php generates a series of inserts like the following.
INSERT INTO nomsing(word,postress,gender) VALUES (''велосипед","8","mask").
INSERT INTO nomplu(word,postress,NOMSING?REFERENCE) VALUES (''велосипеды","2",#the reference to the id of the first insert#).
There are more inserts but this one gets the point across. The second insert should reference the auto generated id for the first insert. I was this to work as a transaction so all inserts should complete or none.
One idea I have is to not auto generate the id and generate it myself in php. That way would know the id given before the transaction but then I would have to check if the id was already in the db.
Another idea I have is to do the first insert and then query for the row id of that insert in php and then make the second insert. I mean both should work but they don't seem like an optimal solution. I am not too familiar with the database transactional features but what would be the best approach to do in this case. I don't like the idea of inserting then querying for the id and then running the rest of the queries. Just seems very inefficient or perhaps I am wrong.
Just insert a row in the master table. Then you can fetch the insert id ( lastInserId when on PDO) and use that to populate your other queries.
You could use the php version as given by JvdBerg , or Mysql's LAST_INSERT_ID. I usually use the former option.
See a similar SO question here.
You could add a new column to the nomsing table, called 'insert_order' (or similar) with a default value of 0, then instead of generating one SQL statement per insert create a bulk insert statement e.g.
INSERT INTO nomsing(word,postress,gender, insert_order)
VALUES (''велосипед","8","mask",1), (''abcd'',"9","hat",2).....
you generate the insert_order number with a counter in your loop starting at one. Then you can perform one SELECT on the table to get the ids e.g.
SELECT row_id
FROM nomsing
WHERE insert_order > 0;
now you have all the IDs you can now do a bulk insert for your following queries. At the end of your script just do an update to reset the insert_order column back to 0
UPDATE nomsing SET insert_order = 0 WHERE insert_order > 0;
It may seem messy to add an extra column to do this but it will add a significant speed increase over performing one query at a time.
Ok, here's what I wanna do.
Table1(globalID [primary key, autoincremented] (is a global id i would use in PHP))
Table2 & table3(ID* [foreign primary key, referencing to globalID], other columns after this, different for each of the tables 2 and 3);
What I wannt to know is if there is a way, and how, to make it so that when I insert a row in table2 or table3 the globalID in talbe1 gets populated and it's value also inserted in table2 or table3 accordingly as an ID. I assume it would be accomplished with a trigger, but triggers are not my forte so if that's the only way it can be done please provide an example.
I was also thinking of expanding it to this scenario:
Table1(globalID [primary key, autoincremented] (is a global id i would use in PHP),
OtherID [an UUID]);
Table2 & table3(ID* [foreign primary key, referencing to OtherID], other columns after this, different for each of the tables 2 and 3);
But in that case there is still the almost nonexistent possibility that 2 users might generate the same UUID while inserting a new row in some table. I was wondering if i could avoid that by making the keys generated by the sql server automatically without me having to code it PHP server side.
If anyone has had this problem worked out and can also point out other things i need to watch out for or take into account please point them out. Also please provide an example of the solution.
EDIT:
#John B.
EDIT BY TosheX
thank you for the reply. I have been reading up on this in the meantime and i already have a solution outside of SQL (in php). Basically I use Yii PHP framework in which I create a model of the table, and create an active record to populate a new row. now when I do that the ID in table1 is generated, and afterwards I still have a variable that points to that populated row and I can just read the generated ID since it's automatically fetched (without having to check for the last record in the database, which may return the record someone created milliseconds after me). Afterwards I just need to create a row inside the appropriate table2 or 3 and assign the already generated value for the ID.
i found the idea here:
http://www.yiiframework.com/forum/index.php/topic/14593-alternate-for-mysql-insert-id/
I really wanted an inbuilt solution in the database, but since I have experience with databases and do know enough about triggers (enough to not like them :D ) i know it's a very tricky code to get right so I was wondering if there are alternatives to auto-populate or something.
Still, I really appreciate your response, since you took your time and all. I have been working in msSQL more too, and I find logic in what you said. UUID in mySQL is the same as GUID in msSQL and I have considered that too as you can see.
Taking all this into account I will go with the solution i found, but I will accept your answer since you did try to help out, and you did bring some good ideas to the table.
Thanks again.
I'm a mssql person, but I dont think it is possible, unless you can create a "BEFORE INSERT" trigger for your db (and even then it will be problematic), because:
Whatever the ID of Tables 2 & 3 is, be it an integer or a guid, these are primary keys: Being a primary key, this column will not allow nulls, yet what you require is essentially to create new rows in these tables, with this column null, and then generate a new row in Table1, and use the new value in that table to update the new row in either Table2 or Table3.
Therefore, you'll probably need to either:
Do a solution in code (ie outside of SQL) whereby you cause the creation of a new row in Table1 first, then use this value when subsequently creating in Table2/3, or:
On tables2/3 have a autoincremented column for your primary key lets call it "PK", then have ID as a separate ordinary FK but which allows nulls, plus a trigger as follows (though you might need to fiddle with the syntax a little bit, because I am a mssql person, and I don't know your db's syntax) :-(
Also this example is designed to work with a multiple row insert, and to achieve this uses a cursor, and my syntax for cursors is dodgy at best even in mssql!
CREATE TRIGGER trg_Table2 ON dbo.Table2
FOR INSERT /* might be AFTER INSERT etc */
AS BEGIN
SET nocount on;
Declare ICursor Scroll For SELECT PK FROM Inserted;
Declare #PK int,
#GlobalID int,
#ID guid;
OPEN ICursor
FETCH Next From ICursor INTO #PK
WHILE ##Fetch_Status BEGIN
INSERT Table1 (OtherID) VALUES (NewID());
SELECT #GlobalID = ##Identity
SELECT #ID = OtherID FROM Table1 WHERE GlobalID = #GlobalID
UPDATE Table2 SET ID = #ID WHERE PK = #PK
FETCH Next FROM ICursor INTO #PK
END
Close ICursor
Deallocate ICursor
END
Hope this helps
I decided back when I was coding to have different tables for each type of content. Now I am stuck solving this. Basically my notification system ranks the newest content by its timestamp currently. This is inaccurate however because there is a small chance that someone would submit content at the same time as another person, and incorrect ranking would occur.
Now if I had all my content in a single table, I would simply rank it by an auto-incrementing variable. Is there a way to implement this auto-increment integer across multiple tables (e.g. When something is inserted into table1, id=0, something is inserted into table2, id=1). Or do I have to recode all my stuff into a single table.
NOTE:
The reason I have content in multiple tables is because its organized and it would reduce load stress. I don't really care about the organization anymore, because I can just access the data through a GUI I coded, I'm just wondering about the load stress.
EDIT:
I'm using PHP 5 with MySQL.
Your question, particularly the need for ID spanning over multiple tables, is clearly signalizing that your database design needs change. You should make one table for all content types (as a generalization), with autoincrementing ID. Then, for each particular content type, you can define other table (equivalent of inheritance in OOP) with extra fields, and foreign key pointing to the basic table.
In other words, you need something like inheritance in SQL.
You can create a table with auto increment id just to keep track of ids. Your program would do an insert on that table, get the id, use it as necessary.
Something along the lines of:
function getNextId() {
$res = mysql_query("INSERT INTO seq_table(id) VALUES (NULL)");
$id = mysql_insert_id();
if ($id % 10 == 0) {
mysql_query("DELETE FROM seq_table");
}
return $id;
}
Where seq_table is a table that you've to create just to get the ids. Make it a function so it can be used whenever you need. Every 10 ids generated I delete all generated ids, anyway you don't need them there. I don't delete every time since it would slow down. If another insert happen in the meantime and I delete 11 or more records, it doesn't affect the behaviour of this procedure. It's safe for the purpose it has to reach.
Even if the table is empty new ids will just keep on growing since you've declared id as auto-increment.
UPDATE: I want to clarify why the ID generation is not wrapped in a transaction and why it shouldn't.
If you generate an auto id and you rollback the transaction, the next auto id, will be incremented anyway. Excerpt from a MySQL bug report:
[...] this is not a bug but expected behavior that happens in every RDBMS we know. Generated values are not a part of transaction and they don't care about other statements.
Getting the ID with this procedure is perfectly thread safe. Your logic after the ID is obtained should be wrapped in a transaction, especially if you deal with multiple tables.
Getting a sequence in this way isn't a new concept, for instance, the code of metabase_mysql.php which is a stable DB access library has a method called GetSequenceNextValue() which is quite similar.
In a single table, you could have a field for the content type and clustered index that includes the content type field. This effectively keeps all of one content type in one place on the disc, and another content type in another place, etc. (It's actually organised into pages, but this physical organisation is still true.)
Assuming that each content type has the same fields, this would likely meet your needs and behave similarly to multiple tables. In some cases you may even find that, with appropriate indexes, a single table solution can be faster, more convenient and maintainable, etc. Such as trying to create global unique identifiers across all content types.
If you're unable to merge these back into a single table, you could create a central link table...
CREATE TABLE content_link (
id INT IDENTITY(1,1), -- MS SQL SERVER syntax
content_type INT,
content_id INT -- The id from the real table
)
As you insert into the content tables, also insert into the link table to create your globally unique id.
More simply, but even more manually, just hold a single value somewhere in the database. Whenever you need a new id, use that centrally stored value and increment it by one. Be sure to wrap the increment and collection in a single transaction to stop race conditions. (This can be done in a number of ways, depending on your flavor of SQL.)
EDIT
A couple of MySQL example lines of code from the web...
START TRANSACTION;
INSERT INTO foo (auto,text)
VALUES(NULL,'text'); # generate ID by inserting NULL
INSERT INTO foo2 (id,text)
VALUES(LAST_INSERT_ID(),'text'); # use ID in second table
COMMIT TRANSACTION;
Personally, I'd actually store the value in a variable, commit the transaction, and then continue with my business logic. This would keep the locks on the tables to a minimum.
You could have a separate ID table, insert into that, and use the newly-inserted ID.
e.g.
CREATE TABLE ids (INT UNSIGNED AUTO INCREMENT PRIMARY KEY, timeadded DATETIME);
In the script:
<?php
$r = mysql_query('INSERT INTO ids (timeadded) VALUES (NOW())');
$id = mysql_insert_id();
mysql_query("INSERT INTO someOtherTable (id, data) VALUES ('$id', '$data)");
Add error checking etc. to taste.
The MySQL manual states:
The ID that was generated is maintained in the server on a
per-connection basis. This means that the value returned by the
function to a given client is the first AUTO_INCREMENT value generated
for most recent statement affecting an AUTO_INCREMENT column by that
client. This value cannot be affected by other clients, even if they
generate AUTO_INCREMENT values of their own. This behavior ensures
that each client can retrieve its own ID without concern for the
activity of other clients, and without the need for locks or
transactions.
(Source) So I don't think concerns about ACID complians are a problem.
How can we re-use the deleted id from any MySQL-DB table?
If I want to rollback the deleted ID , can we do it anyhow?
It may be possible by finding the lowest unused ID and forcing it, but it's terribly bad practice, mainly because of referential integrity: It could be, for example, that relationships from other tables point to a deleted record, which would not be recognizable as "deleted" any more if IDs were reused.
Bottom line: Don't do it. It's a really bad idea.
Related reading: Using auto_increment in the mySQL manual
Re your update: Even if you have a legitimate reason to do this, I don't think there is an automatic way to re-use values in an auto_increment field. If at all, you would have to find the lowest unused value (maybe using a stored procedure or an external script) and force that as the ID (if that's even possible.).
You shouldn't do it.
Don't think of it as a number at all.
It is not a number. It's unique identifier. Think of this word - unique. No record should be identified with the same id.
1.
As per your explanation provided "#Pekka, I am tracking the INsert Update and delete query..." I assume you just some how want to put your old data back to the same ID.
In that case you may consider using a delete-flag column in your table.
If the delete-flag is set for some row, you shall consider program to consider it deleted. Further you may make it available by setting the delete-flat(false).
Similar way is to move whole row to some temporary table and you can bring it back when required with the same data and ID.
Prev. idea is better though.
2.
If this is not what you meant by your explanation; and you want to delete and still use all the values of ID(auto-generated); i have a few ideas you may implement:
- Create a table (IDSTORE) for storing Deleted IDs.
- Create a trigger activated on row delete which will note the ID and store it to the table.
- While inserting take minimum ID from IDSTORE and insert it with that value. If IDSTORE is empty you can pass NULL ID to generate Auto Incremented number.
Of course if you have references / relations (FK) implemented, you manually have to look after it, as your requirement is so.
Further Read:
http://www.databasejournal.com/features/mysql/article.php/10897_2201621_3/Deleting-Duplicate-Rows-in-a-MySQL-Database.htm
Here is the my case for mysql DB:
I had menu table and the menu id was being used in content table as a foreign key. But there was no direct relation between tables (bad table design, i know but the project was done by other developer and later my client approached me to handle it). So, one day my client realised that some of the contents are not showing up. I looked at the problem and found that one of the menu is deleted from menu table, but luckily the menu id exist in cotent table. I found the menu id from content table that was deleted and run the normal insert query for menu table with same menu id along with other fields. (Id is primary key) and it worked.
insert into tbl_menu(id, col1, col2, ...) values(12, val1, val2, ...)