So my app needs to let users generate random alphanumeric codes like A6BU31, 38QV3B, R6RK7T. Currently they consist of 6 chars, whereas I and O are not used (so we got 34^6 possibilities). These codes are then printed out and used for something else.
I must now ensure that many users can "reserve" up to 100 codes per request, so user A might want to get 50 codes, user B wants to generate 10 and so on. These codes must be unique across all users, so user A and user B may not both receive the code ABC123.
My current approach (using PHP and MySQL) is to have two InnoDB tables for this:
One (the "repository") contains a large list of pre-generated codes (since the possibility of collisions will increase over time and I do not want to go the try-insert-if-fails-try-another-code approach). The repository contains just the codes and an auto-incremented ID (so I can sort them, see below).
The other table holds the reserved keys (i.e. code + owning user).
Whenever a user wants to reserve N keys, I planned to do the following
BEGIN;
INSERT INTO revered_codes (code,user_id)
SELECT code FROM repository WHERE 1 ORDER BY id LIMIT N;
DELETE FROM repository WHERE 1 ORDER BY id LIMIT N;
COMMIT;
This should work, but I'm not sure. It seems like I'm building a WTF solution.
After insertion I must select the just reserved codes to display them to the user. And that's the tricky part, since I don't really know how to identify the just reserved codes after my transaction is done. I could of course add just another column to my reserved_codes table, holding some kind of random token, but this seems even more WTFy.
My favorite solution would be to have a random number sequence, so that I can just perform INSERT operations in the reserved_codes table.
So, how to do this unique, random and transactional-safe sequence in MySQL? One idea was to have a regular auto-increment on the reserved_codes table and derive the random code value from that numeric column, but I was wondering whether there was a better way.
UPDATE: I forgot to mention that it would be advantagous to have a rather small table of reserved codes, as I later have to find single codes again for updating them (reserved_codes has a couple of more attributes to it). So letting the reserved table grow slowly is good (instead of having a huge index over ~1mio pre-generated codes).
If you already have a repository table, I would just add a user column and then run this query:
UPDATE repository SET user_id = ? WHERE user_id IS NULL LIMIT N;
Afterwards, you can select the records again. This had two distinct disadvantages:
you need an index on user_id
you can't use the codes in your table for anything else but binding it to users.
Related
I have got a table which has an id (primary key with auto increment), uid (key refering to users id for example) and something else which for my question won’t matter.
I want to make, lets call it, different auto-increment keys on id for each uid entry.
So, I will add an entry with uid 10, and the id field for this entry will have a 1 because there were no previous entries with a value of 10 in uid. I will add a new one with uid 4 and its id will be 3 because I there were already two entried with uid 4.
...Very obvious explanation, but I am trying to be as explainative an clear as I can to demonstrate the idea... clearly.
What SQL engine can provide such a functionality natively? (non Microsoft/Oracle based)
If there is none, how could I best replicate it? Triggers perhaps?
Does this functionality have a more suitable name?
In case you know about a non SQL database engine providing such a functioality, name it anyway, I am curious.
Thanks.
MySQL's MyISAM engine can do this. See their manual, in section Using AUTO_INCREMENT:
For MyISAM tables you can specify AUTO_INCREMENT on a secondary column in a multiple-column index. In this case, the generated value for the AUTO_INCREMENT column is calculated as MAX(auto_increment_column) + 1 WHERE prefix=given-prefix. This is useful when you want to put data into ordered groups.
The docs go on after that paragraph, showing an example.
The InnoDB engine in MySQL does not support this feature, which is unfortunate because it's better to use InnoDB in almost all cases.
You can't emulate this behavior using triggers (or any SQL statements limited to transaction scope) without locking tables on INSERT. Consider this sequence of actions:
Mario starts transaction and inserts a new row for user 4.
Bill starts transaction and inserts a new row for user 4.
Mario's session fires a trigger to computes MAX(id)+1 for user 4. You get 3.
Bill's session fires a trigger to compute MAX(id). I get 3.
Bill's session finishes his INSERT and commits.
Mario's session tries to finish his INSERT, but the row with (userid=4, id=3) now exists, so Mario gets a primary key conflict.
In general, you can't control the order of execution of these steps without some kind of synchronization.
The solutions to this are either:
Get an exclusive table lock. Before trying an INSERT, lock the table. This is necessary to prevent concurrent INSERTs from creating a race condition like in the example above. It's necessary to lock the whole table, since you're trying to restrict INSERT there's no specific row to lock (if you were trying to govern access to a given row with UPDATE, you could lock just the specific row). But locking the table causes access to the table to become serial, which limits your throughput.
Do it outside transaction scope. Generate the id number in a way that won't be hidden from two concurrent transactions. By the way, this is what AUTO_INCREMENT does. Two concurrent sessions will each get a unique id value, regardless of their order of execution or order of commit. But tracking the last generated id per userid requires access to the database, or a duplicate data store. For example, a memcached key per userid, which can be incremented atomically.
It's relatively easy to ensure that inserts get unique values. But it's hard to ensure they will get consecutive ordinal values. Also consider:
What happens if you INSERT in a transaction but then roll back? You've allocated id value 3 in that transaction, and then I allocated value 4, so if you roll back and I commit, now there's a gap.
What happens if an INSERT fails because of other constraints on the table (e.g. another column is NOT NULL)? You could get gaps this way too.
If you ever DELETE a row, do you need to renumber all the following rows for the same userid? What does that do to your memcached entries if you use that solution?
SQL Server should allow you to do this. If you can't implement this using a computed column (probably not - there are some restrictions), surely you can implement it in a trigger.
MySQL also would allow you to implement this via triggers.
In a comment you ask the question about efficiency. Unless you are dealing with extreme volumes, storing an 8 byte DATETIME isn't much of an overhead compared to using, for example, a 4 byte INT.
It also massively simplifies your data inserts, as well as being able to cope with records being deleted without creating 'holes' in your sequence.
If you DO need this, be careful with the field names. If you have uid and id in a table, I'd expect id to be unique in that table, and uid to refer to something else. Perhaps, instead, use the field names property_id and amendment_id.
In terms of implementation, there are generally two options.
1). A trigger
Implementations vary, but the logic remains the same. As you don't specify an RDBMS (other than NOT MS/Oracle) the general logic is simple...
Start a transaction (often this is Implicitly already started inside triggers)
Find the MAX(amendment_id) for the property_id being inserted
Update the newly inserted value with MAX(amendment_id) + 1
Commit the transaction
Things to be aware of are...
- multiple records being inserted at the same time
- records being inserted with amendment_id being already populated
- updates altering existing records
2). A Stored Procedure
If you use a stored procedure to control writes to the table, you gain a lot more control.
Implicitly, you know you're only dealing with one record.
You simply don't provide a parameter for DEFAULT fields.
You know what updates / deletes can and can't happen.
You can implement all the business logic you like without hidden triggers
I personally recommend the Stored Procedure route, but triggers do work.
It is important to get your data types right.
What you are describing is a multi-part key. So use a multi-part key. Don't try to encode everything into a magic integer, you will poison the rest of your code.
If a record is identified by (entity_id,version_number) then embrace that description and use it directly instead of mangling the meaning of your keys. You will have to write queries which constrain the version number but that's OK. Databases are good at this sort of thing.
version_number could be a timestamp, as a_horse_with_no_name suggests. This is quite a good idea. There is no meaningful performance disadvantage to using timestamps instead of plain integers. What you gain is meaning, which is more important.
You could maintain a "latest version" table which contains, for each entity_id, only the record with the most-recent version_number. This will be more work for you, so only do it if you really need the performance.
I have a database in mysql, and a table called Animals, I use this condition to add news records.
public function create()
{
$animals = Animals::all();
$last_animal_id = collect($animals)->last();
if ($last_animal_id->id == $last_animal_id->id) {
$last_animal_id->id = $last_animal_id->id + 1;
} else {
return false;
}
return view('animal.create-animals')->with('last_animal_id', $last_animal_id);
}
I work in laravel and php, and that is my controller 'AnimalsController', the condition add +1 to the last id that is registered in the table.
For example, I have 4 records and I delete the last record, without my condition, after I have added a new record the new record will take the value 6.
And that is the reason that I add manually new records, with this condition, the condition find the last id, and add +1 to the last id, not +2 if I not have this condition. Not directly, I pass the value to an input and then I send the form in my view.
Is possible to add +1 id in the table, if I delete a record in the middle, or before the last record? As the following example explains:
Table Animals
/*NOTE: The field 'id' HAVE THE FOLLOWING ATTRIBUTES:
AUTO_INCREMENT, IS 'NOT NULL','PRIMARY KEY', AND HIS TYPE IS 'INT'*/
id|name |class
1 |Dog |Mammal
2 |Cat |Mammal
3 |Sparrow|Bird
4 |Whale |Mammal
5 |Frog |Amphibian
6 |Snake |Reptile
Then I delete the id, 2, and 3.
In addition to the condition that already exists, I would like to create another condition that allows to add new records among the others, only if there are missing records in between of others.
Using the previous example:
I said that I will delete the id 2 and 3 right? The new condition must allow to create again the records with the id 2 and 3 between the records with the id 1 and 4.
If I delete another record the condition must perform the same function. Certainly replacing the records with corresponding id that were previously deleted.
For more details: I use a form to create new animals to the table Animals, previously I said in the example, that I will delete the records with id 2 and 3, then If the condition in my controller, and my form in my view, work properly then I can add again the animal with id 2, and then in a new form add again the animal with id 3.
Thus, if my question was not understood very well or you thought that my function should add the record(s) simultaneously, you understood it wrong, because It's that not the function that I would like to do in the function.
One thing to keep in mind when working with relational databases is that the id column is usually used to relate this data and as such it can and will appear in other tables. If you arbitrarily renumber things here, you're damaging those links and potentially scrambling up your data.
If ordering is important, create a column for that purpose, for example one called position or something similar. This one you can manipulate freely without concern about altering relations.
Generally your id value should be:
Always populated (e.g. NOT NULL)
Integer (e.g. INT or BIGINT)
Set as your primary key (e.g. PRIMARY KEY)
Generated automatically (e.g. AUTO_INCREMENT)
Never changed, it's permanently assigned
Never recycled and used for another record
Recycling id values is how you create enormous security problems. It's all too easy for a user to "inherit" all the data that came with an old user ID value you've recycled. The safest thing is to never, ever re-use these values.
They're just IDs. Forget about holes or lack of ordering. Any production database will end up with lots of interesting patterns there that are unavoidable, but it doesn't matter.
One exception to this is when creating seed databases. Here you can fuss over the ordering to get things arranged as you want because this is before you insert the data into the database.
At the end of the day you'll want to ensure that:
These numbers don't overflow (e.g. INT keyed table at 2.1 billion)
These numbers aren't exposed to users in a way that makes it possible to enumerate your table (e.g. ID value in a URL)
Just think of them as internal identifiers, like a serial number, and you'll be fine. In fact, MySQL now supports SERIAL as a datatype for this reason, that's an alias for BIGINT UNSIGNED NOT NULL AUTO_INCREMENT UNIQUE which is a good default for systems designed in 2018.
There is a really great answer from Tadman about the implications of your solution.
To give you an alternative to your own solution, you can do something like this....
First, create an order column, an int.
Them, instead of looking at the latest id value, do this...
$highestOrder = Animal::max('order');
And then 'up it'... :-) Just an idea.
BTW: to give you more options, you can look directly in a table as well:
DB::table('animals')->max('order');
... but I would not do that in this case. The model class is the best 'gateway' to this information, not the DB facade directly.
I'm trying to build a very simple login system for my site (just for practice for a project i'm working on). The way I've decided to implement it is use a table with fields for ID, Name, Password, and username and search for the entered information in the existing table.
For registration, it simply injects the information supplied into the table, and I would like to assign a customer ID number. My idea for assigning an ID number is to simply find the size of the ID column (which will contain the ID's 1,2,3..etc up to the end) and assign the new registration to the length +1. For this purpose i'll need a way to get the size of the column, but I'm just learning php and sql so i'm not sure what the syntax would be.
TLDR; is there a funtion in sql that I can use in php to get the length of a particular column? (i.e the number of entries stored in that column?)
Set the ID column to Primary and Auto increment.
you don't include that in your query it is created on its own.
You'd probably be better off just using an IDENTITY or AUTO_INCREMENT column. The problem with checking for the "size of the column" (by which I assume you mean the count of rows in that column) is that you could end up inserting duplicate IDs, for example:
ID | ...
---------
1
2
4
So if you did a SELECT COUNT(ID)+1 FROM MyTable, it would return 4, and you have an ID collision.
You could do something like SELECT MAX(ID)+1 FROM MyTable, but even then there could be concurrency problems (process A and process B both try to run that query at the same time, before either has a chance to insert the new ID of 5). You're really best off just letting your RDBMS take care of it..
I have a table called "posts" and it contain 500 posts but the ids are not sequence
like:
1
3
9
22
446
....
etc.
That's because I deleted some of the posts from the table.
So how can I re-correct the ids?
Primary Key IDs are not supposed to be changed, especially when they are referenced in other tables.
If you need a property that is like a row number, you can add another field for that.
For example invoices are numbered, but the invoice number should not be the primary key, since you want the freedom to re-number one of them without losing other connected information, such as invoice details in other tables.
The easiest way to fix it is to create a quick script to loop through the table and update that the id column and then run on your database: ALTER TABLE tbl AUTO_INCREMENT = 100;
NEVER EVER CHANGE THE ID!
Id is something the record borns with and dies with. That's why it's called id, it is an IDENTITY!
As in real life you cannot change the identity of things, you won't do it in database.
It is a very bad idea from the philosophic perspective, which also results in practical problems. Even if you would renumber the ID in all your tables in your database, the old IDs might still survive somewhere (and make a big mess then):
in URLs all over the internet
in your logs
in your backups
in other database copies.
Also, ID must serve only for identification and nothing else. For example: you use IDs to define order of some dictionary, which you normally present sorted. Then you need to add a new item, which must be presented between items with id 20 and 21. The BAD solution would be to change ID for records with ID >= 21. The GOOD solution is to add a new column Order, which defines the order of items and can be changed whenever needed.
Remember:
ID must serve only for identification and nothing else!
NEVER CHANGE THE ID!
Okay, so let's say I have a mysql database table with two columns, one is for id and the other is for password. If I have three rows of data and the id values go from 1 to 3 and I delete row 3 and then create another row of data, I will see id=4 instead of id=3 on the newly created row. I know this has to do with the auto increment value but I was wondering if I can add some code in a php file that will automatically reset all the id numbers such that you start at id=1 and go up to the last id number in increments of 1 after a row has been deleted?
My goal is to create a form where the user enters a password and the system will match the password with a password value in the database. If there is a match, the row with the matched password will be deleted and the column of id numbers will be reordered such that no id numbers are skipped.
Update: I'm making a rotating banner ad system by setting a random number from 1 to 4 to a variable so that the php file will retrieve a random ad from id=1 to id=4 by using the random number variable. If the random number happens to be 3 and id=3 does not exist, there will be a gap in the row of banner ads. If there is a way to work around big gaps in this situation, please tell me. thanks in advance
Just execute the following SQL query:
ALTER TABLE `tbl_name` AUTO_INCREMENT = 1;
…but it sounds like a terrible idea, so don't do it. Why is the value of your primary key so important? Uniqueness is far more important, and reseting it undermines that.
You can only use
ALTER TABLE 'tbl' AUTO_INCREMENT=#
to reset to a number above the highest value number. If you have 1, 2, 3, and you delete 2, you cannot use this to fill 2. If you delete 3, you could use this to re-use 3 (assuming you haven't put anything higher). That is the best you can do.
ALTER TABLE 'table' AUTO_INCREMENT = 1;
However running this code is not the best idea. There is something wrong with your application if you depend on the column having no gaps. Are you trying to count the number of users? if so use COUNT(id)? Are you trying to deal with other tables? If so use a foreign key.
If you are dead set on doing this the Wrong Way you could try to look for the lowest free number and do the incrementing on your own. Keep in mind the race conditions involves however.
Also, keep in mind that if you change the actual numbers in the database you will need to change all references to it in other tables and in your code.
Well, you can actually just specify the id number you'd like a record to have as part of your insert statement, for example:
INSERT INTO person VALUES(1,'John','Smith','jsmith#devnull.fake','+19995559999');
And if there's not a primary key collision (no record in the database with id=1), then MySQL will happily execute it.
The ALTER TABLE 'tbl' AUTO_INCREMENT=# thing also works, and means you don't have to keep track of the counter.
While you're thinking about this, though, you might want to read some of the discussion on natural vs surrogate keys. The idea of having your id # be specifically important is a bit unusual and might be a sign of a troubled design.
You could do that by:
Inventing a mechanism that provides the next available id when you want to insert (e.g. a transaction involving reading and incrementing an integer column somewhere -- pay special attention to the transaction isolation level!)
Using UPDATE to decrement all ids greater than the one you just deleted (again, with a transaction -- don't forget that foreign keys must be ON UPDATE CASCADE!)
But it begs the question: why do you want this? is it going to be worth the trouble?
It's almost certain that you can achieve whatever your goal is without such witchery.
Update (to address comment):
To select a random number of rows, you can do e.g. in MySQL
SELECT id FROM banners ORDER BY RAND() LIMIT 5
to select 5 random, guaranteed existing banner ids.
A word of caution: there are quite a few people who view ORDER BY RAND() as a bad performance hog. However, it is IMHO not quite right to put every case in the same basket. If the number of rows in the table is manageable (I would consider anything below 10K to be not that many) then ORDER BY RAND() provides a very nice and succint solution. Also, the documentation itself suggests this approach:
However, you can retrieve rows in
random order like this:
mysql> SELECT * FROM tbl_name ORDER BY RAND();
ORDER BY RAND() combined with
LIMIT is useful for selecting a random
sample from a set of rows:
mysql> SELECT * FROM table1, table2 WHERE a=b AND c ORDER BY RAND() LIMIT 1000;
RAND() is not meant to be
a perfect random generator. It is a
fast way to generate random numbers on
demand that is portable between
platforms for the same MySQL version.