I am creating an app that needs to store project IDs (in a MySql DB). The Admin user should be able to set the format for the ID numbers (#####, ##-####, YY-##-####, etc)...only before any projects are created of coarse. The ID must also autoinc. when a normal user creates a new project.
I am thinking the auto increment will have to be done with PHP, by reading the last record and adding 1.
Since the ID format will defined by a user, I think it would probably have to be a single field (varchar?). Or should the app create multiple fields to accommodate the user defined format? Any thoughts on this?
Looking for any ideas...
The problem with reading the previous id and just incrementing by one is that you risk repeated use of the same ID. consider:
You create a new project, which takes its ID from the previous one.
You use, and possibly distribute the project ID
You decide you don't want that project after all, and delete it
You create a new project. The new project will have the same ID as the old, deleted one, because it is derived by incrementing the same source.
If this is not a problem for you, then that is your solution.
Otherwise, you might try this:
Create two columns in your table - one for the ID, and one for an autoincrementing integer.
When you create the project, don't add the ID. The integer value will autoincrement
Fetch that integer value (possibly with mysql_insert_id)
Update your project with the ID generated from the autoincremented integer
This is more effort for the system to make, and at extremely high volumes you may encounter race conditions, but if you add a Unique index on the project ID, that should catch any issues.
Related
Maybe this is a stupid question because I should defer PK increments to MySql itself, but I'm in a weird situation.
Basically to handle versioning and approvals in my system, I have revision_batch table which is a collection of things in a submission that a user wishes to insert or update to the database. It has columns like batch_id, the user_id of the submitter, and an approved value.
It also is parent to a collection of items in the revisions table. The revisions table has things like table_name, key, old_value, and new_value. I use this to store the changes someone wishes to make that may not be approved automatically.
When someone who doesn't have permission to, say, a "task" table, and they change the name of an task, a new revision_batch will be created, and a new revision will be created with table_name="tasks", key=[whatever the task's ID is], old_value="my old task name", new_value="my new task name".
When an approver approves of this batch, my code will rocket through the revisions in the batch and perform the update or inserts to the database.
My problem is when performing parent-child relationships within the same batch. If I'm creating a new task and want to assign a task_item to it, in the same batch, then I need to know what PK the task is getting so that I can give the task_item a "task_id".
If I'm handing the creation of a new revision for a task, I might do something like a
select max(id)+1 as newId from tasks
to inject as the new id. But since I might already have a pending task insert revision with that ID or higher, I also check
select max(key) + 1 as newId
from revisions
inner join revision_batches on revisions.batch_id = revision_batches.id
where table_name='revisions' and approved = 'P'
for a higher id to assign. That way of I have ids 1-9 in a tasks table and 10-12 pending in the revisions table, any new direct insert using Laravel's Eloquent model class is overridden to check both tasks and revisions and will insert with id 13. This avoids collisions between actual cemented rows and possible revision rows. It also allows me to create a parent and many layers of children within a single batch because I determine their ID as I go along.
This is all works fine.
My problem is that if I have two revisions creations happening at the exact same time (like, within a millisecond) , they'll asynchronously both fetch the same next ID to use, both create revisions where key = the same number, and then only one will get through and the other fails on a PK collision.
My question is: is there a way to force this to be thread safe or to be done synchronously, to avoid two instances of the same controller method executing at the same time and both fetching the same ID to use? Can I lock a method down to a single instance at a time? If not, is there a better way I could be handling PK generation? The only reason I do this is to know beforehand the key to insert. But since custom code in the framework is handling PK generation and not the database, it's causing me this major issue. Happens sporadically, but only when I force the same method to execute maybe 4 times at the same time.
I know that I could avoid the majority of cases where I have many things being inserted at the exact same time, but that doesn't mean that randomly in the future that two users won't hit enter at the same time and recreate this issue.
Any ideas?
Thanks!
For this type of issues I use UUID 4, (Universally unique identifier), my case is a little bit different because I have a system in 74 different locations, but need to extract all the transaction records and integrate in a consolidation system, so my PKs needs to be unique across all servers to avoid collisions.
In laravel I use this excelent package to generate the UUID
I hope this works for you.
Use Queues for saving your revisions.
Queues are synchronous, and hence the key collision will never occur.
Source: http://laravel.com/docs/4.2/queues
I have web application built on CakePHP 1.2.11. and mysql database. In this application I have two tables, namely, users and actions. Users hasMany actions and the user_id (id in users tables) is the foreign key in actions table. The id field is autoincrement integer.
CakePHP documentation said that setting the id field to be Char(36) will make CakePHP able to generate Unique string for each record to be the id.
My application is running and I don't want to loss the data records that my application already has. I need to know if it is safely possible to migrate from autoincrement integer id to char(36) keeping in mind the related table?
In other word, How could I change integer value to the unique string id that cakephp do? Is there any rules? If there any tool automate this kind of migration, I will be appreciated to know it.
Yes, simply alter the table to use a varchar. An INT column can be translated into a char, so you won't lose the original IDs (you will end up with a mix of both old regular ints and new uuids). You will need to make sure the change is also made to any foreign keys on any other tables that will need to store VARCHAR(36) as well.
Then make sure to push the new code live immediately otherwise the new records will not be able to be created, because a varchar field can't be auto-increment.
Lastly, immediately after pushing the new code, clear your model cache so Cake doesn't still think it's an INT.
Are you sure you want to switch?
Honestly, unless you have a really good reason to change to UUIDs (CHAR(36)), then I would recommend just staying with auto-incrementing IDs. There are plenty of people that tout the benefits of each, but it boils down to auto-incrementing IDs can be faster, and unless you have multiple databases where you're worried about overlapping data, auto-ids are just fine. (And it's not a simple "switch")
Not crazy-simple:
If you still are sure you want to switch to UUIDs, there is no automated process, but be careful - it's not just about switching the field types and voila - you'll have to create a script or something to update the id fields as well as all the associated fields (ie 'user_id' in the 'actions' table won't be updated..etc etc).
If so, here's how:
So - create a duplicate of your database (or tables) as back-up. You'll then probably want to rename the 'id' field to 'autoid', create another id field CHAR(36), run a script to populate all the UUIDs, then another script that populates the associated ids (ie 'user_id' in the 'actions' table) with the corresponding UUID.
CakePHP code that generates UUIDs:
Here's the link to creating a UUID in CakePHP 1.2: http://book.cakephp.org/1.2/en/view/826/uuid
There's this project I am working on. This is like a social network where we can have users, posts, pictures etc and then this problem came up. We are used to Mysql and the "almost magical" auto-increment field and now we cannot count on it anymore. I know the _id object in Mongo gives an easy way for identifying a document as it guarantee uniqueness. But the key is not user friendly and that's what we need, so we can make urls like:
http://website.com/posts/{post_id}
http://website.com/{user_id}
I developed a solution but I don't think this is the best way of doing this. I first create a mysql table with only one column. This column stores the user_id and it's an auto-increment field. For every new record on mongo I insert a new row in this mysql table and get the user_id with "LAST_INSERT_ID" function, now I can insert my data in my mongo collection with a numeric ID. And other benefit is that I can erase my mysql table let's say, after a million rows because the id's are already stored in mongo.
Am I doing it wrong?
Why not using slugs for posts and usernames for users? That should be human readable.
First, I don't see any benefit to using an arbitrary auto incrementing number over the generated id mongo provides. Not only is not again just a arbitrary id, but you have to maintain the sequence.
That said, why not let mongo manage the id, and use another unique identifier for your URLs. If your users have a 'username', I'm assuming you've already made sure that's unique across the collection. Just query by that unique property, instead of finding by id.
That also allows the user to change their unique identifier, without you having to remap associations in the database.
And for the post, just generate a unique slug from the title.
You can also create the id's in Mongo instead of MySQL, ...here's some documentation and articles on how to achieve it
http://www.mongodb.org/display/DOCS/How+to+Make+an+Auto+Incrementing+Field
http://shiflett.org/blog/2010/jul/auto-increment-with-mongodb
In the following situation, what would the database design look like?
This is for some sort of inventory system.
I want users to be able to create a item-type (lets say, a Laptop). I also want users to be able to say the serial number and MAC address must be unique. This part confuses me where to check for unique values, since I have no idea how to make a table with all items in it, with unique values..
Let's say a user creates another item-type that has no serial number or any unique fields, this means I can't build my DB with property1 till property10 fields in the database.
I also don't want to build a table for every item type, since that would involve too advanced table management in PHP.
Any suggestions on how to build this DB?
Just to clarify I understand your requirements correctly, you meant to create a table that unique rule only applies to a subset of the table instead of the entire table?
If so, I think there will be two options
have two tables, one with unique rules and one without OR
Enforce the unique rules in application level as business rules instead of database level.
If i got your question correctly , i would just do something simple and maintainable, such as :
you can achieve it by,
assume ,table name is item,here item id is primary key.it makes easy to pick up which item you want.then while posting(inserting) serial number and mac address,you should check with php ,that there is no duplication of data.
did i got u !
I'm used to building websites with user accounts, so I can simply auto-increment the user id, then let them log in while I identify that user by user id internally. What I need to do in this case is a bit different. I need to anonymously collect a few rows of data from people, and tie those rows together so I can easily discern which data rows belong to which user.
The difficulty I'm having is in generating the id to tie the data rows together. My first thought was to poll the database for the highest user ID in existence, and write to the database with user ID +1. This will fail, however, if two submissions poll the database before either of them writes to it - they will each share the same user ID.
Another thought I had was to create a separate user ID table that would be set to auto-increment, and simply generate a new row, then poll that table for the id of the last row created. That also fails for the same reason as above - if two submissions create a row before either of them polls for the latest user ID, then they'll end up sharing an ID.
Any ideas? I get the impression I'm missing something obvious.
I think I'm understanding you right; I was having a similar issue. There's a super handy php function, though. After you query the database to insert a new row and auto-incrementing their user ID, do:
$user_id = mysql_insert_id();
That just returns the auto-increment value from the previous query on the current mysql connection. You can read more about it here if you need to.
You can then use this to populate the second table's data, being sure nobody will get a duplicate ID from the first one.
You need to insert the user, get the auto-generated id, and then use that id as a foreign key in the couple of rows you need to associate with the parent record. The hat rack must exist before you can hang hats on it.
This is a common issue, and to solve it, you would use a transaction. This gives you the atomic idea being being able to do more than one thing, but have it tied to either a success or fail as a package. It's an advanced db feature, and does require awareness of some more advanced programming in order to implement it in as fault-tolerant a manner as possible.