There's this project I am working on. This is like a social network where we can have users, posts, pictures etc and then this problem came up. We are used to Mysql and the "almost magical" auto-increment field and now we cannot count on it anymore. I know the _id object in Mongo gives an easy way for identifying a document as it guarantee uniqueness. But the key is not user friendly and that's what we need, so we can make urls like:
http://website.com/posts/{post_id}
http://website.com/{user_id}
I developed a solution but I don't think this is the best way of doing this. I first create a mysql table with only one column. This column stores the user_id and it's an auto-increment field. For every new record on mongo I insert a new row in this mysql table and get the user_id with "LAST_INSERT_ID" function, now I can insert my data in my mongo collection with a numeric ID. And other benefit is that I can erase my mysql table let's say, after a million rows because the id's are already stored in mongo.
Am I doing it wrong?
Why not using slugs for posts and usernames for users? That should be human readable.
First, I don't see any benefit to using an arbitrary auto incrementing number over the generated id mongo provides. Not only is not again just a arbitrary id, but you have to maintain the sequence.
That said, why not let mongo manage the id, and use another unique identifier for your URLs. If your users have a 'username', I'm assuming you've already made sure that's unique across the collection. Just query by that unique property, instead of finding by id.
That also allows the user to change their unique identifier, without you having to remap associations in the database.
And for the post, just generate a unique slug from the title.
You can also create the id's in Mongo instead of MySQL, ...here's some documentation and articles on how to achieve it
http://www.mongodb.org/display/DOCS/How+to+Make+an+Auto+Incrementing+Field
http://shiflett.org/blog/2010/jul/auto-increment-with-mongodb
Related
I am switching my database from MySQL to dynamo and I'm reading everywhere that Dynamo does not support auto-increment and has no concept of queuing writes. If that is the case, what is the best approach to ensure that my keys are unique no matter how many processes are hitting the database at the same time?
I thought of using epoch time stamp, but that does not guarantee uniqueness.
You should figure out a unique id mechanism depending on your needs.
Here is a link with some options in php.
You can also read about twitter snowflake unique id generator.
If session handling is your problem this post from AWS might help.
I have some areas in my db that aren't write intensive so I use DynamoDB table to mimic auto-increment numbers using AttributeAction ADD 1.
DynamoDB is a key-value storage, and it is impossible to have 2 records with the same primary key. But you need to be carefull because without additional checks on put you could overwrite you old record.
To avoid this you could use attribute_not_exists condition expression to ensure that there is no other record with the same id.
For other columns there is no build-in mechanism to ensure uniqueness, but you could implement it by yourself.
In short you need to create several records for each time, first to save you payload and one to handle single unique index. Also it would be required to add conditional expression each time when you insert new or update the record.
e.g. we have a user table and we want to ensure id and email. Here how our records will looks like
{
pk: 'some-user-id',
record_type: 'record'
email: 'some#some.com'
}
{
pk: 'unique-index#some#some.com',
record_type: 'unique-index'
}
Each time when you would insert new user to db, you would need to insert both records using DynamoDB write transaction and check that there is no any other record with the same pk, for both of them.
Here is my article on how. You could find detailed description and wide code examples there
I have web application built on CakePHP 1.2.11. and mysql database. In this application I have two tables, namely, users and actions. Users hasMany actions and the user_id (id in users tables) is the foreign key in actions table. The id field is autoincrement integer.
CakePHP documentation said that setting the id field to be Char(36) will make CakePHP able to generate Unique string for each record to be the id.
My application is running and I don't want to loss the data records that my application already has. I need to know if it is safely possible to migrate from autoincrement integer id to char(36) keeping in mind the related table?
In other word, How could I change integer value to the unique string id that cakephp do? Is there any rules? If there any tool automate this kind of migration, I will be appreciated to know it.
Yes, simply alter the table to use a varchar. An INT column can be translated into a char, so you won't lose the original IDs (you will end up with a mix of both old regular ints and new uuids). You will need to make sure the change is also made to any foreign keys on any other tables that will need to store VARCHAR(36) as well.
Then make sure to push the new code live immediately otherwise the new records will not be able to be created, because a varchar field can't be auto-increment.
Lastly, immediately after pushing the new code, clear your model cache so Cake doesn't still think it's an INT.
Are you sure you want to switch?
Honestly, unless you have a really good reason to change to UUIDs (CHAR(36)), then I would recommend just staying with auto-incrementing IDs. There are plenty of people that tout the benefits of each, but it boils down to auto-incrementing IDs can be faster, and unless you have multiple databases where you're worried about overlapping data, auto-ids are just fine. (And it's not a simple "switch")
Not crazy-simple:
If you still are sure you want to switch to UUIDs, there is no automated process, but be careful - it's not just about switching the field types and voila - you'll have to create a script or something to update the id fields as well as all the associated fields (ie 'user_id' in the 'actions' table won't be updated..etc etc).
If so, here's how:
So - create a duplicate of your database (or tables) as back-up. You'll then probably want to rename the 'id' field to 'autoid', create another id field CHAR(36), run a script to populate all the UUIDs, then another script that populates the associated ids (ie 'user_id' in the 'actions' table) with the corresponding UUID.
CakePHP code that generates UUIDs:
Here's the link to creating a UUID in CakePHP 1.2: http://book.cakephp.org/1.2/en/view/826/uuid
I am working on creating a favorites section on my website where users can simply store certain items in their favorites section for easy access. Each of the items are already well-defined and have multiple attributes. So my question is lets say I had 10,000 users and I would like to implement a 'favorites' system, what would be the best way to keep track of what favorite items have been added by each user?
I was thinking implementing this the following way: link each favorited item id to a username and then run a query for if the user with a particular username is logged in than retrieve all the favorited items by that username.
I appreciate any help with figuring out of a good way to do this. My goal is to store in a way that is later easy to retrieve and use the data and minimize redundant information.
It's pretty easy, you need to create a new table with 3 fields:
id
favoriteID
userID
Every time a user adds a new favourite, it adds a new record to this table, storing both the ID of the favorite, and the ID of the user. There is no redundant information and it's easy to retrieve the details of either the favorite or the user by implementing a join query. This is what relational databases are for.
Within an RDBMS you would probably have a many to many table with the user id and article id. You do not need an independent id column:
create table favourites (user_id int, article_id int);
These of course reference your user table and articles table. (Or whatever you have in place of articles.)
You would then need to retrieve all rows for a single user when wanting to show that user's favourites. You might also want to make a combined UNIQUE index on the columns to prevent duplicates.
You may have faster response with something like cassandra where you can simply retrieve based on the key of the user_id and get all their favourites in one easy spot. But then you're dealing with mutilple systems.
I've heard, but haven't had a chance to look into, that MySQL can now support a Key-Value system similar to Cassandra and that may be your best bet.
In the following situation, what would the database design look like?
This is for some sort of inventory system.
I want users to be able to create a item-type (lets say, a Laptop). I also want users to be able to say the serial number and MAC address must be unique. This part confuses me where to check for unique values, since I have no idea how to make a table with all items in it, with unique values..
Let's say a user creates another item-type that has no serial number or any unique fields, this means I can't build my DB with property1 till property10 fields in the database.
I also don't want to build a table for every item type, since that would involve too advanced table management in PHP.
Any suggestions on how to build this DB?
Just to clarify I understand your requirements correctly, you meant to create a table that unique rule only applies to a subset of the table instead of the entire table?
If so, I think there will be two options
have two tables, one with unique rules and one without OR
Enforce the unique rules in application level as business rules instead of database level.
If i got your question correctly , i would just do something simple and maintainable, such as :
you can achieve it by,
assume ,table name is item,here item id is primary key.it makes easy to pick up which item you want.then while posting(inserting) serial number and mac address,you should check with php ,that there is no duplication of data.
did i got u !
I'm used to building websites with user accounts, so I can simply auto-increment the user id, then let them log in while I identify that user by user id internally. What I need to do in this case is a bit different. I need to anonymously collect a few rows of data from people, and tie those rows together so I can easily discern which data rows belong to which user.
The difficulty I'm having is in generating the id to tie the data rows together. My first thought was to poll the database for the highest user ID in existence, and write to the database with user ID +1. This will fail, however, if two submissions poll the database before either of them writes to it - they will each share the same user ID.
Another thought I had was to create a separate user ID table that would be set to auto-increment, and simply generate a new row, then poll that table for the id of the last row created. That also fails for the same reason as above - if two submissions create a row before either of them polls for the latest user ID, then they'll end up sharing an ID.
Any ideas? I get the impression I'm missing something obvious.
I think I'm understanding you right; I was having a similar issue. There's a super handy php function, though. After you query the database to insert a new row and auto-incrementing their user ID, do:
$user_id = mysql_insert_id();
That just returns the auto-increment value from the previous query on the current mysql connection. You can read more about it here if you need to.
You can then use this to populate the second table's data, being sure nobody will get a duplicate ID from the first one.
You need to insert the user, get the auto-generated id, and then use that id as a foreign key in the couple of rows you need to associate with the parent record. The hat rack must exist before you can hang hats on it.
This is a common issue, and to solve it, you would use a transaction. This gives you the atomic idea being being able to do more than one thing, but have it tied to either a success or fail as a package. It's an advanced db feature, and does require awareness of some more advanced programming in order to implement it in as fault-tolerant a manner as possible.