Good day, I have developed a system using php and mysql. From 5 systems turned into 1 system. Basically, there is a huge possibility that in system 1 there will be a user table and has primary keys. The same with other systems.
My problem is there are identical primary ids to be migrated in current developed system.
In system 1, there are 70,000 data in user table. System 2, 22,000 records. and less than 3000 in other systems. Unfortunately, primary keys were foreign keys in other tables.
How can I migrate those data without conflict in primary keys and how can I update foreign keys?
Please help.
You have to create a mapping between the various user records and the new consolidated user master table. Pls note that this may be true not only for the users, but also for other type of data as well.
When you create the mapping table, then have a new user id field, an old user id field and a field identifying the source system. Then copy over the users by source system to the mapping table and during the copy specify the source system. This way you can distinguish between the users with the same user ids in the various source systems and generate the new user ids.
When you migrate other data from the source systems containing user id, you need to use the mapping table to replace the old user ids with the new one.
Related
Scenario:
I Created a POS (point of sale) system using mysql database. I am managing all shops data in one database. All operation was on server before but now the requirement is changed and i want to make it local too. The challenge i face is Duplicate entry for key primary
For example:
The system is used by two shop. If one shop added record where id=1 in item table in his local database and the second shop also added record where id = 1 in same table in his local database. Now when i send both data to my server database, it will give me error on Duplicate entry for key primary.
Conclusion:
I am not using MYSQL replication because it not suit my database structure so what will be the best solution for this issue?
You can solve this problem in many ways:
You should not sync the primary key as well from the local to remote, rather you can have some order ID (SHOPID_SOMERANDOM-NUMBER) which will be unique for shops .
Otherwise you can keep a composite key as primary key like Autoincrement_ID+SHOP_ID so that while syncing this will never be duplicate.
This shop_ID should be generated from the server at the time of installation and should not be duplicate.
What do you think is the best approach for a PHP and SQL based web application that will be used by a number of people?
For example, say we have a table called "sales" and a user wants to access his sales. The table should contain a foreign key of the user_id or it will be better to make a separate table for each user?
Any other implementations and opinions are also welcome!
In my opinion best approach would be using two tables and refer from a foreign key. Make sure to use indexes as well. MySQL has done various optimizations to WHERE clause on a PRIMARY KEY or a UNIQUE index[1]. So you will be fine when working with considerable number of records(ex: handling 100000 records won't be a issue if you have capable hardware for database instance and optimized database configurations accordingly).
Make sure to do database optimizations based on your system to increase performance as well. Better to do in-house testing to make sure system is upto your expectations in long run.
[1] http://dev.mysql.com/doc/refman/5.7/en/where-optimizations.html
By sales I take it you mean the users invoices or billing history. You would want to separate them out into two tables using foreign keys. You could have a users table and an invoices table with the userid as the foreign key in your invoices table.
Whenever the user wanted to view their invoices, you would select rows from the invoices table where the userid is that users id.
I am developing a inventory software with MySQL and PHP where local database will sync to online database.
Suppose i have a table sell and sell_id is the primary key of the table. I usually use INT and auto increment with primary key.
In local database 1 sell table has 2 entry(sell_id 1,2) and local database 2 sell table has 2 entry(sell_id 1,2).
If i sync/insert these 2 local sell table entries to online sell table it will become (sell_id 1,2,3,4).
As sell id changes it effects those entries in other table which are using sell_id as foreign key.
How should i plan to create primary key in this situation.
I am planning to use alpha-numeric id which will be unique for both database. Will it create any problem or slow my db query further for millions of sell_id??
Are there any other ways to solve the problem ?
This is too long for a comment.
Often, when you have a replicated system, the goal is to maintain the same data on all servers. That does not seem to be your business requirement.
Instead, you might consider having a composite primary key on all the servers. This would combine the auto incremented primary key with a server id. All tables referencing the foreign key would need to incorporate the "server" column as well as the "id".
In general, I'm not a fan of composite primary keys. However, you have a distributed database and need to identify the specific database "partition" where the data is located. This seems like a good use-case for composite primary keys.
An alternative approach -- if you are willing to take the risk -- is to set the auto numbering to a different start value on each server. Use a big int and a big value such as 1,000,000,000,000 for one server, 2,000,000,000,000 for the next, and so on. My preference is to have the "server" explicitly represented as a column, however.
I'm making a table (with MySQL) to store some data, but i'm not sure of the way to do it properly, because of the amount of data. For example if it's adress book database.
so there is a table for users and a table for contacts. Each users can own hundreds of contacts, and there could be thousans of users. Should I add a new row for every single contact (it will make a lot of rows!), or can i just concatenate all of them in one row with the user id.
uuh, this is just an example, but in my case once contacts are INSERTED they will never be UPDATED so, no modifications, they can only be DELETED.
To go by the normal forms, you should have three tables
1) Users -> {User_id} (primary key)
2) Contacts -> {Contact_id} (primary key)
3) Users_Contacts -> {User_id, Contact_id} (Compound key)
The Junction table Users_Contacts will have one record per contact - meaning for each unique value of User_id+Contact_id, there will be one record.
However, In practice, it is not always necessary to stick to the rule book. Depending on the use case, it is often advisable to have a denormalized table. The call is yours.
There is also another option of using NoSQL with MySQL. For example, the contacts can be serialized into JSON and stored. Mysql 5.7 seem to support this data format (with some external help). See this for details.
Say for eg: If you add 3 contacts for a single user and as you mentioned you would be deleting contacts the its better to insert all three contacts, each in a new row with its user id. Because if you want to delete any one of the contact from 3 of them, then it will be easy.
If you concatenate all the contacts for an user and add them in one row could land up many issues. What in future the requirement changes and you need to make a layout all the contacts for an user with edit/delete individual contacts. So you should have one contact in each row.
You can optimize your query by indexing the columns.
Say userid#1234 has 1000 contacts in contact table where the primary key in contact table is idcontact (Indexed by default) and then in contact table another field called "iduser" which is also indexed, then the select performance over an iduser on contact table will be fast.
Ideally its the best approach using mysql database. There are examples of many apps where it maintains millions of data so it should be fine with a contact table and for each contact a new row.
I wouldn't worry about lots of rows. You have to keep in-mind the granularity of control the user would expect (deleting / adding a contact, rearranging the list based on different factors, etc). It's always better to break things out into their owns rows if they are going to be treated independently from a similar item (contacts, users, addresses, etc). Additionally, if you were to concatenate your data, re-ordering for display or removing data becomes extremely resource intensive. Where as MySQL is designed to do exactly that "on the cheap".
MySQL can easily handle millions of rows of data. If you are worried about speed, just make sure your indexes are in-place before your data collection is too big (I would venture a guess, and say you'll need to index the user ID the contact belongs to and the first/last names). Indexes are a double-edged sword, however, as they take up disk space, but allow fast querying of large data sets. So you don't want to go over-board and index everything, only what you'll be sorting/searching by.
(Why on earth will contacts never be updated?...)
I have web application built on CakePHP 1.2.11. and mysql database. In this application I have two tables, namely, users and actions. Users hasMany actions and the user_id (id in users tables) is the foreign key in actions table. The id field is autoincrement integer.
CakePHP documentation said that setting the id field to be Char(36) will make CakePHP able to generate Unique string for each record to be the id.
My application is running and I don't want to loss the data records that my application already has. I need to know if it is safely possible to migrate from autoincrement integer id to char(36) keeping in mind the related table?
In other word, How could I change integer value to the unique string id that cakephp do? Is there any rules? If there any tool automate this kind of migration, I will be appreciated to know it.
Yes, simply alter the table to use a varchar. An INT column can be translated into a char, so you won't lose the original IDs (you will end up with a mix of both old regular ints and new uuids). You will need to make sure the change is also made to any foreign keys on any other tables that will need to store VARCHAR(36) as well.
Then make sure to push the new code live immediately otherwise the new records will not be able to be created, because a varchar field can't be auto-increment.
Lastly, immediately after pushing the new code, clear your model cache so Cake doesn't still think it's an INT.
Are you sure you want to switch?
Honestly, unless you have a really good reason to change to UUIDs (CHAR(36)), then I would recommend just staying with auto-incrementing IDs. There are plenty of people that tout the benefits of each, but it boils down to auto-incrementing IDs can be faster, and unless you have multiple databases where you're worried about overlapping data, auto-ids are just fine. (And it's not a simple "switch")
Not crazy-simple:
If you still are sure you want to switch to UUIDs, there is no automated process, but be careful - it's not just about switching the field types and voila - you'll have to create a script or something to update the id fields as well as all the associated fields (ie 'user_id' in the 'actions' table won't be updated..etc etc).
If so, here's how:
So - create a duplicate of your database (or tables) as back-up. You'll then probably want to rename the 'id' field to 'autoid', create another id field CHAR(36), run a script to populate all the UUIDs, then another script that populates the associated ids (ie 'user_id' in the 'actions' table) with the corresponding UUID.
CakePHP code that generates UUIDs:
Here's the link to creating a UUID in CakePHP 1.2: http://book.cakephp.org/1.2/en/view/826/uuid