I can achieve what I want to do it's just terribly slow and I'm pretty sure there is a better way. Here is an abstract:
I have two tables I imported from two text files (via LOAD DATA INFILE). There is no primary and foreign key relationship between these tables yet.
Table 1 - Events: 1.5 millions rows (person_id values are empty)
+--------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+----------------+
| event_id | int(11) | NO | PRI | NULL | auto_increment |
| person_id | int(11) | NO | UNI | NULL | |
| event_details| longtext | NO | | NULL | |
| person_name | varchar(30) | NO | | NULL | |
| mother_name | varchar(30) | NO | | NULL | |
+--------------+--------------+------+-----+---------+----------------+
Table 2 - People: 140 000 rows
+--------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+----------------+
| person_id | int(11) | NO | PRI | NULL | auto_increment |
| person_name | varchar(30) | NO | | NULL | |
| mother_name | varchar(30) | NO | | NULL | |
+--------------+--------------+------+-----+---------+----------------+
I'm trying to insert the person_id from the people table into the events table.
I've tried a few methods, basically loop through each row of the events table find the person_id in the people table using the person_name and mother_name in the current row from the events table. Then insert the person_id into the events table.
My current query takes a bout 5 days to update all 1.5 million rows in the events table. How can I make this faster?
I'm using laravel and php but any suggestions welcome.
Related
I tried searching for this but I could not find anything about this but design choices.
So my question is like the title. What is faster? Create 1 table with many columns or create 2 or 3 (for many to many) tables with join(s).
I like the idea of have multiple tables so the data is separated. Mostly for many to many like data. But my friend told me having 5 columns with boolean is just fine. But I like the idea of have a table with the settings and then a table between with user.id and setting.id. But my question is also, does it have a impact on the query?
example:
Users
- id
- Email
- SettingA
- SettingB
- SettingC
OR example:
Uers
- id
- email
Users_Settings
- user_id
- setting_id
Settings
- id
- someSettingsValue
What woult be faster for Mysql to query the data? (retrieving settings for user)
not at all.. Only joins between 2 r 3 will take time compared to single table fields.
It's not about preferring single table with many columns or preferring multiple tables. It's about Normalization.
According to your provided schema, if all users will always have three settings, i.e. Setting A, B and C, then it's better to create single table with these three columns for settings.
users table:
+----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| Email | varchar(128) | NO | | NULL | |
| SettingA | tinyint(1) | NO | | NULL | |
| SettingB | tinyint(1) | NO | | NULL | |
| SettingC | tinyint(1) | NO | | NULL | |
+----------+--------------+------+-----+---------+----------------+
But if any of the setting is saved is null, then better is to create separate table for settings and then a pivot table for maintaining users' settings without primary key.
users table:
+-------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| Email | varchar(128) | NO | | NULL | |
+-------+--------------+------+-----+---------+----------------+
settings table:
+---------------+------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| setting_value | tinyint(1) | NO | | NULL | |
+---------------+------------+------+-----+---------+----------------+
setting_user pivot table:
+------------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+---------+------+-----+---------+-------+
| setting_id | int(11) | NO | | NULL | |
| user_id | int(11) | NO | | NULL | |
+------------+---------+------+-----+---------+-------+
Where setting_user is the pivot table.
One more thing is considered when creating schema, that, will there be always three settings, or will there be more in future, when application is expanded!
I'm new to Laravel (using 5.1). I have my entire DB schema (MySQL 5.5) diagrammed and have begin implementing it. The problem is, I need to adapt Laravel to use my sessions table. After making a new migration to bring the table more in line with what Laravel expects, I have this table:
+---------------+---------------------------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+---------------------------------------+------+-----+---------+----------------+
| id | bigint(20) | NO | PRI | NULL | auto_increment |
| id_hash | varchar(255) | NO | UNI | NULL | |
| user_id | bigint(20) unsigned | NO | | 0 | |
| created_at | int(10) unsigned | NO | | 0 | |
| updated_at | int(10) unsigned | NO | | 0 | |
| expires_at | int(10) unsigned | NO | | 0 | |
| last_activity | int(10) unsigned | NO | | 0 | |
| platform | enum('d','p','t','b','a','i','w','k') | NO | | d | |
| ip_address | varchar(40) | NO | | 0.0.0.0 | |
| payload | text | NO | | NULL | |
| user_agent | text | NO | | NULL | |
+---------------+---------------------------------------+------+-----+---------+----------------+
The main thing I need to accomplish is to have id as an auto-incrementing integer (because my Session model has relationships to other models) and use id_hash as the publicly identifying string (I also plan to cut id_hash back to 64), which I think is the token in the payload.
At session creation, id_hash, platform, ip_address, and user_agent will be set, never to change again. After authentication, user_id will be populated, then cleared at logout.
I'm ok with keeping the payload handling as-is.
Is this just a matter of creating a custom class that implements SessionHandlerInterface? What else needs to be in it for handling my extra fields that's not obvious from the session docs?
Having some trouble with foreign keys, I am new to MySQL so please forgive. I am not sure how to do what I want.
I have 4 tables:
Users
Tasks
Day
Week (includes Week number and date commencing).
What I want to do is, reference each table to pull the correct data.
E.G. pseudo code:
find user liam, find select day associated with liam, find the task associated with the selected day, find the week from the selected day.
This should then allow me to sieve through the data.
Here are the tables:
+------------+-----------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+-----------------+------+-----+---------+----------------+
| user_id | int(1) unsigned | NO | PRI | NULL | auto_increment |
| username | varchar(255) | NO | UNI | NULL | |
| password | varchar(255) | NO | | NULL | |
| first_name | varchar(255) | NO | | NULL | |
| last_name | varchar(255) | NO | | NULL | |
| email | varchar(255) | NO | UNI | NULL | |
+------------+-----------------+------+-----+---------+----------------+
+------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+------------------+------+-----+---------+----------------+
| day_id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| day | varchar(255) | NO | | NULL | |
| user-id-fk | int(10) unsigned | NO | | NULL | |
+------------+------------------+------+-----+---------+----------------+
+-------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+----------------+
| am_id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| am_task | varchar(255) | NO | | NULL | |
| description | varchar(255) | NO | | NULL | |
| am_color | varchar(20) | YES | | NULL | |
+-------------+------------------+------+-----+---------+----------------+
+---------+-----------------+------+-----+------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+-----------------+------+-----+------------+----------------+
| week_id | int(2) unsigned | NO | MUL | NULL | auto_increment |
| week | varchar(255) | NO | | NULL | |
| date | date | NO | | 2014-12-29 | |
+---------+-----------------+------+-----+------------+----------------+
So to start off I understand that I have to reference the first two tables, this is why I have created a field called 'user-id-fk'.
I have tried to create the foreign key inside of phpMyAdmin: http://tinypic.com/r/mkar20/8
But not sure if it is working or if I have created it properly.
As I already said I am new to MySQL so even if I had created it properly, I'm not sure how to test?
I can add any more data if required.
Please note that Foreign keys are only ways ensuring solid data structure within SQL. They aren't technically necessary for querying your data across two or more tables, as long as you have keys that are in common across both tables.
The use of foreign keys may make it easier to make sure you are correctly choosing items from a controlled list. However, as they are good ideas as they ensure integrity in your database, the easiest way to test these, is to set up these keys as RESTRICT and test adding a record from the table that has a foreign key restriction that does not exist on the referenced table It should provide an error.
For instance given one of your examples:
+------------+-----------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+-----------------+------+-----+---------+----------------+
| user_id | int(1) unsigned | NO | PRI | NULL | auto_increment |
| username | varchar(255) | NO | UNI | NULL | |
| password | varchar(255) | NO | | NULL | |
| first_name | varchar(255) | NO | | NULL | |
| last_name | varchar(255) | NO | | NULL | |
| email | varchar(255) | NO | UNI | NULL | |
+------------+-----------------+------+-----+---------+----------------+
+------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+------------------+------+-----+---------+----------------+
| day_id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| day | varchar(255) | NO | | NULL | |
| user-id-fk | int(10) unsigned | NO | | NULL | |
+------------+------------------+------+-----+---------+----------------+
If user-id-fk is a foreign key on that second table referencing user_id on the first table, and you set it up as restrict, you will not be able to add any records into user-id-fk without entering the same user_id into the first table.
Assuming user_id on the first table only values 1, 2, or 3 entered. If you try to enter a row 4 into the second table, it should err, because there is no referenced key on the first table.
Note:
You have created different sized integers for these tables, so you could run into a problem after you have entered 10 people. I recommend making the integer size on user_id to be larger than 1.
I have a table hardware_description:
+------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------+--------------+------+-----+---------+----------------+
| Computer_id | int(11) | NO | PRI | NULL | auto_increment |
| Emp_id | int(11) | NO | MUL | NULL | |
| PC_type | varchar(20) | YES | | NULL | |
| Operating_system | varchar(20) | YES | | NULL | |
| Product_key | varchar(30) | YES | | NULL | |
| Assign_date | date | YES | | NULL | |
| DVD_ROM | varchar(20) | YES | | NULL | |
| CPU | varchar(30) | YES | | NULL | |
| IP_address | varchar(30) | YES | | NULL | |
| MAC_address | varchar(30) | YES | | NULL | |
| Model_name | varchar(30) | YES | | NULL | |
| Model_number | varchar(30) | YES | | NULL | |
| Monitor | varchar(30) | YES | | NULL | |
| Processor | varchar(30) | YES | | NULL | |
| Product_name | varchar(30) | YES | | NULL | |
| RAM | varchar(20) | YES | | NULL | |
| Serial_number | varchar(30) | YES | | NULL | |
| Vendor_id | varchar(30) | YES | | NULL | |
Emp_id is foreign key from employees table.
When I update a particular row, I want the existing data for that row to be saved in another table along with the timestamp of that update action. Now,
a) Shall I use PHP code (PDO transaction) to first grab that row & insert in another table then perform the UPDATE query on that particular row?
b) Use trigger on this table.
Which process is better practice & more efficient? Is there another way of achieving this?
I have not used trigger in my short career so far but I can do it if it is better practice.
If you can do a trigger, it would be a lot better to use that.
The reason for this is that if for some reason you forget to write the PHP code to do this (in some weird situation) - you would have missing, unrelated data - otherwise known as orphaned data, which does not have a corresponding row or set of rows.
Here's the link to the MySQL documentation page for triggers: http://dev.mysql.com/doc/refman/5.0/en/create-trigger.html
I am getting this error whenever I try to Insert Data into my "Student" table.
Below are the two tables, I am using MySql :
student table:
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+-------------------+----------------+
| S_id | int(20) | NO | PRI | NULL | auto_increment |
| U_id | int(11) | YES | MUL | NULL | |
| sname | varchar(20) | YES | | NULL | |
| gender | varchar(20) | YES | | NULL | |
| email | varchar(320) | YES | UNI | NULL | |
| Phone_Number | int(20) | YES | | NULL | |
user table:
Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+-------------------+----------------+
| U_id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(100) | NO | | NULL | |
| password | varchar(20) | NO | | NULL | |
| email | varchar(255) | NO | UNI | NULL | |
When I try to run this query ,
INSERT INTO student(U_id,sname,gender,email,Phone_Number) VALUES ('$U_id','$sname','$gender','$email','$Phone_Number');
I am getting this error:
Cannot add or update a child row: a foreign key constraint fails (`Learn`.`student`, CONSTRAINT `student_ibfk_1` FOREIGN KEY (`U_id`) REFERENCES `user` (`U_id`))
I tried to look for the solution but most of them are saying keep a common Engine like InnoDB on both the tables but in my table its already defined.I have even followed the steps given here https://stackoverflow.com/a/9139206/2545197
but nothing worked for me.
I made an SQL Fiddle and everything works perferctly.
The only way I can reproduce the error is when a row is inserted into student before the corresponding row in user exists. Can you please double check that this is really true for each inserted row.
Also, you should var_dump() your PHP variables used for the INSERT and check if the problem lies there. IMHO, it can only have to do with a non-existing record in the user table.