I'm making a notification scheme for my social networking app. I've different kind of notification which are categorized in two groups: Friends-related and Events-related. Currently, my database schema is like this:
+---------------------+------------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+------------------------+------+-----+---------+----------------+
| notification_id | int(11) | NO | PRI | NULL | auto_increment |
| notification_type | enum('event','friend') | NO | | NULL | |
| notification_date | datetime | NO | | NULL | |
| notification_viewed | bit(1) | NO | | NULL | |
| user_id | int(11) | NO | MUL | NULL | |
+---------------------+------------------------+------+-----+---------+----------------+
Now, I've two different tables fro event-related notification and friend-related notification. Below is schema for event-related notification table:
+-------------------------+----------------------------------------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------------------+----------------------------------------------------+------+-----+---------+-------+
| notification_id | int(11) | NO | PRI | NULL | |
| event_id | int(11) | NO | MUL | NULL | |
| event_notification_type | enum('added','kicked','new-message','info-edited') | NO | | NULL | |
+-------------------------+----------------------------------------------------+------+-----+---------+-------+
And again I've 4 more tables for each kicked, added, new-message, info-edited type of notification, since each requires to have it different kind of property (for example kicked requires a reason).
Now, I want to write a conditional SQL query such that it joins the notification with event_notification if notification_type is event otherwise different.
SELECT * FROM notification_table t WHERE t.seen = FALSE AND t.user_id = ? INNER JOIN event_notification en ON(t.notification_type='event' AND en.notification_id = t.notification_id) INNER JOIN .....
There is going to be so many inner joins is there any better way of doing it? I think my query is not very optimized either, would appreciate if any help could be provided.
You can use the joins. However, you want to create the query using left outer joins rather than inner joins:
SELECT *
FROM notification_table t
WHERE t.seen = FALSE AND t.user_id = ? left JOIN
event_notification en
ON(t.notification_type='event' AND en.notification_id = t.notification_id) left JOIN ...
Don't worry about the proliferation of joins. If your tables have proper indexing, they will perform fine.
Do consider changing the data structure so you have only one table for the different notification types. Having a few fields that are not used does not add much performance overhead, especially when you consider the complications of having so many joins and the additional management overhead of having more tables.
Related
I have SQL that in my head, would and should run in under 1 second:
SELECT mem.`epid`,
mem.`model_id`,
em.`UKM_Make`,
em.`UKM_Model`,
em.`UKM_CCM`,
em.`UKM_Submodel`,
em.`Year`,
em.`UKM_StreetName`,
f.`fit_part_number`
FROM `table_one` AS mem
INNER JOIN `table_two` em ON mem.`epid` = em.`ePID`
INNER JOIN `table_three` f ON `mem`.`model_id` = f.`fit_model_id`
LIMIT 1;
When I run in the terminal this SQL executes in 16 seconds. However, if I remove the line:
INNER JOIN `table_three` f ON `mem`.`model_id` = f.`fit_model_id`
then it executes in 0.03 seconds. Unfortunately for me, I'm not to sure how to debug MYSQL performance issues. This causes my PHP script to run out of memory trying to execute the query.
Here are my table structures:
table_one
+----------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+---------+------+-----+---------+-------+
| epid | int(11) | YES | | NULL | |
| model_id | int(11) | YES | | NULL | |
+----------+---------+------+-----+---------+-------+
table_two
+----------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------+--------------+------+-----+---------+-------+
| id | int(11) | NO | PRI | NULL | |
| ePID | int(11) | NO | | NULL | |
| UKM_Make | varchar(100) | NO | | NULL | |
| UKM_Model | varchar(100) | NO | | NULL | |
| UKM_CCM | int(11) | NO | | NULL | |
| UKM_Submodel | varchar(100) | NO | | NULL | |
| Year | int(11) | NO | | NULL | |
| UKM_StreetName | varchar(100) | NO | | NULL | |
| Vehicle Type | varchar(100) | NO | | NULL | |
+----------------+--------------+------+-----+---------+-------+
table_three
+-----------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+-------------+------+-----+---------+----------------+
| fit_fitment_id | int(11) | NO | PRI | NULL | auto_increment |
| fit_part_number | varchar(50) | NO | | NULL | |
| fit_model_id | int(11) | YES | | NULL | |
| fit_year_start | varchar(4) | YES | | NULL | |
| fit_year_end | varchar(4) | YES | | NULL | |
+-----------------+-------------+------+-----+---------+----------------+
The above is output from describe $table_name
Is there anything that I'm obviously missing and if not, how can I try to find out why including table_three causes such a slow response time?
EDIT ONE:
After the indexing suggestion (used CREATE INDEX fit_model ON table_three (fit_model_id), it performs the query in 0.00 seconds (in MYSQL). Removing the limit, is still running from after doing the suggestion ... so not quite there. Anton's suggestion about using EXPLAIN I used it and got this output:
+------+-------------+-------+------+---------------+-----------+---------+----------------------+-------+-------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+------+---------------+-----------+---------+----------------------+-------+-------------------------------------------------+
| 1 | SIMPLE | mem | ALL | NULL | NULL | NULL | NULL | 5587 | Using where |
| 1 | SIMPLE | f | ref | fit_model | fit_model | 5 | mastern.mem.model_id | 14 | |
| 1 | SIMPLE | em | ALL | NULL | NULL | NULL | NULL | 36773 | Using where; Using join buffer (flat, BNL join) |
+------+-------------+-------+------+---------------+-----------+---------+----------------------+-------+-------------------------------------------------+
EDIT TWO
I've added a Foreign Key based on suggestions using the below query:
ALTER TABLE `table_one`
ADD CONSTRAINT `model_id_fk_tbl_three`
FOREIGN KEY (`model_id`)
REFERENCES `table_three` (`fit_model_id`)
MYSQL is still executing the command - there are a lot of rows, so half-expecting this behaviour. With PHP I can break up the query and build my array like that, so I guess that possibly solves the issue - thought is there anything more I can do to try and reduce execution time?
Based on everyone's comments etc. I managed to perform a few things that made my query run a hell of a lot quicker and not crash my script.
1) Indexes
I created an index on my table_three for the field fit_model_id:
CREATE INDEX fit_model ON `table_three` (`fit_model_id`);
This made my LIMIT 1 query go from 16 seconds to 0.03 seconds execution time (in MYSQL CLI).
However, 100 rows or so would still take a lot longer than I thought.
2) Foreign Keys
I created a foreign key that linked table_one.model_id = table_three.fit_model_id using the below query:
ALTER TABLE `table_one`
ADD CONSTRAINT `model_id_fk_tbl_three`
FOREIGN KEY (`model_id`)
REFERENCES `table_three` (`fit_model_id`)
This definitely helped, but still felt like more could be done.
3) OPTIMIZE TABLE
I then used OPTIMIZE TABLE on these tables:
table_one
table_three
This then made my script work and my query fast as ever. However, the issue I had was a large data set, so I let, the query run in MYSQL CLI whilst increasing the LIMIT by 1000 each script run time to help the indexing process, got all the way to 30K rows before it started crashing.
CLI took 31 minutes and 8 seconds to complete. So I did this:
31 x 60 = 1860
1860 + 8 = 1868
1868 / 448476 = 0.0042
So each row took 0.0042 seconds to complete - which is fast enough in my eyes.
Thanks to everyone for commenting and helping me debug and fix the issue :)
Based on comments correct answer is as follows:
In case of long execution of select statement add EXPLAIN statement before SELECT
Check whether possible_keys are empty in subqueries for specific tables.
Add FOREIGN KEYs for tables found in step 2. In case of vast table it's recommended to adjust MAX_EXECUTION_TIME variable (can be done for single query)
In case of massive insert/update/delete operations OPTIMIZE TABLE can adjust performance also.
I tried searching for this but I could not find anything about this but design choices.
So my question is like the title. What is faster? Create 1 table with many columns or create 2 or 3 (for many to many) tables with join(s).
I like the idea of have multiple tables so the data is separated. Mostly for many to many like data. But my friend told me having 5 columns with boolean is just fine. But I like the idea of have a table with the settings and then a table between with user.id and setting.id. But my question is also, does it have a impact on the query?
example:
Users
- id
- Email
- SettingA
- SettingB
- SettingC
OR example:
Uers
- id
- email
Users_Settings
- user_id
- setting_id
Settings
- id
- someSettingsValue
What woult be faster for Mysql to query the data? (retrieving settings for user)
not at all.. Only joins between 2 r 3 will take time compared to single table fields.
It's not about preferring single table with many columns or preferring multiple tables. It's about Normalization.
According to your provided schema, if all users will always have three settings, i.e. Setting A, B and C, then it's better to create single table with these three columns for settings.
users table:
+----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| Email | varchar(128) | NO | | NULL | |
| SettingA | tinyint(1) | NO | | NULL | |
| SettingB | tinyint(1) | NO | | NULL | |
| SettingC | tinyint(1) | NO | | NULL | |
+----------+--------------+------+-----+---------+----------------+
But if any of the setting is saved is null, then better is to create separate table for settings and then a pivot table for maintaining users' settings without primary key.
users table:
+-------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| Email | varchar(128) | NO | | NULL | |
+-------+--------------+------+-----+---------+----------------+
settings table:
+---------------+------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| setting_value | tinyint(1) | NO | | NULL | |
+---------------+------------+------+-----+---------+----------------+
setting_user pivot table:
+------------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+---------+------+-----+---------+-------+
| setting_id | int(11) | NO | | NULL | |
| user_id | int(11) | NO | | NULL | |
+------------+---------+------+-----+---------+-------+
Where setting_user is the pivot table.
One more thing is considered when creating schema, that, will there be always three settings, or will there be more in future, when application is expanded!
Hello so I have an application in Laravel in which a lot of user data is stored in separate tables across several models. I now have a requirement to create an activity feed, which means ordering the various data across tables by date.
For illustrative purposes, imagine I have two models, Comment and Like.
I want a feed that combines both by date. merge() is not an option because they may have the same id.
Therefore I could UNION them, but my problem is I won't know what came from what.
My likes table looks like this:
+------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| user_id | int(11) | NO | | NULL | |
| asset_id | int(11) | NO | | NULL | |
| created_at | timestamp | YES | | NULL | |
| updated_at | timestamp | YES | | NULL | |
+------------+------------------+------+-----+---------+----------------+
My comments table looks like this:
+------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| asset_id | int(11) | NO | | NULL | |
| content | longtext | NO | | NULL | |
| user_id | int(11) | NO | | NULL | |
| created_at | timestamp | YES | | NULL | |
| updated_at | timestamp | YES | | NULL | |
+------------+------------------+------+-----+---------+----------------+
My issue is if I just union the two queries, I would only know they are different based on the presence of the content column, which could be in another model, such as Blurb or whatever.
Basically, how do I get multiple queries across models while keeping straight what belongs where, because in my activity feed I want to say, 10 minutes ago you commented, 5 minutes ago you liked, etc.
I don't want to do multiple queries because of inefficiency and I don't want to store all the activities (likes and comments, etc.) in one table either. Is there some kind of alias I can use where instead of renaming a column I insert data using the query for the purposes of the query, so for example a comment selection would add "comment" in a temporary field so that I can access it like
$data->type? I could put a type in all of the tables but then I'd have space being taken up needlessly, as obviously I know a comment is a comment if its in the comment table when that is my only query, but now I am rethinking my structure given I need one query to span multiple tables.
you can use following code to get user activity feed.
$userId = Auth::id(); //or whatever you want.
$activity = DB::table('comment as ac')
->select(DB::raw('ac.user_id , ac.asset_id , ac.comment , ac.created_at , ac.updated_at , "comment" as activity_type'))
->where("ac.user_id", $userId)
->union(
DB::table('like as al')
->select(DB::raw('al.user_id , al.asset_id , NULL as comment , al.created_at , al.updated_at , "like" as activity_type'))
->where("al.user_id", $userId)
)
->latest()
->get();
When performing your query, select an addition raw value based on the table name. For example, in raw SQL:
SELECT likes.*, '' AS content, 'like' AS type
FROM likes
WHERE likes.user_id = 1
UNION
SELECT comments.*, 'comment' AS type
FROM comments
WHERE likes.user_id = 1
ORDER BY created_at DESC
The Laravel code (untested) will look something like:
$activity = DB::table('comments')
->select("comments.*, 'comment' AS type")
->where('comments.user_id', $user->id)
->union(
DB::table('likes')
->select("likes.*, '' AS content, 'like' AS type")
->where('likes.user_id', $user->id)
)
->orderBy('created_at', 'ASC')
->get();
I'm trying to build a similar facebook style messaging system (conversations).
This is the conversation table.
DESCRIBE conversation;
+----------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+-------------+------+-----+---------+----------------+
| c_id | int(11) | NO | PRI | NULL | auto_increment |
| user_one | int(11) | NO | | NULL | |
| user_two | int(11) | NO | | NULL | |
| ip | varchar(30) | NO | | NULL | |
| time | int(11) | NO | | NULL | |
+----------+-------------+------+-----+---------+----------------+
Now before the user can read a conversation, I need to check if the conversation (c_id) exists, and if the user is the owner of the given conversation id. What is the best possible way to write this query?
Example of what I have, which is not working:
$cid = intval($_GET['cid']);
$conv = $this->db->fetchRow('SELECT c_id FROM `conversation` WHERE
user_one=? OR
user_two=? AND
c_id=?',
array($this->user->id, $this->user->id, $cid));
if ($conv) {
// get the conversation replies etc..
}
I see a couple of problems.
One is that you seem to have overlooked that AND has a higher precedence than OR. So the logic of your condition works as if you had written it this way:
WHERE user_one=? OR (user_two=? AND c_id=?)
Whereas I would guess that you intended the logic to work this way:
WHERE (user_one=? OR user_two=?) AND c_id=?
But if that's how you intended it to work, I wonder why you need to search for the user id's at all, since the condition on c_id=? will select only one row (or zero rows if there's no match), because it's searching for one specific primary key value.
I'm facing the following situation.
We've got an CMS with an entity with translations. These translations are stored in a different table with a one-to-many relationship. For example newsarticles and newsarticle_translations. The amount of available languages is dynamically determined by the same CMS.
When entering a new newsarticle the editor is required to enter at least one translation, which one of the available languages he chooses is up to him.
In the newsarticle overview in our CMS we would like to show a column with the (translated) article title, but since none of the languages are mandatory (one of them is mandatory but i don't know which one) i don't really know how to construct my mysql query to select a title for each newsarticle, regardless of the entered language.
And to make it all a little harder, our manager asked for the possibilty to also be able to sort on title, so fetching the translations in a separate query is ruled out as far as i know.
Anyone has an idea on how to solve this in the most efficient way?
Here are my table schema's it it might help
> desc news;
+-----------------+----------------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+----------------+------+-----+-------------------+----------------+
| id | int(10) | NO | PRI | NULL | auto_increment |
| category_id | int(1) | YES | | NULL | |
| created | timestamp | NO | | CURRENT_TIMESTAMP | |
| user_id | int(10) | YES | | NULL | |
+-----------------+----------------+------+-----+-------------------+----------------+
> desc news_translations;
+-----------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| enabled | tinyint(1) | NO | | 0 | |
| news_id | int(1) unsigned | NO | | NULL | |
| title | varchar(255) | NO | | | |
| summary | text | YES | | NULL | |
| body | text | NO | | NULL | |
| language | varchar(2) | NO | | NULL | |
+-----------------+------------------+------+-----+---------+----------------+
PS: i've though about subqueries and coalesce() solutions but those seem rather dirty tricks, wondering if something better is know that i'm not thinking of?
This is not a fast approach, but I think it gives you what you want.
Let me know how it works, and we can work on speed next :)
select nt.title
from news n
join news_translations nt on(n.id = nt.news_id)
where nt.title is not null
and nt.language = (
select max(x.language)
from news_translations x
where x.title is not null
and x.new_id = nt.news_id)
order
by nt.title;
Assuming I've read your problem aright, you want to get a list of titles for articles, preferring the "required" language? A query for that might go along the lines of ...
SELECT * FROM (
SELECT nt.`title`, nt.news_id
FROM news n
INNER JOIN news_translations nt ON (n.id = nt.news_id)
WHERE title != ''
ORDER BY
CASE
WHEN nt.language = 'en' THEN 3
WHEN nt.language = 'jp' THEN 2
WHEN nt.language = 'de' THEN 1
ELSE 0 END DESC
) AS t1
GROUP BY `news_id`
This example prefers a title in English (en) if available, Japanese (jp) as a second preference, and German (de) as a third, but will display the first 'other' entry if none of the requested languages are available.