I am concerned about optimizing my queries as far as the amount of data that is sent back from the database after doing a query.
Let's say that I have 2 tables. One called "artists" and another called "albums".
Let's say "artists" table columns are: id and name
while "albums" table columns are: id, artist_id, title
Let's say that I want a page to diplay the artist's name as the heading. And then below that, I want to display a list of the artist's albums.
I can get that done easily by doing something like:
SELECT artists.name AS artist_name, albums.title AS album_title
LEFT JOIN albums
ON albums.artist_id = artists.id
WHERE artists.id = 3;
This would give me a result that could look something like:
artist_name | album_title
|
Justin Bieber | My First Crappy Album
Justin Bieber | Another Crappy Album
Justin Bieber | Yet Another Crappy Album
The problem with this result is that it gives me back the artist name multiple times, when I only really need it once. I am concerned that this way of doing things is not very efficient. Especially if the artist already has plenty of albums. I could be wrong about this, and if I am, someone please correct me.
In any case, would there be a better way of doing this? One where I don't have to retrieve the artist name multiple times?
You can use the aggregate functions.
SELECT ar.name AS artist_name, GROUP_CONCAT(al.title) AS album_titles
FROM artists ar, albums al
WHERE ar.id = al.artist_id
AND ar.id = 3
GROUP BY artist_name;
This should give you something like:
artist_name | album_titles
Justin Bieber | Album1, Album2, Album3
I haven't used this command in a while, but you can find more documentation on it here: http://dev.mysql.com/doc/refman/5.7/en/group-by-functions.html#function_group-concat
Also, I prefer to list my tables in the FROM clause and use the WHERE clause to join them, but it's the same as your JOIN above... just a different syntax.
Then do two seperate requests.
First request the artist name
SELECT name
FROM artists
WHERE id = 3
Then the album titles..
SELECT title
FROM albums
WHERE artist_id = 3
I don't know about any bandwith logging inside php, but you can calculate witch way is the fastest in executing using microtime()
Related
I am quite new to programming and am having some problems in trying to get data from a SQL database into my PHP website.
I will make something quite similar to the scenario below.
Let's say my tables include:
pets
pet_id pet_type
1 cat
2 dog
3 rabbit
pet_symptoms
symptom_id symptom
1 not playing
2 not eating
3 not sleeping
...
Action_call
call_id advice
1 take to vet
2 change food
3 give a shower
...
I need my last page to pull session data from the first and second page to get (1) what the problem is with the pet and (2) what pet it is, in order to (3) give the correct advice.
here is my SQL on the 3rd page - calling vet
$rows=$db->query("SELECT call_id, advice pet_id, symptom_id, symptom, pet_type
FROM pets, pet_symptoms, call_vet
WHERE pets.pet_id = pet_symptoms.pet_id
AND pets.pet_type='$_SESSION[pet_type]'
");
the problem I am having is that I want to see the problem as text and not an id on the second page, so I am not sure how to match the ids from different tables according to the problem selected.
thanks for any help
EDIT:
so that would look like
id pet_type pet_symptom
1 cat not sleeping
2 dog not sleeping
.....
CREATE TABLE Pet_health
(Pet_id number(3),
Pet_name Varchar2(15,)
Symptom_id number(3),
Call_id number(3))
Create this table. and this table will get the information from the other 3 table,
you do not have to UPDATE Pet, Pet Symptoms and Action call table unless u want to add new type of pet, new symptom and new advice. do not add duplicates.
Now Pet Health can have all the pets health issues
Once this is done then you can use this SELECT query
SELECT P.pet_type, PH.pet_name, S.symptom, A.advice
FROM Pet_Health PH
INNER JOIN Pets P
ON PH.pet_id =P.pet_id
INNER JOIN Symptom S
ON PH.symptom_id = S.symptom_id
INNER JOIN Action_call A
ON PH.call_id = A.call_id
This query will show everything. if you want to show the ID, you can add it in SELECT statement
I have the following tables
ea_users
id
first_name
last_name
email
password
id_roles
ea_user_cfields
id
c_id = custom field ID
u_id = user ID
data
ea_customfields
id
name = name of custom field
description
I want to get all users which have a certain role, but I also want to retrieve all the custom fields per user. This is for the backend of my software where all the ea_users and custom fields should be shown.
I tried the following, but for each custom field, it duplicates the same user
$this->db->join('(SELECT GROUP_CONCAT(data) AS custom_data, id AS dataid, u_id, c_id
FROM ea_user_cfields userc
GROUP BY id) AS tt', 'tt.u_id = ea.id','left');
$this->db->join('(SELECT GROUP_CONCAT(name) AS custom_name, id AS customid
FROM ea_customfields AS cf
GROUP BY id) AS te', 'tt.c_id = te.customid','left');
$this->db->where('id_roles', $customers_role_id);
return $this->db->get('ea_users ea')->result_array();
the problem that u did not understand properly how join works.
its ok, that u have duplicates in select when u have relation one to many.
in few words your case: engine tries to fetch data from table "A" (ea_users) then JOIN according to the conditions another table "B" (ea_customfields). If u have one to many relation between tables (it means that one record from table "A" (lets say that we have in this table A1 record) can contain few related rows in table "B", lets call them as B1.1, B1.2 and B1.3 and B1.4), in this case it will join this records and put join result in memory. So in memory u would see something like
| FromTable A | FromTableB |
| A1 | B1.1 |
| A1 | B1.2 |
| A1 | B1.3 |
| A1 | B1.4 |
if u have 10 records in table "B", which related to the table "A" it would put 10 times in memory copy of data from table "A" during fetching. And then will render it to u.
depending on join type rows, with missing related records, can be skipped at all (INNER JOIN), or can be filled up with NULLs (LEFT JOIN or RIGHT JOIN), etc.
When u think about JOINs, try to imagine yourself, when u try to join on the paper few big tables. U would always need to mark somehow which data come from which table in order to be able to operate with it later, so its quite logically to write row "A1" from table "A" as many times as u need to fill up empty spaces when u find appropriate record in table "B". Otherwise u would have on your paper something like:
| FromTable A | FromTableB |
| A1 | B1.1 |
| | B1.2 |
| | B1.3 |
| | B1.4 |
Yes, its looks ok even when column "FromTable A" contains empty data, when u have 5-10 records and u can easily operate with it (for example u can sort it in your head - u just need to imagine what should be instead of empty space, but for it, u need to remember all the time order how did u wrote the data on the paper). But lets assume that u have 100-1000 records. if u still can sort it easily, lets make things more complicated and tell, that values in table "A" can be empty, etc, etc.. Thats why for mysql engine simpler to repeat many times data from table..
Basically, I always stick to examples when u try to imagine how would u join huge tables on paper or will try to select something from this tables and then make sorting there or something, how would u look through the tables, etc.
GROUP_CONCAT, grouping
Then, next mistake, u did not understand how GROUP_CONCAT works:
The thing is that mysqlEngine fetch on the first step structure into memory using all where conditions, evaluating subqueries + appends all joins. When structure is loaded, it tried to perform GROUPing. It means that it will select from temporary table all rows related to the "A1". Then will try to apply aggregation function to selected data. GROUP_CONCAT function means that we want to apply concatenation on selected group, thus we would see something like "B1.1, B1.2, B1.3, B1.4". Its in few words, but I hope it will help a little to understand it.
I googled table structure so u can write some queries there.
http://www.mysqltutorial.org/tryit/query/mysql-left-join/#1
and here is example how GROUP_CONCAT works, try to execute there query:
SELECT
c.customerNumber, c.customerName, GROUP_CONCAT(orderNumber) AS allOrders
FROM customers c
LEFT JOIN orders o ON (c.customerNumber = o.customerNumber)
GROUP BY 1,2
;
can compare with results with previous one.
power of GROUP in aggregation functions which u can use with it. For example, u can use "COUNT()", "MAX()", "GROUP_CONCAT()" or many many others.
or example of fetching of count (try to execute it):
SELECT c.customerName, count(*) AS ordersCount
FROM customers AS c
LEFT JOIN orders AS o ON (c.customerNumber = o.customerNumber)
GROUP BY 1
;
so my opinion:
simpler and better to solve this issue on client side or on backend, after fetching. because in term of mysql engine response with duplication in column is absolutely correct. BUT of course, u can also solve it using grouping with concatenations for example. but I have a feeling that for your task its overcomplicating of logic
PS.
"GROUP BY 1" - means that I want to group using column 1, so after selecting data into memory mySql will try to group all data using first column, better not to use this format of writing on prod. Its the same as "GROUP BY c.customerNumber".
PPS. Also I read comments like "use DISTINCT", etc.
To use DISTINCT or order functions, u need to understand how does it work, because of incorrect usage it can remove some data from your selection, (same as GROUP or INNER JOINS, etc). On the first look, you code might work fine, but it can cause bugs in logic, which is the most complicated to find out later.
Moreover DISTINCT will not help u, when u have one-to-many relation(in your particular case). U can try to execute queries:
SELECT
c.customerName, orderNumber AS nr
FROM customers c
INNER JOIN orders o ON (c.customerNumber = o.customerNumber)
WHERE c.customerName='Alpha Cognac'
;
SELECT
DISTINCT(c.customerName), orderNumber AS nr
FROM customers c
INNER JOIN orders o ON (c.customerNumber = o.customerNumber)
WHERE c.customerName='Alpha Cognac'
;
the result should be the same. Duplication in customer name column and orders numbers.
and example how to loose data with incorrect query ;):
SELECT
c.customerName, orderNumber AS nr
FROM customers c
INNER JOIN orders o ON (c.customerNumber = o.customerNumber)
WHERE c.customerName='Alpha Cognac'
GROUP BY 1
;
I have to select all videos, which are related (1:N Relation) to many categories.
But finally I only want these videos, which are related to all given categories.
For example:
Request: cat[]=5&cat[]=6
Results: All videos, which are in category 5 AND 6
In fact I know how to handle this within 2 given categories (IN Statement, WHERE Exists etc). But we want to do it as dynamically as possible. Which means, that a customer is able to select about 30 categories at a time.
I see a little bit of a performance issue in that case.
Currently I am handling these issue in PHP where I build several arrays, to validate each selected video (which results of an IN select) against each selected category.
But of course I have other problems with that. For example the pagination, limits etc. A lot of overhead I think.
My other idea was to create a cache_table, where the categories are concatenated and stored in an additional column which I would query with a INSTR or FULLTEXT search.
Does anybody know better ways to handle such a Query?
I think you should create a table with possible categories, a table with the videos, and a table which links the 2.
Categories Videos VideoCategories
1 cats 1 Cat being lazy 1 1
2 dogs 2 Dog barking 2 2
3 Dog chasing Cat 3 1
3 2
If you index those tables a query like the following is quite fast:
Select * from videos
where videoId in
( select videoId, count(*) numberOfMatches
from VideoCategories
where categoryid in (1,2)
group by videoId
having numberOfMatches >= 2 )
I don't claim this is the fastest solution, but i think its a good way to get started, and its definitely faster than a fulltext search
Since IN ( SELECT ... ) is very poorly optimized, turn it into a JOIN:
Select b.*
from videos b
JOIN
( SELECT videoId, count(*) numberOfMatches
from VideoCategories
where categoryid in (5,6)
group by videoId
having numberOfMatches >= 2
) a USING videoId;
I come forth bearing, rather obviously, a brain-stormy question. That, or I'm just too much of a noob to see any obvious answers to it:
How can I achieve a tagging system in which each tag has a specific relationship to each user of a website?
A very plain example of what I'm trying to achieve would be a situation in which the system keeps track of page-hits a user makes for each tag. For example, multiple tag keywords are "tagged" to every page of the website. A page about environmental science might be tagged with keywords like "biology" and "sociology". When User A visits that page, a number is incremented by 1 that counts each time User A has viewed "biology", and another count for "sociology".
Thus, we end up with records that exhibit how much a person likes a particular tag.
What would be the most efficient way to do this with MySQL and would it be possible to work it into a two-table system such as the following?
Table 1: Tags
- tag_title
- tagged_pages
- user_likes
Table 2: Users
- user_id
Adding more table columns is completely fine. What I'm trying to avoid is duplicating the tags table and all the records in it.
All that I've come up with so far is having the user_likes field be formatted something like this for each tag: 101-12, 99-3, 156-14...etc. Then using PHP explode() to separate the user_id from the number of likes.
Thanks a lot for your creative insight!
Don't ever put more than one information in a column. It's the first rule of normalization and you really shouldn't break it.
I think you need some more tables to correctly build this (and have it normalized correctly)
users
id
name
tags
id
name
pages
id
page_tags
page_id
tag_id
user_page_hits
user_id
page_id
hits
Now, if you want to know how much a user likes a certain tag, you can create a VIEW on those tables like
user_tag_hits
as select u.name, t.name, sum(uph.hits) hits from users u
join user_page_hits uph on uph.user_id = u.id
join page_tags pt on pt.page_id = uph.page_id
join tags t on t.id = pt.tag_id
group by u.name, t.name
Which would produce results like
username tagname hits
me tag1 112
me tag2 112
me tag3 70
me tag4 4
me tag5 4
I have two tables, categories and info.
categories looks like:
id | name
info looks like:
id | parent_id | name | url
parent_id is the id from categories.
I have a form where a user can add more than one url. There is a button that can be pressed to make more url fields appear. If there is more than one url, then url2, url3, etc are added to the database.
info will then look like:
id | parent_id | name | url | url2 | url3
Is this an appropriate approach?
If so, what if info is like this:
id | parent_id |name |url |url2 |url3
1 | 1 |One |http://cnn.com |
2 | 1 |Two |http://msn.com |http://aol.com|
When I view One or Two or w/e, how do I query to get their urls?
I know I can do:
$mysqli->query("SELECT i.url FROM info AS i LEFT JOIN categories AS c ON i.parent_id = c.id");
But this is dependent on me putting i.url, i.url2, i.url3 and I would have to create a separate query for each info. I want it so PHP determines how many and what i.url to select.
So it should be:
$mysqli->query("SELECT (PHP determines what i.url to put here depending on what info page I'm viewing) FROM info AS i LEFT JOIN categories AS c ON i.parent_id = c.id");
If you go that path you will have a bunch of columns with null values and encounter issues with queries, which is pretty much what you are describing.
I would create a different table (perhaps urls) that maps info with urls. For example:
url_id | info_id | url
This is the Repeated Attributes design pattern which is concisely explained here:
http://www.tomjewett.com/dbdesign/dbdesign.php?page=phone.php
Here is an example on how to query:
SELECT i.*, url
FROM info i JOIN urls u ON i.id = u.info_id
No, the right solution here is another table, perhaps called "info_urls", which models a one-to-many with info-to-urls.
This is a "normalized" design, and will allow unlimited URLs per "info", and is quite easy to query:
SELECT name, GROUP_CONCAT(info_urls.url) as urls
FROM info JOIN info_urls ON info.id = info_urls.info_id;
The columns will look like this:
id
info_id
url
This solution models the reality that you do not know how many URLs the user will enter, and your RDBMS is made to CRUD rows , not columns :-) (and is very good at joining them together for queries)