SQL inner join performance issues - php

I have a WordPress installation and I'm making a custom plugin. My plugin stores a good bit of usermeta. I'm using an inner join to combine data from the "users" and "usermeta" table at which point I spit the result back to my PHP script. Here's what my query looks like:
select
# Users table stuff
users.ID,
users.user_email,
users.display_name,
# Meta stuff
first_name.meta_value as first_name,
last_name.meta_value as last_name,
phone_number.meta_value as phone_number,
country.meta_value as country,
years_of_experience.meta_value as years_of_experience,
highest_degree_obtained.meta_value as highest_degree_obtained,
availability_for_work.meta_value as availability_for_work,
english_proficiency.meta_value as english_proficiency,
disciplines.meta_value as disciplines,
profile_picture.meta_value as profile_picture,
resume.meta_value as resume,
description.meta_value as description,
hourly_rate.meta_value as hourly_rate,
satisfaction_rating.meta_value as satisfaction_rating,
invited_projects.meta_value as invited_projects,
completed_project_count.meta_value as completed_project_count
# The table we're selecting from
from tsd_retro_users as users
# Join in our usermeta table for each individual meta value
inner join tsd_retro_usermeta first_name on users.ID = first_name.user_id
inner join tsd_retro_usermeta last_name on users.ID = last_name.user_id
inner join tsd_retro_usermeta phone_number on users.ID = phone_number.user_id
inner join tsd_retro_usermeta country on users.ID = country.user_id
inner join tsd_retro_usermeta years_of_experience on users.ID = years_of_experience.user_id
inner join tsd_retro_usermeta highest_degree_obtained on users.ID = highest_degree_obtained.user_id
inner join tsd_retro_usermeta availability_for_work on users.ID = availability_for_work.user_id
inner join tsd_retro_usermeta english_proficiency on users.ID = english_proficiency.user_id
inner join tsd_retro_usermeta disciplines on users.ID = disciplines.user_id
inner join tsd_retro_usermeta profile_picture on users.ID = profile_picture.user_id
inner join tsd_retro_usermeta resume on users.ID = resume.user_id
inner join tsd_retro_usermeta description on users.ID = description.user_id
inner join tsd_retro_usermeta hourly_rate on users.ID = hourly_rate.user_id
inner join tsd_retro_usermeta satisfaction_rating on users.ID = satisfaction_rating.user_id
inner join tsd_retro_usermeta invited_projects on users.ID = invited_projects.user_id
inner join tsd_retro_usermeta completed_project_count on users.ID = completed_project_count.user_id
# Define our select stipulations
where
(users.ID = 20)
and
(first_name.meta_key = 'first_name')
and
(last_name.meta_key = 'last_name')
and
(phone_number.meta_key = 'phone_number')
and
(country.meta_key = 'country')
and
(years_of_experience.meta_key = 'years_of_experience')
and
(highest_degree_obtained.meta_key = 'highest_degree_obtained')
and
(availability_for_work.meta_key = 'availability_for_work')
and
(english_proficiency.meta_key = 'english_proficiency')
and
(disciplines.meta_key = 'disciplines')
and
(profile_picture.meta_key = 'profile_picture')
and
(resume.meta_key = 'resume')
and
(description.meta_key = 'description')
and
(hourly_rate.meta_key = 'hourly_rate')
and
(satisfaction_rating.meta_key = 'satisfaction_rating')
and
(invited_projects.meta_key = 'invited_projects')
and
(completed_project_count.meta_key = 'completed_project_count')
As you can see, I inner join the usermeta table for each value that I'm trying to obtain from the database. It all seems to work fine, but after a certain number of inner joins, the query seems to slow to a crawl. Right now, the above query is taking about a second on average, and I only have about twenty users in my user table.
My question is: what are the performance ramifications of inner joins? Is there a better way to run the above query and achieve the same result?

Try this way:
select
# Users table stuff
users.ID,
users.user_email,
users.display_name,
# Meta stuff
MAX(CASE WHEN meta_table.meta_key='first_name' THEN meta_table.first_name ELSE NULL END) as first_name,
MAX(CASE WHEN meta_table.meta_key='last_name' THEN meta_table.last_name ELSE NULL END) as last_name,
MAX(CASE WHEN meta_table.meta_key='phone_number' THEN meta_table.phone_number ELSE NULL END) as phone_number,
MAX(CASE WHEN meta_table.meta_key='country' THEN meta_table.country ELSE NULL END) as country,
MAX(CASE WHEN meta_table.meta_key='years_of_experience' THEN meta_table.years_of_experience ELSE NULL END) as years_of_experience,
MAX(CASE WHEN meta_table.meta_key='highest_degree_obtained' THEN meta_table.highest_degree_obtained ELSE NULL END) as highest_degree_obtained,
MAX(CASE WHEN meta_table.meta_key='availability_for_work' THEN meta_table.availability_for_work ELSE NULL END) as availability_for_work,
MAX(CASE WHEN meta_table.meta_key='english_proficiency' THEN meta_table.english_proficiency ELSE NULL END) as english_proficiency,
MAX(CASE WHEN meta_table.meta_key='disciplines' THEN meta_table.disciplines ELSE NULL END) as disciplines,
MAX(CASE WHEN meta_table.meta_key='profile_picture' THEN meta_table.profile_picture ELSE NULL END) as profile_picture,
MAX(CASE WHEN meta_table.meta_key='resume' THEN meta_table.resume ELSE NULL END) as resume,
MAX(CASE WHEN meta_table.meta_key='description' THEN meta_table.description ELSE NULL END) as description,
MAX(CASE WHEN meta_table.meta_key='hourly_rate' THEN meta_table.hourly_rate ELSE NULL END) as hourly_rate,
MAX(CASE WHEN meta_table.meta_key='satisfaction_rating' THEN meta_table.satisfaction_rating ELSE NULL END) as satisfaction_rating,
MAX(CASE WHEN meta_table.meta_key='invited_projects' THEN meta_table.invited_projects ELSE NULL END) as invited_projects,
MAX(CASE WHEN meta_table.meta_key='completed_project_count' THEN meta_table.completed_project_count ELSE NULL END) as completed_project_count
# The table we're selecting from
from tsd_retro_users as users
# Join in our usermeta table for each individual meta value
inner join tsd_retro_usermeta as meta_table
ON users.ID = meta_table.user_id
# Define our select stipulations
where
(users.ID = 20)
GROUP BY users.ID
And as soon as we use php you can do this way:
$requiredMetaFields = array(
'first_name',
'last_name',
'phone_number',
'country',
'years_of_experience',
'highest_degree_obtained',
'availability_for_work',
'english_proficiency',
'disciplines',
'profile_picture',
'resume',
'description',
'hourly_rate',
'satisfaction_rating',
'invited_projects',
'completed_project_count'
);
$query = "select
users.ID,
users.user_email,
users.display_name,";
foreach($requiredMetaFields as $metafield) {
$query .= "MAX(CASE WHEN meta_table.meta_key='$metafield' THEN meta_table.$metafield ELSE NULL END) as $metafield, ";
}
$query =substr($query,0,-1);
$query .= "from tsd_retro_users as users
inner join tsd_retro_usermeta as meta_table
ON users.ID = meta_table.user_id
where
(users.ID = 20)
GROUP BY users.ID";
Edited syntax errors

Related

Sort mysqli from a many-to-many table

I'm trying to make a internal web based message system, with a *amp system, primarily for learning purposes. I don't know if this is a trivial topic, but I'm having difficulties so please bear with me.
The goal is to list all the contacts ordered by the last message sent / received.
Currently without sorting it the SQL looks like this
$query = "SELECT username, user.id as user_id,
(SELECT COUNT(message_read)
FROM message_user
WHERE message_read = 0
AND sent_id = user_id
AND receive_id = {$userId}) as unread
FROM user
WHERE user.id IN
(SELECT contact_id FROM allowed_contact WHERE user_id = {$userId})
;";
The structure of the tables are:
The user table has an id,
That links to the message_user table which has a sent_id and a receive_id,
The message_user has a message_id that corresponds to the message.id,
The message table has a timestamp.
I would like this to be done in SQL but if it comes down to PHP I resign to resort to that.
This works.
SELECT `u`.`id` AS user_id, username,
(SELECT COUNT(message_user.message_read)
FROM message_user
WHERE message_user.message_read = 0
AND sent_id = user_id
AND receive_id = {$userId}) as unread
FROM `user` AS `u`
LEFT JOIN `message_user` AS `mu`
ON
(CASE WHEN `u`.`id` != {$userId}
THEN `u`.`id` = `mu`.`sent_id`
WHEN `mu`.`sent_id` = {$userId} AND `mu`.`receive_id` = {$userId}
THEN `u`.`id` = `mu`.`sent_id`
END)
OR
(CASE WHEN `u`.`id` != {$userId}
THEN `u`.`id` = `mu`.`receive_id`
END)
LEFT JOIN `message` AS `m` ON `m`.`id` = `mu`.`message_id`
WHERE u.id IN
(SELECT contact_id FROM allowed_contact WHERE user_id = {$userId})
GROUP BY u.id
ORDER BY MAX(`m`.`timestamp`) DESC;
This broke down the problem I was having.
#Andreas thanks for time and help.
Use 2 LEFT JOIN with a DISTINCT (untested):
SELECT DISTINCT `u`.`id`
FROM `user` AS `u`
LEFT JOIN `message_user` AS `mu` ON `u`.`id` = `mu`.`sent_id` OR `u`.`id` = `mu`.`receive_id`
LEFT JOIN `message` AS `m` ON `m`.`id` = `mu`.`message_id`
ORDER BY `m`.`timestamp` DESC;

Selecting Max Value and Corresponding Columns

I am trying to grab the max date from the updates column but also return the corresponding fullname from that table, what is currently happening is that the latest updates.date is being returned but the first updates.consultant is currently being returned as well and we need the correct fullname for the MAX date.
SELECT customer.id,
customer.name,
customer.retainer_value,
customer.customer_type,
clientdetails.performance,
clientdetails.url,
members.fullname AS acc_manager,
u.maxdate,
u.fullname
FROM customer
LEFT JOIN clientdetails
ON clientdetails.id = customer.id
LEFT JOIN members
ON members.id = customer.consultant_name
LEFT JOIN (SELECT updates.clientid,
members.fullname,
Max(updates.`date`) AS MaxDate
FROM updates
LEFT JOIN members
ON members.id = updates.consultant
GROUP BY updates.clientid
ORDER BY updates.date DESC) u
ON customer.id = u.clientid
WHERE customer.switchedoff = 'N'
AND customer.companyid <> '3'
I think the easiest way in your case is to use the substring_index()/group_concat() method:
SELECT customer.id,
customer.name,
customer.retainer_value,
customer.customer_type,
clientdetails.performance,
clientdetails.url,
members.fullname AS acc_manager,
u.maxdate,
u.fullname
FROM customer
LEFT JOIN clientdetails
ON clientdetails.id = customer.id
LEFT JOIN members
ON members.id = customer.consultant_name
LEFT JOIN (SELECT updates.clientid,
substring_index(group_concat(m.fullname order by u.date desc separator '|'), '|', 1) as full_name
Max(updates.`date`) AS MaxDate
FROM updates u
LEFT JOIN members m
ON m.id = u.consultant
GROUP BY u.clientid
) u
ON customer.id = u.clientid
WHERE customer.switchedoff = 'N'
AND customer.companyid <> '3' ;

can I EXCLUDE rows in my complex query where criteria is met? (combining WHERE + multi AND)

I need to get results from a table where the grouped IDs exist in rows where field1 = a and field2 IS NOT NULL, but NOT rows where field1 = b and field2 IS NULL. just getting syntax error, don't know quite how to combine these criteria...
here's what I'm trying to do:
SELECT a.post_title AS title, a.id,
MAX(CASE WHEN b.meta_key = 'endorser' THEN b.meta_value END) endorser,
MAX(CASE WHEN b.meta_key = 'trail' THEN b.meta_value END) trail,
MAX(CASE WHEN b.meta_key = 'townarea' THEN b.meta_value END) townarea,
MAX(CASE WHEN b.meta_key = 'state' THEN b.meta_value END) state,
MAX(CASE WHEN b.meta_key = 'start-date' THEN b.meta_value END) startdate,
MAX(CASE WHEN b.meta_key = 'description' THEN b.meta_value END) description,
MAX(CASE WHEN b.meta_key = 'organizer-name' THEN b.meta_value END) organizer,
MAX(CASE WHEN b.meta_key = 'info-email' THEN b.meta_value END) infoemail
FROM wp_posts a LEFT JOIN wp_postmeta b ON a.id = b.post_id
WHERE b.post_id IN
(SELECT Post_id FROM wp_postmeta
WHERE (meta_key = 'endorser' AND meta_value IS NOT NULL)
AND (meta_key = 'trail' AND meta_value IS NULL)) group by b.post_id
AND the table looks something like this:
meta_id | post_id | meta_key | meta_value
---------|-------------|------------|------------
1 | 53 | endorser | joe
2 | 54 | trail | trail name
So I would get rows containing post_id 53, but NOT 54
I'm very tired, so I may be missing something simple, like a simple OUTER JOIN?
Here's the fiddle: http://sqlfiddle.com/#!2/ee5420/1
You can get rid of your subquery by using b.meta_key IN ('endorser','trail') and in where clause you can use CASE for your is null and is not null criteria
SELECT a.post_title AS title, a.id,
(SELECT MAX(meta_value) FROM wp_postmeta WHERE meta_key = 'endorser' AND post_id =a.id) endorser,
(SELECT MAX(meta_value) FROM wp_postmeta WHERE meta_key = 'trail' AND post_id =a.id) trail,
(SELECT MAX(meta_value) FROM wp_postmeta WHERE meta_key = 'townarea' AND post_id =a.id) townarea,
(SELECT MAX(meta_value) FROM wp_postmeta WHERE meta_key = 'state' AND post_id =a.id) state,
(SELECT MAX(meta_value) FROM wp_postmeta WHERE meta_key = 'start-date' AND post_id =a.id) startdate,
(SELECT MAX(meta_value) FROM wp_postmeta WHERE meta_key = 'description' AND post_id =a.id) description,
(SELECT MAX(meta_value) FROM wp_postmeta WHERE meta_key = 'organizer-name' AND post_id =a.id) organizer,
(SELECT MAX(meta_value) FROM wp_postmeta WHERE meta_key = 'info-email' AND post_id =a.id) infoemail
FROM wp_posts a
LEFT JOIN wp_postmeta b ON a.id = b.post_id
WHERE b.meta_key IN ('endorser','trail')
AND (
CASE
WHEN b.meta_key = 'endorser'
THEN b.meta_value IS NOT NULL
WHEN b.meta_key = 'trail'
THEN b.meta_value IS NULL
END
)
HAVING trail =''
Fiddle Demo
Dont use LEFT JOIN, use INNER JOIN
FROM wp_posts a INNER JOIN wp_postmeta b ON a.id = b.post_id

Yii CDbCommandBuilder query

I'm trying to perform a query in Yii with CDbCommandBuilder so I can have the resultset in an array.
Problem is that I don't understand how to convert my SQL (pgsql) to the Yii CDbCommandBuilder syntax. Mainly my problem is with nested joins.
The query:
SELECT p.id
up.id as fid,
sum(CASE
WHEN v1.count>v2.count THEN v2.count
ELSE v1.count
END
) as res
FROM product as v1
INNER JOIN (
SELECT p_id, count
FROM product
WHERE user_id = {$user_id}) as v2 on v1.p_id = v2.p_id and v1.user_id <> {$user_id}
RIGHT JOIN users as p on p.id = v1.user_id
INNER JOIN uf on uf.friend_id = p.id and uf.user_id = {$user_id} and is_active = true
INNER JOIN up on up.user_id = p.id and is_active = true
GROUP BY p.id, up.id
ORDER BY res desc
Can anyone help?
Thanks
$sql = "SELECT p.id
up.id as fid,
sum(CASE
WHEN v1.count>v2.count THEN v2.count
ELSE v1.count
END
) as res
FROM product as v1
INNER JOIN (
SELECT p_id, count
FROM product
WHERE user_id = :user_id) as v2 on v1.p_id = v2.p_id and v1.user_id <> :user_id
RIGHT JOIN users as p on p.id = v1.user_id
INNER JOIN uf on uf.friend_id = p.id and uf.user_id = :user_id and is_active = true
INNER JOIN up on up.user_id = p.id and is_active = true
GROUP BY p.id, up.id
ORDER BY res desc";
$params = array(':user_id' => $user_id);
$aArrayOfRows = Yii::app()->db->createCommand($sql)->queryAll(true, $params);

Muliple Joins sum values based on two different critiea with sort

I have a database that has 4 tables.
Table 1 - "company" table with company_id as the key
Table 2 - "users" table with the user_id as the key
Table 3 - "teams" table that references the company_id and the user_id (So that user can belong to multiple teams.
Table 4 - "points" table that references the company_id, the user_id, points_earned (Numeric value of points given), exchange_dte (0 - if the user has not used the points, otherwise a unixtime value)
Given a known company_id, I am trying to call all users that belong to that "team" and show their total points and their un-exchanged points. The following MySQL will only give the first user on company's #1 team. There are currently 5 users in the database all with a number of points earned, some exchanged, some not.
SELECT
users.user_id AS u_id,
SUM(points.points_earned) AS ttl_points,
SUM(case when exchange_dte = '0' then points.points_earned else 0 end) AS unused_points
FROM users
INNER JOIN teams ON teams.user_id = users.user_id
INNER JOIN points ON points.user_id = users.user_id
WHERE (teams.company_id = '1' AND points.company_id = '1' AND users.user_active = '1');
So then I tried to add the user_id to the Sum calls. And end up with the same thing.
SELECT
users.user_id AS u_id,
SUM(case when points.user_id = users.user_id then points.points_earned else 0 end) AS ttl_points,
SUM(case when points.exchange_dte = '0' AND points.user_id = users.user_id then points.points_earned else 0 end) AS unused_points
FROM users
INNER JOIN teams ON teams.user_id = users.user_id
INNER JOIN points ON points.user_id = users.user_id
WHERE (teams.company_id = '1' AND points.company_id = '1' AND users.user_active = '1')
ORDER BY ttl_points;
The interesting thing is, the point totals for the first user appear to be all the points in the database, even though they have a user_id and company_id associated with them
Thoughts?
You're trying to do a SUM without using GROUP BY: not sure if it will work for you but try this adding a GROUP BY users.user_id after the end of the query and see if that helps you out.
SELECT
users.user_id AS u_id,
SUM(case when points.user_id = users.user_id then points.points_earned else 0 end) AS ttl_points,
SUM(case when points.exchange_dte = '0' AND points.user_id = users.user_id then points.points_earned else 0 end) AS unused_points
FROM users
INNER JOIN teams ON teams.user_id = users.user_id
INNER JOIN points ON points.user_id = users.user_id
WHERE (teams.company_id = '1' AND points.company_id = '1' AND users.user_active = '1') GROUP BY users.user_id
ORDER BY ttl_points;

Categories