Merging two complex queries - php

What I ended up doing was taking two SQL queries and using the array_intersect() in PHP to filter out the results:
$sql1 = 'SELECT z.*, u.username, u.user_colour, u.username_clean, u.user_avatar, u.user_avatar_type
FROM ' . ZEBRA_TABLE . ' z, ' . USERS_TABLE . ' u
WHERE (( z.user_id = ' . $user->data['user_id'] . '
AND z.friend = 1
AND u.user_id = z.zebra_id )
OR ( z.zebra_id = ' . $user->data['user_id'] . '
AND z.friend = 1
AND u.user_id = z.user_id ))
ORDER BY u.username_clean ASC';
$sql2 = 'SELECT z.*, u.username, u.user_colour, u.username_clean, u.user_avatar, u.user_avatar_type
FROM ' . ZEBRA_TABLE . ' z, ' . USERS_TABLE . ' u
WHERE (( z.user_id = ' . $user_id . '
AND z.friend = 1
AND u.user_id = z.zebra_id )
OR ( z.zebra_id = ' . $user_id . '
AND z.friend = 1
AND u.user_id = z.user_id ))
ORDER BY u.username_clean ASC';
The structure of both queries are the same and the only difference is $user->data['user_id] (first person) is replaced with $user_id (second person) in the second query. I want to retrieve friends that both users have in common. Could anyone merge this into a single query so that I don't have to use two queries and call array_intersect()?

Well, you could always just subquery both:
$sql = 'SELECT a.*
FROM ('.$sql1.') AS a
JOIN ('.$sql2.') AS b ON a.user_id = b.user_id AND a.username = b.username';
You may want to add u.user_id to the field list of both queries u.user_id AS u_user_id then change the second join clause from a.username = b.username to a.u_user_id = b.u_user_id...
EDIT: Now that I really look at it closer, those two queries are almost identical... Why not just do something like this (replace the where clause to this):
WHERE z.friend = 1
AND (
( z.user_id = '.$user_id.' AND u.user_id = z.zebra_id )
OR
(z.zebra_id = '.$user_id.' AND u.user_id = z.user_id )
) AND (
( z.user_id = '.$user->data['user_id'].' AND u.user_id = z.zebra_id )
OR
(z.zebra_id = '.$user->data['user_id'].' AND u.user_id = z.user_id )
)
That should give you the result of both queries intersected, and be faster since it can optimize better (hopefully)...
Oh, and they are in different where blocks because there's a few cases where z.user_id matches $user_id, but z.zebra_id matches $user->data['user_id']... So rather than list all the permutations, I just layed it out like this...

You could select users who are friends with both users by linking the user table to the zebra table twice:
SELECT u.username, u.user_colour, u.username_clean, u.user_avatar, u.user_avatar_type
FROM users u
JOIN zebra z1 ON z1.friend=1 AND (
(u.user_id = z1.user_id AND z1.zebra_id = #user_id1)
OR (u.user_id = z1.zebra_id AND z1.user_id = #user_id1)
)
JOIN zebra z2 ON z2.friend=1 AND (
(u.user_id = z2.user_id AND z2.zebra_id = #user_id2)
OR (u.user_id = z2.zebra_id AND z2.user_id = #user_id2)
)
ORDER BY u.username_clean ASC
The JOIN takes all the rows from the users table, and all the rows from the zebra table, and looks for the combinations that satisfy the ON clause. In this case, the first join finds all users who are friends with #user_id1, the second join further restricts it to users who are also friends with #user_id2.
This query will perform much faster than using subqueries will. The query would be even faster if the zebra table stored friendships in both directions, allowing you to take more advantage of table indexes, and you could remove the OR portion of the ON clauses:
SELECT u.username, u.user_colour, u.username_clean, u.user_avatar, u.user_avatar_type
FROM users u
JOIN zebra z1 ON u.user_id = z1.user_id AND z1.friend=1 AND z1.zebra_id = #user_id1
JOIN zebra z2 ON u.user_id = z2.user_id AND z2.friend=1 AND z2.zebra_id = #user_id2
ORDER BY u.username_clean ASC

Related

MySQL Query for HTML table using filters, COUNT, GROUP BY / LIMIT / ORDER BY and sub-queries

I got a query that blows my head off for about 2 days, and I need some help. Pleeeaaze !
What I need to do is get data form described tables below and the output must display every message for each application using filters on customer.(name, codeX/Y/Z), application.(name, wave), migrationLabel.label, migrationMessage.(dates) using pagination (50 per pages).
Each application can display 0 to n messages and I have to make an HTML table using attribute "rowspan".
Here are the tables :
"# : PK"
"# : FK"
application(#id, label, #customerId)
customer(#id, name, codeX, codeY, codeZ)
erpApplication(#id, label, #customerId)
migrationMessage(#applicationId, creationTime, closureDate, previsionalDate, #labelId, #authorId)
migrationLabel(#id, label, ordering)
works(#employeeId, #applicationId, #roleId)
employee(#id, username)
Here is the current Query :
<?php
$query = 'SELECT
c.id AS customerId, c.name AS customerName, c.X3Code, c.mfgProCode, c.as400Code, c.jdeCode,
a.id AS appId, a.label AS appName, a.wave,
mm.closureDate, mm.previsionalDate, mm.content,
ml.id, ml.label,
q.msgCount
FROM (
SELECT a.id
FROM `application` a
JOIN `customer` c ON c.id = a.customerId
ORDER BY c.name
) l
JOIN application a ON a.id = l.id
JOIN `customer` c ON c.id = a.customerId
LEFT JOIN (
SELECT applicationId, COUNT(*) AS msgCount
FROM application a
JOIN migrationMessage mm ON mm.applicationId = a.id
JOIN migrationLabel ml ON ml.id = mm.labelId
JOIN customer c ON c.id = a.customerId
' . $where . '
GROUP BY applicationId
LIMIT ' . $limit . ' OFFSET ' . $offset . '
) q ON q.applicationId = a.id
LEFT JOIN `migrationMessage`mm ON mm.applicationId = q.applicationId
LEFT JOIN `migrationLabel` ml ON ml.id = mm.labelId
JOIN `works` w ON w.applicationId = a.id '
. ($user->isOperator() ? ' AND w.roleId = ' . Role::Rri : ' AND w.employeeId = ' . $user->id)
. ' JOIN employee e ON e.id = w.employeeId'
. ($user->isJustUser() ? ' AND e.siteId = ' . $user->siteId : '')
. $where
. ' ORDER BY c.name, a.label, ml.ordering'
. 'LIMIT ' . $limit . ' OFFSET ' . $offset;
Almost everything works using this query, the problem happens when I got an application with several messages which get itself on 2 pages...
For example, "rowspan" (msgCount in the query) is 5 but the 3 last messages are on a different page than the first 2 and then the HTML display is completely erroneous. See HTML result
I wrote my query from scratch and come with a result that completely fit my needs.
For those who wondered, the key is the RIGHT JOIN'ed query :
<?php
$query = 'SELECT
c.id AS customerId, c.name AS customerName, c.X3Code, c.mfgProCode, c.as400Code, c.jdeCode,
a.id AS appId, a.label AS appName, a.wave,
mm.closureDate, mm.previsionalDate, mm.content,
ml.id, ml.label,
q0.msgCount
FROM application a
JOIN customer c ON c.id = a.customerId
LEFT JOIN migrationMessage mm ON mm.applicationId = a.id
LEFT JOIN migrationLabel ml ON ml.id = mm.labelId
JOIN `works` w ON w.applicationId = a.id '
. ($user->isOperator() ? ' AND w.roleId = ' . Role::Rri : ' AND w.employeeId = ' . $user->id)
. ' JOIN employee e ON e.id = w.employeeId'
. ($user->isJustUser() ? ' AND e.siteId = ' . $user->siteId : '')
. ' LEFT JOIN (
SELECT applicationId, COUNT(*) AS msgCount
FROM application a
JOIN migrationMessage mm ON mm.applicationId = a.id
JOIN migrationLabel ml ON ml.id = mm.labelId
JOIN customer c ON c.id = a.customerId
GROUP BY applicationId
) q0 ON q0.applicationId = a.id
RIGHT JOIN (
SELECT a.id
FROM application a
JOIN customer c ON c.id = a.customerId
LEFT JOIN migrationMessage mm ON mm.applicationId = a.id
LEFT JOIN migrationLabel ml ON ml.id = mm.labelId
' . $where . '
GROUP BY a.id
ORDER BY c.name, a.label, ml.ordering
LIMIT ' . $limit . ' OFFSET ' . $offset . '
) q1 ON q1.id = a.id
ORDER BY c.name, a.label, ml.ordering'

How to join with unknown condition like pivot in mysql

I have the following code to
public function get_posted_questions($data) {
include dirname(__FILE__) . "/database.php";
$user_db_name = $dbconfig[$data['college_id']]['database'];
if (isset($data['start'])) {
$start = $data['start'];
} else {
$start = 0;
}
if (isset($data['end']) and ! empty($data['end'])) {
$end = $data['end'];
} else {
$end = 30;
}
/*
* multiple college database funda here goes
*/
$max_college_id = "
SELECT DISTINCT college_id
FROM just_ask_question
WHERE
status = '1'
AND isDeleted = '0'
AND college_id !='0'
ORDER BY id DESC
LIMIT $start, $end
";
$max_college_id_run = mysql_query($max_college_id);
$question_data = array();
$question_query = "
SELECT
Q.id,
Q.title,
Q.description,
Q.user_id,
Q.college_id,
Q.datetime,
IFNULL(GROUP_CONCAT(DISTINCT T.name),'') AS tags,
IFNULL(CONCAT_WS(' ',U.firstName,U.lastName),'') AS user_name
IFNULL(U.image,'') AS image,
IFNULL(V.id,'') AS no_of_view,
IFNULL(Vote.upvote,'') AS up_vote,
IFNULL(answer.id,0) AS no_of_answer,
IFNULL(is_upvote,0) AS upvote_status,
category_name
FROM just_ask_question AS Q
LEFT JOIN just_ask_question_tag Qt
ON Qt.question_id = Q.id
LEFT JOIN just_ask_tag T
ON T.id = Qt.tag_id
LEFT JOIN just_ask_category
ON just_ask_category.id = Q.category_id
LEFT JOIN
(
SELECT COUNT(id) as id, question_id FROM just_ask_answer
) AS answer ON answer.question_id = Q.id
";
while ( $row = mysql_fetch_assoc($max_college_id_run) ) {
$user_db_name = $dbconfig[$row['college_id']]['database'];
$question_query .= "
CASE WHEN Q.college_id = '".$row['college_id']."'
THEN LEFT JOIN $user_db_name.users U ON U.id = Q.user_id
";
}
$question_query .= "
LEFT JOIN
(
SELECT count(id) AS id, question_id FROM just_ask_view
) AS V ON V.question_id = Q.id
LEFT JOIN
(
SELECT
COUNT(upvote) as upvote,
question_id
FROM just_ask_upvote_downvote
WHERE upvote = '1' AND is_question = '1'
GROUP BY question_id
) AS Vote
ON Vote.question_id = Q.id
LEFT JOIN
(
SELECT
IF(COUNT(id) > 0,1,0) AS is_upvote,
question_id
FROM just_ask_upvote_downvote
WHERE
upvote = '1'
AND college_id = '" . $data['college_id'] . "'
AND user_id = '" . $data['user_id'] . "'
AND is_question = '1'
) AS Is_Vote
ON Is_Vote.question_id = Q.id
WHERE
Q.status = '1'
AND Q.isDeleted = '0'
AND CASE
WHEN Q.visibility = 0 AND Q.college_id != 0 THEN Q.college_id = '" . $data['college_id'] . "'
ELSE true
END
";
if (!empty($data['search_text'])) {
$search_text = $data['search_text'];
$question_query .= " and (Q.title like '%$search_text%' or Q.description like '%$search_text%' or T.name like '%$search_text%')";
}
$question_query .= " group by Q.id order by Q.id desc limit $start,$end";
$question_query_run = mysql_query($question_query);
/* get weather question exist */
$check_num_rows = mysql_num_rows($question_query_run);
if ($check_num_rows > 0) {
while ($row = mysql_fetch_assoc($question_query_run)) {
if ($row['image'] != '') {
$row['thumbnail'] = USER_THUMBNAIL_URL . $row['image'];
$row['image'] = IMAGE_URL . $row['image'];
} else
$row['thumbnail'] = '';
$question_data[] = $row;
}
$status['statuscode'] = "1";
$status['statusmessage'] = "ok";
$status['question_data'] = $question_data;
$response['response'] = $status;
echo json_encode($response);
die();
}else {
$status['statuscode'] = "2";
$status['statusmessage'] = "There is no record found";
$response['response'] = $status;
echo json_encode($response);
die();
}
}
AND the db config file like which contain the database name information
like
$dbconfig['1'] = array(
'host' => 'localhost',
'user_name' => 'root',
'password' => 'test123*',
'database' => 'staging_myuniversity'
);
$dbconfig['2'] = array(
'host' => 'localhost',
'user_name' => 'root',
'password' => 'test123*',
'database' => 'staging_myuniversity_dias'
);
the database are reside on same server and with full permission.
the query generate like this
select Q.id,Q.title,Q.description,Q.user_id,Q.college_id,
Q.datetime,
ifnull(group_concat(distinct T.name),'') as tags,
ifnull(CONCAT_WS(' ',
U.firstName,U.lastName),'') as user_name,
ifnull(U.image,
'') as image,ifnull(V.id,'') as no_of_view,
ifnull(Vote.upvote,
'') as up_vote,
ifnull(answer.id,0) as no_of_answer,
ifnull(is_upvote,
0
) as upvote_status,
category_name
from just_ask_question as Q
left join just_ask_question_tag Qt on Qt.question_id = Q.id
left join just_ask_tag T on T.id = Qt.tag_id
left join just_ask_category on just_ask_category.id = Q.category_id
left join
( SELECT count(id) as id,question_id
from just_ask_answer
) as answer on answer.question_id = Q.id case when Q.college_id = '1' then
left join staging_myuniversity.users U on U.id = Q.user_id case when Q.college_id = '12'then
left join campify_solutions_mathura.users U on U.id = Q.user_id case when Q.college_id = '4' then
left join staging_myuniversity_nit_kkr.users U on U.id = Q.user_id case when Q.college_id = '2' then
left join staging_myuniversity_dias.users U on U.id = Q.user_id
left join
( SELECT count(id) as id,question_id
from just_ask_view
) as V on V.question_id = Q.id
left join
( SELECT count(upvote) as upvote,question_id
from just_ask_upvote_downvote
where upvote = '1'
and is_question = '1'
group by question_id
) as Vote on Vote.question_id = Q.id
left join
( SELECT if(count(id) > 0,1,0) as is_upvote,question_id
from just_ask_upvote_downvote
where upvote = '1'
and college_id = '1'
and user_id = '1'
and is_question = '1'
) as Is_Vote on Is_Vote.question_id = Q.id
where Q.status = '1'
and Q.isDeleted = '0'
and case when Q.visibility = 0
and Q.college_id != 0 then Q.college_id = '1' else true end
group by Q.id
order by Q.id desc
limit 0,30
I need to make the join on different databases based on college id
first i fetch the distinct college id from the question then make a loop
for college id and try to make join condition but i am getting error there is any suggestion and help
I won't help you with entire query (I'll just list some concerns), but this "case-join" might be solved with UNION ALL subquery this way:
INNER JOIN (
SELECT 1 AS college_id, id, firstName, lastName, image
FROM staging_myuniversity.users
UNION ALL
SELECT 12 AS college_id, id, firstName, lastName, image
FROM campify_solutions_mathura.users
UNION ALL
...
) AS U ON U.college_id = Q.college_id AND U.id = Q.user_id
When you join counters with subqueries like SELECT count(), some_id use GROUP_BY some_id as well. Otherwise you'll count all rows in table and join this result for one (first/random) id only (if it passes without error).
It looks like You use LEFT JOIN too much. It means attach to the result on left regardless if any row on the right matches (joins null columns then). It's slow, gives large data sets and results with nulls for most of these columns seems useless.
Join subquery with GROUP BY clause to get concatenated tags
If large portion of this data will be repeated across the rows (see: N+1 problem) splitting queries (and merging subsets to common "header") might be a good idea.
Build your query one step at a time. Test subqueries if you're not sure of its results. Start with resource consuming parts - it might get slow at some point and you'll end up splitting it anyway.
... case when Q.college_id = '1' then left join ...
does not make sense. What are you trying to do?
Performance tip: Don't use LEFT JOIN ( SELECT ... ) Instead of that and IFNULL(...), simply do
SELECT ...,
IFNULL(( SELECT ... ), '') AS upvotes,
As for pivoting, I must say simply, and strongly, that some tasks are better left to your application code, not SQL.

Laravel / MySQL console different results

I am using laravel and I have this method inside one model.
When I run the query in PhpMyAdmin or in the MySQL console I get the expected result.
When running it in Laravel, I always get the same row which doesn't even fulfill the "where" statements in the query. Same luck when using ORDER BY RAND()
public static function getPotentialMatch($id, $genderId = NULL)
{
$query = "SELECT
student.id AS id,
(SELECT COUNT(*) FROM matches AS m WHERE m.local_student_id = student.id) AS matches
FROM
users AS user
JOIN students AS student
ON user.id = student.user_id
LEFT OUTER JOIN languages_student AS ls
ON student.id = ls.student_id
LEFT OUTER JOIN languages AS language
ON ls.language_id = language.id
LEFT OUTER JOIN courses AS course
ON student.course_id = course.id
WHERE ls.language_id IN
(
SELECT ls.language_id AS languageId
FROM languages_student AS ls
JOIN students AS student1
ON ls.student_id = student1.id
JOIN users AS user1
ON student1.user_id = user1.id
WHERE student1.id = " . $id . "
)
AND student.course_id IN
(
SELECT courseR.id AS affinityId
FROM courses AS course
JOIN course_affinities
ON course.id = course_affinities.course1_id
OR course.id = course_affinities.course2_id
JOIN courses AS courseR
ON (courseR.id = course_affinities.course1_id
AND courseR.id <> course.id)
OR (courseR.id = course_affinities.course2_id
AND courseR.id <> course.id)
WHERE course.id IN
(
SELECT student.course_id
FROM students AS student
WHERE student.id = " . $id . "
)
)
AND student.incoming = 0
AND student.active = 1
AND (
SELECT COUNT(*)
FROM matches AS m
WHERE m.local_student_id = student.id
) < 3
";
if ($genderId != NULL) {
$query .= " AND student.gender_id = " . $genderId;
}
$query .= " ORDER BY (SELECT COUNT(*) FROM matches AS m WHERE m.local_student_id = student.id)
LIMIT 1";
return DB::select(DB::raw($query));
}
Any ideas?

SQL Query with multiple joins only returning first result on final join

So I have a prepared SQL query (below) that joins 4 tables together based on the user's ID. The final relationship however is a one to many (a user can have more than one skill) so I need all rows returned where the skill ID equals the user ID. At the moment only the first row that matches the user ID in the freelancer_skill table returns. How do I get all rows to return?
SELECT
u.user_id, u.firstname, u.lastname, u.email, u.bio, u.portfolio, u.location, u.time_joined, u.image_location,
f.freelancer_id, f.jobtitle, f.priceperhour,
ut.*,
ft.testimonial, ft.testimonial_source,
fs.skill, fs.skill_rating
FROM ((((users AS u
LEFT JOIN freelancers AS f
ON u.user_id = f.freelancer_id)
LEFT JOIN user_types AS ut
ON u.user_id = ut.user_type_id)
LEFT JOIN freelancer_testimonials AS ft
ON u.user_id = ft.testimonial_id)
LEFT JOIN freelancer_skills AS fs
ON u.user_id = fs.skill_id)
WHERE
u.confirmed = :confirmed
AND u.user_id = :userID
AND ut.user_type = :userType
AND u.granted_access = :grantedAccess
EDIT
Updated code with user type WHERE clause moved into join:
SELECT
u.firstname, u.lastname, u.email, u.bio, u.portfolio, u.location, u.time_joined, u.image_location,
f.jobtitle, f.priceperhour,
ut.user_type,
ft.testimonial, ft.testimonial_source,
fs.skill, fs.skill_rating
FROM ((((" . DB_NAME . ".users AS u
LEFT JOIN " . DB_NAME . ".freelancers AS f
ON u.user_id = f.freelancer_id)
LEFT JOIN " . DB_NAME . ".user_types AS ut
ON u.user_id = ut.user_type_id AND ut.user_type = :userType)
LEFT JOIN " . DB_NAME . ".freelancer_testimonials AS ft
ON u.user_id = ft.testimonial_id)
RIGHT JOIN " . DB_NAME . ".freelancer_skills AS fs
ON u.user_id = fs.skill_id)
WHERE
u.confirmed = :confirmed
AND u.user_id = :userID
AND u.granted_access = :grantedAccess
To fetch the results I am using fetch (below). I've tried using fetchALL which does return each skill a user holds but also returns duplicates of their data for each skill.
$results->execute();
$user = $results->fetch(PDO::FETCH_ASSOC);
return $user;
Here is an SQLFiddle: http://sqlfiddle.com/#!2/d7bb3/4
You have a filter in the wrong place. You are declaring a left join to user_types but are filtering on that table in the where clause. That makes it an implicit inner join. Move
and ut.user_type = :userType
to follow this:
LEFT JOIN user_types AS ut ON u.user_id = ut.user_type_id
That might be the cause of your problem, but even if it's not, you will still get unexpected results with that filter in the where clause.
change the order of the joins. Swap the Users table and freelancer_skills table.
-This apparently does not work.
Maybe try something like this instead. Not ideal if you want to list a hundred skills for one person.
SELECT
u.firstname,
u.lastname,
u.email, u.bio, u.portfolio, u.location, u.time_joined, u.image_location,
f.jobtitle, f.priceperhour,
ut.user_type,
ft.testimonial, ft.testimonial_source,
count(case when fs.skill = 'HTML5' then 1 end) as HTML5,
count(case when fs.skill = 'CSS3' then 1 end) as CSS3,
count(case when fs.skill = 'PHP' then 1 end) as PHP,
fs.skill_rating
FROM
(
(
(
(
users AS u
LEFT JOIN freelancers AS f ON
u.user_id = f.freelancer_id
)
LEFT JOIN user_types AS ut ON
u.user_id = ut.user_type_id AND
ut.user_type = 'developer'
)
LEFT JOIN freelancer_testimonials AS ft ON
u.user_id = ft.testimonial_id
)
RIGHT JOIN freelancer_skills AS fs ON
u.user_id = fs.skill_id
)
WHERE
u.confirmed = '1'
AND u.user_id = '3'
AND u.granted_access = '1'
group by
u.firstname,
u.lastname,
u.email, u.bio, u.portfolio,
u.location,
u.time_joined, u.image_location,
f.jobtitle, f.priceperhour,
ut.user_type,
fs.skill_rating

MySQL UNION query separation

I have this query that uses a UNION:
$this->db->query('
SELECT DISTINCT users.user_pic, users.id, users.username, contacts.accepted
FROM users
LEFT JOIN contacts ON users.id = contacts.user_1
WHERE contacts.user_2 = ' . $this->session->userdata('user_id') . '
UNION DISTINCT
SELECT DISTINCT users.user_pic, users.id, users.username, contacts.accepted
FROM users
LEFT JOIN contacts ON users.id = contacts.user_2
WHERE user_1 = ' . $this->session->userdata('user_id')
);
Is there a way to see if the sessions 'user_id' was encountered in contacts.user_1 or contacts.user_2? Maybe by changing the query, or if i can use some if statements in the view?
Thanks in advance
George
yes you can add another field to select , that you can recognise like (1,2)
$this->db->query('
SELECT 1 as table_id,DISTINCT users.user_pic, users.id, users.username, contacts.accepted
FROM users
LEFT JOIN contacts ON users.id = contacts.user_1
WHERE contacts.user_2 = ' . $this->session->userdata('user_id') . '
UNION DISTINCT
SELECT 2 as table_id,DISTINCT users.user_pic, users.id, users.username, contacts.accepted
FROM users
LEFT JOIN contacts ON users.id = contacts.user_2
WHERE user_1 = ' . $this->session->userdata('user_id')
);
I typically use:
$id = $this->session->userdata('user_id');
then use $id in the query. Reads cleaner and on MySQL at least, if the query fails, it lists the value of $id. Never tried calling the session in the query before.

Categories