I have a json field that stores a list of ids (not best practice here I know), I want to know if it's possible to use do operations on this JSON field and use them in the sql.
Below is a fictitious example of what I'm trying to achieve, is something like this doable?
CREATE TABLE user (
user_id INT,
user_name VARCHAR(50),
user_groups JSON
);
CREATE TABLE user_group (
user_group_id INT,
group_name VARCHAR(50)
);
INSERT INTO user_group (user_group_id, group_name) VALUES (1, 'Group A');
INSERT INTO user_group (user_group_id, group_name) VALUES (2, 'Group B');
INSERT INTO user_group (user_group_id, group_name) VALUES (3, 'Group C');
INSERT INTO user (user_id, user_name, user_groups) VALUES (101, 'John', '[1,3]');
With the above data I would like to fashion a query that gives me the results like this:
user_id | user_name | user_group_id | group_name|
-------------------------------------------------
101 | John | 1 | Group A
101 | John | 3 | Group C
Some psuedo style SQL I'm thinking is below, though I still have no clue if this is possible, or what JSON functions mysql offers I would use to achieve this
SELECT
u.user_id,
u.user_name,
g.user_group_id
g.group_name
FROM users u
LEFT JOIN user_group g on g.user_group_id in some_json_function?(u.user_groups)
Let me know if the question isn't clear.
With the help of Feras's comment and some fiddling:
SELECT
u.user_id,
u.user_name,
g.user_group_id,
g.group_name
FROM user u
LEFT JOIN user_group g on JSON_CONTAINS(u.user_groups, CAST(g.user_group_id as JSON), '$')
This appears to work, let me know if there's a better way.
Funny, I got to the opposite solution compared to Kyle's.
I wrote my query like this:
SELECT
u.user_id,
u.user_name,
g.user_group_id,
g.group_name
FROM user u
LEFT JOIN user_group g on JSON_UNQUOTE(JSON_EXTRACT(u.user_groups, '$')) = g.user_group_id;
It also works, and this solution doesn't need any transforming on the right side of the expression, this could provide a benefit in query optimizing in certain cases.
For arrays like ["1", "2", "3"] that values are in string type, JSON_SEARCH function is the way for your question:
SELECT
u.user_id,
u.user_name,
g.user_group_id
g.group_name
FROM users u
LEFT JOIN user_group g ON (JSON_SEARCH(u.user_groups, 'one', g.user_group_id))
JSON_CONTAINS function does not return true for integers as candidate parameter:
SELECT JSON_CONTAINS(CAST('["1", "2", "3"]' AS JSON), CAST(1 AS JSON), '$')
returns 0 (false). You need to change it to this:
SELECT JSON_CONTAINS(CAST('["1", "2", "3"]' AS JSON), CAST(CONCAT('"', 1, '"') AS JSON), '$')
But JSON_SEARCH can find the result:
SELECT JSON_SEARCH(CAST('["1", "2", "3"]' AS JSON), 'one', 1)
returns "$[0]" that means "true".
I have just tried the following and it worked in mysql 8.0.26:
-- index
ALTER TABLE user ADD KEY ( (CAST(user_groups -> '$' AS UNSIGNED ARRAY)) );
SELECT *
FROM user as a
left join user_group as t on t.user_group_id MEMBER OF (a.user_groups)
where t.group_name in ('Group A', 'Group C');
The query is based on the following select syntax:
SELECT * FROM t_json_arrays WHERE 3 MEMBER OF (c_array);
See examples on: https://saveriomiroddi.github.io/Storage-and-indexed-access-of-denormalized-columns-arrays-on-mysql-8.0-via-multi-valued-indexes/
Related
I am creating a Friend System in my Forum.
I am having a tough time trying to figure out how I would grab users and order by the mutual_friend count.
I am trying to build a page that shows a list of recommended friends.
Here is my structure of tables:
users table
-----
user_id | name |
friends table
-----
friend_id | from_id | to_id
Here is an example of what is happening.
Suppose there are total of A,B,C,D,E,F = 6 people in the site.
I am A, and B,C are my friends.
D and E in turn are friends of B.
D is also a friend of C but E is not a friend of C.
F is not a friend of anyone in the site.
Therefore from above data it looks like D and E are mutual friends of me (A). F is not a mutual friend of mine.
Since D is a friend of both B and C and E is friend of only B:
A and D has 2 mutual friends.
A and E has 1 mutual friend.
A and F has 0 mutual friend.
Now if I want to search (remember i am A) for people who are not my friends I can do something like:
$myfriends = "2,3"; //putting all my friends in a variable
SELECT * FROM users WHERE user_id NOT IN( $myfriends )
But it will yield in terms of user_id ASC .
How can I make it search in DESC order of mutual_friends. ?
I am A i.e user_id = 1
i.e. Person with more mutual friends comes first
Please can anyone show me how can I do this? I have been stuck here for a long while. I searched for lots of thing but can't figure it out.
This query will take the reciprocity of relationships into account, so it doesn't matter if the relationship goes "From A to B" or "From B to A", it will still return the expected result. So given tables like this:
CREATE TABLE people
(`id` int, `name` varchar(1))
;
INSERT INTO people
(`id`, `name`)
VALUES
(1, 'A'),
(2, 'B'),
(3, 'C'),
(4, 'D'),
(5, 'E'),
(6, 'F')
;
CREATE TABLE friends
(`id` int, `personId1` int, `personId2` int)
;
INSERT INTO friends
(`id`, `personId1`, `personId2`)
VALUES
(1, 1, 2),
(2, 3, 1),
(3, 2, 4),
(4, 5, 2),
(5, 3, 4)
;
I believe this is set up as you described: A and B are friends, A and C are friends (notice the inverted relationship), B and D are friends, E and B are friends (another inverted relationship), and C and D are friends.
Assume the id of the person you want is in #personId:
SELECT StrangerId, COUNT(MutualFriendId) AS TotalMutualFriends
FROM
(SELECT
CASE WHEN f.personId2 = mf.friendId THEN f.personId1 ELSE f.personId2 END AS StrangerId,
CASE WHEN f.personId1 = mf.friendId THEN f.personId1 ELSE f.personId2 END AS MutualFriendId
FROM
(SELECT
CASE
WHEN personId1 = #personId THEN personId2
ELSE personId1
END AS friendId
FROM friends
WHERE personId1 = #personId OR personId2 = #personId) AS mf
INNER JOIN friends f
ON (personId1 != #personId AND personId2 = mf.friendId)
OR (personId1 = mf.friendId AND personId2 != #personId)
) AS totals
GROUP BY StrangerId
ORDER BY TotalMutualFriends DESC;
Results for #personId = 1 are:
StrangerId TotalMutualFriends
4 2
5 1
And here is a SQLFiddle to demonstrate (I couldn't get it to allow me to set up a variable, so there is a 1 in its place).
Something like this, perhaps:
Select user_id, friends.to_id, count(friend_of_friend.to_id)
from users left outer join
friends on users.user_id = friends.from_id left outer join
users as friend_user on friends.to_id = friend_user.user_id left outer join
friends as friend_of_friend on friend_user.user_id = friend_of_friend.from_id and friend_of_friend.to_id in (select to_id from friends where from_id = users.user_id)
Group by USER_ID, friends.to_id
Order by 3
Edit for clarity:
The logic of this query depends on joining the same table multiple times. The first two joins are pretty straight-forward, we are starting with a table of users, then joining in the friends table that links each user with all their friends. But then we join in the user table again, but this time using the "to" column - we are getting the user info for each friend. Since we can't have the same table name twice in a query, we give it an alias of "friend_user". Then we join the friends table again, based on the id in the friend_user table - this gives us all the friends of each original user's friends. Then we limit the friends of friends that we get back using the "Friend_of_friend.to_id in ..." to compare the friends of friends to a list of all of the original user's friends, which we bring in by a subquery - the section just after the "in" that is enclosed in parenthesis.
I know this is quite complicated, but I sincerely hope someone will check this out.
I made short version (to better understand the problem) and full version (with original SQL)
Short version:
[TABLE A] [TABLE B]
|1|a|b| |1|x
|2|c|d| |1|y
|3| | | |2|z
|5| | | |2|v
|4|w
How can I make MySQL query to get rows like that:
1|a|b|x|y
2|c|d|z|v
2 columns from A and 2 rows from B as columns, only with keys 1 and 2, no empty results
Subquery?
Full version:
I tried to get from Prestashop db in one row:
product id
ean13 code
upc code
feature with id 24
feature with id 25
It's easy to get id_product, ean13 and upc, as it's one row in ps_product table. To get features I used subqueries (JOIN didn't work out).
So, I selected id_product, ean13, upc, (subquery1) as code1, (subquery2) as code2.
Then I needed to throw out empty rows. But couldn't just put code1 or code2 in WHERE.
To make it work I had to put everything in subquery.
This code WORKS, but it is terribly ugly and I bet this should be done differently.
How can I make it BETTER?
SELECT * FROM(
SELECT
p.id_product as idp, p.ean13 as ean13, p.upc as upc, (
SELECT
fvl.value
FROM
`ps_feature_product` fp
LEFT JOIN
`ps_feature_value_lang` fvl ON (fp.id_feature_value = fvl.id_feature_value)
WHERE fp.id_feature = 24 AND fp.id_product = idp
) AS code1, (
SELECT
fvl.value
FROM
`ps_feature_product` fp
LEFT JOIN
`ps_feature_value_lang` fvl ON (fp.id_feature_value = fvl.id_feature_value)
WHERE fp.id_feature = 25 AND fp.id_product = idp
) AS code2,
m.name
FROM
`ps_product` p
LEFT JOIN
`ps_manufacturer` m ON (p.id_manufacturer = m.id_manufacturer)
) mainq
WHERE
ean13 != '' OR upc != '' OR code1 IS NOT NULL OR code2 IS NOT NULL
create table tablea
( id int,
col1 varchar(1),
col2 varchar(1));
create table tableb
( id int,
feature int,
cola varchar(1));
insert into tablea (id, col1, col2)
select 1,'a','b' union
select 2,'c','d' union
select 3,null,null union
select 5,null,null;
insert into tableb (id, feature, cola)
select 1,24,'x' union
select 1,25,'y' union
select 2,24,'z' union
select 2,25,'v' union
select 4,24,'w';
select a.id, a.col1, a.col2, b1.cola b1a, b2.cola b2a
from tablea a
inner join tableb b1 on (b1.id = a.id and b1.feature = 24)
inner join tableb b2 on (b2.id = a.id and b2.feature = 25);
SQLFiddle here.
What you want to do is called a Pivot Query. MySQL has no native support for pivot queries, though other RDBMSen do.
You can simulate a pivot query with derived columns, but you must specify each derived column. That is, it is impossible in MySQL itself to have the number of columns match rows of another table. This has to be known ahead of time.
It would be much easier to query the results as rows and then use PHP to do the aggregation into columns. For example:
while ($row = $result->fetch()) {
if (!isset($table[$row->id])) {
$table[$row->id] = array();
}
$table[$row->id][] = $row->feature;
This is not a simple question because it's not a standard query, by the way if you can make use of views you can do the following procedure. Assuming you're starting from this tables:
CREATE TABLE `A` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`firstA` char(1) NOT NULL DEFAULT '',
`secondA` char(1) NOT NULL DEFAULT '',
PRIMARY KEY (`id`)
);
CREATE TABLE `B` (
`id` int(11) unsigned NOT NULL,
`firstB` char(1) NOT NULL DEFAULT ''
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `A` (`id`, `firstA`, `secondA`)
VALUES (1, 'a', 'b'), (2, 'c', 'd');
INSERT INTO `B` (`id`, `firstB`)
VALUES (1, 'x'), (1, 'y'), (2, 'z'), (2, 'v'), (4, 'w');
First create a view that joins the two tables:
create or replace view C_join as
select A.firstA, A.secondA, B.firstB
from A
join B on B.id=A.id;
Create the view that groups the rows in table B:
create or replace view d_group_concat as
select firstA, secondA, group_concat(firstB) groupconcat
from c_join
group by firstA, secondA
Create the view that does what you need:
create or replace view e_result as
select firstA, secondA, SUBSTRING_INDEX(groupconcat,',',1) firstB, SUBSTRING_INDEX(SUBSTRING_INDEX(groupconcat,',',2),',',-1) secondB
from d_group_concat
And that's all. Hope this helps you.
If you can't create views, this could be the query:
select firstA, secondA, SUBSTRING_INDEX(groupconcat,',',1) firstB, SUBSTRING_INDEX(SUBSTRING_INDEX(groupconcat,',',2),',',-1) secondB
from (
select firstA, secondA, group_concat(firstB) groupconcat
from (
select A.firstA, A.secondA, B.firstB
from A
join B on B.id=A.id
) c_join
group by firstA, secondA
) d_group_concat
Big thanks to everyone for the answers. James's answer was first, simplest and works perfectly in my case. The query runs several times faster than mine, with subqueries. Thanks, James!
Just a few words why I needed that:
It's a part of integration component for Prestashop and wholesale exchange platform. There are 4 product code systems that wholesalers use on the platform (ean13, upc and 2 other systems). Those 2 other product codes are added as product feature in Prestashop. There are thousands of products on the shop and hundreds of thousands of products on the platform. Which is why speed is crucial.
Here is the code for full version of my question. Maybe someone will find this helpful.
Query to get Prestashop product codes and certain features in one row:
SELECT
p.id_product, p.ean13, p.upc, fvl1.value as code1, fvl2.value as code2
FROM `ps_product` p
LEFT JOIN
`ps_feature_product` fp1 ON (p.id_product = fp1.id_product and fp1.id_feature = 24)
LEFT JOIN
`ps_feature_value_lang` fvl1 ON (fvl1.id_feature_value = fp1.id_feature_value)
LEFT JOIN
`ps_feature_product` fp2 ON (p.id_product = fp2.id_product and fp2.id_feature = 25)
LEFT JOIN
`ps_feature_value_lang` fvl2 ON (fvl2.id_feature_value = fp2.id_feature_value)
WHERE
ean13 != '' OR upc != '' OR fvl1.value IS NOT NULL OR fvl2.value IS NOT NULL;
I have a row with some values hyphen-delimited:
table: live_customers
row: areas
id | areas
1 | 10-20-30
2 | 40-50-60
...
Using this...
LEFT JOIN $table5 AS table5 ON live.areas REGEXP CONCAT('(^|-) ?',table5.id,' ?($|-)')
My results looks like:
(tab id:1) area: 10
(tab id:1) area: 20
...
(tab id:2) area: 40
...
But i expect:
(tab id:1) area: 10,20,30
(tab id:2) area: 40,50,60
How could i solve that?
EDIT:
The full query looks like:
SELECT live.*,
live.id AS lid,
table1.id, table1.value AS tn_val,
table2.id, table2.value AS tp_val,
table3.id, table3.value AS ht_val,
table5.id, table5.value AS ar_val
FROM $dblist AS live
LEFT JOIN $table1 AS table1 ON live.town = table1.id
LEFT JOIN $table2 AS table2 ON live.htype = table2.id
LEFT JOIN $table3 AS table3 ON live.ht = table3.id
LEFT JOIN $table5 AS table5 ON live.areas REGEXP CONCAT('(^|-) ?',table5.id,' ?($|-)')
ORDER BY live.id ASC
PHP echoes:
...
if ($post['areas']){ // Debugging areas stuff
echo '<strong>'.$_areas.': (ar_val)</strong> '.$post['ar_val'].'<p>';
echo '<strong>'.$_areas.': (areas)</strong> '.$post['areas'].'<p>';
}
...
EDIT2:
It's quite hard for me to explain my issue in English, but i'm trying the best i can :)
in the table "live_customers" i does have this:
id | areas
1 | 10-20-30
2 | 40-50-60
...
in the table "areas" (that is a completely different table):
id | value
38 | Zone1
39 | Zone2
40 | Zone3
...
In the SQL query you see just tables variables because i previousvly declared them at the top of page:
$table5 = 'areas';
$dblist = 'live_customers';
etc..
Solution
Thanks anyone for their answers and for let me know "GROUP_CONCAT".
Here is my solution:
SELECT live.*,
live.id AS lid,
table1.id, table1.value AS tn_val,
table2.id, table2.value AS tp_val,
table3.id, table3.value AS ht_val,
table5.id, GROUP_CONCAT(table5.value) AS ar_val
FROM $dblist AS live
LEFT JOIN $table1 AS table1 ON live.town = table1.id
LEFT JOIN $table2 AS table2 ON live.htype = table2.id
LEFT JOIN $table3 AS table3 ON live.ht = table3.id
LEFT JOIN $table5 AS table5 ON FIND_IN_SET(table5.id, REPLACE(live.areas, '-', ','))
GROUP BY live.id
Result is what i expected ^^
Take it together with GROUP_CONCAT()
First thing to say is that your schema violates First Normal Form (1NF) in that the column areas is not atomic. You should not be putting 3 different values in one column.
Next you say you have a table called live_customers with a row called areas. This is nonsense. Rows do not have names, columns do. You show a bit of table with 2 columns id and areas. What table is this?
Next in the query there is no mention of a table called live_customers.
Next, if there is a column called areas in the table with the alias of live, then the output should contain that column since you are selecting live.*. That being the case, your results cannot be what you showed us, since it would contain a results column with data like 10-20-30
Finally those cannot be the results of the posted query since I can see a results column of lid specified.
If you would care to take some time over ensuring that the questionyou post makes sense, then you might get a reasonable answer.
I'm very new with SQL and need assistance on how I can accomplish this task using the correct query.
I have 2 tables that I need to use. Table "TB1" has:
id Name
1 bob
2 blow
3 joe
table "TB2" has:
compid property
1 bob
2 blow
I am trying to get which compid is missing in "TB2" and insert it from "TB1"
the query I am doing is:
SELECT id, name from TB1, TB2 where id <> compid
what I get is 2 ouputs of Id 1, and 2, and 3 outputs from id 3. by using php:
for($i=0;$i <= mysql_num_rows($comp)-1; $i++)
{
echo mysql_result($comp, $i, 0)."<br>";
}
and I expected the ouput 3 but instead got this:
1
1
2
2
3
3
3
I understand its comparing all the rows within the table but is there a way to achieve what I am looking for?
Thanks for your time.
You are performing an implicit Cartesian JOIN which results in every row against every other row. You need to specify what attribute JOINs the two tables.
Using implicit syntax (not recommended):
SELECT id, name
FROM TB1, TB2
WHERE id <> compid
AND TB1.Name = TB2.property <-- Column join
Using explicit syntax:
SELECT id, name
FROM TB1
JOIN TB2
ON TB2.property = TB1.Name <-- Column join
WHERE id <> compid
To accomplish your goal you would need something along the lines of:
SELECT TB1.id, TB1.name
FROM TB1
LEFT JOIN TB2
ON TB2.property = TB1.Name
WHERE TB2.compid IS NULL
See it in action
It's best practice to always alias the columns you select to prevent ambiguity.
To select it you can do:
SELECT *
FROM TB1
WHERE id NOT IN (
SELECT compid
FROM TB2
);
I have a hard nut to crack with joing 3 tables.
I have a newsletter_items, newsletter_fields and newsletter_mailgroups which I want to be joined to get a list of newsletters.
The newsletter_items contains the fields:
letter_id, letter_date, receivers, template, status
That can look like
1, 1234567899, 1,2 (comma separated), standard.html, 1
newsletter_fields contains the fields:
field_uid, field_name, field_content, field_letter_uid
That can look like
1, letter_headline, A great headline, 1
where field_letter_uid is the newsletter for which the field belongs to.
and newsletter_mailgroups contains the fields:
mailgroup_id, mailgroup_name, number_of_members
That can look like
1, Group1, 233
2, Group2, 124
3, Group3, 54
What I want is to combine these 3 tables to that I can get a list of all the newsletter like this:
Letter date | Letter headline | Receivers | Status
2008-01-01 12:00:00 | A great headline | Group1, Group 2 | 1
So in short I want my SQL query to join the 3 tables and in that process select the receivers from the mailgroup table and display them comma separated like Group1, Group 2
This what I got now
SELECT A.*, B.* FROM newsletter_items A, newsletter_fields B, WHERE B.field_letter_uid = A.letter_id AND field_name = 'letter_headline' AND A.template = '". $template ."';
But I can't seem to figure out how to get the mailgroups into that.
I recommend that you make your joins explicit.
It makes it easier to debug your query and to change inner with left joins.
There is absolutely never a good reason to use SQL '89 implicit join syntax.
SELECT ni.*
, nf.*
, group_concat(nm.mailgroup_name) as mailgroups
FROM newsletter_items ni
INNER JOIN newsletter_fields nf
ON (nf.field_letter_uid = ni.letter_id)
INNER JOIN newsletter_mailgroups nm
ON (find_in_set(nm.mailgroup_id, ni.receivers))
WHERE
nf.field_name = 'letter_headline'
ni.template = '". $template ."'
GROUP BY ni.letter_id;
Regarding your database design.
I recommend you normalize your database, that means that you move the comma separated fields into a different table.
So you make a table receivers
Receivers
----------
id integer auto_increment primary key
letter_id integer not null foreign key references newsletter_items(letter_id)
value integer not null
You then remove the field receiver from the table newsletter_items
Your query then changes into:
SELECT ni.*
, group_concat(r.value) as receivers
, nf.*
, group_concat(nm.mailgroup_name) as mailgroups
FROM newsletter_items ni
INNER JOIN newsletter_fields nf
ON (nf.field_letter_uid = ni.letter_id)
INNER JOIN newsletter_mailgroups nm
ON (find_in_set(nm.mailgroup_id, ni.receivers))
LEFT JOIN receiver r ON (r.letter_id = ni.letter_id)
WHERE
nf.field_name = 'letter_headline'
ni.template = '". $template ."'
GROUP BY ni.letter_id;
This change should also speed up your query significantly.
If it's allowed, why don't you create a new table called newsletter_item_receivers where you could store letter_id, receiver_id fields?
Having comma separated values in a field like this usually means you're missing a table :)
Edit:
By using CSV, you are making your life miserable when you want to retrieve an answer to "give me all newsletters that receiver_id=5 receives" :)
Here's a good answer to a similar question on SO: Comma separated values in a database field
Edit2:
If I understand your table relationships correctly then it would be something like this:
SELECT
a.letter_date,
b.receiver_id,
a.status
FROM newsletter_items_receivers b
LEFT OUTER JOIN newsletter_items a ON (a.letter_id = b.letter_id)
LEFT OUTER JOIN newsletter_mailgroups m ON (m.mailgroup_id = b.receiver_id)
NOTE! This query WILL NOT return a newsletter when there are no receivers of that newsletter.
If you need that functionality you can try something like this:
SELECT
x.letter_date,
y.mailgroup_name,
x.status
FROM (
SELECT
a.letter_date,
b.receiver_id,
a.status
FROM newsletter_items a
LEFT OUTER JOIN newsletter_items_rec b ON (b.letter_id = a.letter_id)) x
LEFT OUTER JOIN newsletter_mailgroups y ON (y.mailgroup_id = x.receiver_id)
I don't have access to SQL right now so I might have made some syntax errors (hopefully not logical ones :)).
As for why we are doing it like this, as #Konerak pointed out, you'd be well advised to read up on database normalization and why it's important.
You can start with this article from about.com, just glanced over it seems an OK read
http://databases.about.com/od/specificproducts/a/normalization.htm
Also, it would be good if you'd keep fields names the same across multiple tables.
For example you have letter_id in newsletter_items, but you have field_letter_uid in newsletter_fields. Just a thought :)
Try to use
SELECT A.*, B.*, group_concat(C.mailgroup_name SEPARATOR ',')
FROM newsletter_items A, newsletter_fields B, newsletter_mailgroups C
WHERE B.field_letter_uid = A.letter_id
AND field_name = 'letter_headline'
AND A.template = '". $template ."'
and find_in_set(c.mailgroup_id, A.receivers)
group by A.letter_id;