MySQL query to retrieve related data from another table - php

I'm not too experienced with SQL queries (most times I used query builder from frameworks, like CodeIgniter and Laravel). Now, I need to retrieve data from a relational DB which have two tables: one for entity entries and other for complemental data of entities. For example, see below:
tbl_posts
id
name
slug
1
Lorem ipsum
lorem-ipsum
2
Testing post
testing-post
3
Hello world
hello-world
tbl_posts_meta
id
post_id
key
value
1
1
first_name
John
2
1
last_name
Doe
3
1
rating
5
4
2
parent
1
5
2
rating
3
6
3
rating
4
In this example, I need to retrieve an array of objects in this format:
[
{
id: 1,
name: "Lorem ipsum",
slug: "lorem-ipsum",
data: {
first_name: "John",
last_name: "Doe",
rating: 5,
}
},
{
id: 2,
name: "Testing post",
slug: "testing-post",
data: {
parent: 1,
rating: 3,
}
},
{
id: 3,
name: "Hello World",
slug: "hello-world",
data: {
rating: 4,
}
},
]
I've tried using subquery to handle this, but I'm reveiving Cardinality violation: 1241 Operand should contain 1 column(s) error. My query looks like this:
SELECT *,(SELECT * FROM tbl_posts_meta WHERE "post_id" = "tbl_posts"."id") as data FROM tbl_posts;
I've already tried using JOIN, but the result looks more away from expected (I have one "key" property and one "value" property in result, containing the last found entry in tbl_posts_meta).
SELECT * FROM tbl_posts INNER JOIN tbl_posts_meta ON tbl_posts_meta.post_id = tbl_posts.id;
Is there someway I can retrieve desired results with one query? I don't want to do this by applogic (like first retrieving data from tbl_posts and appending another query on "data" property, that returns all data from tbl_posts_meta), since this way may cause database overload.
Thanks!

The code is not shown right in the comment so here it is for you to check, sorry I can't contribute more to the theoretical debate:
select
tp.id,
tp.name,
tp.slug,
(select group_concat(`key`, "=",value separator ';' ) from tbl_posts_meta where post_id= tp.id )as datas
from tbl_posts tp

You can follow these simple steps to get to your needed json:
join the two tables to have centralized data
generate a tbl_posts_meta JSON-like string by aggregating on your columns of interest (tbl_posts.id, tbl_posts.name, tbl_posts.slug) using a GROUP_CONCAT aggregation function. Since your keys can either be integers or strings, you need to do a quick check using the CAST(<col> AS UNSIGNED), which will return 0 if the string is not a number.
use the JSON_OBJECT function to transform each row into a JSON
use the JSON_ARRAYAGG function to merge all jsons into a single json.
WITH cte AS(
SELECT p.id,
p.name,
p.slug,
CONCAT('{',
GROUP_CONCAT(
CONCAT('"',
m.key_,
'": ',
IF(CAST(m.value_ AS UNSIGNED)=0,
CONCAT('"', m.value_, '"'),
CAST(m.value_ AS UNSIGNED)) )
ORDER BY m.id ),
'}') AS data_
FROM tbl_posts p
INNER JOIN tbl_posts_meta m
ON p.id = m.post_id
GROUP BY p.id,
p.name,
p.slug
)
SELECT JSON_ARRAYAGG(
JSON_OBJECT("id", id,
"name", name,
"slug", slug,
"data", CAST(data_ AS JSON)))
FROM cte
Check the demo here.

After some research, testing and reading the queries you posted here, I've finally reached the result I needed. The final query is structured this way:
SELECT *, (
SELECT JSON_ARRAYAGG(
JSON_OBJECT(
key, value
)
) FROM tbl_posts_meta WHERE post_id = post.id
) AS data
FROM tbl_posts;
Translating it to Laravel's Eloquent query builder, and using PHP's serialize/unserialize functions, the end result looks like this:
$posts = Post::query()
->select("*")
->selectRaw('(SELECT JSON_ARRAYAGG(JSON_OBJECT(key, value)) FROM tbl_posts_meta WHERE post_id = posts.id) as data');
return $posts->get()->map(function($post){
$meta = [];
foreach($post->data as $dataItem){
foreach($dataItem as $key => $value){
$meta[$key] = unserialize($value);
}
}
$post->data = $meta;
return $post;
});
Thank you for your support and attention! :)

Related

Querying a table with an array of json objects in RedbeanPHP

I'm having trouble working in Redbean PHP with querying a table with an array of json objects in a single field, and producing a report on it.
I want to get a report with subtotals of all staff with notes by the category. I know this should be easy/obvious but I'm just not getting it properly.
I have a database, with:
table clients
with columns:(int) client_id, (string) client_name, (array of json) notes
notes is an array of json with
(int) note_id, (int) note_category_id, (int) staff_id, (string) description, (memo) content, (date) note_date
table staff with columns (int) sid, (string) sname
table categories with columns (int) cat_id, (string) cat_name
So in pseudocode (since I'm still trying to figure this all out)
I need to run a query like: (with parameters in brackets)
R::getAll('Select * from Join (staff, categories, clients)
On (staff.sid=clients.services.staff_id, categories.cat_id=clients.services.note_category_id)
Where (clients.services.note_date Between [startdate] and [enddate],
categories.cat_name IN [chosencateg], staff.sname IN [pickednames])
Orderby sname Asc, cat_name Asc, note_date Desc ');
report output format:
Filters used: [picked filter choices if any]
-----------
[sname]
-- note category: [cat_name] 1
[note_date] 1 [description] 1 [content] 1
[note_date] 2 [description] 2 [content] 2
note category 1 subtotal
-- note category: [cat_name] 2
[note_date] 3 [description] 3 [content] 3
[note_date] 4 [description] 4 [content] 4
note category 2 subtotal
staff subtotal
[sname] 2 ...
I'm asking a fairly generic one because I'll have to work with a number of similar tables, and maybe seeing a query template will help my understanding.
Thanks for any help.
redbean is fantastic and - getAll is just scratching the surface and truly isn't working with redbean at all really... Read up on it here:
Here's a query template to get you started:
Query Template:
1)
R::getAll('Select * from Join (staff, categories, clients)
On (staff.sid=clients.services.staff_id, categories.cat_id=clients.services.note_category_id)
Where (clients.services.note_date Between :startdate and :enddate,
categories.cat_name IN (:chosencateg), staff.sname IN (:pickednames))
Orderby sname Asc, cat_name Asc, note_date Desc ');
You could also simply use:
2)
R::getAll('Select * from Join (staff, categories, clients)
On (staff.sid=clients.services.staff_id, categories.cat_id=clients.services.note_category_id)
Where (clients.services.note_date Between ? and ?,
categories.cat_name IN (?), staff.sname IN (?))
Orderby sname Asc, cat_name Asc, note_date Desc ');
The only difference is that query template 1 uses named parameters (so it's going to look to the array of params that you pass it to contain an associative array with parameters named in the same way as they are in the query). While template 2 requires simply an array of parameters with the indexes lined up with the order in which the ? marks appear in your query.
Anyway... the query should return an associative array of your columns representing rows. a var_dump would look something like this:
Array
(
[client_id] => 1,
[client_name] => "joe",
[noes] => "[
{note_id=1
,note_category_id=1
,staff_id=1
,description=blah blah
,content=blah blah blah blah
,content=some content for this note
,note_date=12/06/2018
}
]"
[sid] => 100,
[sname] => "some staff name"
[cat_id] => 100
[cat_name] => "some category name"
)
Notice how the notes field has just come out as a string (I know the json above is not properly formed json, I'm just trying to show an example).
I assume that what you want is to have that string converted into an array so you can work with it as if it were data and not a string. So the below should get you started with that:
Once you have it out of the database you should be able to access it like this:
$result = R::getAll($query,
['startdate'=> $mystartDate
,'enddate' => $myEndDate
,'chosencateg'=>$myChosenCategory
,'pickednames'=>$myPickedNames
]);
// this would output the json string to your screen
echo $result['notes'];
but it seems like you want to work with the json as if it were part of your data - so... you would need to decode it first.
// decode my notes field:
foreach(array_key($result) as $key) {
/* you are working with a multidimensional array in this loop
, use $key to access the row index. Each row index
will contain named column indexes that are column names from the database
*/
$result[$key]['decoded_notes'] = json_decode($result[$key]['notes'],true);
}
// I now have a new column in each row index, containing 'notes'
as another associative array
// the statement below now results in an array to string conversion error:
echo $result[someIndexNumber]['decoded_notes'];
So, I decided I would want this in MySQL (5.7) so as to use its capabilities. To do this I used string manipulation. MySQL 8 adds json_table functions which would have been nice to have.
I converted each array of JSON notes into lines of 'INSERT INTO temptable' to convert the array list into temptable rows,
one JSON object per row, adding the client_id to each object, then
EXECUTEing those statements.
SET #allnotes = (
SELECT json_arrayagg(REPLACE(`notes`,'{', CONCAT('{"id_client": ', id_client, ', '))) /* add id_client to each note object */
FROM clients
WHERE `notes` != '' AND `notes` != '[]' ); /* no empty note cases */
SET #allnotes = REPLACE(REPLACE(#allnotes ,'"[',''),']"','' ); /* flatten out outer array of each note */
SET #allnotes = REPLACE(REPLACE(#allnotes ,'{','("{'),'}','}")' ); /* INSERT INTO string formatting for the objects */
DROP TEMPORARY TABLE IF EXISTS jsonTemporary;
CREATE TEMPORARY TABLE IF NOT EXISTS jsonTemporary (anote json);
SET #allnotes = REPLACE(REPLACE(#allnotes,'[','INSERT INTO jsonTemporary (anote) VALUES '),']',';');
PREPARE astatement FROM #allnotes;
EXECUTE astatement;
/* totals */
SELECT concat(staff.last_name,", ",staff.first_name) AS sname,
categories.name AS cat_name,
count(anote->'$.id_client') AS cat_total,
FROM jsonTemporary
JOIN categories ON cast(anote->'$.note_category_id' as unsigned)=categories.id
JOIN clients ON clients.id_client=anote->'$.id_client'
JOIN staff ON staff.id=anote->'$.staff_id'
WHERE anote->'$.note_date' >= "2018-10-01" AND anote->'$.note_date' <= "2018-12-31"
GROUP BY sname, cat_name;
/* all notes */
SELECT concat(staff.last_name,", ",staff.first_name) AS sname,
categories.name AS cat_name,
anote->'$.note_date' AS n_date,
anote->'$.description' AS description,
anote->'$.content' AS content,
FROM jsonTemporary
JOIN categories ON cast(anote->'$.note_category_id' as unsigned)=categories.id
JOIN clients ON clients.id_client=anote->'$.id_client'
JOIN staff ON staff.id=anote->'$.staff_id'
WHERE anote->'$.note_date' >= "2018-10-01" AND anote->'$.note_date' <= "2018-12-31"
GROUP BY sname, cat_name;
DROP TEMPORARY TABLE IF EXISTS jsonTemporary;

PHP multi array (JSON) from SQL query on three tables

I hope someone can help me with this specific problem.
I'm new to PHP and MySQL but I'm trying as best as I can. Also, I know that probably similar questions have been asked, but unfortunately I tried every angle I could think of to modify those tutorials/answers to suit my needs, but unfortunately I've failed miserably..
So, here's my problem: I have 3 MySQL tables(contacts, phone numbers, and phone types) for simple phonebook, structured like this:
|ID |name | |ID |cID |tID |number| |ID |type |
-----------------------------------------------------------------------
1 John 1 1 2 123456 1 Home
2 Mary 2 2 1 234567 2 Mobile
3 Sue 3 1 3 213234 3 Work
4 2 2 444321
5 3 2 555543
The first table contains contact names, second holds the number details, and third is "static" table for referencing phone number types.
Now, I'm creating an api for simple crud app in PHP and I'm stuck at creating the array that will give me the result structured as I envisioned:
[
{"ContactID": 1,
"ContactName": "John",
"PhoneNumbers": {
"PhoneNumberID": 1,
"PhoneType": 2,
"PhoneNumber": 123456
}
},
{...},
{...}
]
The query I'm using is:
SELECT contacts.*, pt.type, pn.number, pn.id
FROM contacts
LEFT JOIN phonenumbers pn ON c.ID = pn.cID
LEFT JOIN phonetypes pt ON pn.tID = pt.ID
And now I'm stuck at PHP syntax for creating the array mentioned above. Can you help point me in the right direction please?
Also, as this is a small assignment demonstrating the CRUD functions, I'm not sure about my database, is the three table structure OK? Do I need to change it to something else?
Thanks in advance! Cheers!
If all the tables have ID columns, you need to use an alias in the SQL to distinguish phonenumbers.id from contacts.id. So change the query to:
SELECT contacts.*, pt.type, pn.number, pn.id AS phoneid
FROM contacts
LEFT JOIN phonenumbers pn ON c.ID = pn.cID
LEFT JOIN phonetypes pt ON pn.tID = pt.ID
Here's the code assuming you're using PDO; mysqli will be similar.
$result = array();
while ($row = $stmt->fetch(PDO::FETCH_ASSOC) {
if (!isset($result[$row['ContactID']]) {
// If this is the first row for this contact, create an entry in the results
$result[$row['ContactID']] = array(
'ContactID' => $row['ID'],
'ContactName' => $row['name'],
'PhoneNumbers' => array()
);
}
// Add this phone number to the `PhoneNumbers` array
$result[$row['ContactID']]['PhoneNumbers'][] = array(
'PhoneNumberID' => $row['phoneid'],
'PhoneType' => $row['type'],
'PhoneNumber' => $row['number']
);
}
$result = array_values($result); // Convert from associative to indexed array
// Convert to JSON
echo json_encode($result);
The resulting JSON will look like this:
[
{"ContactID": 1,
"ContactName": "John",
"PhoneNumbers": [
{
"PhoneNumberID": 1,
"PhoneType": "Mobile",
"PhoneNumber": "123456"
},
{
"PhoneNumberID": 3,
"PhoneType": "Work",
"PhoneNumber": "213234"
}
},
{...},
{...}
]
PhoneNumbers is an array of all the phone numbers, and PhoneType is the type name, not its ID. If you only want the type ID, you don't need to join with phonetypes.

MySQL query select rows from A and combine multiple rows from B

After looking into this, apparently it's a complex thing to do? It doesn't look like it's a pivot query, or should it be?? Anyway, two tables like this:
articles:
id col2 col3
1 .... ....
2 .... ....
3 .... ....
articleImages:
id imgFile artRef
1 img1.jpg 1
2 img2.jpg 1
3 img3.jpg 2
4 ....... 3
5 ....... 3
6 ....... 3
I want to select everything form "articles" and combine the "articleImages" into the row so it will look something like this:
[{
"id":"1",
"col2":"...",
"col3":"...",
"imgFiles":{
"img1.jpg",
"img2.jpg"
}
},{
"id":"2",
"col2":"...",
"col3":"...",
"imgFiles":{
"img3.jpg"
}
},{
"id":"3",
...etc.
}]
I tried:
"SELECT art.*, ai.imgFile FROM articles art LEFT JOIN articleImages ai ON art.id = ai.artRef ORDER BY art.id desc LIMIT $lim, $limit"
And that's wrong...I got separate rows for each image. It's not a LEFT JOIN...what's the syntax for this? Or is there a more optimal way of organizing the table? I didn't want to make an "articleImages" column and concatenate inside "articles"...
If I got your goal correctly you can group your query and use GROUP_CONCAT function to get all coma separated img names in one field:
SELECT art.*,
GROUP_CONCAT(ai.imgFile) imgFiles
FROM articles art
LEFT JOIN articleImages ai
ON art.id = ai.artRef
GROUP BY art.id
ORDER BY art.id desc
LIMIT $lim, $limit

mySQL search for id in array

I want to filter my SQL query with some array of IDs.
so I have IDs for example: 2,3,4 and activity_meta.value 1,2,5;
And I want to find every activity where it has id in activity_meta.value
SELECT DISTINCT a.*, u.user_email, u.user_nicename, u.user_login, u.display_name
FROM wp_bp_activity a
LEFT JOIN wp_users u
ON a.user_id = u.ID
INNER JOIN wp_bp_activity_meta
ON (a.id = wp_bp_activity_meta.activity_id)
WHERE a.is_spam = 0
AND a.hide_sitewide = 0
AND a.type != 'activity_comment'
AND (wp_bp_activity_meta.meta_key = 'activity_tagz' )
DESC LIMIT 0, 20
I want to add AND (1,2,3 IN wp_bp_activity_meta.meta_value)
I just dont know how to treat serialized array;
The value of activity_meta.value is not normalized. It would be a better choice, to create a second table, where you assign the meta tags to your element.
So, it would look like:
wp_bp_activity_id | meta
1 | 1
1 | 2
1 | 3
let's cal this table "meta_relation". Then you simple could either use a subselect, like this:
... WHERE wp_bp_activity.id in (SELECT wp_bp_activity_id FROM meta_relation WHERE meta in (1,2,5))
Or you could implement this with another join, like
... INNER JOIN meta_relation ON wp_bp_activitiy.id = meta_relation.wp_bp_activity_id
(will return 3 rows then for 3 matching tags)
... WHERE meta in (1,2,5)
... GROUP BY wp_bp_activity.id
(will remove any duplicate result and unwanted tags)
For your current scenario you could use a workaround. However this requires to build up the query programmatically:
For each "Tag", you want to find (i.e. 1,2,5) you need to add another or-condition.
To make sure you are not matching the 2 within 125 you can sourround it by ,.
To make sure, you are not missing the FIRST or LAST item (which has no leading / trailing , you need to concatenate the column with 2 more , in the first place:
Query then contains the following additional criterias.
SELECT
....
WHERE
...
AND
(
CONCAT(CONCAT(",", activity_meta.value ), ",") LIKE "%,1,%" OR
CONCAT(CONCAT(",", activity_meta.value ), ",") LIKE "%,2,%" OR
CONCAT(CONCAT(",", activity_meta.value ), ",") LIKE "%,5,%"
)
if you want all 3 tags to appear, use AND instead of OR.
in your example, this will match :
,2,3,4, against ,1,
,2,3,4, against ,2, //match
,2,3,4, against ,5,
Order and / or gaps don't matter with this approach.
(Depending on whether its a 10 User Website or a Million-Customer-Portal, you can use either. Preferred Solution is to normalize your table.)

I am getting 2 rows returned when i'm really expecting one merged row, MySQL and INNER JOIN

I have the following SQL:
$queryString = "
SELECT
c.id,
c.object,
c.ref,
c.title,
o.value
FROM
organisations c
INNER JOIN
organisationType oT ON
oT.ref = c.ref
INNER JOIN
options o ON
oT.value = o.ref
WHERE c.ref = '585567'
ORDER BY
title
";
My organisations table contains information about an organisation (The following is one organisation)
id, dateModified, object, ref, title
2312, 2013-04-22 17:34:55,
{"__attributes":{"item_id":6868447,"external_id":..., 585567, Test
organisation
My organisationType table contains multiple entries for one organisation as an organisation can be a number of different types. (The following is 2 types assigned to the organisation above)
id, dateModified, relativeTable, ref, value
5, 2013-04-22 17:34:55, organisations, 585567, 3
6, 2013-04-22 17:34:55, organisations, 585567, 2
Finally my options table contains one row for each type that can be for reference:
id, dateModified, relativeTable, ref, value
1, 2013-04-22 16:36:59, organisations, 2, Client
2, 2013-04-22 16:36:59, organisations, 3, Partner
3, 2013-04-22 16:38:27, organisations, 4, Vendor
I would like to be able to return one organisation with its type embedded onto the end (o.value)
so my result might be:
[1] => stdClass Object
(
[id] => 2312
[object] => ""
[ref] => "585567"
[title] => "Test organisation"
[type] => array( [0] => "Client", [1] => "Vendor" )
)
With my current SQL, I am getting back an array of 2 rows with a duplicate of all components of the row apart from the type which has changed. It's fine, but it's not perfect.
You can't really bring back a sub array directly like that in SQL. You can bring back a concatenated field of all the values though. Something like this:-
$queryString = "
SELECT c.id,
c.object,
c.ref,
c.title,
GROUP_CONCAT(o.value) AS type
FROM organisations c
INNER JOIN organisationType oT ON oT.ref = c.ref
INNER JOIN options o ON oT.value = o.ref
WHERE c.ref = '585567'
GROUP BY c.id, c.object, c.ref, c.title
ORDER BY title
";
afaik. it is impossible by mysql to return an array as the field value. you can do it by your own code, adding "order by 1,2,3,4" to your sql expression and collecing an array in the fetch loop, comparing 1-4 columns with previous row.

Categories