I have two tables: Workplans and Progress. Here is the Workplan table.
Workplan Table:
id
division_name
division_chief
Progress Table:
id
score
workplan_id foreign_key
Here workplan_id is foreign key of workplan table.
Now I want to get average from progress table grouping by the division_name. But I am really messed up with my query. What is I have done:
$report = DB::table('workplans')
->join('progress','workplans.id','=','progress.workplan_id')
->avg('progress.score')
->groupBy('workplans.division_name')
->get();
dd($report);
The SQL you want is something like this...
SELECT workplans.division_name, AVG(progress.score) as average_score
FROM `workplans`
LEFT JOIN `progress` ON progress.workplan_id = workplans.id
GROUP BY workplans.id
With the laravel query builder...
$report = DB::table ('workplans')
->join ('progress', 'progress.workplan_id', '=', 'workplans.id')
->select ('workplans.division_name', DB::raw ('AVG(progress.score) as average_score'))
->groupBy ('workplans.id')
->get ();
So for division 1 with scores 5 and 15, division 2 with scores 15 and 25 this returns...
Array
(
[0] => stdClass Object
(
[division_name] => Division 1
[average_score] => 10
)
[1] => stdClass Object
(
[division_name] => Division 2
[average_score] => 20
)
)
The problem with your query is that the avg aggregate function returns the average value for all columns on the table, not a reference back to the query so that you can continue to chain additional functions. The solution to this is to manually specify the average using the MySQL AVG function using the DB::raw method.
Try to do it via raw select:
$report = DB::table('workplans')
->join('progress','workplans.id','=','progress.activity_id')
->select(DB::raw('avg(progress.score) as avg_progress_score'))
->groupBy('workplans.division_name')
->get();
I'm not sure how to write it using Laravel's query builder, but a static MySQL query might be easier:
$report = DB::select("SELECT AVG(progress.score) AS 'avg' FROM workplans JOIN progress ON workplans.id=progress.activity_id GROUP BY workplans.division_name;");
Related
I have a table (tbl_operations) with rows of where the id column values may be comma-delimited. I want to get the count of each OpId for each month. I am trying to accomplish this through pure sql, but without success.
from this view
OpId
OpDate
3
2022-01-03
5,3
2022-01-15
4
2022-01-27
5
2022-02-01
7
2022-02-09
3,2
2022-01-16
to this
OpId
count
Month
2
1
01
3
3
01
4
1
01
5
1
01
5
1
02
7
1
02
I am stuck here. Can someone enlighten me on how to do this with sql? If not, maybe use php to display the result?
SELECT tbl_operations.OpId,
tbl_operations.OpDate ,
COUNT(tbl_operations.OpId) AS `count`
FROM tbl_operations
WHERE MONTH(OpDate)=1
GROUP BY tbl_operations.OpId
Here’s a quick example. The first part just creates an array of arrays which simulates what you’d get from the database.
The gist is that $counts is an array with a unique OpID for a keys. The values for those arrays are sub-arrays with keys of the month and values of how many times they are found.
Display should just be a simple loop again, however you might want to sort this.
$rows = [
['3', '2022-01-03'],
['5,3', '2022-01-15'],
['4', '2022-01-27'],
['5', '2022-02-01'],
['7', '2022-02-09'],
['3,2', '2022-01-16'],
];
$counts = [];
foreach($rows as $row){
$ids = explode(',', $row[0]);
$month = date('m', strtotime($row[1]));
foreach($ids as $id){
if(!array_key_exists($id, $counts)){
$counts[$id] = [];
}
if(!array_key_exists($month, $counts[$id])){
$counts[$id][$month] = 0;
}
$counts[$id][$month]++;
}
}
Demo here: https://3v4l.org/mVaBB
edit
From #mickmackusa, you can shorten the inner loop by using isset:
if(!isset($counts[$id][$month])){
$counts[$id][$month] = 0;
}
See their comment for a demo link
If you're going to query the data in PHP, you might as well return a better result to work with in the first place:
SQL
SELECT GROUP_CONCAT(OpId), MONTH(OpDate)
FROM tbl_operations
GROUP BY MONTH(OpDate)
PHP
// Result from MySQL query
$rows = [
['3,5,3,4,3,2', 1],
['5,7', 2]
];
And you can perform a count of those grouped results like this:
$results = [];
foreach ($rows as $row) {
$counts = array_count_values(explode(',', $row[0]));
$results[$row[1]] = $counts;
}
Result
Array
(
[1] => Array
(
[3] => 3
[5] => 1
[4] => 1
[2] => 1
)
[2] => Array
(
[5] => 1
[7] => 1
)
)
What you really want to do though is normalise your data, then you can do this easily in SQL alone.
If you are using at least MYSQL8 and you are not going to normalize your table design, then you can actually use the following CTE query to split, group, format, and sort your result set (no PHP processing).
This approach makes recursive calls on the denormalized table and progressively isolates the rightmost id from comma-delimited values and generates new rows for the individual id values. The recursion continues until there are no commas left.
This solution is built on top of the basic technique demonstrated here.
SQL: (Demo)
WITH RECURSIVE norm AS (
SELECT OpId,
OpDate
FROM tbl_operations
UNION ALL
SELECT REGEXP_REPLACE(OpId, '^[^,]*,', '') AS OpId,
OpDate
FROM norm
WHERE OpId LIKE '%,%'
)
SELECT Id,
Mo,
COUNT(*) AS Cnt
FROM (
SELECT REGEXP_REPLACE(norm.OpId, ',.*', '') AS Id,
MONTH(norm.OpDate) AS Mo
FROM norm
) formatted
GROUP BY formatted.Id,
formatted.Mo
Result Set:
Id
Mo
Cnt
2
1
1
3
1
3
4
1
1
5
1
1
5
2
1
7
2
1
That said, this is a lot of unnecessary voodoo mumbo jumbo for a task that is mega-easy once you've normalized your table --- just normalize it A.S.A.P.
I'm having trouble working in Redbean PHP with querying a table with an array of json objects in a single field, and producing a report on it.
I want to get a report with subtotals of all staff with notes by the category. I know this should be easy/obvious but I'm just not getting it properly.
I have a database, with:
table clients
with columns:(int) client_id, (string) client_name, (array of json) notes
notes is an array of json with
(int) note_id, (int) note_category_id, (int) staff_id, (string) description, (memo) content, (date) note_date
table staff with columns (int) sid, (string) sname
table categories with columns (int) cat_id, (string) cat_name
So in pseudocode (since I'm still trying to figure this all out)
I need to run a query like: (with parameters in brackets)
R::getAll('Select * from Join (staff, categories, clients)
On (staff.sid=clients.services.staff_id, categories.cat_id=clients.services.note_category_id)
Where (clients.services.note_date Between [startdate] and [enddate],
categories.cat_name IN [chosencateg], staff.sname IN [pickednames])
Orderby sname Asc, cat_name Asc, note_date Desc ');
report output format:
Filters used: [picked filter choices if any]
-----------
[sname]
-- note category: [cat_name] 1
[note_date] 1 [description] 1 [content] 1
[note_date] 2 [description] 2 [content] 2
note category 1 subtotal
-- note category: [cat_name] 2
[note_date] 3 [description] 3 [content] 3
[note_date] 4 [description] 4 [content] 4
note category 2 subtotal
staff subtotal
[sname] 2 ...
I'm asking a fairly generic one because I'll have to work with a number of similar tables, and maybe seeing a query template will help my understanding.
Thanks for any help.
redbean is fantastic and - getAll is just scratching the surface and truly isn't working with redbean at all really... Read up on it here:
Here's a query template to get you started:
Query Template:
1)
R::getAll('Select * from Join (staff, categories, clients)
On (staff.sid=clients.services.staff_id, categories.cat_id=clients.services.note_category_id)
Where (clients.services.note_date Between :startdate and :enddate,
categories.cat_name IN (:chosencateg), staff.sname IN (:pickednames))
Orderby sname Asc, cat_name Asc, note_date Desc ');
You could also simply use:
2)
R::getAll('Select * from Join (staff, categories, clients)
On (staff.sid=clients.services.staff_id, categories.cat_id=clients.services.note_category_id)
Where (clients.services.note_date Between ? and ?,
categories.cat_name IN (?), staff.sname IN (?))
Orderby sname Asc, cat_name Asc, note_date Desc ');
The only difference is that query template 1 uses named parameters (so it's going to look to the array of params that you pass it to contain an associative array with parameters named in the same way as they are in the query). While template 2 requires simply an array of parameters with the indexes lined up with the order in which the ? marks appear in your query.
Anyway... the query should return an associative array of your columns representing rows. a var_dump would look something like this:
Array
(
[client_id] => 1,
[client_name] => "joe",
[noes] => "[
{note_id=1
,note_category_id=1
,staff_id=1
,description=blah blah
,content=blah blah blah blah
,content=some content for this note
,note_date=12/06/2018
}
]"
[sid] => 100,
[sname] => "some staff name"
[cat_id] => 100
[cat_name] => "some category name"
)
Notice how the notes field has just come out as a string (I know the json above is not properly formed json, I'm just trying to show an example).
I assume that what you want is to have that string converted into an array so you can work with it as if it were data and not a string. So the below should get you started with that:
Once you have it out of the database you should be able to access it like this:
$result = R::getAll($query,
['startdate'=> $mystartDate
,'enddate' => $myEndDate
,'chosencateg'=>$myChosenCategory
,'pickednames'=>$myPickedNames
]);
// this would output the json string to your screen
echo $result['notes'];
but it seems like you want to work with the json as if it were part of your data - so... you would need to decode it first.
// decode my notes field:
foreach(array_key($result) as $key) {
/* you are working with a multidimensional array in this loop
, use $key to access the row index. Each row index
will contain named column indexes that are column names from the database
*/
$result[$key]['decoded_notes'] = json_decode($result[$key]['notes'],true);
}
// I now have a new column in each row index, containing 'notes'
as another associative array
// the statement below now results in an array to string conversion error:
echo $result[someIndexNumber]['decoded_notes'];
So, I decided I would want this in MySQL (5.7) so as to use its capabilities. To do this I used string manipulation. MySQL 8 adds json_table functions which would have been nice to have.
I converted each array of JSON notes into lines of 'INSERT INTO temptable' to convert the array list into temptable rows,
one JSON object per row, adding the client_id to each object, then
EXECUTEing those statements.
SET #allnotes = (
SELECT json_arrayagg(REPLACE(`notes`,'{', CONCAT('{"id_client": ', id_client, ', '))) /* add id_client to each note object */
FROM clients
WHERE `notes` != '' AND `notes` != '[]' ); /* no empty note cases */
SET #allnotes = REPLACE(REPLACE(#allnotes ,'"[',''),']"','' ); /* flatten out outer array of each note */
SET #allnotes = REPLACE(REPLACE(#allnotes ,'{','("{'),'}','}")' ); /* INSERT INTO string formatting for the objects */
DROP TEMPORARY TABLE IF EXISTS jsonTemporary;
CREATE TEMPORARY TABLE IF NOT EXISTS jsonTemporary (anote json);
SET #allnotes = REPLACE(REPLACE(#allnotes,'[','INSERT INTO jsonTemporary (anote) VALUES '),']',';');
PREPARE astatement FROM #allnotes;
EXECUTE astatement;
/* totals */
SELECT concat(staff.last_name,", ",staff.first_name) AS sname,
categories.name AS cat_name,
count(anote->'$.id_client') AS cat_total,
FROM jsonTemporary
JOIN categories ON cast(anote->'$.note_category_id' as unsigned)=categories.id
JOIN clients ON clients.id_client=anote->'$.id_client'
JOIN staff ON staff.id=anote->'$.staff_id'
WHERE anote->'$.note_date' >= "2018-10-01" AND anote->'$.note_date' <= "2018-12-31"
GROUP BY sname, cat_name;
/* all notes */
SELECT concat(staff.last_name,", ",staff.first_name) AS sname,
categories.name AS cat_name,
anote->'$.note_date' AS n_date,
anote->'$.description' AS description,
anote->'$.content' AS content,
FROM jsonTemporary
JOIN categories ON cast(anote->'$.note_category_id' as unsigned)=categories.id
JOIN clients ON clients.id_client=anote->'$.id_client'
JOIN staff ON staff.id=anote->'$.staff_id'
WHERE anote->'$.note_date' >= "2018-10-01" AND anote->'$.note_date' <= "2018-12-31"
GROUP BY sname, cat_name;
DROP TEMPORARY TABLE IF EXISTS jsonTemporary;
What I'm trying to achieve is doing 3 queries in one go, to limit the n1+ problem :
given we have 3 models :
trips
id => int
price => float
city_id => uint
........
cities
id => int
name => varchar
........
ratings:
id => int
ratable_id => int
rate => small-int
......
pseudocode:
select from tours where price >= 100
-then from the result
select from cities where id in result.city_id as cities
select count from ratings where ratable_id in result.id as rates groupBy rate
so the result is
[
trips => list of the trips where price more than or equal 100
cities=> list of the cities those trips belongs to
rates => list of rating with it's count so like [1 => 5, 2 => 100] assuming that '1 and 2' are the actual rating , and '5,100' is the trips count
]
how would I achieve that?
Two ways to go, Use eloquent methods which is preferred approach or use joins a single query to get your desired results
Moving forward with eloquent way i assume you have defined your models and their mappings based on their type of relationship (1:m, m:m)
$trips= Trips::with('city')
->withCount('ratings')
->where('price', '>=', 100)
->get();
Moving forward with join
$trips = DB::table('trips as t')
->select('t.id', 't.price','c.name',DB::raw('count(*) as rating_count'))
->join('cities as c' ,'t.city_id', '=' , 'c.id')
->join('ratings as r' ,'t.ratable_id', '=' , 'r.id')
->where('t.price', '>=', 100)
->groupBy('t.id', 't.price','c.name')
->get();
Trip Model Relationship
public function city(){
return $this->belongsTo(City::class);
}
public function ratings(){
return $this->hasMany(Rating::class, 'ratable_id'); //assuming ratable_id is an id of trips table
}
Fetch Data
$trips= Trip::with('city', 'ratings')
->where('price', '>=', 100)
->get();
Print Data
foreach($trips as $trip){
$trip->city->name." - ". $trip->price." - ". $trip->ratings()->avg('rate');
}
I have two relational tables of which i both need data. The first (Main) table contains about 90k rows. The secondary table contains about 200k plus rows.
I need to add some data of the secondary table to the results i get from the first table currently i do this in two parts:
First: i select the required rows from the Main table this goes super fast.
$records = getData($conn, "
SELECT id
from Main
where contains_tags is not null
and contains_mediums is not null
and contains_techniques is not null
limit 100
");
-
Then i go over each record to add the extra data from the secondary table but this takes ages (1 min for 100 records 50min - 100 min for 5k records. In the end i need to run this query for about 5-10k rows of the Main table). Is there a way to speed this process up?
foreach ($records as $r => $record) {
$records[$r]['mediums'] = getData($conn, "SELECT medium from mediums where kleding_id = ".$record['id']."");
kleding_id = ".$record['id']."");
}
This is the desired output after both queries finish:
[1] => Array
(
[id] => 15
[mediums] => Array
(
[0] => Array
(
[medium] => wol
)
[1] => Array
(
[medium] => katoen
)
)
)
So my question is how to run this query structure efficiently in as little as time as possible.
the Main table look like this:
the Secondary table looks like this:
If anything is unclear let me know so i can clarify.
If you need improve performance
be sure you have a proper index table mediums column kleding_id
CREATE INDEX my_index ON mediums (kleding_id);
remember that limit (for not recent db version) normally work on the result ..a and not break after the first 100 is reached
You can do it in a single query. But it's for mysql Verison 5.7 & above.
SELECT id, GROUP_CONCAT(json_object('medium', mediums.medium))
FROM Main
LEFT JOIN mediums ON Main.id = mediums.kleding_id
WHERE contains_tags IS NOT NULL AND contains_mediums IS NOT NULL AND contains_techniques IS NOT NULL
GROUP BY Main.id
LIMIT 100
Also make sure you have a index on kleding_id.
There is no need to iterate over the first result and call the 2nd table multiple times. This will always be slow.
Try inner join to aggregate data from both tables in one sql statement.
SELECT
Main.id,
mediums.medium
from
Main
inner join mediums on
(
Main.id = mediums.kleding_id
)
where
contains_tags is not null
and contains_mediums is not null
and contains_techniques is not null limit 100;
I am using Laravel 5.4's Query Builder to perform a series of leftJoins on three tables. Here are my tables:
items
id type title visibility status created_at
-- ---- ----- ---------- ------ ----------
1 1 This is a Title 1 1 2017-06-20 06:39:20
2 1 Here's Another Item 1 1 2017-06-24 18:12:13
3 1 A Third Item 1 1 2017-06-26 10:10:34
count_loves
id items_id user_id
-- ------- -------
1 1 2
2 1 57
3 1 18
count_downloads
id items_id user_id
-- ------- -------
1 1 879
2 1 323
And here is the code I am running in Laravel:
$items_output = DB::table('items')
->leftJoin('count_loves', 'items.id', '=', 'count_loves.items_id')
->leftJoin('count_downloads', 'items.id', '=', 'count_downloads.items_id')
->where('items.visibility', '=', '1')
->where('items.status', '=', '1')
->orderBy('items.created_at', 'desc')
->select('items.*', DB::raw('count(count_loves.id) as loveCount'), DB::raw('count(count_downloads.id) as downloadCount'))
->groupBy('items.id')
->get();
When I return the results for this query, I am getting the following counts:
count_loves: 6
count_downloads: 6
As you can see, the actual count values should be:
count_loves: 3
count_downloads: 2
If I add another entry to the count_loves table, as an example, the totals move to 8. If I add another entry to the count_downloads table after that, the totals jump to 12. So, the two counts are multiplying together.
If I die and dump the query, here's what I get:
"query" => "select 'items'.*, count(count_loves.id) as loveCount,
count(count_downloads.id) as downloadCount from 'items' left join
'count_loves' on 'items'.'id' = 'count_loves'.'items_id' left join
'count_downloads' on 'items'.'id' = 'count_downloads'.'items_id'
where 'items'.'visibility' = ? and 'items'.'status' = ? group by
'items'.'id' order by 'items'.'created_at' desc"
How do I perform multiple leftJoins using Query Builder and count on several tables to return the proper sums?
NOTE:
This is intended as a HELP answer not the total absolute answer but I could not write the code in a comment. I am not asking for votes (for those who just can't wait to downvote me). I have created your tables and tried a UNION query on raw sql. I got correct results. I dont have laravel installed, but maybe you could try a UNION query in Laravel.
https://laravel.com/docs/5.4/queries#unions
select count(count_downloads.user_id)
from count_downloads
join items
on items.id = count_downloads.items_id
UNION
select count(count_loves.user_id)
from count_loves
join items
on items.id = count_loves.items_id