Laravel groupBy performance slow - php

Question

I'm working in one of my Laravel 8 projects and need to parse dates of my returned query for formatting with my graph. My query, which returns around 70,000 rows takes around 100ms, but upon further debugging, the getUptimeTimeline function in my project which takes the results of my query as $uptimeChecks is taking 800ms for just the groupBy part.
How can I improve this performance or maybe exclude it entirely?
The query results passed to my function are:
$uptimeChecks = UptimeCheck::where('user_id', $user->id)
->where('monitor_id', $monitor['id'])
->where('checked_at', '>=', $from)
->where('checked_at', '<=', $to)
->orderBy('checked_at', 'asc')
->select('event', 'response_time', 'checked_at')
->get();
Here's my function:
/**
* Get uptime timeline
*
* #return Response
*/
protected function getUptimeTimeline($user, $id, $uptimeChecks, $period, $days)
{
try {
$start = microtime(true);
$dates = collect($period->toArray())->mapWithKeys(function ($date) {
return [$date->format('Y-m-d') => [
'total_events' => 0,
'down_events' => 0,
'up_events' => 0,
'uptime' => 'No Data',
'fill' => '#ced1d7',
]];
});
$end = microtime(true);
Log::debug('timeline_1', [
'diff' => ($end - $start) * 1000
]);
$start = microtime(true);
$uptimeDates = $uptimeChecks->groupBy(function ($item, $key) {
$date = Carbon::parse($item->checked_at);
return $date->format('Y-m-d');
});
$end = microtime(true);
Log::debug('timeline_2', [
'diff' => ($end - $start) * 1000
]);
return $uptimeDates;
} catch (\Exception $e) { }
}
It's the timeline_2 results which is slow.
UPDATE
$responseTimes = UptimeCheck::where('user_id', $user->id)
->where('monitor_id', $monitor['id'])
->where('checked_at', '>=', $from)
->where('checked_at', '<=', $to)
->orderBy('checked_at', 'asc')
->select('event', 'response_time', 'checked_at', DB::raw('DATE(checked_at) as check_date'))
->groupBy('check_date')
->limit(10)
->get();
Attempting to run a select containing DB::raw() throws an error:
[2023-01-30 20:42:31] local.ERROR: SQLSTATE[42000]: Syntax error or access violation: 1055 Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'domainmonitor_db.uptime_checks.event' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by (SQL: select event, response_time, checked_at, DATE(checked_at) as check_date from uptime_checks where user_id = 1 and monitor_id = 1 and checked_at >= 2022-12-26 20:42:31 and checked_at <= 2023-01-30 20:42:31 group by check_date order by checked_at asc limit 10) {"userId":1,"exception":"[object] (Illuminate\Database\QueryException(code: 42000): SQLSTATE[42000]: Syntax error or access violation: 1055 Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'domainmonitor_db.uptime_checks.event' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by (SQL: select event, response_time, checked_at, DATE(checked_at) as check_date from uptime_checks where user_id = 1 and monitor_id = 1 and checked_at >= 2022-12-26 20:42:31 and checked_at <= 2023-01-30 20:42:31 group by check_date order by checked_at asc limit 10) at C:\Users\Ryan\Desktop\web-projects\domain-monitor\domain-monitor-api\vendor\laravel\framework\src\Illuminate\Database\Connection.php:760)

Answer 1

You should group the item during the query and handle other column properly on how you want like combine, sum, min, max or whatever that belongs to a group
here's an example grouping your query by checked_at and combing the values of other columns you want to select.
$uptimeChecks = UptimeCheck::select(
DB::raw('group_concat(event) as events'), // this combines event column values belongs to a group
DB::raw('group_concat(response_time) as responses_time'), // this combines response_time column values belongs to a group
'checked_at'
)
->where('user_id', $user->id)
->where('monitor_id', $monitor['id'])
->whereBetween('checked_at', [$from, $to])
->groupBy('checked_at')
->orderBy('checked_at', 'asc')
->get();