I am trying to figure out how to form my database query to build up a collection of projects, each of which has a set of personnel with their assigned working periods and the daily shift hours that they submit.
In the code I have so far, I select the projects that are running for the current month. For each of those I select the personnel that are working on the project during that month. Then for each of the personnel I make another 2 queries to get their assignment dates and submitted shift details.
This does work, and creates the data structure I want, but it's slow due to the large number of individual database queries.
There must be a more efficient way to do this through subqueries or something?
$projects = Project::select([
'projects.id',
'projects.name',
'projects.start',
'projects.finish'
])
->where('projects.start', '<=', Carbon::now()->endOfMonth())
->where('projects.finish', '>=', Carbon::now()->startOfMonth())
->get();
foreach($projects as $project)
{
$project->personnel = ProjectUser::select([
'project_users.id',
'project_users.user_id',
'users.name',
'project_users.role'
])
->join('users', 'users.id', '=', 'project_users.user_id')
->join('assignments', 'assignments.project_user_id', '=', 'project_users.id')
->where('project_users.project_id', $project->id)
->where('assignments.start', '<=', Carbon::now()->endOfMonth())
->where('assignments.finish', '>=', Carbon::now()->startOfMonth())
->distinct('users.name')
->get();
foreach($project->personnel as $person)
{
$person->assignments = Assignment::select([
'assignments.start',
'assignments.finish',
'assignments.travel_out',
'assignments.travel_home'
])
->where('assignments.project_user_id', $person->project_user_id)
->get();
$person->shifts = WorkShift::select([
'work_shifts.id',
'work_shifts.role',
'work_shifts.date',
'work_shifts.start',
'work_shifts.hours',
'work_shifts.status',
'work_shifts.remarks'
])
->where('work_shifts.user_id', $person->user_id)
->where('work_shifts.project_id', $project->id)
->get();
}
}
Update
I now have this mostly working through the use of model relationships. My last problem is that I can filter the projects within a date range at the top level in my controller, but I also need to apply that filter to the assignments and work_shifts.
So, I only want to retrieve project_users that have assignments within the requested date range and then pick out only the assignments and work_shifts that match.
What I have so far is...
// Controller
public function load(Request $request)
{
$projects = Project::select([
Project::ID,
Project::NAME,
Project::START,
Project::FINISH
])
// Only able to filter by date range here?!?
->where(Project::START, '<=', Carbon::now()->endOfYear())
->where(Project::FINISH, '>=', Carbon::now()->startOfYear())
->with('project_staff')
->orderBy(Project::START)
->get();
return response()->json([
'projects' => $projects
]);
}
// Project Model
// Just want to get the staff that are assigned to the project
// between the selected date range
public function project_staff()
{
return $this->hasMany(ProjectUser::class)
->select([
ProjectUser::ID,
ProjectUser::PROJECT_ID,
ProjectUser::USER_ID,
User::NAME,
ProjectUser::ROLE
])
->whereIn(ProjectUser::STATUS, [
ProjectUser::STATUS_RESERVED,
ProjectUser::STATUS_ASSIGNED,
ProjectUser::STATUS_ACCEPTED
])
->join(User::TABLE, User::ID, '=', ProjectUser::USER_ID)
->with([
'assignments',
'shifts'
]);
}
// Assignments Model
// Again, just want the assignments and workshifts that fall
// within the selected date range
public function assignments()
{
return $this->hasMany(Assignment::class, 'projectUser_id')
->select([
Assignment::PROJECT_USER_ID,
Assignment::START,
Assignment::FINISH,
Assignment::TRAVEL_OUT,
Assignment::TRAVEL_HOME
]);
}
public function shifts()
{
return $this->hasMany(WorkShift::class, ['project_id','user_id'], ['project_id','user_id'])
->select([
WorkShift::ID,
WorkShift::USER_ID,
WorkShift::PROJECT_ID,
WorkShift::ROLE,
WorkShift::DATE,
WorkShift::START,
WorkShift::HOURS,
WorkShift::STATUS,
WorkShift::REMARKS
]);
}
// WorkShift Model
public function projectUser()
{
return $this->belongsTo(ProjectUser::class, ['project_id','user_id'], ['project_id','user_id']);
}
As mentioned before, you can use Laravel built in relationships, or, you can construct a giant query and join all the data together;
Like so:
<?php
$projectUsers = ProjectUser::select([
'project_users.id', // define aliases for all columns
'project_users.user_id',
'users.name',
'project_users.role',
'projects.id',
'projects.name',
'projects.start',
'projects.finish',
'assignments.start',
'assignments.finish',
'assignments.travel_out',
'assignments.travel_home',
'work_shifts.id',
'work_shifts.role',
'work_shifts.date',
'work_shifts.start',
'work_shifts.hours',
'work_shifts.status',
'work_shifts.remarks'
])
->join('users', 'users.id', 'project_users.user_id'
->leftJoin('assignments', 'assignment.project_user_id', 'project_users.id')
->leftJoin('projects', fn ($query) => $query
->where('projects.start', '<=', Carbon::now()->endOfMonth())
->where('projects.finish', '>=', Carbon::now()->startOfMonth())
)
->leftJoin('work_shifts', fn ($query) => $query
->whereColumn('work_shifts.id', 'project_users.id')
->whereColumn('work_shifts.project_id', 'projects.id')
)
->whereColumn('project_users.project_id', 'projects.id')
->where('assignments.start', '<=', Carbon::now()->endOfMonth())
->where('assignments.finish', '>=', Carbon::now()->startOfMonth())
->distinct('users.name')
->cursor();
foreach ($projectUsers as $user) {
// ...
}
After a day of much learning, the solution I've found is to have very simple relationships between the models and implement all the additional query logic in filter functions attached to the relationship query when it's called using with().
See https://laravel.com/docs/8.x/eloquent-relationships#constraining-eager-loads
My code is now working far more efficiently, making only 4 calls to the database in about 10-20ms in my dev-env, and structuring the results data in the correct form to simply return the whole collection as a JSON response.
// Controller
public function load(Request $request)
{
$start = Carbon::parse($request->input('start_date'));
$end = Carbon::parse($request->input('end_date'));
$projects = Project::select([
Project::ID,
Project::NAME,
Project::START,
Project::FINISH
])
->where(Project::START, '<=', $end)
->where(Project::FINISH, '>=', $start)
->with([
// Filter results of the model relationship, passing $start and
// $end dates down to propagate through to the nested layers
'project_staff' => $this->filter_project_staff($start, $end)
])
->orderBy(Project::START)
->get();
return response()->json([
'projects' => $projects
]);
}
private function filter_project_staff($start, $end)
{
return function ($query) use ($start, $end) {
$query
->select([
ProjectUser::ID,
ProjectUser::PROJECT_ID,
ProjectUser::USER_ID,
User::NAME,
ProjectUser::ROLE
])
->whereIn(ProjectUser::STATUS, [
ProjectUser::STATUS_RESERVED,
ProjectUser::STATUS_ASSIGNED,
ProjectUser::STATUS_ACCEPTED
])
->join(User::TABLE, User::ID, '=', ProjectUser::USER_ID)
->join(Assignment::TABLE, Assignment::PROJECT_USER_ID, '=', ProjectUser::ID)
->where(Assignment::START, '<=', $end)
->where(Assignment::FINISH, '>=', $start)
->distinct(User::NAME)
->orderBy(ProjectUser::ROLE, 'desc')
->orderBy(User::NAME)
->with([
// Filter results of the nested model relationships
'assignments' => $this->filter_assignments($start, $end),
'shifts' => $this->filter_shifts($start, $end)
]);
};
}
private function filter_assignments($start, $end)
{
return function ($query) use ($start, $end) {
$query->select([
Assignment::PROJECT_USER_ID,
Assignment::START,
Assignment::FINISH,
Assignment::TRAVEL_OUT,
Assignment::TRAVEL_HOME
])
->where(Assignment::START, '<=', $end)
->where(Assignment::FINISH, '>=', $start)
->orderBy(Assignment::START);
};
}
private function filter_shifts($start, $end)
{
return function ($query) use ($start, $end) {
$query->select([
WorkShift::ID,
WorkShift::USER_ID,
WorkShift::PROJECT_ID,
WorkShift::ROLE,
WorkShift::DATE,
WorkShift::START,
WorkShift::HOURS,
WorkShift::STATUS,
WorkShift::REMARKS
])
->where(WorkShift::DATE, '>=', $start)
->where(WorkShift::DATE, '<=', $end)
->orderBy(WorkShift::DATE);
};
}
///////////////
//Project Model
public function project_staff(): HasMany
{
return $this->hasMany(ProjectUser::class);
}
////////////////////
// ProjectUser Model
public function project(): BelongsTo
{
return $this->belongsTo(Project::class);
}
public function assignments(): HasMany
{
return $this->hasMany(Assignment::class, 'projectUser_id');
}
public function shifts()
{
// Using awobaz/compoships to simplify creating the relationship using a composite key
return $this->hasMany(WorkShift::class, ['project_id','user_id'], ['project_id','user_id']);
}
///////////////////
// Assignment Model
public function project_user(): BelongsTo
{
return $this->belongsTo(ProjectUser::class, 'projectUser_id');
}
// WorkShift Model
public function project_user()
{
return $this->belongsTo(ProjectUser::class, ['project_id','user_id'], ['project_id','user_id']);
}
Many thanks to anyone that put time and thought into helping me with this problem. Your effort is very much appreciated.
I have this query
$t_items_between = TransferItem::whereHas('transfer', function ($query) use($old_until_date, $inventory_id, $old_from_date_interval) {
$query->where('document_date', '>=', $old_from_date_interval);
$query->where('document_date', '<=', $old_until_date);
$query->where('to_inventory_id', $inventory_id);
})
->with(['transfer' => function($query) use($old_until_date, $inventory_id, $old_from_date_interval) {
$query->where('document_date', '>=', $old_from_date_interval);
$query->where('document_date', '<=', $old_until_date);
$query->where('to_inventory_id', $inventory_id);
}
])
->whereIn('item_id', $subset)
->addSelect(['quantity' => TransferItem::selectRaw('sum(quantity)')
->groupBy('item_stock_id')
->from((new TransferItem)->getTable() . ' as ti')
->whereColumn('ti.item_stock_id', (new TransferItem)->getTable() . '.item_stock_id')
])
->get();
So TransferItem is the model of the table transfer_items. transfer_items has 4 columns: id, transfer_id, item_stock_id and quantity.
What I am trying to do is to sum the quantity of each result, grouping by the item_stock_id, but the query above doesn't work. How would I approach it?
try using map
$t_items_between = TransferItem::whereHas('transfer', function ($query) use($old_until_date, $inventory_id, $old_from_date_interval) {
$query->where('document_date', '>=', $old_from_date_interval);
$query->where('document_date', '<=', $old_until_date); $query->where('to_inventory_id', $inventory_id);
})
->with(['transfer' => function($query) use($old_until_date, $inventory_id, $old_from_date_interval) {
$query->where('document_date', '>=', $old_from_date_interval);
$query->where('document_date', '<=', $old_until_date); $query->where('to_inventory_id', $inventory_id);
}
])
->whereIn('item_id', $subset)
->get()
->groupBy('item_stock_id')
->map(function ($q) {
return $q->sum('quantity');
});
Visit https://laravel.com/docs/9.x/collections
I'm struggling to understand how to use Eloquent/Query Builder with relationships.
I have the following:
$plantLots = PlantLot::with('controlNumber')
->whereHas('controlNumber', function ($q) use ($fromDate, $toDate) {
$q->where('created_at', '>=', $fromDate);
if($toDate != '') {
$q->where('created_at', '=<', $toDate);
}
$q->groupBy('creator_id');
})->get();
I want to group by creator_id, but I still just get a single collection of data.
What am I doing wrong?
change it to
$plantLots = PlantLot::with(['controlNumber' => function($q){
$q->groupBy('creator_id');
}])
->whereHas('controlNumber', function ($q) use ($fromDate, $toDate) {
$q->where('created_at', '>=', $fromDate)
->when($toDate != '',function($q){
$q->where('created_at', '=<', $toDate);
});
})->get();
Hope it helps.
I'm using Laravel 5.3 and trying to return a heist with it's product and only the latest order and with the latest price history. Both joins don't return anything but if I remove the $q->latest()->first(); and replace it with a simple orderBy() I get all results. My query is:
$data = $heist->with(['product'=> function($query) {
$query->with(['orders' => function($q) {
return $q->latest()->first();
}]);
$query->with(['price_history' => function($q) {
return $q->latest()->first();
}]);
}])->orderBy('completed_at', 'DESC')->orderBy('active', 'DESC')->get();
As discussed in the comments, I believe the simplest way of doing this is
$heists = $heist->with(['product'=> function($query) {
$query->with([
'orders' => function($q) {
return $q->orderBy('created_at', 'desc')->take(1)->get();
},
'price_history' => function($q) {
return $q->orderBy('created_at', 'desc')->take(1)->get();
}
]);
}])->orderBy('completed_at', 'desc')->orderBy('active', 'desc')->get();
Hope this helps :)
Calling first() is the same as calling take(1)->get()[0];
Which means limit the amount returned to 1 and return it. What you want is just the limit part. So if you change first() to take(1).
Update
$data = $heist->with([
'product'=> function($query) {
$query->with(
[
'orders' => function($q) {
$q->latest()->take(1);
},
'price_history' => function($q) {
$q->latest()->take(1);
}
]
);
}
])->orderBy('completed_at', 'DESC')->orderBy('active', 'DESC')->get();
$jobs = EventJob::with(['applications' => function ($query) {
$query->where('accepted', false);
}, 'job' => function($query) {
$query->where('division_id', 1);
}, 'event' => function($query) use ($date) {
$query->where('date', '>=', $date);
}])->get();
Complete overhaul of the question.
I wan't only the EventJob elements where the conditions in the $query are true but I get all the elements.
Seems like the function you're looking for is whereHas, it is documented here.
Your code would then be :
$jobs = EventJob::whereHas('applications', function ($query) {
$query->where('accepted', false);
})->whereHas('job', function($query) {
$query->where('division_id', 1);
})->wherehas('event', function($query) use ($date) {
$query->where('date', '>=', $date);
})->get();