I am trying to figure out how to form my database query to build up a collection of projects, each of which has a set of personnel with their assigned working periods and the daily shift hours that they submit.
In the code I have so far, I select the projects that are running for the current month. For each of those I select the personnel that are working on the project during that month. Then for each of the personnel I make another 2 queries to get their assignment dates and submitted shift details.
This does work, and creates the data structure I want, but it's slow due to the large number of individual database queries.
There must be a more efficient way to do this through subqueries or something?
$projects = Project::select([
'projects.id',
'projects.name',
'projects.start',
'projects.finish'
])
->where('projects.start', '<=', Carbon::now()->endOfMonth())
->where('projects.finish', '>=', Carbon::now()->startOfMonth())
->get();
foreach($projects as $project)
{
$project->personnel = ProjectUser::select([
'project_users.id',
'project_users.user_id',
'users.name',
'project_users.role'
])
->join('users', 'users.id', '=', 'project_users.user_id')
->join('assignments', 'assignments.project_user_id', '=', 'project_users.id')
->where('project_users.project_id', $project->id)
->where('assignments.start', '<=', Carbon::now()->endOfMonth())
->where('assignments.finish', '>=', Carbon::now()->startOfMonth())
->distinct('users.name')
->get();
foreach($project->personnel as $person)
{
$person->assignments = Assignment::select([
'assignments.start',
'assignments.finish',
'assignments.travel_out',
'assignments.travel_home'
])
->where('assignments.project_user_id', $person->project_user_id)
->get();
$person->shifts = WorkShift::select([
'work_shifts.id',
'work_shifts.role',
'work_shifts.date',
'work_shifts.start',
'work_shifts.hours',
'work_shifts.status',
'work_shifts.remarks'
])
->where('work_shifts.user_id', $person->user_id)
->where('work_shifts.project_id', $project->id)
->get();
}
}
Update
I now have this mostly working through the use of model relationships. My last problem is that I can filter the projects within a date range at the top level in my controller, but I also need to apply that filter to the assignments and work_shifts.
So, I only want to retrieve project_users that have assignments within the requested date range and then pick out only the assignments and work_shifts that match.
What I have so far is...
// Controller
public function load(Request $request)
{
$projects = Project::select([
Project::ID,
Project::NAME,
Project::START,
Project::FINISH
])
// Only able to filter by date range here?!?
->where(Project::START, '<=', Carbon::now()->endOfYear())
->where(Project::FINISH, '>=', Carbon::now()->startOfYear())
->with('project_staff')
->orderBy(Project::START)
->get();
return response()->json([
'projects' => $projects
]);
}
// Project Model
// Just want to get the staff that are assigned to the project
// between the selected date range
public function project_staff()
{
return $this->hasMany(ProjectUser::class)
->select([
ProjectUser::ID,
ProjectUser::PROJECT_ID,
ProjectUser::USER_ID,
User::NAME,
ProjectUser::ROLE
])
->whereIn(ProjectUser::STATUS, [
ProjectUser::STATUS_RESERVED,
ProjectUser::STATUS_ASSIGNED,
ProjectUser::STATUS_ACCEPTED
])
->join(User::TABLE, User::ID, '=', ProjectUser::USER_ID)
->with([
'assignments',
'shifts'
]);
}
// Assignments Model
// Again, just want the assignments and workshifts that fall
// within the selected date range
public function assignments()
{
return $this->hasMany(Assignment::class, 'projectUser_id')
->select([
Assignment::PROJECT_USER_ID,
Assignment::START,
Assignment::FINISH,
Assignment::TRAVEL_OUT,
Assignment::TRAVEL_HOME
]);
}
public function shifts()
{
return $this->hasMany(WorkShift::class, ['project_id','user_id'], ['project_id','user_id'])
->select([
WorkShift::ID,
WorkShift::USER_ID,
WorkShift::PROJECT_ID,
WorkShift::ROLE,
WorkShift::DATE,
WorkShift::START,
WorkShift::HOURS,
WorkShift::STATUS,
WorkShift::REMARKS
]);
}
// WorkShift Model
public function projectUser()
{
return $this->belongsTo(ProjectUser::class, ['project_id','user_id'], ['project_id','user_id']);
}
As mentioned before, you can use Laravel built in relationships, or, you can construct a giant query and join all the data together;
Like so:
<?php
$projectUsers = ProjectUser::select([
'project_users.id', // define aliases for all columns
'project_users.user_id',
'users.name',
'project_users.role',
'projects.id',
'projects.name',
'projects.start',
'projects.finish',
'assignments.start',
'assignments.finish',
'assignments.travel_out',
'assignments.travel_home',
'work_shifts.id',
'work_shifts.role',
'work_shifts.date',
'work_shifts.start',
'work_shifts.hours',
'work_shifts.status',
'work_shifts.remarks'
])
->join('users', 'users.id', 'project_users.user_id'
->leftJoin('assignments', 'assignment.project_user_id', 'project_users.id')
->leftJoin('projects', fn ($query) => $query
->where('projects.start', '<=', Carbon::now()->endOfMonth())
->where('projects.finish', '>=', Carbon::now()->startOfMonth())
)
->leftJoin('work_shifts', fn ($query) => $query
->whereColumn('work_shifts.id', 'project_users.id')
->whereColumn('work_shifts.project_id', 'projects.id')
)
->whereColumn('project_users.project_id', 'projects.id')
->where('assignments.start', '<=', Carbon::now()->endOfMonth())
->where('assignments.finish', '>=', Carbon::now()->startOfMonth())
->distinct('users.name')
->cursor();
foreach ($projectUsers as $user) {
// ...
}
After a day of much learning, the solution I've found is to have very simple relationships between the models and implement all the additional query logic in filter functions attached to the relationship query when it's called using with().
See https://laravel.com/docs/8.x/eloquent-relationships#constraining-eager-loads
My code is now working far more efficiently, making only 4 calls to the database in about 10-20ms in my dev-env, and structuring the results data in the correct form to simply return the whole collection as a JSON response.
// Controller
public function load(Request $request)
{
$start = Carbon::parse($request->input('start_date'));
$end = Carbon::parse($request->input('end_date'));
$projects = Project::select([
Project::ID,
Project::NAME,
Project::START,
Project::FINISH
])
->where(Project::START, '<=', $end)
->where(Project::FINISH, '>=', $start)
->with([
// Filter results of the model relationship, passing $start and
// $end dates down to propagate through to the nested layers
'project_staff' => $this->filter_project_staff($start, $end)
])
->orderBy(Project::START)
->get();
return response()->json([
'projects' => $projects
]);
}
private function filter_project_staff($start, $end)
{
return function ($query) use ($start, $end) {
$query
->select([
ProjectUser::ID,
ProjectUser::PROJECT_ID,
ProjectUser::USER_ID,
User::NAME,
ProjectUser::ROLE
])
->whereIn(ProjectUser::STATUS, [
ProjectUser::STATUS_RESERVED,
ProjectUser::STATUS_ASSIGNED,
ProjectUser::STATUS_ACCEPTED
])
->join(User::TABLE, User::ID, '=', ProjectUser::USER_ID)
->join(Assignment::TABLE, Assignment::PROJECT_USER_ID, '=', ProjectUser::ID)
->where(Assignment::START, '<=', $end)
->where(Assignment::FINISH, '>=', $start)
->distinct(User::NAME)
->orderBy(ProjectUser::ROLE, 'desc')
->orderBy(User::NAME)
->with([
// Filter results of the nested model relationships
'assignments' => $this->filter_assignments($start, $end),
'shifts' => $this->filter_shifts($start, $end)
]);
};
}
private function filter_assignments($start, $end)
{
return function ($query) use ($start, $end) {
$query->select([
Assignment::PROJECT_USER_ID,
Assignment::START,
Assignment::FINISH,
Assignment::TRAVEL_OUT,
Assignment::TRAVEL_HOME
])
->where(Assignment::START, '<=', $end)
->where(Assignment::FINISH, '>=', $start)
->orderBy(Assignment::START);
};
}
private function filter_shifts($start, $end)
{
return function ($query) use ($start, $end) {
$query->select([
WorkShift::ID,
WorkShift::USER_ID,
WorkShift::PROJECT_ID,
WorkShift::ROLE,
WorkShift::DATE,
WorkShift::START,
WorkShift::HOURS,
WorkShift::STATUS,
WorkShift::REMARKS
])
->where(WorkShift::DATE, '>=', $start)
->where(WorkShift::DATE, '<=', $end)
->orderBy(WorkShift::DATE);
};
}
///////////////
//Project Model
public function project_staff(): HasMany
{
return $this->hasMany(ProjectUser::class);
}
////////////////////
// ProjectUser Model
public function project(): BelongsTo
{
return $this->belongsTo(Project::class);
}
public function assignments(): HasMany
{
return $this->hasMany(Assignment::class, 'projectUser_id');
}
public function shifts()
{
// Using awobaz/compoships to simplify creating the relationship using a composite key
return $this->hasMany(WorkShift::class, ['project_id','user_id'], ['project_id','user_id']);
}
///////////////////
// Assignment Model
public function project_user(): BelongsTo
{
return $this->belongsTo(ProjectUser::class, 'projectUser_id');
}
// WorkShift Model
public function project_user()
{
return $this->belongsTo(ProjectUser::class, ['project_id','user_id'], ['project_id','user_id']);
}
Many thanks to anyone that put time and thought into helping me with this problem. Your effort is very much appreciated.
I'm trying to filter products based on query string. My goal is to get products from a collection if it's given, otherwise get every product. Could someone help me what's wrong with the following code?
$products = \App\Product::where([
'collection' => (request()->has('collection')) ? request('collection') : '[a-z]+',
'type' => (request()->has('type')) ? request('type') : '[a-z]+'
])->get();
PS.: I've also tried with 'regex:/[a-z]+', it's not working...
$products = \App\Product::where(['collection' => (request()->has('collection')) ? request('collection') : 'regex:/[a-z]+'])->get();
What you can do is use when eloquent clause, so your where clause for collections will be triggered only when the request('collection') exists, same logis applie to type as well.
$products = \App\Product::
when(request()->has('collection'), function ($q) {
return $q->where('collection', request('collection'));
});
->when(request()->has('type'), function ($q) {
return $q->where('type', request('type'));
})
->get();
Or another way if you have your request values assigned to a variable something like:
$collection = request('collection');
$type= request('type');
$products = \App\Product::
when(!empty($collection), function ($q) use ($collection) {
return $q->where('collection', $collection);
});
->when(!empty($type), function ($q) use ($type) {
return $q->where('type', $type);
})
->get();
I have this function:
public function show($id)
{
if (count($post = Post::find($id))) {
$post = $post->load(['comments' => function ($q) {
$q->latest();
$q->with(['author' => function ($q) {
$q->select('id', 'username');
}]);
}, 'user' => function ($q) {
$q->select('id', 'username');
}]);
$this->authorize('seePost', $post);
return view('post.show', ['post' => $post]);
} else {
dd('no post');
}
}
I added the if statement as if I try to open a route to a non existent post id I get the error Call to a member function load() on null.
However now I have two queries, one looks for the Post in the DB and if it finds one then I have to load the relations with the second one. What can I do to go back to just one query with all the relations loaded and avoid the error? Any clue?
You can use Constraining Eager Loads do it like this:
https://laravel.com/docs/5.8/eloquent-relationships#constraining-eager-loads
$post = Post::with(["comments" => function ($query) {
// Order by created_at, query comment author & select id, username
$query->latest()->with(["author" => function ($q) {
$q->select("id", "username");
}]);
}, "user" => function ($query) {
// Query post author & select id,username
$query->select("id", "username");
}])
// Fetch post or throw a 404 if post is missing
->findOrFail($id);
// You can also return an empty post instance like this if post is missing
// ->findOrNew([]);
// Or return the post or null if post is missing
// ->find($id);
// Authorize
$this->authorize('seePost', $post);
return view("post.show", ["post" => $post]);
Laravel has an Eager Loading feature that would be helpfull in your case. Eager Loading allows you to autoload relations along with the same query that you use to retrieve your main model info. https://laravel.com/docs/5.8/eloquent-relationships#eager-loading
You could a below codes.
Easiest way is :
$post = Post::with('comments.author', 'user')
->find($id);
Or fine tune query with callback :
$post = Post::with(['comments' => function ($q) {
// if you use comments select, then you need to specify foreign key too
$q->select('id', 'author_id', 'details') // comment fields
->latest(); // Use chaining method
// OR use $q = $q->latest();
},
'comments.author' => function ($q) {
$q->select('id', 'username'); // author fields
},
'user' => function ($) {
$q->select('id', 'username'); // user fields
}])
->find($id);
In some cases you might need some modifications, bu in overall that should avoid you N+1 queries problem.
I would like to assign a group to users which have a role student and have also a particular selected group. There is a users table, which has pivot tables: role_user and group_user with roles and groups tables. Below is the code for the controller where I am trying to execute the query:
$this->validate($request, [
'add-group' => 'required',
'where-group' => 'required'
]);
$selectedGroup = $request->input('add-group');
$whereGroupId = $request->input('where-group');
$users = User::whereHas('roles', function($q) {
$q->where('name', 'student');
})->whereHas('groups', function($q) {
$q->where('id', $whereGroupId);
})->get();
$selectedGroup = Group::whereId($selectedGroup)->first();
$users->assignGroup($selectedGroup);
You need to use the orWhereHas clause for the second half of the query.
Secondly, your $whereGroupId variable is not in the inner-function's scope, add a use($whereGroupId) statement to include it in the function's scope.
$users = User::whereHas('roles', function($q) {
$q->where('name', 'student');
})->orWhereHas('groups', function($q) use ($whereGroupId) { // <-- Change this
$q->where('id', $whereGroupId);
})->get();
You had syntax error and a missing use to pass whereGroupId. I have no clue what assignGroup does, but this should fix the code you have.
$this->validate($request, [
'add-group' => 'required',
'where-group' => 'required'
]);
$selectedGroup = $request->input('add-group');
$whereGroupId = $request->input('where-group');
$users = User::whereHas('roles', function ($q) {
$q->where('name', 'student');
})
->whereHas('groups', function ($q) use ($whereGroupId) {
$q->where('id', $whereGroupId);
})->get();
$selectedGroup = Group::whereId($selectedGroup)->first();
$users->assignGroup($selectedGroup);
I'm using Laravel 5.3 and trying to return a heist with it's product and only the latest order and with the latest price history. Both joins don't return anything but if I remove the $q->latest()->first(); and replace it with a simple orderBy() I get all results. My query is:
$data = $heist->with(['product'=> function($query) {
$query->with(['orders' => function($q) {
return $q->latest()->first();
}]);
$query->with(['price_history' => function($q) {
return $q->latest()->first();
}]);
}])->orderBy('completed_at', 'DESC')->orderBy('active', 'DESC')->get();
As discussed in the comments, I believe the simplest way of doing this is
$heists = $heist->with(['product'=> function($query) {
$query->with([
'orders' => function($q) {
return $q->orderBy('created_at', 'desc')->take(1)->get();
},
'price_history' => function($q) {
return $q->orderBy('created_at', 'desc')->take(1)->get();
}
]);
}])->orderBy('completed_at', 'desc')->orderBy('active', 'desc')->get();
Hope this helps :)
Calling first() is the same as calling take(1)->get()[0];
Which means limit the amount returned to 1 and return it. What you want is just the limit part. So if you change first() to take(1).
Update
$data = $heist->with([
'product'=> function($query) {
$query->with(
[
'orders' => function($q) {
$q->latest()->take(1);
},
'price_history' => function($q) {
$q->latest()->take(1);
}
]
);
}
])->orderBy('completed_at', 'DESC')->orderBy('active', 'DESC')->get();