I have a scenario like this:
I have User model that has an OneToMany relationships with the Post model.
I have a Hashtag model that has an OneToMany relationships with the Post model.
Recap: ONE user has MANY posts, ONE post belongs to ONE hashtag, ONE hashtag has MANY posts.
I would like to fetch only unique users records of an hashtag.
I'm able to do it in non scalable way (fetching all the posts first and then iterate filtering by user id), but I need to maintain scalability for large record numbers.
Edit: I saw a partial solution in Laravel docs.
Laravel eloquent has a method called unique().
With that method I can specify the parameter which should be unique in query.
In my case figured it out with:
$users = $hashtag->posts->unique('user_id');
But I can't paginate query in this way...
Has anyone a solution for that?
You have to paginate your main model caller.
I don't know exactly if this is can reproduce your actual scenario, but let's see this example:
Let's say you have the model hashtags
So let's say the code could be something like this:
$users = DB::table('hashtags AS tags')
->select('tags.*')
->join('[posts AS post','post.id','=','tags.id')
->distinct()
->paginate(5, ['tags.*']);
I don't know if your query it'll be very accurate, but for this, I believe the best approach it'll be you do a raw query, could look like it'll be costly to your database and the operation, but you can work around this approach indexing and partitioning your database.
Remembering always the eloquent sometimes even when we're building join queries could be more even costly to your database.
Since you're worried about scalability so the best thing could be for too could write a view to fetch all the data with proper indexing and partitioning.
Try to use DISTINCT
$users = $hashtag->posts()->selectRaw('DISTINCT(user_id) AS unique_user_id')->paginate(10);
OR
use groupBy()
$users = $hashtag->posts()->groupBy('name')->select('user_id')->paginate(10);
Related
I've created migrations and also their relationships. looks like I'm using polymorphic relations, because in one table, I could have the value of one model or i could have the value of another model. as you might guess, I have something like this
ID Mappable_type Mappable_id value
Mappable_type could be one or many other models. mappable_id will be appropriate id from that model. I needed this because I'm creating dynamic form builder and that's how I designed a database.
Now I need to fetch some records from 5 tables together, so I gotta use joins. the table where I have mappable_type is at the third place from those 5 joins. now What should I do also to only fetch data from these 5 joins that have specific model type that is in that morph table? I'm doing something like that for now:
$final = Dynamic::join('dynamic_properties_values','dynamic_properties_values.dynamic_id','=','dynamic.id')
->join('dynamic_properties_mapping','dynamic_properties_mapping.id','=','dynamic_properties_values.mapping_id')
->where('dynamic_properties_mapping.mappable_type','=','App\Driver')
->get();
As you see, I have written by hand something like
"App\Driver"
. It doesn't matter I have written it by hand or had it saved into a variable and than used it here. WHat really bothers me is that let's say when I was inserting data into these morph tables, Model got saved as "App\Driver". what if I make a mistake and when fetching with joins, I write something like App\Http\Driver? or what If I changed the location of the model itself in my code project?
What's the best practice not to make those kind of errors by the time my project reaches huge amount of users?
The scenario:
I am building a search system for a given set of data. Most of this is straight forward - where record title contains X, where record date before Y, etc. Where I am running into difficulty is essentially a category search. Each record belongs to zero or more categories (a relationship pivot table exists such that each row contains a record and a category), and when the user searches for Category A, I want to return all of the records that belong to that category. I've gotten this working with a whereHas, but it seems inordinately slow. In this instance, assume $category is a numeric id that is correctly validated as a category, and $records is an eloquent query builder that has not yet been executed by a get, pagination, all, etc (my function checks to see if several $request->input parameters are defined, then attaches a where to $record as required by the specified parameters, only executing it after all parameters have been considered):
if(!empty($category)) {
$records = $records->whereHas('categories', function($query) use ($category)
{
$query->where('category_id', $category);
});
}
This works, but as there are 7000+ records, 7000+ relationships defined in the pivot, and roughly 30 'categories', the search takes longer than I am comfortable leaving it. My unconfirmed thought is that the where query is executing for every record, thus leading to hundreds or thousands of queries.
I've debated approaching this using the raw query builder and just passing the list of record id's that have that category and using a simple where to filter the record collection before it's executed, but it seems counter-intuitive, leading me to believe there must be a better way.
The Question
How do I efficiently limit the records returned by $records->get() to those records with a defined relationship to category $category.
Edit 2018-01-16
To clarify, while I could simply do $category->records to return all records belonging to a category, this is part of a larger search engine. The full structure of the code looks like this:
If($subject_search_term) {
$records->where('subject', $subject_search_term)
}
If(some other search criteria is defined) {
$records->where(someothercriteria);
}
If(Category search criteria is defined) {
$records->whereHas(something);
}
$records->paginate(20);
Furthermore, there are two of these many-to-many relationships that I need to query (in addition to 'categories', lets say there is also a 'subject' that is independent of it, but similar structure and idea). As far as I know, I need to build the query off records and filter it accordingly.
EDIT 2
For anyone else with this problem, it seems the vastly more efficient way (and the most efficient that I've found) is Joel Hinz's comment - use the DB facade to build a raw query, pluck the id's from it, and use that in a whereIn clause.
Try this:
mpyw/eloquent-has-by-non-dependent-subquery: Convert has() and whereHas() constraints to non-dependent subqueries.
mpyw/eloquent-has-by-join: Convert has() and whereHas() constraints to join() ones for single-result relations.
if(!empty($category)) {
$records = $records->hasByNonDependentSubquery('categories', function($query) use ($category)
{
$query->where('category_id', $category);
});
}
That's all. Happy Eloquent Life!
I have a table for articles and categories, along with a pivot table; I got all related articles within a category. Using
$category = Category::first();
return $category->articles()->paginate(10); // many to many relationship $this->belongsToMany(Article::class, 'article_category');
It returns the right articles, but it is so slow when there are more than 20000 articles in a category. Is there any way to make the query faster?
note: All tables have indexes.
I have face this quite few times.
So far the only solution that I know is TO NOT USE ELOQUENT in COMPLEX QUERY.
You have to change to DB Query Builder.
I assume you are using "whereHas" in your query. Using it will slow down a lot on your queries. If you do use it, change to DB Query with Join methods. It would be a lot faster.
The only problem is that there is no relationship you can use which you have already declared in the model. You have to manually link it back.
But recently I heard they add this relationship feature already in 5.3. But I have not yet check it out.
Hopefully this will solve the problem.
The problem
Is there a way to get an eloquent model and all its relationships using only one query?
Example scenario:
You have both Post and Comment Eloquent models.
You add their relationships in the model class using hasMany('Comment') and belongsTo('Post') respectively.
Now this is what i've been doing to retrieve both the post AND its comments:
$post = Post::find($id);
$post->comments;
return $post;
This returns a beautiful jsonobject. The problem is, this way, i would be using two queries. That's not so great.
Workaround
Two alternatives come to mind:
Using joinstatements in order to make the query i want. But Eloquent is so much more elegant.
Leveraging the Cache class in order to make even fewer queries in the future (this, i will do anyway later on).
Any ideas?
In eloquent you can do it like this
$post = Post::with('comments')->find($id);
return $post;
Edit:
This will not run the query with join in mysql but its just a single query in Eloquent.
What kind of logic is behind getting a list of users that is not a member of the selected group.
Or get a list of users that is not in my contacts list.
Using laravel I have came up with a code.
The code:
// I get all the uid of the members of the user group.
$members_list = GroupMembers::where('group_id', $group_id)->lists('uid');
// I compare it to the list of all users and get those that are not in the list
$nonMembers = Users:whereNotIn('uid', $members_list)->paginate(10);
But is this the most efficient way of doing it?
I'm not sure it really matters too much because they are both pretty simple queries so they shouldn't take too long. It would probably take some experimenting to figure out which way is the fastest if you are really concerned about performance.
Here is a way you can do it using the query builder and table joins. This will use one query, but because it's joining, it will probably take just about the same amount of time as using what you have already. Also note that this won't allow you to traverse the collections like you usually would ($user->group->name for example would just become $user->name, which if there is a name column on your users table, something may get overwritten)
$users = DB::table('group_members')
->join('users','users.uid','=','group_members.uid')
->where('group_members.group_id', '<>', $group_id)
->paginate(10);
What you are doing now is pretty much exactly what eager loading is doing, except you are doing it by hand. The correct way to do this is as follows and is probably your best solution, though it probably won't save you anything on performance.
$users = User::with(array('groupMembers' => function($query) use ($group_id)
{
$query->where('group_id', '<>', $group_id);
}))->paginate(10);
definitely not that best..
Assuming the relationship is
Group members->has many->Users
you could to to use joins
DB::table('group_members')
->join('users','group_members.uid','<>','users.uid')