i hope you are having a good time. i am learning laravel and the inscuctor talked about when you load relationships in laravel, like so
public function timeline()
{
$ids = $this->follows()->pluck('id');
$ids->push($this->id);
return Tweet::whereIn('user_id', $ids)->latest()->get();
}
and i have a follows relationship in my model, and he talked about this line
$ids = $this->follows()->pluck('id');
being better for performance than this line
$ids = $this->follows->pluck('id');
my question is, how does laravel pluck the ids in the first case, and how it queries the database
i hope im making sense, thanks for your time, and answer.
the following one executes a select query on database
$this->follows()->pluck('id');
the follows() returns a query builder (which is a not yet executed sql statement) and then on the result select the id column and returns a collection of ids
you can see the query by dumping the query builder by $this->follows()->dd()
Whereas in the second option
$this->follows->pluck('id')
up until $this->follows laravel executes a query and returns all the records as a collection instance, You will be able to see all the attributes on each of the records. And then ->pluck('id') is getting executed on the laravel collection class, which will do an operation I assume similar to the array_column function does and returns only the id column.
as you can easily see in the second operation the whole data set was retrieved first from the DB and then selected the required attribute/column (2 distinct and heavy operations). Where as in the first option we directly told eloquent to select only the required column, which is only one lighter operation compared to the second option.
Related
In a nutshell, the title best discribes my question, but here I am showing the core of the problem.
I have two databases in my web application, One is MariaDB, the other is MongoDB, To give some context, the "user" table in MariaDB stores user information with column "id" it's primary key, there is another "badge" table which stores badge information with also column "id" it's primary key, at last there is "user_badge" collection in MongoDB having documents of fields
{_id, user_id, badge_id, date}
which just links the User with his/her Badges. This is what I meant by pseudo-relation, Unfortunately I don't know what is it called in this situation.
An example:
I want to query and get all users that have a badge with ID 1. So my pseudo-query should do something like "Select all fields from user table where badge_id in user_badge collection is 1". I highlighted like here because this is impossible to be done in a query (based on my knowledge) somehow a query ought to be made on the MongoDB database first then a second have to be made in the MariaDB database against the results of the former query.
Edit: My original question was about how to implement this in Yii2 PHP framework, but when I googled for sometime and I found out no information to do such a thing even in pure PHP, So I decide to end my edited question here, asking for a way to query between a table in an sql database and a collection in a no-sql database, Yet below I leave my old question which just asks for how to do this more specifically in the PHP framework. really if I knew how to do this in pure PHP I can just make a function somehow that does that in the framework if there wasn't any.
Obviously there cannot be a direct primarykey-foriegnkey relation between two database types but I overrided this issue by having a ::hasMany ActiveRecord method in my User Model, and that worked perfectly fine; When I have a User model between hands I just call $model->userBadges to get from MongoDB all documents having that User ID, also vice versa. The problem is when I do a Query involving this relation, I get error
Calling unknown method: yii\mongodb\ActiveQuery::getTableNameAndAlias()
Parts of my Application
User getUserBadges method in User model
public function getUserBadges(){
return $this->hasMany(UserBadge::className(), ['user_id' => 'id']);
}
UserBadge model extending yii\mongodb\ActiveRecord
class UserBadge extends ActiveRecord
{
public static function collectionName()
{
return 'user_badge';
}
public function attributes()
{
return ['_id', 'user_id', 'badge_id', 'date'];
}
public function getUser(){
return $this->hasOne(User::className(), ['id' => 'user_id']);
}
public function getBadge(){
return $this->hasOne(Badge::className(), ['id' => 'badge_id']);
}
}
My query
$query = User::find()->joinWith(['userBadges']);
Edit: I figured out that the previous query is not really what I want, I simplified it to be clear but the real query that I want to do and you will get the point of why I am doing all of this is
$query = User::find()->joinWith(['userBadges'])->where(['badge_id' => 1]);
And with that I can get users from the user table who have a certain badge with id for example 1.
And here the code fails and throws the error stated above. After inspecting for sometime I found the API for the joinWith method
This method allows you to reuse existing relation definitions to perform JOIN queries. Based on the definition of the specified relation(s), the method will append one or multiple JOIN statements to the current query.
And here I knew that it's normal for this error to occur, In my query I am joining a document in a collection of the MongoDB database not a record in a table in a SQL database which definitely wouldn't work. I got stuck here and don't know what to exactly do, I am sticking to have user table in a SQL database and having the user_badge collection in a no-SQL database, what shall I do in such scenario? query on the no-SQL first and then query a SQL query against the result of the former query? or there is already a solution to such a problem in the methods of AcitveQuery? Or my Database structure is invalid?
Thanks in advance.
So after some good time I knew how to do it with the help of this question, where a SQL query is made against a PHP array.
So, first MongoDB will be queried and the results will be stored in an array, then A MariaDB SQL query will be made against the array generated from former query, I am pretty sure that this is not the best option; what if the result of the MongoDB query 100,000? well an array will be made with 100,000 entries, the SQL query will be made using also that 100,000 item array. Yet this is the best answer I could get (until now).
How to implement it in Yii2
// This line query from the MongoDB database and format the data returned well
$userBadges = UserBadge::find()->select(['user_id'])->where(['badge_id' => 1])->column();
// This line make the SQL query using the array generated from the former line
$userQuery = User::find()->where(['id' => $userBadges]);
I hope there can be a better answer for this question that someone can know, But I thought of sharing what I have reached so far.
Have a formatResults callback function that adds a "custom calculated" field into the entities post returned from a model query in my Cakephp. I would like to sort by this field and use this on a paginate is this possible?
So far i cannot accomplish this because the paginate limits the records fetched and therefore only records less than the paginator limit get sorted and not all the resultset...
Current code:
$owners = $this->Owners->find('all');
$owners->formatResults(function (\Cake\Collection\CollectionInterface $owners) {
$owners = $owners->map(function ($entity) {
$entity->random = rand(0,1);
return $entity;
});
return $owners->sortBy(function ($item){
return $item->random;
},SORT_DESC);
});
This works as expected:
$owners->toArray();
This does not:
$owners = $this->paginate($owners);
$owners->toArray();
Mainly because its "callback processing" only the first 10 records, i would like to process the whole resultset.
After diggin around ive found a similar topic opened by a previous user on the this link, it seems that is not possible to use pagination sort in other than the fields in the database.
As a result, i would suggest:
1 - Either alter your model logic, to accommodate your requirements by creating virtual fields or alter database schema to include this data.
2 - If the data requires further or live processing and it cannot be added or calculated in the database, perhaps programming a component that will replicate the paginate functionality on a cakephp collection would be a good option.The downside of this approach is that all records will be returned from the database which may present performance issues on large resultsets.
The scenario:
I am building a search system for a given set of data. Most of this is straight forward - where record title contains X, where record date before Y, etc. Where I am running into difficulty is essentially a category search. Each record belongs to zero or more categories (a relationship pivot table exists such that each row contains a record and a category), and when the user searches for Category A, I want to return all of the records that belong to that category. I've gotten this working with a whereHas, but it seems inordinately slow. In this instance, assume $category is a numeric id that is correctly validated as a category, and $records is an eloquent query builder that has not yet been executed by a get, pagination, all, etc (my function checks to see if several $request->input parameters are defined, then attaches a where to $record as required by the specified parameters, only executing it after all parameters have been considered):
if(!empty($category)) {
$records = $records->whereHas('categories', function($query) use ($category)
{
$query->where('category_id', $category);
});
}
This works, but as there are 7000+ records, 7000+ relationships defined in the pivot, and roughly 30 'categories', the search takes longer than I am comfortable leaving it. My unconfirmed thought is that the where query is executing for every record, thus leading to hundreds or thousands of queries.
I've debated approaching this using the raw query builder and just passing the list of record id's that have that category and using a simple where to filter the record collection before it's executed, but it seems counter-intuitive, leading me to believe there must be a better way.
The Question
How do I efficiently limit the records returned by $records->get() to those records with a defined relationship to category $category.
Edit 2018-01-16
To clarify, while I could simply do $category->records to return all records belonging to a category, this is part of a larger search engine. The full structure of the code looks like this:
If($subject_search_term) {
$records->where('subject', $subject_search_term)
}
If(some other search criteria is defined) {
$records->where(someothercriteria);
}
If(Category search criteria is defined) {
$records->whereHas(something);
}
$records->paginate(20);
Furthermore, there are two of these many-to-many relationships that I need to query (in addition to 'categories', lets say there is also a 'subject' that is independent of it, but similar structure and idea). As far as I know, I need to build the query off records and filter it accordingly.
EDIT 2
For anyone else with this problem, it seems the vastly more efficient way (and the most efficient that I've found) is Joel Hinz's comment - use the DB facade to build a raw query, pluck the id's from it, and use that in a whereIn clause.
Try this:
mpyw/eloquent-has-by-non-dependent-subquery: Convert has() and whereHas() constraints to non-dependent subqueries.
mpyw/eloquent-has-by-join: Convert has() and whereHas() constraints to join() ones for single-result relations.
if(!empty($category)) {
$records = $records->hasByNonDependentSubquery('categories', function($query) use ($category)
{
$query->where('category_id', $category);
});
}
That's all. Happy Eloquent Life!
Have I got it correctly that when I query a Laravel collection, it doesn't query the database but executes the query on what was already fetched?
For example, I have a relation that returns a collection:
public function permissions()
{
return $this->belongsToMany(Permission::class, RolePermission::getModelTable(), 'role_id', 'permission_id');
}
Does the following code query the database or it works with the collection using php facilities?
$role->permissions->where('code','global.test')->count()
And, as far as I understand, if I query the relationship then the database will be queried instead of working with the results that were already fetched:
$role->permissions()->where('code','global.test')->count()
So basically, $role->permissions - working with the fetched results "offline", but $role->permissions() - querying the database
What way is generally more efficient and when?
You're basically right. The difference between calling $role->permissions and $role->permissions() is that the first returns an instance of Collection, while the second returns an instance of BelongsToMany.
Collection is a collection (really?) of related objects and BelongsToMany is the relation itself. So yes, by calling the method (and not the magic property) you are querying the database.
Update
I didn't get the last question, sorry.
The first time you call $role->permissions (magic property), Laravel fetches all of the permissions associated with $role, if they were not eager loaded. If you need only a subset of those permissions, you can filter them by using any of the magic property and the method. Let me do some examples.
$role = Role::first();
// Fetch all the permissions and count a subset.
$role->permissions->where('code', 'global.test')->count();
// Count another subset.
$role->permissions->where('code', 'another.test')->count();
The same can be done using the method:
$role = Role::first();
// Fetch a subset of the permissions and count it.
$role->permissions()->where('code', 'global.test')->count();
// Fetch another subset and count it.
$role->permissions()->where('code', 'another.test')->count();
As you can see, in the first example you make only one query and filter the results differently. In the second example you make two queries. The first one is obviously more efficient.
If you need only a subset during the same execution, though, things change. Here we are using eager loading:
$role = Role::with('permissions', function($query) {
// Here we filter the related permissions.
$query->where('code', 'global.test');
})->first();
// We have what we want. No need to filter the collection.
$role->permissions->count();
// Let's do something else with this subset.
$role->permissions->all();
What if you fetch all the related objects, but need only that subset?
$role = Role::first();
// Filter the collection to count the needed subset.
$role->permissions->where('code', 'global.test')->count();
// Filter the collection to get the needed subset.
$role->permissions->where('code', 'global.test')->all();
As you can see, in the second example we are much less DRY, and we are also doing the same operation multiple times. Less efficient, of course.
is there a way for Eloquent/raw queries to execute a function before a query is fired? It would also be nice if I could extend the functionality to pass a parameter if the function should be run before or not. Depending on the outcome of the function (true/false) the query shouldn't be executed.
I would be nice to use the principal of "DB::listen", but I'm not sure if I can stop or run the query from within this function.
The reason for this is that I would like to build a little data warehouse myself for permanently saving results to a warehouse (db) and not query a huge database all the time.
The method I'm would like to use is to create a hash of a query, check if the hash exists in the warehouse. If it exists, then the value is returned. If not the query is executed and the output is saved together with the hash into the warehouse.
Any ideas?
///// EDIT /////
I should clarify, that I would like to access the queries and update the value if the calculated value needs to be updated. i.e.: Number of cars in december: While I'm in december, I need to keep updating the value every so often. So I store the executed query in the db and just retrieve it, run it and then update the value.
//// EDIT 2 /////
Github: https://github.com/khwerhahn/datawarehouselibrary/blob/master/DataWareHouseLib.php
What I would like to achieve is to hook into Laravels query/Eloquent logic and use the data warehouse logic in the background.
Maybe something like this:
$invalid_until = '2014-12-31 23:59:59'; // date until query needs to be updated every ten minutes
$cars = Cars::where('sales_month', '=', 12)->dw($invalid_until)->get();
If the dw($date_parameter) is added I would like Laravel to execute the data warehouse logic in the background and if found in the db then not execute the query again.
You don't need to use events to accomplish this. From the 4.2 docs:
Caching Queries
You may easily cache the results of a query using the remember method:
$users = DB::table('users')->remember(10)->get();
In this example, the results of the query will be cached for ten
minutes. While the results are cached, the query will not be run
against the database, and the results will be loaded from the default
cache driver specified for your application.
http://laravel.com/docs/4.2/queries#caching-queries
You can also use this for Eloquent objects,
eg: User::where($condition)->remember(60)->get()
I get what you're trying to do, but as I view it (I might not still be getting it right, though) you still can get away with using rememberForever() (if you don't want a specific time limit)
So, let's pretend you have a Cars table.
$cars = Cars::where('sales_month', '=', 12)->rememberForever()->get();
To work around the problem of deleting the cache, you can assign a key to the caching method, and then retrievit by that key. Now, the above query becomes:
$cars = Cars::where('sales_month', '=', 12)->rememberForever('cars')->get();
Every time you run that query you will be getting the same results, first time from the DB, all the others from the cache.
Now you say you're going to update the table, and you want to reset the cache, right?
So, run your update, then forget() the Cache with the cars index:
// Update query
Cache::forget('cars');
Your next call to the Cars query will issue a new resultset, and it will be cached. In case you're wondering, the remember() and rememberForever() are methods of the QueryBuilder class that use the same Cache class you can see in the docs in its own section.
Alternatively, in fact, you could also use the Cache class directly (it gives you a better control):
if (null == $cars= Cache::get('cars')) {
$cars = Cars::where('sales_month', '=', 12)->get();
Cache::forever('cars', $cars);
}
By overriding the method runSelect exsists in Illuminate\Database\Query\Builder that runs in every select query in Laravel.
see this package:
https://github.com/TheGeekyM/caching-queries