Is it possible to have an eloquent query builder return StdClass rather then Model?
For example User::where('age', '>', 34)->get() returns a Collection of User models.
Whereas DB::table('users')->where('age', '>', 34)->get() returns a Collection of StdClass objects. Much faster.
Therefore:
Is it possible to prevent hydrating eloquent models and return StdClass objects as a database query builder would, but still leverage the usefulness of an eloquent query builder syntax?
Yes, is possible using the 'getQuery' or 'toBase' method. For example:
User::where('age', '>', 34)->getQuery()->get();
or
User::where('age', '>', 34)->toBase()->get();
In my opinion,
Hydrating models rarely affects application performance
There are so many ORMs out there and if you look at any framework, these questions keep popping up - but the truth, as I've come to realize, is that ORMs hardly affect performance.
More often than not the culprits are the queries themselves and not
the ORM
Let me give you a few examples of why Eloquent models may perhaps be slower than DB facade queries:
1. Model events:
When you have model events (such as saving, creating, etc.) in your models, they sometimes slow down processing. Not to say that events should be avoided, you just need to be careful when and when not to use them
2. Loading Relationships:
Countless times have I seen folks load relationships using appends lists provided by Eloquent and sometimes models have 5-10 relationships. That's 5-10 joins each time you fire an Eloquent query! If you compare that with a DB facade query, it would definitely be faster. But then again, who's the real culprit? Not the ORM, it's the queries (with the extra joins!)
As an example, not so long someone asked a question on this and he/she wondered why an Eloquent query was slower than a raw one. Check it out!
3. Not understanding what triggers an Eloquent query
This is by far the most prominent reason why people think ORMs are slower. They usually (not always) don't understand what triggers a query.
As an example, lets say you want to update a products table and set the price of product #25 to $250.
Perhaps, you write in your controller, the following:
$id = 25;
$product = Product::findOrFail($id);
$product->price = 250;
$product->save();
Then, your colleague says hey, this is super slow. Try using DB facade. So you write:
$id = 25;
DB::table('products')->where('product_id', $id)->update(['price' => 250]);
And boom! It's faster. Again, the culprit isn't the ORM. It's the query. The one above is actually 2 queries, the findOrFail triggers a select * query and the save triggers an update query.
You can and should write this as a single query using Eloquent ORM like so:
Product::where('product_id', 25)->update(['price' => 250]);
Some Good Practices for Query Optimization
Have your database do most of the work instead of PHP: E.g. instead of iterating over Eloquent collections, perhaps frame your DB query in such a manner that the database does the work for you.
Mass Updates Over Single Updates: Pretty obvious. Avoid saving models in for loops, yuk!
For heavy queries, use transactions: DB transactions avoid re-indexing on every insert. If you really need to call say thousands of inserts/update queries in a single function call, wrap them into a transaction
Last but not the least, when in doubt check your query: If you're ever ever ever in doubt, that perhaps the ORM is the real culprit - think again! Check your query, try and optimize it.
If the ORM is slowing things down, use obervers or the Laravel debugbar to compare the queries with and without the ORM. More often than not, you'll find that the queries are different, and the difference isnt in hydration but the actual queries themselves!
It is inefficient to have small models and load them. What ->toBase() does is lowering the inefficiency. Memory inefficiency for 4-5 model persisted attributes with average length between 5 and 10 when loading model is 90+ percent. What is even more inefficient is having many and small models and as definition of hell - when there is a lot of traffic on them. Then you should think of another persistence design for that data. Wisely choose when model is the right home for a piece of data.
Related
The first sentence of the Eager Loading section from the Laravel docs is:
When accessing Eloquent relationships as properties, the relationship
data is "lazy loaded". This means the relationship data is not
actually loaded until you first access the property.
In the last paragraph of this section it is stated:
To load a relationship only when it has not already been loaded, use
the loadMissing method:
public function format(Book $book)
{
$book->loadMissing('author');
return [
'name' => $book->name,
'author' => $book->author->name
];
}
But I don't see the purpose of $book->loadMissing('author'). Is it doing anything here?
What would be the difference if I just remove this line? According to the first sentence, the author in $book->author->name would be lazy-loaded anyway, right?
Very good question; there are subtle differences which are not getting reflected instantly by reading through the documentation.
You are comparing "Lazy Eager Loading" using loadMissing() to "Lazy Loading" using magic properties on the model.
The only difference, as the name suggests, is that:
"Lazy loading" only happens upon the relation usage.
"Eager lazy loading" can happen before the usage.
So, practically, there's no difference unless you want to explicitly load the relation before its usage.
It also worths a note that both load and loadMissing methods give you the opportunity to customize the relation loading logic by passing a closure which is not an option when using magic properties.
$book->loadMissing(['author' => function (Builder $query) {
$query->where('approved', true);
}]);
Which translates to "Load missing approved author if not already loaded" which is not achievable using $book->author unless you define an approvedAuthor relation on the model (which is a better practice, though).
To answer your question directly; yeah, there won't be any difference if you remove:
$book->loadMissing('author');
in that particular example as it's being used right after the loading. However, there might be few use cases where one wants to load the relation before its being used.
So, to overview how relation loading methods work:
Eager loading
Through the usage of with() you can "eager load" relationships at the time you query the parent model:
$book = Book::with('author')->find($id);
Lazy eager loading
To eager load a relationship after the parent model has already been retrieved:
$book->load('author');
Which also might be used in a way to only eager load missing ones:
$book->loadMissing('author');
Contrary to the load() method, loadMissing() method filters through the given relations and lazily "eager" loads them only if not already loaded.
Through accepting closures, both methods support custom relation loading logics.
Lazy loading
Lazy loading which happens through the usage of magic properties, is there for developer's convenience. It loads the relation upon its usage, so that you won't be needing to load it beforehand.
#rzb has mentioned a very good point in his answer as well. Have a look.
I believe the accepted answer is missing one important fact that may mislead some: you cannot run loadMissing($relation) on a collection.
This is important because most use cases of lazy eager loading relationships are when you already have a collection and you don't want to commit the n+1 sin - i.e. unnecessarily hit the DB multiple times in a loop.
So while you can use load($relation) on a collection, if you only want to do it if the relationships haven't already been loaded before, you're out of luck.
its mean do not repeat the query
to be clear about it
if you use : load() 2 times the query will repeat even if the relationships exists
while : loadMissing() is check if the relationship has loaded . it will not repeat the query . beacuse it has already loaded before by [ load() or with() ] = egear load
DB::enableQueryLog();
$user = User::find(1);
// see the query
$user->load('posts');
$user->load('posts');
$user->loadMissing('posts'); // put it on top to see the difference
dd(DB::getQueryLog());
that's what i think its purpose
Very useful for APIs
The use of with, loadMissing or load can has more importance when use it in API environment, where the results are passed to json. On this case, lazy loading hasn't any effect.
Lets say you have multiple relationships.
book belongs to an author and book belongs to a publisher.
so first you might load it with one relationship.
$books->load('author');
and later on certain condition you want to load another relationship into it.
$book->loadMissing('publisher');
But I don't see the purpose of $book->loadMissing('author');. Is it
doing anything here? What would be the difference if I just remove
this line? According to the first sentence, the author in
$book->author->name would be lazy-loaded anyway, right?
Suppose say
public function format(Book $book)
{
//book will not have the author relationship yet
return [
'name' => $book->name, //book will not have the author relationship loaded yet
'author' => $book->author->name //book will now have the author relationship
];
}
Difference between above and below code is when will the relationship be loaded and how much control you have over the property.
public function format(Book $book)
{
$book->loadMissing('author'); // book will now have the author relationship
return [
'name' => $book->name, // book have the author relationship loaded
'author' => $book->author->name // book have the author relationship loaded
];
}
Both answers here have covered pretty well what the technical difference is, so I'd refer you to them first. But the "why" isn't very evident.
Something I find myself preaching a lot lately is that Eloquent is really good at giving you enough rope to hang yourself with. By abstracting the developer so far away from the actual SQL queries being produced, especially with dynamic properties, it's easy to forget when your database hits are hurting your performance more than they need to.
Here's the thing. One query using an IN() statement on 1000 values takes about the same execution time as one query running on one value. SQL is really good at what it does- the performance hit usually comes with opening and closing the DB connection. It's a bit like going grocery shopping by way of making one trip to the market for each item, as opposed to getting it all done at once. Eager-loads use the IN statements.
Lazy-loading is good for instances where you're handling too much data for your server's RAM to cope with, and in my opinion, not good for much else. It handles only one entry at any given moment. But it's reconnecting each time. I can't tell you the number of times I've seen Transformer classes, which should be responsible only for reformatting data as opposed to retrieving it, leveraging those dynamic properties and not realizing that the data wasn't already there. I've seen improvements as dramatic reducing execution time from 30 minutes to 30 seconds just by adding a single line of eager-loading prior to the Transformer being called.
(By the way, batching might be considered the happy-medium, and Eloquent's chunk() method offers that too.)
To answer your question a little more directly; if you're dealing with an instance where it's a one-to-one relationship, and it's going to be used in only one place, then functionally there is no difference between load, loadMissing, or a lazy-loading dynamic property. But if you have a many-to-many, it may be worthwhile to gather up all that data all at once. One book can have many co-authors. One author can write many books. And if you're about to loop through large sets of either, go ahead and make the most of your trip to the market before you start cooking.
I'm doing a web app here using Laravel + AngularJS and I have a question.Do I need a model for each table that I have in my database? There are 87 tables in my database and I need to query all of them according to with the input that the User wants.
I just want to make sure with all tables must have a model file or if just one is enough.
There are 2 ways by which you can access your DB tables:
Eloquent ORM (dependent on Models)
DB Facade Query Builder(independent on Models)
Former, is more clean and best approach to perform DB query and related task, whereas latter is not clean, and it is going to be difficult for you to manage large application, as you told there are 80+ tables in your application.
Also, if you're using Eloquent way, then it's also a better to have a base model, which will have common code which you can inherit in child models. Like if you want to store "user id" who did some DB changes, then in the boot function, you can write Auth::id() and assign that value to changed_by field on your table.
In DB Facade way, you've to hard code table name every time you're performing DB operation, and which leads to inconsistency when you found that you've to change the name of the table, it's a rare scenario still it'll be difficult to manage if in a file there are multiple tables DB operation going on. There are options like, creating a global table name variable which can be accessed to perform DB operation.
Conclusion:
Yes, creating 80+ model for implementing Eloquent way is painful, but for a short term, as the application grows it will be easy for you to manage, it will be good for other developer if they start working on it, as it will give a overview of DB and it will improves code readability.
It depends on how you'd like to handle queries.
If you'd like to use Eloquent ORM, you need model classes to handle objects and relationships. That is a model for a table, except intermediate relationship tables, which may be accessed through pivot attribute.
Raw SQL queries are also supported. You don't really need model classes for them, as each result within the result array will be a PHP StdClass object. You need to write raw SQL though.
See Laravel documentation.
I am working on a project that uses the ORM heavily instead of model relationships in the controller to get to the data (for eg: using leftJoins instead of establishing proper model relationships using hasMany etc and then retrieving through that)
My question is actually to do with the performance. Is using model relationships to access data faster than using leftJoins? I am not sure if Laravel actually does perform a leftJoin under the hood.
I am on a tight schedule so I want to decide if its actually worth trying to refactor the code to use the model relationships and if it would provide gains in performance?
Thanks in advance
Laravel anbd Eloquent may execure queries differently using the relationship as opposed to the join. You can use the barryvdh/laravel-debugbar to see query execution.
My observation is that Laravel will tend to use two queries instead of a join if possible. It will execute the first query and then using the IDs returned from that query execute a second query on the "joined" table as a "where myID in (1,2,3,4,5..)" and then stitch the results together.
As others have suggested, building up a quick test and seeing what works best in your environment would be the best bet. I prefer to use the relationships as much as possible becuase you can easily read what your code is doing.
Joins and relationship are not opposite of each other. Relationships are for ease of development, cleaner, more readable code, easy to maintain. Performance difference is almost non existant unless you are developing a data miner of some sort.
Edited
If you are focusing on performance, you can work on implementing caching in a efficient way.
Laravel's eloquent models are set to lazy load by default. The problem is that it makes a lot of query to the database and especially during high traffic, the laravel application crashes whereas a similar application build on Yii 1 has no issues.
After installing the Laravel's debug bar, the problem is too many queries being made on every page load. The next step is to query optimization. I have been using eager loading as directed in the Laravel's documentation but still too many queries.
I was wondering if there is a way to set Eloquent to only "Eager Load" in dev environment. That way when the page fails to load, identifying issue would be easier.
You could set defaults relations to "eager load" directly on the models :
Class MyModel extends Model {
protected $with = ['relation'];
}
The solution for high database load is Cache.
Caching properly could give you incredible performance during high traffic, because it reduces common database queries to zero, and redirect them to RAM ones, which are faster.
Enabling Route Caching will increase perfomance too:
php artisan route:cache
EDIT:
As Fx32 points, you should make sure that you need Eloquent and wouldn't be better to make the same query directly to the DB, joining the tables you need and making a single query instead of a lot:
Cache is not a good solution as a fix for bad database querying.
Eloquent is great, but often it's better to write proper queries with
some joins. Don't just bash away at your DB and then throw all the
results in a cache, as it will introduce new problems. If your use
case is not a flat CRUD API, ActiveRecord patterns might not be the
best solution anyway. If you carefully select and join results from
the DB, and want to speed up retrieval of such items, caching can
help.
I am working on an architecture redesign at my work and we've basically settled on a loosely-basic MVC custom solution. The intentions are to have the standard CRUD operations plus additional list operations defined in each of the models in our system.
Unfortunately about 30% of the code in our system uses complex joins and otherwise advanced querying that doesn't fit this model. Which is to say it could fit the model, but the list function would be huge and certainly error prone which is something we are trying to solve with the rewrite.
Given that, where would you place complex and very specific queries in such a system? We've been toying with a few options.
Add multiple versions of list/get in addition to the basic ones
Add in custom models for these queries that reside as siblings to the model directory
Don't use models in this situation and add the work directly in the action
We have outsourced help as well so we are attempting to keep it as simple as we can in terms of implementation and maintainability. ORM solutions or other heavyweights are out of the question.
Where would you want to see such things placed as a developer?
I apparently lack the privileges necessary to comment, so I'm posting this as answer...
Could you provide an example or two of the kinds of queries you have that don't fit into a model? Generally speaking: a good ORM will get you a long way, but some queries really are just too hairy to map easily, and if your team already has strong SQL skills the ORM can also seem like it's getting in the way.
First , all you're queries should stay in you're model .
Second , most of mvc frameworks provide more than just simple crud for you're database operations like a query functionality that where you can pass the query string , in this case you can build you're queryes manualy or with a query builder like for example Zend_Db_Table_Select and that handles multiple joins prety well . Or again if we look some place else than Zend let's say Codeigniter , it still provides a query builder for the model where you can add you're joins or build any other kind of complex queries .
That being sayd , it looks like you're base model class ( the one you extend each of you're models ) needs a query builder functionality , then you should be all good as you would be able to build any query you like inside any model you like .
I have similar issues in am MVC framework I've been building from scratch.
I don't particularly like the overhead of SELECT * on complex queries so I didn't build any of that functionality in whatsoever.
It's slower to code, but I code every query by hand in the relevant class (my model calls a Class 99% of the time).
For really complex queries shared amongst various routines, I have functions that return the generic joins and then concat the additional parameters for that particular query.
Example provided as requested:
private function returnFindClientRequests(){
$query = "SELECT
SR.sign_project_name, SR.module_signregister_id_pk
,SRI.module_signregister_sign_id_pk,SRI.sign_location_address
,SRR.status, SRR.module_signregister_item_client_request_id_pk, SRR.client_comment, SRR.requested_by_user, SRR.date_created
,SRR.admin_comment, SRR.date_actioned
,CL.client_name, CL.module_client_id_pk
FROM
`module_signregister` SR, `module_signregister_item` SRI, `module_signregister_item_client_request` SRR, `module_client` CL
WHERE
SR.module_signregister_id_pk = SRR.module_signregister_id_pk
AND SRR.module_signregister_sign_id_pk = SRI.module_signregister_sign_id_pk
AND SRR.requested_by_group = CL.module_client_id_pk
AND " . Database::groupQuery('CL');
return $query;
}
This query is shared amongst some other functions but also uses a call to Database::groupQuery() that us used to return session specific variables to many of the queries.
Models are workers - if you have 100 reports you're potentially going to need 100 models. Joins have nothing to do with MVC - how your data is addressed is another pattern altogether. If you're not using ORM and you're not using active records then all that's left is sending the SQL straight to the server via a model. Probably via a dedicated database class but the model will handle the query and its results.