Laravel's eloquent models are set to lazy load by default. The problem is that it makes a lot of query to the database and especially during high traffic, the laravel application crashes whereas a similar application build on Yii 1 has no issues.
After installing the Laravel's debug bar, the problem is too many queries being made on every page load. The next step is to query optimization. I have been using eager loading as directed in the Laravel's documentation but still too many queries.
I was wondering if there is a way to set Eloquent to only "Eager Load" in dev environment. That way when the page fails to load, identifying issue would be easier.
You could set defaults relations to "eager load" directly on the models :
Class MyModel extends Model {
protected $with = ['relation'];
}
The solution for high database load is Cache.
Caching properly could give you incredible performance during high traffic, because it reduces common database queries to zero, and redirect them to RAM ones, which are faster.
Enabling Route Caching will increase perfomance too:
php artisan route:cache
EDIT:
As Fx32 points, you should make sure that you need Eloquent and wouldn't be better to make the same query directly to the DB, joining the tables you need and making a single query instead of a lot:
Cache is not a good solution as a fix for bad database querying.
Eloquent is great, but often it's better to write proper queries with
some joins. Don't just bash away at your DB and then throw all the
results in a cache, as it will introduce new problems. If your use
case is not a flat CRUD API, ActiveRecord patterns might not be the
best solution anyway. If you carefully select and join results from
the DB, and want to speed up retrieval of such items, caching can
help.
Related
I'm developing a Laravel application & started using Redis as a caching system. I'm thinking of caching the data of all of a specific model I have, as a user may make an API request that this model is involved in quite often. Would a valid solution be storing each model in a hash, where the field is that record's unique ID, and the values are just the unique model's data, or is this use case too complicated for a simple key value database like Redis? I"m also curious as to how I would create model instances from the hash, when I retrieve all the data from it. Replies are appreciated!
Short answer: Yes, you can store a model, or collections, or basically anything in the key-value caching of Redis. As long as the key provided is unique and can be retraced. Redis could even be used as a primary database.
Long answer
Ultimately, I think it depends on the implementation. There is a lot of optimization that can be done before someone can/should consider caching all models. For "simple" records that involve large datasets, I would advise to first optimize your queries and code and check the results. Examples:
Select only data you need, not entire models.
Use the Database Query Builder for interacting with the database when targeting large records, rather than Eloquent (Eloquent is significantly slower due to the Active Record pattern).
Consider using the toBase() method. This retrieves all data but does not create the Eloquent model, saving precious resources.
Use tools like the Laravel debugbar to analyze and discover potential long query loads.
For large datasets that do not change often or optimization is not possible anymore: caching is the way to go!
There is no right answer here, but maybe this helps you on your way! There are plenty of packages that implement similar behaviour.
I am calculating basketball stats and have the models Stat, User (which the basketball players are held within), Team, Stat_Meta, Game, Season, Substitution.
I have a view called statTable that can be added to any other view on the app. statTable basically just iterates through each player on the team and retrieves the calculation for each stat type (found in Stat_Meta model). Within those calculation, there are queries run for the Stat, Game, Season, etc. tables. By the time it iterates through every player and all their stats, we are looking at like 500 queries PER game (often we are going through like ~30 queries/view, so you do the math, it's bad).
My question: With the Laravel debug bar installed, I can see that in my test environment, I've got 3,116 queries running when loading the front page, and 2,432 of them are duplicates. It takes forever to load as well. So, how can I re-work this system of queries to reduce the number of them?
Full disclosure, I'm not a CS person, so efficiency isn't something I'm trained in. Right now, I'm super happy this even works, but not it is going to cost me an arm and a leg to do all these queries at scale (not to mention horrible UX).
You could do some optimization of your queries by using Laravel's eager loading. Definition of eager loading from official documentation:
When accessing Eloquent relationships as properties, the relationship
data is "lazy loaded". This means the relationship data is not
actually loaded until you first access the property. However, Eloquent
can "eager load" relationships at the time you query the parent model.
Eager loading alleviates the N + 1 query problem.
You can read some great examples from the link I provided. I believe this will optimize your queries a lot.
Beside eager loading, you should always aim to accomplish as much as you can with your queries instead of processing data with PHP, Laravel collections, etc.
Is it possible to have an eloquent query builder return StdClass rather then Model?
For example User::where('age', '>', 34)->get() returns a Collection of User models.
Whereas DB::table('users')->where('age', '>', 34)->get() returns a Collection of StdClass objects. Much faster.
Therefore:
Is it possible to prevent hydrating eloquent models and return StdClass objects as a database query builder would, but still leverage the usefulness of an eloquent query builder syntax?
Yes, is possible using the 'getQuery' or 'toBase' method. For example:
User::where('age', '>', 34)->getQuery()->get();
or
User::where('age', '>', 34)->toBase()->get();
In my opinion,
Hydrating models rarely affects application performance
There are so many ORMs out there and if you look at any framework, these questions keep popping up - but the truth, as I've come to realize, is that ORMs hardly affect performance.
More often than not the culprits are the queries themselves and not
the ORM
Let me give you a few examples of why Eloquent models may perhaps be slower than DB facade queries:
1. Model events:
When you have model events (such as saving, creating, etc.) in your models, they sometimes slow down processing. Not to say that events should be avoided, you just need to be careful when and when not to use them
2. Loading Relationships:
Countless times have I seen folks load relationships using appends lists provided by Eloquent and sometimes models have 5-10 relationships. That's 5-10 joins each time you fire an Eloquent query! If you compare that with a DB facade query, it would definitely be faster. But then again, who's the real culprit? Not the ORM, it's the queries (with the extra joins!)
As an example, not so long someone asked a question on this and he/she wondered why an Eloquent query was slower than a raw one. Check it out!
3. Not understanding what triggers an Eloquent query
This is by far the most prominent reason why people think ORMs are slower. They usually (not always) don't understand what triggers a query.
As an example, lets say you want to update a products table and set the price of product #25 to $250.
Perhaps, you write in your controller, the following:
$id = 25;
$product = Product::findOrFail($id);
$product->price = 250;
$product->save();
Then, your colleague says hey, this is super slow. Try using DB facade. So you write:
$id = 25;
DB::table('products')->where('product_id', $id)->update(['price' => 250]);
And boom! It's faster. Again, the culprit isn't the ORM. It's the query. The one above is actually 2 queries, the findOrFail triggers a select * query and the save triggers an update query.
You can and should write this as a single query using Eloquent ORM like so:
Product::where('product_id', 25)->update(['price' => 250]);
Some Good Practices for Query Optimization
Have your database do most of the work instead of PHP: E.g. instead of iterating over Eloquent collections, perhaps frame your DB query in such a manner that the database does the work for you.
Mass Updates Over Single Updates: Pretty obvious. Avoid saving models in for loops, yuk!
For heavy queries, use transactions: DB transactions avoid re-indexing on every insert. If you really need to call say thousands of inserts/update queries in a single function call, wrap them into a transaction
Last but not the least, when in doubt check your query: If you're ever ever ever in doubt, that perhaps the ORM is the real culprit - think again! Check your query, try and optimize it.
If the ORM is slowing things down, use obervers or the Laravel debugbar to compare the queries with and without the ORM. More often than not, you'll find that the queries are different, and the difference isnt in hydration but the actual queries themselves!
It is inefficient to have small models and load them. What ->toBase() does is lowering the inefficiency. Memory inefficiency for 4-5 model persisted attributes with average length between 5 and 10 when loading model is 90+ percent. What is even more inefficient is having many and small models and as definition of hell - when there is a lot of traffic on them. Then you should think of another persistence design for that data. Wisely choose when model is the right home for a piece of data.
I remember reading something similar about Symfony's Doctrine, but I can't find anything about this in Zend 2 documentation.
Here is the question explained:
Let's say that in a single controller action I call two model functions (both in same model): both functions run exactly the same TableGateway query sets. These queries only SELECT data. Furthermore there are no INSERT/UPDATE operations anywhere in this action.
In this case, would Zend run the query sets twice? Or, seeing that they are duplicated and no INSERT/UPDATE operation is done in-between, it would run the query set just once, and the second time return it from some internal cache.
ps. Just in case, please understand that I don't need general best practice advice, just the specific answer from someone who knows the depths of Zend's core.
No. It would be bad to do that anyway as your application has no idea if another application has written to the database and invalidated your cached query.
If you have query cache enabled in my sql then the query could be cached as the rdbms knows if that data can be cached or if it's been changed.
I am working on a project that uses the ORM heavily instead of model relationships in the controller to get to the data (for eg: using leftJoins instead of establishing proper model relationships using hasMany etc and then retrieving through that)
My question is actually to do with the performance. Is using model relationships to access data faster than using leftJoins? I am not sure if Laravel actually does perform a leftJoin under the hood.
I am on a tight schedule so I want to decide if its actually worth trying to refactor the code to use the model relationships and if it would provide gains in performance?
Thanks in advance
Laravel anbd Eloquent may execure queries differently using the relationship as opposed to the join. You can use the barryvdh/laravel-debugbar to see query execution.
My observation is that Laravel will tend to use two queries instead of a join if possible. It will execute the first query and then using the IDs returned from that query execute a second query on the "joined" table as a "where myID in (1,2,3,4,5..)" and then stitch the results together.
As others have suggested, building up a quick test and seeing what works best in your environment would be the best bet. I prefer to use the relationships as much as possible becuase you can easily read what your code is doing.
Joins and relationship are not opposite of each other. Relationships are for ease of development, cleaner, more readable code, easy to maintain. Performance difference is almost non existant unless you are developing a data miner of some sort.
Edited
If you are focusing on performance, you can work on implementing caching in a efficient way.