Laravel - Execute function before query is fired - php

is there a way for Eloquent/raw queries to execute a function before a query is fired? It would also be nice if I could extend the functionality to pass a parameter if the function should be run before or not. Depending on the outcome of the function (true/false) the query shouldn't be executed.
I would be nice to use the principal of "DB::listen", but I'm not sure if I can stop or run the query from within this function.
The reason for this is that I would like to build a little data warehouse myself for permanently saving results to a warehouse (db) and not query a huge database all the time.
The method I'm would like to use is to create a hash of a query, check if the hash exists in the warehouse. If it exists, then the value is returned. If not the query is executed and the output is saved together with the hash into the warehouse.
Any ideas?
///// EDIT /////
I should clarify, that I would like to access the queries and update the value if the calculated value needs to be updated. i.e.: Number of cars in december: While I'm in december, I need to keep updating the value every so often. So I store the executed query in the db and just retrieve it, run it and then update the value.
//// EDIT 2 /////
Github: https://github.com/khwerhahn/datawarehouselibrary/blob/master/DataWareHouseLib.php
What I would like to achieve is to hook into Laravels query/Eloquent logic and use the data warehouse logic in the background.
Maybe something like this:
$invalid_until = '2014-12-31 23:59:59'; // date until query needs to be updated every ten minutes
$cars = Cars::where('sales_month', '=', 12)->dw($invalid_until)->get();
If the dw($date_parameter) is added I would like Laravel to execute the data warehouse logic in the background and if found in the db then not execute the query again.

You don't need to use events to accomplish this. From the 4.2 docs:
Caching Queries
You may easily cache the results of a query using the remember method:
$users = DB::table('users')->remember(10)->get();
In this example, the results of the query will be cached for ten
minutes. While the results are cached, the query will not be run
against the database, and the results will be loaded from the default
cache driver specified for your application.
http://laravel.com/docs/4.2/queries#caching-queries
You can also use this for Eloquent objects,
eg: User::where($condition)->remember(60)->get()

I get what you're trying to do, but as I view it (I might not still be getting it right, though) you still can get away with using rememberForever() (if you don't want a specific time limit)
So, let's pretend you have a Cars table.
$cars = Cars::where('sales_month', '=', 12)->rememberForever()->get();
To work around the problem of deleting the cache, you can assign a key to the caching method, and then retrievit by that key. Now, the above query becomes:
$cars = Cars::where('sales_month', '=', 12)->rememberForever('cars')->get();
Every time you run that query you will be getting the same results, first time from the DB, all the others from the cache.
Now you say you're going to update the table, and you want to reset the cache, right?
So, run your update, then forget() the Cache with the cars index:
// Update query
Cache::forget('cars');
Your next call to the Cars query will issue a new resultset, and it will be cached. In case you're wondering, the remember() and rememberForever() are methods of the QueryBuilder class that use the same Cache class you can see in the docs in its own section.
Alternatively, in fact, you could also use the Cache class directly (it gives you a better control):
if (null == $cars= Cache::get('cars')) {
$cars = Cars::where('sales_month', '=', 12)->get();
Cache::forever('cars', $cars);
}

By overriding the method runSelect exsists in Illuminate\Database\Query\Builder that runs in every select query in Laravel.
see this package:
https://github.com/TheGeekyM/caching-queries

Related

How to get latest record in Laravel 9 [duplicate]

I would like to retrieve the last file inserted into my table. I know that the method first() exists and provides you with the first file in the table but I don't know how to get the last insert.
You'll need to order by the same field you're ordering by now, but descending.
As an example, if you have a time stamp when the upload was done called upload_time, you'd do something like this;
For Pre-Laravel 4
return DB::table('files')->order_by('upload_time', 'desc')->first();
For Laravel 4 and onwards
return DB::table('files')->orderBy('upload_time', 'desc')->first();
For Laravel 5.7 and onwards
return DB::table('files')->latest('upload_time')->first();
This will order the rows in the files table by upload time, descending order, and take the first one. This will be the latest uploaded file.
Use the latest scope provided by Laravel out of the box.
Model::latest()->first();
That way you're not retrieving all the records. A nicer shortcut to orderBy.
You never mentioned whether you are using Eloquent, Laravel's default ORM or not. In case you are, let's say you want to get the latest entry of a User table, by created_at, you probably could do as follow:
User::orderBy('created_at', 'desc')->first();
First it orders users by created_at field, descendingly, and then it takes the first record of the result.
That will return you an instance of the User object, not a collection. Of course, to make use of this alternative, you got to have a User model, extending Eloquent class. This may sound a bit confusing, but it's really easy to get started and ORM can be really helpful.
For more information, check out the official documentation which is pretty rich and well detailed.
To get last record details
Model::all()->last(); or
Model::orderBy('id', 'desc')->first();
To get last record id
Model::all()->last()->id; or
Model::orderBy('id', 'desc')->first()->id;
Many answers and some where I don't quite agree. So I will summarise again with my comments.
In case you have just created a new object.
By default, when you create a new object, Laravel returns the new object.
$lastCreatedModel = $model->create($dataArray);
dd($lastCreatedModel); // will output the new output
echo $lastCreatedModel->key; // will output the value from the last created Object
Then there is the approach to combine the methods all() with (last()and first()) without a condition.
Very bad! Don't do that!
Model::get()->last();` // the most recent entry
Model::all()->last();` // the most recent entry
Model::get()->first();` // the oldest entry
Model::all()->first();` // the oldest entry
Which is basically the wrong approach! You get() all() the records, and in some cases that can be 200,000 or more, and then pick out just one row. Not good! Imagine your site is getting traffic from Facebook and then a query like that. In one month that would probably mean the CO² emissions of a city like Paris in a year. Because the servers have to work unnecessarily hard. So forget this approach and if you find it in your code, replace it/rewrite it. Maybe you don't notice it with 100 data sets but with 1000 and more it can be noticeable.
Very good would be:
Model::orderBy('id', 'desc')->last(); // the most recent record
Model::latest('id')->first(); // the most recent record
Model::latest('id')->limit(1)->get(); // the most recent record
Model::orderBy('id', 'desc')->limit(1)->get(); // the most recent entry
Model::orderBy('id', 'desc')->first(); // the most recent entry
Model::orderBy('id', 'asc')->first(); // the oldest entry
Model::orderBy('id', 'asc')->limit(1)->get(); // the oldest entry
Model::orderBy('id', 'asc')->first(); // the oldest entry
If orderBy is used in this context, the primarykey should always be used as a basis and not create_at.
Laravel collections has method last
Model::all() -> last(); // last element
Model::all() -> last() -> pluck('name'); // extract value from name field.
This is the best way to do it.
You can use the latest scope provided by Laravel with the field you would like to filter, let's say it'll be ordered by ID, then:
Model::latest('id')->first();
So in this way, you can avoid ordering by created_at field by default at Laravel.
Try this :
Model::latest()->get();
Don't use Model::latest()->first(); because if your collection has multiple rows created at the same timestamp (this will happen when you use database transaction DB::beginTransaction(); and DB::commit()) then the first row of the collection will be returned and obviously this will not be the last row.
Suppose row with id 11, 12, 13 are created using transaction then all of them will have the same timestamp so what you will get by Model::latest()->first(); is the row with id: 11.
To get the last record details, use the code below:
Model::where('field', 'value')->get()->last()
Another fancy way to do it in Laravel 6.x (Unsure but must work for 5.x aswell) :
DB::table('your_table')->get()->last();
You can access fields too :
DB::table('your_table')->get()->last()->id;
Honestly this was SO frustrating I almost had to go through the entire collection of answers here to find out that most of them weren't doing what I wanted. In fact I only wanted to display to the browser the following:
The last row ever created on my table
Just 1 resource
I wasn't looking to ordering a set of resources and order that list through in a descending fashion, the below line of code was what worked for me on a Laravel 8 project.
Model::latest()->limit(1)->get();
Use Model::where('user_id', $user_id)->latest()->get()->first();
it will return only one record, if not find, it will return null.
Hope this will help.
Model($where)->get()->last()->id
For laravel 8:
Model::orderBy('id', 'desc')->withTrashed()->take(1)->first()->id
The resulting sql query:
Model::orderBy('id', 'desc')->withTrashed()->take(1)->toSql()
select * from "timetables" order by "id" desc limit 1
If you are looking for the actual row that you just inserted with Laravel 3 and 4 when you perform a save or create action on a new model like:
$user->save();
-or-
$user = User::create(array('email' => 'example#gmail.com'));
then the inserted model instance will be returned and can be used for further action such as redirecting to the profile page of the user just created.
Looking for the last inserted record works on low volume system will work almost all of the time but if you ever have to inserts go in at the same time you can end up querying to find the wrong record. This can really become a problem in a transactional system where multiple tables need updated.
Somehow all the above doesn't seem to work for me in laravel 5.3,
so i solved my own problem using:
Model::where('user_id', '=', $user_id)->orderBy('created_at', 'desc')->get();
hope am able to bail someone out.
be aware that last(), latest() are not deterministic if you are looking for a sequential or event/ordered record. The last/recent records can have the exact same created_at timestamp, and which you get back is not deterministic. So do orderBy(id|foo)->first(). Other ideas/suggestions on how to be deterministic are welcome.
You just need to retrive data and reverse them you will get your desire record let i explain code for laravel 9
return DB::table('files')->orderBy('upload_time', 'desc')->first();
and if you want no. of x last result
return DB::table('files')->orderBy('upload_time', 'desc')->limit(x)->get();
If the table has date field, this(User::orderBy('created_at', 'desc')->first();) is the best solution, I think.
But there is no date field, Model ::orderBy('id', 'desc')->first()->id; is the best solution, I am sure.
you can use this functions using eloquent :
Model::latest()->take(1)->get();
With pdo we can get the last inserted id in the docs
PDO lastInserted
Process
use Illuminate\Support\Facades\DB;
// ...
$pdo = DB::getPdo();
$id = $pdo->lastInsertId();
echo $id;

question about laravel relationships and performance

i hope you are having a good time. i am learning laravel and the inscuctor talked about when you load relationships in laravel, like so
public function timeline()
{
$ids = $this->follows()->pluck('id');
$ids->push($this->id);
return Tweet::whereIn('user_id', $ids)->latest()->get();
}
and i have a follows relationship in my model, and he talked about this line
$ids = $this->follows()->pluck('id');
being better for performance than this line
$ids = $this->follows->pluck('id');
my question is, how does laravel pluck the ids in the first case, and how it queries the database
i hope im making sense, thanks for your time, and answer.
the following one executes a select query on database
$this->follows()->pluck('id');
the follows() returns a query builder (which is a not yet executed sql statement) and then on the result select the id column and returns a collection of ids
you can see the query by dumping the query builder by $this->follows()->dd()
Whereas in the second option
$this->follows->pluck('id')
up until $this->follows laravel executes a query and returns all the records as a collection instance, You will be able to see all the attributes on each of the records. And then ->pluck('id') is getting executed on the laravel collection class, which will do an operation I assume similar to the array_column function does and returns only the id column.
as you can easily see in the second operation the whole data set was retrieved first from the DB and then selected the required attribute/column (2 distinct and heavy operations). Where as in the first option we directly told eloquent to select only the required column, which is only one lighter operation compared to the second option.

Is there a way to invalidate / dirty Yii2 query cached record?

I am using latest (March 2016) Yii2's query caching mechanism in Models with Redis in a form:
$object = $db->cache(function ($db) use($id) {
return self::findOne($id);
});
As a result, an entry with GUID ID (e.g. "bb83d06878206d758eda3e29082dda4f") is set that holds the result of the query.
Is there a way to invalidate just that record (based on id) or the whole Model's table, every time Model's save method is invoked?
E.g. if a User record is saved, we want to dirty that User's record (or "user" table), so next time we fetch that user, cache is no longer valid and record is retrieved from DB.
If possible, I would like to avoid DbDependency (e.g. on "last_updated" field on the record), since that is another DB query, if I am not mistaken.
It seems that the way to go is to use TagDependency. With that tag you can invalidate the cached query when you see fit.
You create the cached query giving it a unique tag like this:
$object = $db->cache(function ($db) use($id) {
return self::findOne($id);
}, 0, new TagDependency(['tags' => 'myquerytag']));
Then when you want to invalidate it you can use the 'invalidate' static method of TagDependency like this:
TagDependency::invalidate(Yii::$app->cache, 'myquerytag');
Keep in mind that in this case I gave a cache expiration time of 0 for this query like in the documentation example, but you can give it any time you see fit.

Is there any CakePHP function that builds and executes the query without returning the result set

I'm using CakePHP 2.1.3. I have a performance problem of looping a large set of array data returned from find('all'). I want to retrieve a query result row by row to eliminate this expensive array processing. I don't want the result set of array returning from find() or query(). What I'm trying to do is like below:
$db = $this->Model->getDataSource();
$sql = 'SELECT * FROM my_table';
if($result = $db->execute($sql){
$db->resultSet($result);
while($row = $db->fetchResult()){
// do something with $row
}
}
However, I don't want to write the raw query. Is there any Cake function that just builds the query according to the association set and executes it without returning the result set?
[Edit]
I'm currently implementing the above script in controller. My model has no associations and so I don't need to use recursive = -1. It is the whole table fetching for the purpose of CSV export.
The Cake's find() has an internal array processing and the returned result set has to be looped again explicitly. I want to optimize the code by avoiding the array processing of large data twice.
Related issue: https://github.com/cakephp/cakephp/issues/6426
At first be sure, that you only fetch the data you really yreally need. Ideally you get everything you need with $this->YourModel->recursive = -1
Often performance problems arise due to many connected data.
When you have checked this I think a loop would be the best solution where you fetch the desired data in chunks via an between condition. Although I am not sure if this will help you.
Why do you want to go through the whole table? Do you perform some maintenance like e.g. filling a new field or updating a counter? Maybe you can achieve the goal better than by trying to fetch a whole table.

mass insert laravel for many records

I'm trying to retrieve data from sql table A, modify some columns then insert the modified columns into sql table B.
However my issue is that when I use:
$customer= new Customer;
$fakecustomer= new Fakecustomer;
$fake_customer_name_records = $fakecustomer->get()->toArray();
foreach($fake_customer_name_records as $record){
//process columns for each record
$fake_customer_name_records_arry[]=array(
'last_name'=>$last_name,
'first_name'=>$first_name,
'home_phone'=>$phonenumber,
);
}
$customer->insert($fake_customer_name_records_arry);
It can only insert around 1000 records. Is there a way in Laravel for me to process about 60,000 records?
Thanks
I would suggest to use the "chunk" option here, and process records in "chunks". It's more native way, to my opinion. Here's what docs say :
Chunking Results
If you need to process a lot (thousands) of Eloquent records, using
the chunk command will allow you to do without eating all of your RAM:
User::chunk(200, function($users)
{
foreach ($users as $user)
{
//
}
});
The first argument passed to the method is the number of records you
wish to receive per "chunk". The Closure passed as the second argument
will be called for each chunk that is pulled from the database.
Link to read more : click
Use an extra variable and sum 1 every iteration, when reaches 1000 (or lower) execute 'insert' and reset the counter.
Have you tried disabling the query log with DB::disableQueryLog(); ? I had the same problem and this prety much solved it.
Also, when working with migrations or some process that's going to take a lot of time try to create a command instead of trying to do it with a controller.

Categories