I have a Product model with id,name,price.
The price value is stored in an external API and i need to fetch it every minute in order to update it in the database.
Looking through the Laravel documentation I found two ways to implement:
Create an artisan command (https://laravel.com/docs/8.x/artisan) and add it to task scheduling (https://laravel.com/docs/8.x/scheduling#scheduling-artisan-commands)
Create a job (https://laravel.com/docs/8.x/queues) and add it to task scheduling (https://laravel.com/docs/8.x/scheduling#scheduling-artisan-commands)
First of all, is there any other approach i should take in consideration?
If not, which one of the above would be the best approach and why is it correct for my use case?
As per my comments on one of your previous questions on this topic, whether you use a queue or not depends on your use case.
An Artisan command is a process that executes once and performs a task or tasks and then exits when that task is complete. It is generally run from the command line rather than through a user action. You can then use the task scheduling of your command's host operating system (e.g. a CRON job) to execute that command periodically. It will faithfully execute it when you schedule it to be done.
A Queued job will execute when the Job turns up next in the queue, in priority order. Let's say you send your API call (from your other post) to the queue to be processed. Another system then decides it needs to send out emails urgently (with a higher priority). Suddenly, your Job, which was next, is now waiting for 2000 other Jobs to finish (which might take a half hour). Then, you're no longer receiving new data until your Job executes.
With a scheduled job, you have a time critical system in place. With queues, you have a "when I get to it" approach.
Hope this makes the difference clearer.
With laravel it is a lot easy to use the built in scheduler. You have to add only one entry to the crontab and that is to run the command php artisan schedule:run EVERY MINUTE on your project. After that you dont have to thing about configuring the crontab on the server, you just add commands to the laravel scheduler and they will work as expected.
You should probably use Cron Job Task Scheduling which would be the first approach you mentioned.
Commonly for this type of use-cases commands are the easiest and cleanest approach.
There are a few things to do in order to make it work as expected:
Create a new command that will need to take care of hitting the endpoint and storing the retrieved data to the database
In Kernel.php file register your command and the frequency of running (each minute)
Run php artisan schedule:run
You can read more about how to create it here:
Related
I have a cron job that run every 5 Hours. It calls a PHP script , this script will do a call to an external API to sync some data.
The problem is sometimes I'm getting timeout from the API and the job will fail.
Are there any mechanisms to let cron tab do auto retry or auto recover the jobs that are failed?
I have tried to do an extra job and call it in case of any failures manually.
What is the best approach to do so?
Cron does only run once at specific time or every minutes/hours/days etc. It doesn't check the return code. So it's not that easy peasy lemon squeezy at all...
In my opinion you have a few options how to do it:
Create a some kind of scheduler where you can write your CRON job again if it fails, in this case you will need one more CRON job to read you scheduler and run proper command. Scheduler can be database / file / NoSQL based. In scheduler you can have flag like (bool) executed which will let scheduler know which tasks are already done.
Use queues (f.ex. Rabbit) to call it self again when fail.
Use framework, I'm using Symfony to manage own created commands to execute them (check second link below) based on database, using also enqueue/enqueue-bundle package to manage queues in Symfony.
I think if you are not so advanced with PHP I'd recommend to go for self made scheduler based on database (MySQL / PostgreSQL / NoSQL) with Symfony (check second link below). In this case you just have to SELECT all non executed record (commands) from database and just run them.
Lecture
Laravel - Queues, retrying failed jobs
Symfony - calling another commands in command
Queues package for PHP (incl. Symfony)
enqueue/enqueue-bundle
What you can do is something like this:
https://crontab.guru/#1_0-23_13__
1 0-23 13 * *
Start the job past 1 minute at every hour on the 13th of each month.
“At minute 1 past every hour from 0 through 23 on day-of-month 13.”
...then in your code you'd have some logic to detect if the process\script already ran correclty... if yes, skip the run attempt; otherwise let it run and then have a flag set to check against on the subsequent attempt run.
Hope you get the idea.
You can use supervisord :
supervisord website
Or handle API timeout in code.
I have a setting section in my laravel application. In this section I have a setting for defining a sync time. So the user can set their preferred time for sync data from laravel application to a third party CRM(Salesforce).
For now, I am storing a sync time into Database table. Now I want to run a cron JOB at that time for a particular user.
I have already created a cron job script and the cron job script is working fine. I am able to test cron job manually I just need to automate it with user setting(Preferred time).
My Cron JOB URL https://my-domain.org/cronjob?user_id=101
I tried to use laravel scheduling but this not fulfillig my requirement.
Is there any other better solution available at laravel?
There is some information missing but I try to assume a few things to hopefully propose a working solution using the Laravel scheduler.
Assuming you store the time at which the CRON runs in the database as 01:00 for 1 o'clock in the night you could do the following:
protected function schedule(Schedule $schedule)
{
User::query()->whereNotNull('cron_time')->each(function (User $user) use ($schedule) {
$schedule->command('app:import', ['user_id' => $user->id])
->withoutOverlapping() // prevents running the same command (+ parameters) twice at the same time, should only be a problem if the command can run > 24 hours, but still a safe thing to do
->dailyAt($user->cron_time);
});
}
I dreamed up that you stored the information on the User model
That you used a flag in the database called cron_time which contains the 01:00 or is null if the cron is disabled
The cron needs to run daily at that user supplied time
The import command is called app:import and is a console command that accept a user_id argument with the user id to run the import for
This might help you adapt the above to how it's structured in you own app.
If you want to use queued jobs to execute the imports you could also create the app:import CLI command to just dispatch that job instead of actually running the import. Depending on the amount of imports it might be a good idea to prevent long running scheduler commands and have the ability to retry and have a timeout.
Another option could also be to have a daily scheduled command that dispatches jobs to the queue scheduled to run at a specific time (which does not work with the SES driver). For more about that see: https://laravel.com/docs/5.8/queues#delayed-dispatching.
I need to run certain commands after 2 days of another time that I read from database in laravel. I checked out the laravel's scheduler documentation but could not find anything related my problem.
How can I set up a scheduler relative to another time? I want to read a time/date from database and schedule a command based on this time. Also, It is not repetitive. It will only run once.
I think the best you can do is to setup a queued job. When you need to dispatch it (whenever it is in a scheduled task or not, it doesn't matter), you just dispatch it with a delay. It is well explained in the queue section of the documentation, right here: https://laravel.com/docs/5.7/queues#delayed-dispatching
In your case, you can dispatch the job using this Carbon instance as a delay: now()->addDays(2). Two days later, the job will be executed :)
I want to implement a queue for sending out emails in Laravel. I have the queue working fine, but am worried about efficiency. These are my settings:
I have created the jobs table and set up the .env file, to use the queues with my local database.
I have set up this crontab on the server:
* * * * * php /var/www/imagine.dev/artisan schedule:run >> /dev/null 2>&1
And have set up a schedule in app\Conosle\Kernel.php, so I dont have to manually enter the 'queue:listen' every time through console.
$schedule->command('queue:listen');
Now to my question. I would like to know if this is efficient? I am worried about having the queue:listen running all the time in the background consuming cpu and memory.
I have been trying to only run the queue:listen once every 5 minutes, and then put it to sleep with
$schedule->command('queue:listen --sleep 300');
but again, am not sure if this is the best approach.
Another thing I tried is using 'queue:work', but this only processes one queue at a time.
Ideally, I would like a way, to process all the queues every 5 minutes, avoiding a constant use of memory and cpu.
What is the best approach?
Not sure which version of Laravel you're using, but I suspect it's 5.2 or earlier.
You do not need to run this every minute, it continues to run until it's manually stopped.
From Laravel 5.2 documentation:
Note that once this task has started, it will continue to run until it is manually stopped. You may use a process monitor such as Supervisor to ensure that the queue listener does not stop running.
So maybe you want to look into Supervisor
Also, if this is helpful at all, you can chain onto $schedule, ->everyFiveMinutes(). There are several other methods available as well. Laravel Scheduling
I'm in need of a way to create dynamic, one-off cron jobs to execute tasks at different times. Ideally, I would like to achieve this using PHP, so that when a user completes a certain action, the cron job is created and scheduled for a time that is calculated based on the time that the user completes said action. At any one time, there could be multiple cron jobs scheduled at once for different times. These cron jobs also need to be deleted upon completion.
I have tried searching around for something appropriate, however haven't encountered anything that works as I need. If anybody could point me in the right direction, that would be greatly appreciated.
Cheers
A possible solution:
Setting a Cron job that calls to CronJobManager.php every second.
Just make a regular daemon that calls for the CronJobManager.php.
Create a cronjob table in your database
The cron job table should contain these basic fields: path (to php file), run_time(datetime), last run (datetime) and type (like suicidal, if as you explain you want some cron jobs to delete themselves)
Connect CronJobManager.php with the cronjob table
Every time CronJobManager.php runs (that is, every second), loads the cron jobs. Then, comparing "now"'s time with each cron job's run_time you'll get which cron jobs to run.
For example, if cron job "foo" run_time is set to 18/04/2014 22:02:01, CronJobManager will run it when reaching that moment.
Notice that if Cron jobs executing time needs a lot of time, they'll get delayed and eventually a second or two will get lost.
Now, for every cron job that needs an execution, you would execute the related php file of that cron job, indicated in the path.
This is a general idea, and of course you would have to extend it with for example cron job states (idle, running, stop, etc).
In order to delete cron jobs you would implement this feature in the cron job object.
That is: the Cron Job class, once it has executed what it had to do, it would check its type (as defined in database). If it is 'suicidal', then it would delete the database row.
UPDATE
I updated the answer but I want to note something. If what you need is several cron jobs to run at once, in a specific second with 0 delay, then you need a cron job per task out of php that runs a specific file.
To achieve this functionality, you need to have a daemon that is running all the time that checks for these dynamic jobs and launches them. The setup is a tad complicated to put together, but if you are ready for such an endeavor, you can use the project PHP Resque Scheduler.
https://github.com/chrisboulton/php-resque-scheduler
You start up a daemon that runs all the time and then you can add jobs to a dynamic queue to be executed at any specified time in the future. I think you will find this suitable to everything you are looking to do.