I have Laravel cron issue ,In Console Kernel I have defined job which will hit Rollovercron.php file every 10 mins and every time it will hit it will pass one country. atleast 100 Countries are defined in an array and will be passed one by one to Rollovercron.php according to foreach loop. Rollovercron.php file takes minimum 2 hrs to run for single country.
I have multiple issues with this cron job:
100 elements in an array not getting fetched one by one means I can see 'GH' country(Ghana) has run for 5 times continuously and many of the countries are skipped.
when ever I get country missing issue I do composer update and clear cache frequently.
I want my cron should run smoothly and fetch all countries not even single country should miss and I should not need to do composer update for this all the time.
Please help me in this ,struggling for this since many months.
bellow is Kernel.php file:
<?php
namespace App\Console;
use Illuminate\Console\Scheduling\Schedule;
use Illuminate\Foundation\Console\Kernel as ConsoleKernel;
use DB;
class Kernel
extends ConsoleKernel
{
/**
* The Artisan commands provided by your application.
*
* #var array
*/
protected $commands = [
\App\Console\Commands\preAlert::class,
\App\Console\Commands\blCron::class,
\App\Console\Commands\mainRollover::class,
\App\Console\Commands\refilingSync::class,
\App\Console\Commands\TestCommand::class,
\App\Console\Commands\rollOverCron::class,
\App\Console\Commands\FrontPageRedis::class,
\App\Console\Commands\filingStatusRejectionQueue::class,
\App\Console\Commands\VesselDashboardRedis::class,
\App\Console\Commands\Bookingcountupdate::class,
// \App\Console\Commands\Voyagetwovisit::class,
];
/**
* Define the application's command schedule.
*
* #param \Illuminate\Console\Scheduling\Schedule $schedule
* #return void
*/
protected function schedule(Schedule $schedule)
{
$countrylist=array('NL','AR','CL','EC','DE','PH','ID','TT','JM','KR','BE','VN','US','BR','CM','MG','ZA','MU','RU','DO','GT','HN','SV', 'PR','SN', 'TN', 'SI','CI','CR','GM','GN','GY','HR','LC','LR','MR','UY','KH','BD','TH','JP','MM','AT','IE','CH','LB','PY','KE','YT','TZ','MZ','NA','GQ','ME');
foreach ($countrylist as $country) {
$schedule->command('rollOverCron:send ' . $country)
->everyTenMinutes()
->withoutOverlapping();
}
foreach ($countrylist as $country) {
$schedule->command('mainRollover:send ' . $country)
->daily()
->withoutOverlapping();
}
$schedule->command('filingStatusRejectionQueue')
->hourly()
->withoutOverlapping();
$schedule->command('Bookingcountupdate')
->everyTenMinutes()
->withoutOverlapping();
$schedule->command('preAlert')
->hourly()
->withoutOverlapping();
}
protected function commands()
{
require base_path('routes/console.php');
}
}
/**
* Register the Closure based commands for the application.
*
* #return void
*/
Laravel scheduling, knowing how it works helps, so you can debug it when it doesn't work as expected. This does involve diving in the source.
You invoke command on the scheduler, this returns an event.
Let's check how Laravel decides what defines overlapping, we see it expires after 1440 minutes, aka 24 hours.
So after one day, if the scheduled items have not run these scheduled items just stop being scheduled.
We see that a mutex is being used here. Let's see where it comes from. It seems it's provided in the constructor.
So lets see which mutex is being provided. In the exec and the call functions the mutex defined in the Scheduler constructor is used.
The mutex used there is an interface, probably used as a Facade, and the real implementation is most likely in CacheSchedulingMutex, which creates a mutex id using the mutexName from the event and the current time in hours and minutes.
Looking at the mutexName we see that the id exists out of the expression and command combined.
To summarise, all events called in one Scheduler function, share the same mutex that is used in checking if method calls don't overlap, but the mutex generates an unique identifier for each command, including differing parameters, and based on the time.
Your scheduled jobs will expire after 24 hours, which means that with jobs that take 2 hours to complete, you'll get about 12 jobs in a day completed. More if the jobs are small, less if the jobs take longer. This is because PHP is a single threaded process by default.
First task 1, then task 2, then task 3. Etc... This means that if each tasks takes 2 hours, then after 12 tasks their queued jobs expire because the job has been running for 1440 minutes and then the new jobs are scheduled and it starts again from the top.
Luckily there is a way to make sure they run simultaneously.
I suggest you add ->runInBackground() to your scheduling calls.
$schedule->command('rollOverCron:send ' . $country)
->everyTenMinutes()
->withoutOverlapping()
->runInBackground()
->emailOutputTo(['ext.amourya#cma-cgm.com','EXT.KKURANKAR#cma-cgm.com']);cgm.com']);
}
It seems that only everyMinute() and cron('* * * * *') are working for me. Any other methods like everyFiveMinutes, everyTenMinutes, daily, dailyAt etc, aren't working at all and always return "No scheduled commands are ready to run". My cron entry is always * * * * * so the other methods should work as well right? And yes; I've actually tried waiting for the other methods including daily, excluding yearly :P
Cron entry: * * * * * /opt/alt/php72/usr/bin/php /home/retracted/domains/retracted/artisan schedule:run >> /dev/null 2>&1
Schedule entry:
$schedule->call(function () {
$stat = new Stat();
$stat->users = User::count();
$stat->reviews = Review::count();
$stat->scholen = School::count();
$stat->save();
})->daily();
So my questions: Why don't the other methods work? How do I make the other methods work, especially daily()?
So first you have to run your cronjob every minute that is correct. With that line you run your Laravel scheduler.
So i don't know the scheduler code but it's possible that the code runs only on that minute and not backwards.
So if you need a 5 minute cronjob you have to run your scheduler every minute and then define your duration in your scheduler task.
$schedule->call(function () {
$stat = new Stat();
$stat->users = User::count();
$stat->reviews = Review::count();
$stat->scholen = School::count();
$stat->save();
})->everyFiveMinutes();
So with the function ->everyFiveMinutes(); you can run the scheduler every five minutes.
For laravel custom cron jobs to work you have to do the following:
First setup an every minute cron by executing the command "crontab
-e" and adding the following line * * * * * php /var/www/html/crontutorial/artisan schedule:run >> /dev/null 2>&1
Configure the appropriate timezone on app/config.php eg 'timezone'
=> 'Europe/Berlin',
Create a custom command that you want to execute at a specific time.
If you don't know how to create custom commands please have a look
at laravel cronjob scheduling tutorial
Schedule custom crons in app/Console/Kernel.php by adding the
following lines of code
protected function schedule(Schedule $schedule)
{
$schedule->command('my:customcommand')
->cron('01 13 * * *');
}
The cron will run every day 1.13pm using the timezone configuration in app/config.php.
Did you try specifying with the timezone? Check the below working snippet from my project:
protected function schedule(Schedule $schedule)
{
try{
$schedule->call(function (){
(new MTSnapshot)->createSnapshot();
})->timezone('Asia/Kolkata')->dailyAt('23:57');
$schedule->call(function (){
(new MTTransaction)->schedulerStatus();
})->hourly();
$schedule->call(function (){
(new MTTransaction)->syncPendingTransactions();
(new MTCommission)->processPendingCommissions();
})->twiceDaily(1, 16);
} catch(\Throwable $t){
Log::emergency($t);
}
}
I want to create a queue (AMAZON SQS) that only runs jobs every X sec. So if suddenly 50 jobs are submitted, the end up in the queue. The queue listener then pulls a job, does something and waits X sec. After that, the next job is pulled. Another X sec pause. Etc etc
For the queue listener, the sleep option option only determines how long the worker will "sleep" if there are no new jobs available. So it will only sleep if there is nothing in the queue.
Or should I just put in a pause(x) in my PHP code?
[edit] I just tested the sleep method with a FIFO and standard AWS SQS queue and this messes up the whole queue. Suddenly jobs are (sucesssfully) resubmitted 3 times after which the go into failed state. Moreover, the delay that is given in my code (3-4 min) was ignored, instead a one minute was taken
<?php
namespace App\Jobs;
use App\City;
class RetrieveStations extends Job
{
protected $cities;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct ($cities)
{
$this->cities = $cities;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
// code here
doSomething()
sleep(X);
}
}
I have the exact same problem to solve. I'm using Laravel 5.8 and I don't see how I can get the queue worker to wait a fixed period between jobs.
I'm now thinking of using a scheduled task to handle this. I can schedule a task to run, say, every 5 minutes and run the following artisan command:
$schedule->command('queue:work --queue=emails --once')->everyFiveMinutes();
This will take one job from the queue and run it. Unfortunately, there's not much more granular control over how often a job is processed.
Exactly, you need to set asleep your php code, there is no other way.
Php sleep
I am trying to configure laravel task schedular , I am priting numbers from 0 to infinity to just test why withoutoverlapping() isn't working.
My code :
protected function schedule(Schedule $schedule)
{
$schedule->call(function (Request $request)
{
$i=0;
while(1)
{
echo $i."-";
sleep(3);
$i++;
}
})->everyMinute()->name('mailfetch')->withoutOverlapping();
}
If my schedular is running and I am trying to run another schedular then that should not execute, but in my case both schedulars start running and start priting data.
Output:
Everything seems to be correct but dont' know why is that happening.
It will not overlap only if you launch them in the same minute. Because, you said to your process: "start an infinite loop every minute". So when you run your new instance, maybe you are in a new minute.
If you want to try something about overlapping, you could try:
$schedule->call(function (Request $request) {
echo 'start';
sleep(120);
echo '- end';
})->dailyAt('13:00')->withoutOverlapping();
I have a multithread, the main idea is to run nmap commands in the console and deliver the results in an orderly manner,
example:
Results after shell_exec
Command 4
Command 1
Command 2
Command 3
How can I get the results in an orderly manner?
Command 1
Command 2
Command 3
Command 4
public function __construct($arg) {
$this->arg = $arg;
}
public function run() {
$salida = shell_exec($comando);
}
`
If you're launching them in separate threads, the jobs are unlikely to finish in the same order that they were started. You'll need to track them and wait until all are done. You didn't include much of your code, but here's a generic example:
// create jobs
$jobs[0] = new nmapJob(args0);
$jobs[1] = new nmapJob(args1);
...
// start jobs
foreach ($jobs as $job)
{
$job.start();
}
// wait for jobs to finish
foreach ($jobs as $job)
{
$job.join();
}
// display results
foreach ($jobs as $job)
{
echo($job.salida);
}
But... I suggest using a different technique. Having a shell command dangle like that isn't the best of practices, especially if it can take a while to run (as nmap jobs often do). It's more complicated, but look into running the scans asynchronously. Spawn them as a separate process and have the results dumped into a directory. A different PHP script can be used to process the results in that directory once the scans are done.