Laravel queue keep processing multiple times for a job - php

Below is what's happening when i run php artisan queue:listen and at my job table only have one job
and this is my code :
public function handle(Xero $xero)
{
$this->getAndCreateXeroSnapshotID();
$this->importInvoices($xero);
$this->importBankTransaction($xero);
$this->importBankStatement($xero);
$this->importBalanceSheet($xero);
$this->importProfitAndLoss($xero);
}

In order for a job to leave the queue, it must reach the end of the handle function -- without errors and exceptions.
There must be something breaking inside one or more of your functions.
If an exception is thrown while the job is being processed, the job will automatically be released back onto the queue so it may be attempted again. https://laravel.com/docs/5.8/queues
The same behavior can be achieved with
$this->release()
If you can't figure out what is breaking, you can set your job to run only once. If an error is thrown, the job will be considered failed and will be put in the failed jobs queue.
The maximum number of attempts is defined by the --tries switch used
on the queue:work Artisan command. https://laravel.com/docs/5.8/queues
php artisan queue:work --tries=1
If you are using the database queue, (awesome for debugging) run this command to create the failed queue table
php artisan queue:failed
Finally, to find out what is wrong with your code. You can catch and log the error.
public function handle(Xero $xero)
{
try{
$this->getAndCreateXeroSnapshotID();
$this->importInvoices($xero);
$this->importBankTransaction($xero);
$this->importBankStatement($xero);
$this->importBalanceSheet($xero);
$this->importProfitAndLoss($xero);
}catch(\Exception $e){
Log::error($e->getMessage());
}
}
You could also set your error log channel to be slack, bugsnag or whatever. Just be sure to check it. Please don't be offended, it's normal to screw up when dealing with laravel queues. How do you think I got here?

Laravel try to run the job again and again.
php artisan queue:work --tries=3
Upper command will only try to run the jobs 3 times.
Hope this helps

In my case the problem was the payload, I've created the variable private, but it needs to by protected.
class EventJob implements ShouldQueue
{
use InteractsWithQueue, Queueable, SerializesModels;
// payload
protected $command;
// Maximum tries of this event
public $tries = 5;
public function __construct(CommandInterface $command)
{
$this->command = $command;
}
public function handle()
{
$event = I_Event::create([
'event_type_id' => $command->getEventTypeId(),
'sender_url' => $command->getSenderUrl(),
'sender_ip' => $command->getSenderIp()
]);
return $event;
}
}

The solution that worked for me to delete the job after pushing them into the queue.
Consider the e.g.
class SomeController extends Controller{
public function uploadProductCsv(){
//process file here and push the code inot Queue
Queue::push('SomeController#processFile', $someDataArray);
}
public function processFile($job, $data){
//logic to process the data
$job->delete(); //after complete the process delete the job
}
}
Note: This is implemented for laravel 4.2

Related

How to keep the code running after return the response early?

I need to save the request into the database after that i have to call an API to sync the data to the other server.
i do the API call using the finally but it seems PHP still processing it, even when i am sending the response in the try clause.
how do i make this asynchronous ? i want to send the response as fast as possible but still processing the API call after the response.
so this what the simple code look like, describing what i am currently doing.
Code with finally =>
public function store(Request $request)
{
try {
//returning the code early
return response("i am speed", 202);
} catch (\Throwable $th) {
return response($th->getMessage(), 500);
} finally {
//lets says this is the super long too run
$i = 0;
$last = 11111111;
while ($i <= $last) {
$i++;
}
}
}
//this code finish in 1000ms
code without finally =>
public function store(Request $request)
{
try {
return response("i am speed", 202);
} catch (\Throwable $th) {
return response($th->getMessage(), 500);
}
} //this code finish in 90ms;
why this is happen ?
i already sending the response but why it not returning early ?
what can i do to send the response first then continue the execution ?
I already finish this problem.
As the comment suggest Laravel have a feature called a queue that will dispatch the job the database, this process won't running until you run the queue worker.
To make queue what i do is :
First change the .env QUEUE_CONNECTION to database.
QUEUE_CONNECTION=database
Then run this artisan command to setup the queue worker.
php artisan queue:table
php artisan migrate
After that make a job for the function that want to run in the queue.
For example i am gonna put the finally code (the one in the question) in the new job.
php artisan make:job exampleJobName
Then go to exampleJobName.php, write the code that will be running in the queue in the handle function.
public function handle()
{
//lets says this is the super long too run code
$i = 0;
$last = $this->data; //variable from constructor
while ($i <= $last) {
$i++;
}
}
//exampleJobName.php
If a variable need to be pass to the handle, then add a contructor in the __construct function.
public function __construct($data)
{
$this->data = $data;
}
//still inside the exampleJobName.php
Then everything is setup go to the controller that want to run this job (i am gonna take example from question) and change the code to this.
public function store(Request $request)
{
try {
$data = 111111111;
exampleJobName::dispatch($data);
//this will insert the job on the jobs table in the database
//therefore the job won't run until the queue worker is running
return response("i am speed", 202);
} catch (\Throwable $th) {
return response($th->getMessage(), 500);
}
}
//this code finish fast
Everything is ready then just run the queue worker in the terminal side by side with artisan serve.
php artisan queue:work
The queue worker will check to the jobs table if there are any job that hasn't run yet. then will run it one by one.
That's what i do, hope this can help someone.
credit to Tim Lewis in the comment for show me this this link.
Call an API to sync the data to the other server potentially can take a long time so I suggest that you create and dispatch a job for that.
If you still want to do so right after the response is sent, you might want to use dispatchAfterResponse.

Laravel queue is not doing job in the background

I am trying to use laravel queue to send bulk emails. So far I have written down the logic and it works fine, but the problem is that when I wrote the logic in controller it takes a lot of time so I thought of using jobs but again the problem persists.
My Problem
My problem is that I am not able send the email in background even if I am using queue.
Controller
public function newsletter(Request $request)
{
//dd($request->all());
dispatch(new SendEmail($request));
Session::flash('message', 'Email Sent');
Session::flash('class', 'success');
return redirect()->route('news');
}
Jobs
public function handle(Request $request)
{
//
$data = array(
'message' => $request->message,
'subject' => $request->subject,
'file' => $request->file("file")
);
$teachingLevel = $request->highest_teaching_level;
$school = $request->school;
$province = $request->province;
$district = $request->district;
$subject = $request->subject;
if ($teachingLevel != "" && $school != "" && $province != "" && $district != "") {
$email = User::where('highest_teaching_level', $teachingLevel)->where('current_school_name', $school)->where('address','LIKE', '%'.$province.'%')->where('address','LIKE', '%'.$district.'%')->pluck('email');
}else{
$email = User::pluck('email');
}
foreach($email as $e)
{
Mail::to($e)->send(new NewsLetter($data, $subject));
}
}
The email is sent but it doesn't happen in the background. Maybe it has to do with the way I have passed $request variable in the handle() function.
Any help will be appreciated. Thanks!
Here is how I'm using Laravel jobs in my project:
SampleJob.php
namespace App\Jobs;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use App\Services\SampleService;
class SampleJob implements ShouldQueue {
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
// if you omit this value, you'll be in trouble, believe me
public $tries = 1;
private $param;
public function __construct($param) {
$this->param = $param;
}
public function handle(SampleService $service) {
// do something with $this->param
// Also, notice that you can type hint classes in handle function declaration for DI
$service->doSomething($this->param);
}
}
SampleController.php
namespace App\Http\Controllers;
use App\Jobs\SampleJob;
class SampleController extends Controller {
public function sampleMethod(Request $request) {
$param = $request->input('param');
SampleJob::dispatch($param); // $param will be passed to SampleJob constructor
// ...
}
}
A few points worth to note are:
Read the comments in my code snippets
If you use db-based queue, migrate first with php artisan queue:table && php artisan migrate
Create jobs with artisan command: php artisan make:job Sample
Don't forget to run queue worker: php artisan queue:work. To make it run in background: sudo nohup php artisan queue:work > ./storage/logs/queue-worker.log &
Highly recommended: In deployment, use Supervisor to keep php artisan queue:work running in the background
If you manage to make the job work, all delayed (queued but not handled because of misconfiguration or not starting queue worker) works will be instantly executed.
Common pitfalls:
If you don't set $tries param, and somehow your job throws an error, laravel will try to retry that job again and again until your database is down :(
If http user and php user is different, and if you used Log in your job, nine out of ten times you face permission problem on storage directory. To avoid this problem, add 'permission' => '0666' to your log channel setting in config/logging.php
Queue worker does not detect your code change, thus restart queue worker by php artisan queue:restart after you make some change to code base.
My laravel version: 5.8
If you are going to work with “database” connection is neccesary to run the next migrations:
php artisan queue:table
php artisan migrate
Also have an Event and a Listener that implements the “ShouldQueue” interface, and by last register the event associated with the listener or listeners in your “providers/EventProvider.php” path and in “EventProvider.php” file add your event and listeners with the next notation as example:
protected $listen = [
Registered::class => [
SendEmailVerificationNotification::class,
],
];
Its important to understant few points related to following queue:restart command
php artisan queue:restart
For that to work, you need to be running the Queue Listener:
php artisan queue:listen
Ref: https://medium.com/ariel-mejia-dev/run-queues-in-the-background-on-development-in-laravel-8d697d812f29

Why Laravel keeps calling schedule() with every Artisan Command?

I have one table called dc_user_meta and I've created one artisan command and scheduled it in kernel.php. Just after cloning the repository, when I try to run PHP artisan migrate, I get this error.
[Illuminate\Database\QueryException]
SQLSTATE[42S02]: Base table or view not found: 1146 Table 'database.dc_user_meta' doesn't exist (SQL: select * from `dc_user_met
a` where `meta_key` = usage_in_days)
Not only php artisan migrate but I am unable to run any artisan command at all! I don't know why PHP keeps calling schedule method every time I try to execute any artisan command.
Here in this case, What I can do to solve this error is put the cover over my logic in schedule method just like this.
if(Schema::hasTable('dc_user_meta')){
// Code here
}
But I don't think it's good in Long run. What's the right way to solve this error?
UPDATE:
I just tried covering call to command in kernel.php just like this but still no success!
if(Schema::hasTable('dc_user_meta')){
$schedule->command('usage:update')->daily();
}
UPDATE:
I got the solution. But I don't think it's the answer to the question. It solves my problem but I don't think it's standard Solution. I just covered by Command login just like this.
if(Schema::hasTable('dc_user_meta')){
// Command Logic
}
Any specific answer to why Laravel calls schedule() with every artisan command and how to solve the error in a standard way if something like this happens!
Technically the schedule method ist called via the constructor of Illuminate\Foundation\Console\Kernel ( This is the parent class of app\Console\Kernel.php)
So every time the console Kernel is instantiated, the schedule() method gets executed.
Let's see what gets executed in which scenario ( $schedule->call() can be replaced with $schedule->command() or $schedule->exec()):
protected function schedule(Schedule $schedule)
{
// everything that is inside the schedule function is executed everytime the console kernel is booted.
// gets exectuted every time
\App\User::where('foo', 1)->get();
$schedule->call(function() {
// gets executed for every call to php artisan schedule:run
\App\User::where('foo', 1)->get();
});
$schedule->call(function() {
// gets executed for every call to php artisan schedule:run
// IF the closure in the when() function is true;
\App\User::where('foo', 1)->get();
})->when(function() {
// if true is returned the scheduled command or closure is executed otherwise it is skipped
\Schema::hasColumn('user', 'foo');
});
}
But why HAS the schedule command to be exectuted with every command?
Well, obviously php artisan schedule:run is a console command itself. So it definitely needs information about scheduled commands.
Also other commands could need information about scheduled commands... For example if you want to write an artisan command list:scheduledTasks. This command would require that all scheduled commands have been added to the console schedule list.
Maybe there are several other (internal) arguments why the schedule function has to run everytime. ( I did not dig too deep in the source code... )
Nevertheless... information about scheduled commands could be useful to a variety of use cases.
Your error is with table dc_user_meta while your logic is of table user_meta you need to do Schema::hasTable('dc_user_meta')
I'm convinced that table dc_user_meta doesn't exist in database.
As I understand, yor have table "user_meta" not "dc_user_meta" but you have written the code to use table "dc_user_meta" hence there is an error saying "dc_user_meta" table not found.
If anyone still cares about this...
<?php
# This is your app/Console/Kernel.php
use ...;
class Kernel extends ConsoleKernel {
# Other stuff...
protected function schedule(Schedule $schedule) {
if( in_array('schedule:run', $_SERVER['argv']) ){
# Your scheduler commands here...
}
}
}

How to sleep PHP(Laravel 5.2) in background

I have created an artisan command that I want to run immediately after a method called. But the command contains a sleep(); command. I want to run that artisan command in background because the method need to return immediately response to user. My sample code is a below:
In route file
Route::get('test', function(){
Artisan::queue('close:bidding', ['applicationId' => 1]);
return 'called close:bidding';
});
In close:bidding command
public function handle()
{
$appId = $this->argument('applicationId');
//following code line is making the problem
sleep(1000 * 10);
//close bidding after certain close time
try{
Application::where('id', $appId)->update(['live_bidding_status' => 'closed']);
}catch (\PDOException $e){
$this->info($e->getMessage());//test purpose
}
$this->info($appId.": bid closed after 10 seconds of creation");
}
Problem
When I hit /test url the return string called close:bidding is being shown after 10 seconds browser loading, because there is a sleep(10 * 1000) inside the command.
What I want
I want to run the command in background. I mean when I hit the /test url it should immediately show up the called close:bidding but close:bidding command will be running in background. After 10 seconds it will update the application though the front-end user won't notice anything anything about it.
Partial Questions
Is it somehow related to multi threading?
Is it something that cannot be solve using PHP, should I think differently?
Is there any way to solve this problem even using Laravel queue?
Create a job
<?php
namespace App\Jobs;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
class JobTest implements ShouldQueue
{
use InteractsWithQueue, Queueable, SerializesModels;
private $payload = [];
public function __construct($payload)
{
$this->payload = $payload;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
$appId = $this->payload('applicationId');
//following code line is making the problem
sleep(1000 * 10);
}
}
To push job background queue
Route::get('test', function(){
dispatch((new JobTest)->onQueue('queue_name'));
return 'called close:bidding';
});
At this state we have job and you dispath the job to the queue. But it not processed yet. We need queue listener or worker to process those job in background
php artisan queue:listen --queue=queue_name --timeout=0
OR
php artisan queue:work --queue=queue_name --timeout=0 //this will run forever
Note:
May be you can try supervisord,beanstakd for manage queue
For more info refer this
If you don't need all advantages of proper queue, which come at a price, it may be sufficient to use terminate middleware. It will do the job after response was sent to the browser.

Laravel 4.2 Queue - force job fail

I want to do something like this in my fire method:
class MyClass{
public function fire($job) {
if(something) {
$job->fail();
}else {
//processing
}
$job->delete();
}
There is no such method as fail(), is it possible to do something like this?
There is no such thing as fail a job but what you can do:
release it back to the queue with
$job->release();
After defined number of attempts it will end up in failed jobs table.
throw an exception. The job will be released back to the queue on it's own.
if you're using beanstalkd as a queue driver you can bury a job
$job->bury();
If your condition is unrecoverable you can log this fact and simply delete the job.

Categories