I'm adding functionality to a pre-existing app, using Laravel 5.8.38 and the SQS queue driver.
I'm looking for a way to log the receipt handle of queue messages as they're processed, so that we can manually delete messages from the queue for jobs that have gone horribly wrong (without the receipt ID, we'd have to wait for the visibility timeout to be reached).
I'm not super familiar with Laravel and am trying to figure things out as I go. We have two types of queued jobs:
a custom class implementing Illuminate\Contracts\Queue\ShouldQueue, that also uses the Illuminate\Queue\InteractsWithQueue, Illuminate\Foundation\Bus\Dispatchable and Illuminate\Bus\Queueable traits (our class gets queued directly)
a custom command, extending Illuminate\Console\Command, that runs via Illuminate\Foundation\Console\QueuedCommand
For the custom class, browsing through the source for InteractsWithQueue and Illuminate/Queue/Jobs/SqsJob I discovered I could access the receipt handle directly:
$sqsJob = $this->job->getSqsJob();
\Log::info("Processing SQS job {$sqsJob["MessageId"]} with handle {$sqsJob["ReceiptHandle"]}");
This works great! However, I can't figure out how to do a similar thing from the console command.
Laravel's QueuedCommand implements ShouldQueue as well as Illuminate\Bus\Queueable, so my current guess is that I'll need to extend this, use InteractsWithQueue, and retrieve and log the receipt handle from there. However if I do that, I can't figure out how I would modify Artisan::queue('app:command', $commandOptions); to queue my custom QueuedCommand class instead.
Am I almost there? If so, how can I queue my custom QueuedCommand class instead of the Laravel one? Or, is there a better way to do this?
Ok I had just posted this question and then realised a suggestion a colleague offered provided a way forward.
So, here's my solution in case it helps anyone else!
Laravel fires a Illuminate\Queue\Events\JobProcessing event when processing any new queue job. I just needed to register a listener in app/Providers/EventServiceProvider.php:
protected $listen = [
'Illuminate\Queue\Events\JobProcessing' => [
'App\Listeners\LogSQSJobDetails',
],
];
and then provide the listener to handle it:
namespace App\Listeners;
use Illuminate\Queue\Events\JobProcessing;
class LogSQSJobDetails
{
public function __construct()
{
}
public function handle(JobProcessing $event)
{
$sqsJob = $this->job->getSqsJob();
\Log::info("Processing SQS job {$sqsJob["MessageId"]} with handle {$sqsJob["ReceiptHandle"]}");
}
}
This works great - and means I can also now remove the addition to my custom class from earlier.
Related
I'm using Symfony 4.4 with Redis for the session.
I have some controllers and I wish to update the DB on back ground after to send a reply to user.
So I have written a code like this:
class GetCatController extends AbstractController
{
public function getCatController(LoggerInterface $logger, ManagerRegistry $doctrine, SessionInterface $session, ValidatorInterface $validator)
{
[...]
$replyToSend = new JsonResponse($reply, 200);
$replyToSend->send();
//My back ground activity like to do some query on the db.
[...]
return null;
}
But I have some problems about the sessions.
Is there a better way to create and run background activity sending before the reply to user?
There are two decent ways to do this.
If you are running PHP under php-fpm (not mod_php), you can dispatch & catch an event, kernel.terminate In Symfony pre-4.4, this is called PostResponseEvent (TerminateEvent from 4.4/5.0).
The better choice may be with Symfony Messenger. Here, you would create a message object, with all the information needed to perform the task, and send it to a background queue (Redis is supported as a queue). A worker then consumes that message, and does the task.
I have an artisan command that fires a job called PasswordResetJob which iterates as it calls a method forcePasswordReset in a repository class OrgRepository, the method updates a user's table. The whole process works fine.
Now I'm trying to write a Laravel test to mock the OrgRepository class and assert that the forcePasswordReset method is called at least once, which should be the case, based on the conditions I provided to the test. In the test, I call the artisan command to fire job; (I'm using sync queue for testing) this works fine as the job gets called and the user's table gets updated as I can view my database updates directly.
However, the test fails with the error: Mockery\Exception\InvalidCountException : Method forcePasswordReset() from Mockery_2_Repositories_OrgRepository should be called
at least 1 times but called 0 times.
The artisan call within the test is:
Artisan::call('shisiah:implement-org-password-reset');
I have tried to make the artisan call before, as well as after this mock initialization, but I still get the same errors. Here is the mock initialization within the test
$this->spy(OrgRepository::class, function ($mock) {
$mock->shouldHaveReceived('forcePasswordReset');
});
What am I missing? I have gone through the documentation and searched through Google for hours. Please let me know if you need any additional information to help. I'm using Laravel version 6.0
edit
I pass the OrgRepository class into the handle method of the job class, like this:
public function handle(OrgRepository $repository)
{
//get orgs
$orgs = Org::where('status', true)->get();
foreach ($orgs as $org){
$repository->forcePasswordReset($org);
}
}
The problem is that you are initializing your spy after your job has already run, which means during the job it will use the real class instead of the spy.
You have to do something like this in your test:
$spy = $this->spy(OrgRepository::class);
// run your job
$spy->shouldHaveReceived('forcePasswordReset');
We tell laravel to use the spy instead of the repository, run the job and then assert that the method was called.
Jeffrey Way explains it pretty well in this screencast.
I am wondering how to deal with this. I have a webhook endpoint which responds to a webhook call from Github.
It starts a long running process in where it clones the repository from which the webhook call was made.
/**
* The webhook endpoint.
*
* #param Request $request
* #return mixed
* #throws \Exception
*/
public function webhook(Request $request)
{
// The type of GitHub event that we receive.
$event = $request->header('X-GitHub-Event');
$url = $this->createCloneUrl();
$this->cloneGitRepo($url);
return new Response('Webhook received succesfully', 200);
}
The problem with this is that Github gives an error when the response is not provided soon enough.
We couldn’t deliver this payload: Service Timeout
This is rightfully so because my cloneGitRepo method is simply blocking the execution of the response and it is taking too long.
How can I still deliver a response to acknowledge to Github that the webhook call has been made successfully and start my long running process?
I am using Laravel for all of this with Redis, maybe something can be accomplished there? I am open to all suggestions.
What you're looking for is a queued job. Laravel makes this very easy with Laravel Queues.
With queues, you setup a queue driver (database, redis, Amazon SQS, etc), and then you have one to many queue workers that are continuously running. When you put a job on the queue from your webhook method, it will be picked up by one of your queue workers and run in a separate process. The act of dispatching a queued job to a queue is very quick, though, so your webhook method will return quickly while the real work is being done by the queue worker.
The linked documentation has all the details, but the general process will be:
Setup a queue connection. You mention you're already using redis, I would start with that.
Use php artisan make:job CloneGitRepo to create a CloneGitRepo job class.
It should implement the Illuminate\Contracts\Queue\ShouldQueue interface so that Laravel knows to send this job to a queue when it is dispatched.
Make sure to define properties on the class for any data you pass into the constructor. This is necessary so the worker can rebuild the job correctly when it is pulled off the queue.
The queue worker will call the handle() method to process the job. Any dependencies can be type hinted here and they will be injected from the IoC container.
To dispatch the job to the queue, you can either use the global dispatch() helper function, or call the static dispatch() method on the job itself.
dispatch(new CloneGitRepo($url));
CloneGitRepo::dispatch($url);
So, your webhook would look like:
public function webhook(Request $request)
{
// The type of GitHub event that we receive.
$event = $request->header('X-GitHub-Event');
$url = $this->createCloneUrl();
CloneGitRepo::dispatch($url);
return new Response('Webhook received succesfully', 200);
}
I'm working on a multi-tenant app where I need to log a lot more data than what I pass to the log facade. What I mean is, every time I do this...
Log::info('something happened');
I get this:
[2017-02-15 18:12:55] local.INFO: something happened
But I want to get this:
[2017-02-15 18:12:55] [my ec2 instance id] [client_id] local.INFO: something happened
As you can see I'm logging the EC2 instance ID and my app's client ID. I'm of course simplifying this as I need to log a lot more stuff in there. When I consume and aggregate these logs, having these extra fields make them incredibly handy to figure what things went wrong and where.
In the Zend Framework, I usually subclass the logger and add these extra fields in my subclass but I'm not sure how I can do that with Laravel. I can't find where the logger is instantiated so that I can plug my custom logger in (if that is even the way to go in Laravel).
So, I'm not asking how to get the EC2 instance ID and the other stuff, I'm only asking what the proper way to "hot wire" the Laravel logger is to be able to plug this in.
Just an idea... the logger in Laravel is really a Monolog instance... You could push a handler on it and do whatever processing you want for each entry... like so...
<?php
$logger->pushProcessor(function ($record) {
$record['extra']['dummy'] = 'Hello world!';
return $record;
});
As per the Laravel doc you can hook up into the monolog config at boot...
Custom Monolog Configuration
If you would like to have complete control over how Monolog is
configured for your application, you may use the application's
configureMonologUsing method. You should place a call to this method
in your bootstrap/app.php file right before the $app variable is
returned by the file:
$app->configureMonologUsing(function ($monolog) {
$monolog->pushHandler(...);
});
return $app;
So instead just push a processor on the $monolog instance passed to the hook...
Just an idea, I have not tried this in Laravel but used Monolog before...
So Laravel 5 was finally released yesterday with the final implementation of the command bus but I was wandering, what's the real difference in using a command bus over event mechanisms that we have in the previous releases?
Ok, I see the reason that it can be used to create commands from Request objects which is pretty useful but beyond that it seems to behave in a similar way even down to the whole queuing functionality for events now?
Can you please provide examples of use cases and where the pros and cons of either are?
Commands are things about to happen right now. i.e. "CreateUser"
Events are things that have just occured right now - i.e. "UserSuccessfullyCreated"
The differences appear minor - but have some key differences.
Commands must be specifically called/dispatched. I.e. if you want to
do CommandX - you must call CommandX somewhere.
Events respond to an event firing anywhere in your application.
The great thing is multiple event handling classes can respond to the
same event.
Lets do an example to illustrate it best. Lets say we create a user, and we want to send them a welcome email and also update our newsletter list.
In a Command Scenario would would do
AdminController {
function create() {
Bus::dispatch(new CreateUser(Auth::user());
}
}
then in our CommandClass - we would do
public function handle(CreateUser $auth)
{
// 1. Create the user here
// 2. Send welcome email
// 3. Update our newsletter
}
But if we use events - we would do something like this in our CommandClass
public function handle(CreateUser $auth)
{
// 1. Create the user here
Event::fire(new UserWasCreated($user));
}
then we can create as many events as we want to listen to that event and do something:
EventClassA
Event::listen('UserWasCreated', function($event)
{
// 2. Send welcome email
});
EventClassB
Event::listen('UserWasCreated', function($event)
{
// 3. Update our newsletter
});
The great thing is separation of concerns. The command "createuser" now does not need to worry itself about what happens after a user is created. It just needs to CreateUser.
Also - if we want to add another function after a user signs up - say enter them in a lotto draw - you can just add another Event Class and add a new event listener.
EventClassC
Event::listen('UserWasCreated', function($event)
{
// 4. Register them in lotto
});
Notice how we didnt need to touch the command CreateUser class code at all? This provides a true separation concerns of classes in a OOP style approach.
I just want to share my understanding of this concept on top of the correct answer:
The main difference is that Commands can change a Model state, while Events just react to a state change.
COMMANDS:
Commands in Laravel represent the implementation of the Command design pattern.
The main adventages of Commands:
The can be accessed from anywhere
They are very easy to read by any other developer
To create a Command in Laravel 5:
You need to generate a command DTO (which can implement the SelfHandling interface). Using php artisan make:command {command-name}
Example: php artisan make:command Course/PostCourseCommand
The naming convention for commands: speak the business language and add postfix Command to it
To call (dispatch) the command from you controller, you can use:
$this->dispatch(new PostCourseCommand())
or
Bus::dispatch(new PostCourseCommand());
Side Note:
The "dispatch from request” feature is a nice way to skip passing the variables to the command constructor one by one, instead it will resolve this for you:
Example:
$test_request = Request::create('/test', 'GET', [
'name' => 'Mahmoud',
'nickname' => 'Mega'
]);
$result = Bus::dispatchFrom(
CreateCourse::class, $test_request
);
Finally:
You can separate the handler function and it’s logic from the command DTO to the Handlers directory, to do so:
Generate a command handler via artisan
art handler:command --command="Course/PoatCourseCommand"
remove the SelfHandling interface from the Command class so it will search for a handler to handle it.
EVENTS:
Events in Laravel represent the implementation of the Observer design pattern.
To create an Event in Laravel 5:
use artisan: art make:event {event-name}
Example: art make:event LogSomething
generate an event handler for that event
art handler:event LogSomething --event="LogSomething"
register the event and it’s handler in the event service provider (app/Providers/EventServiceProvider.php)
Example:
protected $listen = [
\Zzz\Events\LogSomething::class => [ // event.name
\Zzz\Handlers\Events\LogSomething::class, //EventListener
],
],
To call (fire) an Event:
use:
Event::fire(New LogSomething());
or you can use the event helper
event(New LogSomething());
Side Note:
alternatively you can generate an event by simply registering the event in the service provider then running this command.
php artisan event:generate << this will automatically add the two classes for you
Also you can listen to an event without creating an event handler or registering a lister in the listeners array, by just going to the event service prover and inside the boot function writing your event and its action (NOT RECOMMENDED). Example:
Event::listen('XXX\Events\DoSomethingElse', function($event)
{
dd('handle me :)');
});
Finally: you can queue an event or even subscribe to multiple events from within the class itself..