class ProcessComment extends Job implements ShouldQueue
{
use InteractsWithQueue;
/**
* #var int
*/
public $tries = 1;
public function handle(Somedepency $someDependency) {
// method body....
// tries to connect to a database
// deliberately provide the wrong database url so that the job .
// will throw exception and hence faild
}
The problem is that when i run php artisan queue:work --daemon or php artisan queue:work --daemon --tries=1
The tries option doesn't seem to work. In my redis queue I continuously see the attempts it tries to make like. It should try only one time and if the job failed, just ignore that job and move ahead.
"EXEC"
1522044746.165780 [0 172.20.0.5:48992] "WATCH" "queues:comments:reserved"
1522044746.166110 [0 172.20.0.5:48992] "ZRANGEBYSCORE" "queues:comments:reserved" "-inf" "1522044746"
1522044746.166718 [0 172.20.0.5:48992] "UNWATCH"
1522044746.167436 [0 172.20.0.5:48992] "LPOP" "queues:comments"
1522044746.168051 [0 172.20.0.5:48992] "ZADD" "queues:comments:reserved" "1522044806" {"some serialized data here ... "attempts: 4"}
and so on
This is my configs/queue.php
'default' => env('QUEUE_DRIVER', 'redis'),
'connections' => [
'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => 'comments'
],
],
tried to google it alot but couldn't find satisfactory answer.
Thanks
Related
I'm trying to add websockets support to my Laravel 9 application. I've read in their documentation that I could run an open source server on my own instead of relying on commercial solutions. With that in mind, I decided to give Beyondcode's Laravel Websockets a try. My application runs on PHP 8.2, using Laravel 9.19 and Laravel Websockets 1.13, communicating with the sockets through the Pusher Channels HTTP PHP Library 7.2.
Laravel Websockets is meant to match the server's settings with the client's by using the same naming scheme on both ends and launching off the same codebase.
This is how I've set up my .env file:
APP_NAME="My app"
APP_ENV=local
APP_KEY=<redacted>
APP_DEBUG=true
BROADCAST_DRIVER=pusher
PUSHER_APP_ID="${APP_NAME}"
PUSHER_APP_KEY="${APP_KEY}"
PUSHER_APP_SECRET="${APP_KEY}${APP_NAME}"
PUSHER_HOST=ws-server
PUSHER_PORT=6001
PUSHER_SCHEME=http
PUSHER_APP_CLUSTER=mt1
My docker-compose.yml file contains the following content:
version: "3.9"
services:
queue:
build: .
environment:
- ROLE=queue
volumes:
- .:/var/www
ws-server:
build: .
environment:
- ROLE=ws-server
volumes:
- .:/var/www
ports:
- 6001:6001
The ws-server service has an entrypoint script which runs the following command on an infinite loop: php artisan websockets:serve --host 0.0.0.0 --port 6001
Within config/websockets.php I've left everything as it comes out of the box, just so you can rest assured, this is how my apps array looks like:
'apps' => [
[
'id' => env('PUSHER_APP_ID'),
'name' => env('APP_NAME'),
'key' => env('PUSHER_APP_KEY'),
'secret' => env('PUSHER_APP_SECRET'),
'path' => env('PUSHER_APP_PATH'),
'capacity' => null,
'enable_client_messages' => false,
'enable_statistics' => true,
],
]
The queue service hosts a class called TerminalUpdated, which looks like this:
<?php
namespace App\Events;
use App\Models\User;
use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
use Illuminate\Foundation\Events\Dispatchable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Log;
class TerminalUpdated implements ShouldBroadcast
{
use Dispatchable, InteractsWithSockets, SerializesModels;
public string $id;
public string $newLine;
/**
* Create a new event instance.
*
* #return void
*/
public function __construct(string $id, string $newLine)
{
$this->id = $id;
$this->newLine = $newLine;
}
/**
* Get the channels the event should broadcast on.
*
* #return \Illuminate\Broadcasting\Channel|array
*/
public function broadcastOn()
{
return new PrivateChannel('console.' . $this->id);
}
}
Another script running within the service mentioned above executes the following sentence:
<?php
$id = '123';
$line = 'This is a line of text.';
TerminalUpdated::dispatch(
$this->id, // id
$line // newLine
);
?>
Everything looks more or less okay to me, yet I'm getting the following exception:
[2023-01-19 01:05:09] local.ERROR: Pusher error: {"error":"Invalid auth signature provided."}. {"exception":"[object] (Illuminate\\Broadcasting\\BroadcastException(code: 0): Pusher error: {\"error\":\"Invalid auth signature provided.\"}. at /var/www/vendor/laravel/framework/src/Illuminate/Broadcasting/Broadcasters/PusherBroadcaster.php:164)
"}
I don't seem to be able to figure out where is it even trying to generate the signature mentioned in the exception shown above.
I've already tried the following possible fixes:
Restart the server
Reset the framework's caches and temporary files by calling php artisan optimize:clear
This is how I configured the WebSocket:
In the websocket.php file :
apps => [
[
'id' => env('PUSHER_APP_ID'),
'name' => env('APP_NAME'),
'key' => env('PUSHER_APP_KEY'),
'secret' => env('PUSHER_APP_SECRET'),
'path' => env('PUSHER_APP_PATH'),
'capacity' => null,
'enable_client_messages' => false,
'enable_statistics' => true,
],
],
In broadcasting.php I have:
'pusher' => [
'driver' => 'pusher',
'key' => env('PUSHER_APP_KEY'),
'secret' => env('PUSHER_APP_SECRET'),
'app_id' => env('PUSHER_APP_ID'),
'options' => [
'cluster' => env('PUSHER_APP_CLUSTER'),
'useTLS' => false,
'encrypted' => false,
'host'=> '127.0.0.1',
'port' => 6001,
'scheme' => 'http',
],
],
I hope it helps you out.
The problem is that PUSHER_APP_ID can't contain spaces. Using a string without spaces instead fixes the issue. It looks like the library just can't translate the spaces to the expected URL encoding.
I am trying to understand if my issue is a limitation with Laravel's logic or an issue in my config. I have this in my queue.php file
'connections' => [
'sqs' => [
'driver' => 'sqs',
'key' => env('AWS_ACCESS_KEY_ID'),
'secret' => env('AWS_SECRET_ACCESS_KEY'),
'token' => env('AWS_SESSION_TOKEN'),
'prefix' => env('SQS_PREFIX'),
'region' => env('AWS_REGION'),
],
],
Now, when I try to this command
php artisan queue:work sqs --tries 3 --queue='high-priority'
I would expect it to process the outstanding jobs in the 'high-priority' queue, however instead it throws this exception
ErrorException : Undefined index: queue
at {redacted}/vendor/laravel/framework/src/Illuminate/Queue/Connectors/SqsConnector.php:26
22| $config['credentials'] = Arr::only($config, ['key', 'secret', 'token']);
23| }
24|
25| return new SqsQueue(
> 26| new SqsClient($config), $config['queue'], $config['prefix'] ?? ''
27| );
28| }
29|
30| /**
Now I can see that laravel is trying to construct a new SqsClient using the config value in key of queue, so in this case it seems that to use multiple queues here I would need to define an individual connection for every queue (which seems completely overkill).
So really, wondering if this is a limitation or if I misunderstand how the config is being passed into the construction of the SqsClient.
Amazon has announced their new FIFO SQS service and I'd like to use it in Laravel Queue to solve some concurrency issues.
I've created several new queues and changed the configurations. However, I got a MissingParameter error which says
The request must contain the parameter MessageGroupId.
So I modified the file vendor/laravel/framework/src/Illuminate/Queue/SqsQueue.php
public function pushRaw($payload, $queue = null, array $options = [])
{
$response = $this->sqs->sendMessage(['QueueUrl' => $this->getQueue($queue), 'MessageBody' => $payload,
'MessageGroupId' => env('APP_ENV', getenv('APP_ENV'))]);
return $response->get('MessageId');
}
public function later($delay, $job, $data = '', $queue = null)
{
$payload = $this->createPayload($job, $data);
$delay = $this->getSeconds($delay);
return $this->sqs->sendMessage([
'QueueUrl' => $this->getQueue($queue), 'MessageBody' => $payload, 'DelaySeconds' => $delay,
'MessageGroupId' => env('APP_ENV', getenv('APP_ENV'))
])->get('MessageId');
}
I'm using APP_ENV as the group ID (it's a single message queue so actually it doesn't matter a lot. I just want everything to be FIFO).
But I'm still getting the same error message. How could I fix it? Any help would be appreciated.
(btw, where has the SDK defined sendMessage? I can find a stub for it but I didn't find the detailed implementation)
I want to point out to others who might stumble across the same issue that, although editing SqsQueue.php works, it will easily be reset by a composer install or composer update. An alternative is to implement a new Illuminate\Queue\Connectors\ConnectorInterface for SQS FIFO then add it to Laravel's queue manager.
My approach is as follows:
Create a new SqsFifoQueue class that extends Illuminate\Queue\SqsQueue but supports SQS FIFO.
Create a new SqsFifoConnector class that extends Illuminate\Queue\Connectors\SqsConnector that would establish a connection using SqsFifoQueue.
Create a new SqsFifoServiceProvider that registers the SqsFifoConnector to Laravel's queue manager.
Add SqsFifoServiceProvider to your config/app.php.
Update config/queue.php to use the new SQS FIFO Queue driver.
Example:
Create a new SqsFifoQueue class that extends Illuminate\Queue\SqsQueue but supports SQS FIFO.
<?php
class SqsFifoQueue extends \Illuminate\Queue\SqsQueue
{
public function pushRaw($payload, $queue = null, array $options = [])
{
$response = $this->sqs->sendMessage([
'QueueUrl' => $this->getQueue($queue),
'MessageBody' => $payload,
'MessageGroupId' => uniqid(),
'MessageDeduplicationId' => uniqid(),
]);
return $response->get('MessageId');
}
}
Create a new SqsFifoConnector class that extends Illuminate\Queue\Connectors\SqsConnector that would establish a connection using SqsFifoQueue.
<?php
use Aws\Sqs\SqsClient;
use Illuminate\Support\Arr;
class SqsFifoConnector extends \Illuminate\Queue\Connectors\SqsConnector
{
public function connect(array $config)
{
$config = $this->getDefaultConfiguration($config);
if ($config['key'] && $config['secret']) {
$config['credentials'] = Arr::only($config, ['key', 'secret']);
}
return new SqsFifoQueue(
new SqsClient($config), $config['queue'], Arr::get($config, 'prefix', '')
);
}
}
Create a new SqsFifoServiceProvider that registers the SqsFifoConnector to Laravel's queue manager.
<?php
class SqsFifoServiceProvider extends \Illuminate\Support\ServiceProvider
{
public function register()
{
$this->app->afterResolving('queue', function ($manager) {
$manager->addConnector('sqsfifo', function () {
return new SqsFifoConnector;
});
});
}
}
Add SqsFifoServiceProvider to your config/app.php.
<?php
return [
'providers' => [
...
SqsFifoServiceProvider::class,
],
];
Update config/queue.php to use the new SQS FIFO Queue driver.
<?php
return [
'default' => 'sqsfifo',
'connections' => [
'sqsfifo' => [
'driver' => 'sqsfifo',
'key' => 'my_key'
'secret' => 'my_secret',
'queue' => 'my_queue_url',
'region' => 'my_sqs_region',
],
],
];
Then your queue should now support SQS FIFO Queues.
Shameless plug: While working on the steps above I've created a laravel-sqs-fifo composer package to handle this at https://github.com/maqe/laravel-sqs-fifo.
FIFO message works in a different way than standard AWS SQS queues.
You need a separate driver for handling FIFO queues.
I had to face the same situation and the below package was a lifesaver.
https://packagist.org/packages/shiftonelabs/laravel-sqs-fifo-queue
in queue.php
'sqs-fifo' => [
'driver' => 'sqs-fifo',
'key' => env('SQS_KEY'),
'secret' => env('SQS_SECRET'),
'prefix' => env('SQS_PREFIX'),
'queue' => env('SQS_QUEUE'),
'region' => env('SQS_REGION'),
'group' => 'default',
'deduplicator' => 'unique',
],
then
dispatch(new TestJob([]))->onQueue('My_Mail_Queue.fifo');
NB:
you need to specify default queue name you are going to use in your application in the .env
SQS_QUEUE=My_Default_queue.fifo
Also, you need to specify all the queue names you are going to use in your application in the listener. (if you are using the same queue name for the whole application, you don't need to specify the queue name in the listener)
php artisan queue:listen --queue=My_Default_queue.fifo,My_Mail_Queue.fifo,My_Message_Queue.fifo
Apart from the MessageGroupId, it needs a MessageDeduplicationId or enabling content-based deduplication.
I'm using Phinx to execute migrations across 100s of applications on multiple servers. Every application should execute same migrations.
In order to do this there is a instance of app on central server which is aware of all configs and other information needed to do bootstrap process (which is being done based on applicationId).
That central instance (let's call it adminapp) executes command and receives applicationIds through STDIN and then does a loop which bootstraps application and runs migration command.
<?php
namespace Command\Db;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use App\Command\AppCommand;
class MigrateBulkCommand extends AppCommand
{
protected function configure()
{
$this
->setName('command:blah')
->setDescription('Executes SQL migrations accross multiple applications. Expects ApplicationIDs to be passed as new line delimited string on STDIN.')
;
}
protected function execute(InputInterface $input, OutputInterface $output)
{
$stdin = $this->getStdin();
if ($stdin === false) {
throw new \RuntimeException("Bulk migration command requires applicationIds to be passed to STDIN.");
}
$applicationIds = explode("\n", $stdin);
foreach($applicationIds as $applicationId) {
try {
$this->bootstrap($applicationId);
} catch (\Exception $e) {
$output->writeln(sprintf("<error>Bootstrap process failed for applicationId `%s`</error>", $applicationId));
}
$command = new \Phinx\Console\Command\Migrate();
$migrationInput = new \Symfony\Component\Console\Input\ArrayInput([
]);
$returnCode = $command->run($migrationInput, $output);
$output->writeln(sprinf("<info>Migrations for applicationId `%s` executed successfully.</info>", $applicationId));
}
}
}
Now Phinx expects it's configuration to be present in form of a config file. What I'm trying to do is reuse DB connection resource (PDO) and pass it to Phinx command Phinx\Console\Command\Migrate on the fly, together with db name.
I've seen in Phinx documentation that this is an option with PHP config file but I can't find a way to do this on the fly (during Phinx\Console\Command\Migrate class initialization).
Phinx doc suggests:
require 'app/init.php';
global $app;
$pdo = $app->getDatabase()->getPdo();
return array('environments' =>
array(
'default_database' => 'development',
'development' => array(
'name' => 'devdb',
'connection' => $pdo
)
)
);
Is there a way, without horrible hacking to pass PDO connection resource and db name to \Phinx\Console\Command\Migrate
I ended up extending Phinx Config class \Phinx\Config\Config and creating method fromArray.
$command = new \Phinx\Console\Command\Migrate();
$command->setConfig(\MyNamespace\Config::fromArray(
[
'paths' => [
'migrations' => APPLICATION_PATH . "/../db/migrations",
'seeds' => APPLICATION_PATH . "/../db/seeds"
],
'environments' => [
'default_database' => 'production',
'production' => [
'name' => $db->get('dbname'),
'adapter' => 'mysql',
'host' => $db->get('host'),
'port' => $db->get('port'),
'user' => $db->get('username'),
'pass' => $db->get('password'),
]
]
]
));
I am using Laravel 5.2 and php 7. I am testing this on a local Mac OSX environment. I am having no trouble pinging other services or making requests with php. I configured my application by editing broadcasting.php and .env with my api details.
broadcasting.php
'default' => env('BROADCAST_DRIVER', 'pusher'),
'connections' => [
'pusher' => [
'driver' => 'pusher',
'key' => env('mykey'),
'secret' => env('mysecret'),
'app_id' => env('myappid'),
],
...
and in .env
...
CACHE_DRIVER=file
SESSION_DRIVER=file
QUEUE_DRIVER=redis
BROADCAST_DRIVER=pusher
PUSHER_KEY=mykey
PUSHER_SECRET=mysecret
PUSHER_APP_ID=myappid
...
I created a TestEvent like so
<?php
namespace App\Events;
use App\Events\Event;
use Illuminate\Queue\SerializesModels;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
class TestEvent extends Event implements ShouldBroadcast
{
use SerializesModels;
public $data;
public function __construct()
{
$this->data = array(
'power'=> '10'
);
}
public function broadcastOn()
{
return ['test_channel'];
}
}
I then call on the event like so
Event::fire(new TestEvent());
The event shows up and is processed in redis
[2016-02-04 10:06:18] Processed: Illuminate\Broadcasting\BroadcastEvent
[2016-02-04 10:08:44] Processed: Illuminate\Broadcasting\BroadcastEvent
[2016-02-04 10:11:14] Processed: Illuminate\Broadcasting\BroadcastEvent
[2016-02-04 10:11:59] Processed: Illuminate\Broadcasting\BroadcastEvent
[2016-02-04 10:14:22] Processed: Illuminate\Broadcasting\BroadcastEvent
[2016-02-04 11:17:04] Processed: Illuminate\Broadcasting\BroadcastEvent
[2016-02-04 11:38:14] Processed: Illuminate\Broadcasting\BroadcastEvent
But nothing shows up in pusher when viewing the debug console. So it seems that the event is never being sent to pusher for some reason. I also checked my timezone settings to make sure that was not an issue. There are no errors in my in the laravel log file when the event is broadcasted.
How can I solve this problem? Is there a good way to debug and see if there is actually a request being made to pusher?
This might not fix the issue and maybe you changed your details for the question, but if your .env is like this:
PUSHER_KEY=mykey
PUSHER_SECRET=mysecret
PUSHER_APP_ID=myappid
Shouldn't the code for your pusher credentials look like this:
'pusher' => [
'driver' => 'pusher',
'key' => env('PUSHER_KEY'),
'secret' => env('PUSHER_SECRET'),
'app_id' => env('PUSHER_APP_ID'),
],
It's referencing the value in your code, not the key!