I am trying to understand if my issue is a limitation with Laravel's logic or an issue in my config. I have this in my queue.php file
'connections' => [
'sqs' => [
'driver' => 'sqs',
'key' => env('AWS_ACCESS_KEY_ID'),
'secret' => env('AWS_SECRET_ACCESS_KEY'),
'token' => env('AWS_SESSION_TOKEN'),
'prefix' => env('SQS_PREFIX'),
'region' => env('AWS_REGION'),
],
],
Now, when I try to this command
php artisan queue:work sqs --tries 3 --queue='high-priority'
I would expect it to process the outstanding jobs in the 'high-priority' queue, however instead it throws this exception
ErrorException : Undefined index: queue
at {redacted}/vendor/laravel/framework/src/Illuminate/Queue/Connectors/SqsConnector.php:26
22| $config['credentials'] = Arr::only($config, ['key', 'secret', 'token']);
23| }
24|
25| return new SqsQueue(
> 26| new SqsClient($config), $config['queue'], $config['prefix'] ?? ''
27| );
28| }
29|
30| /**
Now I can see that laravel is trying to construct a new SqsClient using the config value in key of queue, so in this case it seems that to use multiple queues here I would need to define an individual connection for every queue (which seems completely overkill).
So really, wondering if this is a limitation or if I misunderstand how the config is being passed into the construction of the SqsClient.
Related
I have found the documentation for it here. I have PHP SDK installed. Now when I go through the documents there is not so much in detail about the PHP one. I have the following questions:
Here how can I specify the $client
$result = $client->createDatabase([
'DatabaseName' => '<string>', // REQUIRED
'KmsKeyId' => '<string>',
'Tags' => [
[
'Key' => '<string>', // REQUIRED
'Value' => '<string>', // REQUIRED
],
// ...
],
]);
Is there any good documents or videos regarding the timestream in PHP from where I can get some help.
There are two client classes. One for writing and one for reading.
TimestreamWriteClient
https://docs.aws.amazon.com/aws-sdk-php/v3/api/class-Aws.TimestreamWrite.TimestreamWriteClient.html
and
TimestreamQueryClient
https://docs.aws.amazon.com/aws-sdk-php/v3/api/class-Aws.TimestreamQuery.TimestreamQueryClient.html
You can use the function createTimestreamQuery and createTimestreamWrite on the $sdk object to instantiate those classes.
A sample Timestream client and query below.
// Create client
$client = new \Aws\TimestreamQuery\TimestreamQueryClient([
'version' => 'latest',
'region' => AWS_REGION, /* eg: eu-west-1 */
'endpoint' => AWS_TIMESTREAM_ENDPOINT, /* eg: https://query-cell3.timestream.eu-west-1.amazonaws.com */
'credentials' => new \Aws\Credentials\Credentials(AWS_KEY, AWS_SECRET)
]);
// Perform a basic query with the client
$client->query([
'QueryString' => 'select * from "db_timestream"."tbl_usage_logs"',
'ValidateOnly' => true,
]);
If you receive endpoint warning, such as "The endpoint required for this service is currently unable to be retrieved"
You can find endpoint using AWS CLI command,
aws timestream-query describe-endpoints --region eu-west-1
Sample response:
{
"Endpoints": [
{
"Address": "query-cell3.timestream.eu-west-1.amazonaws.com",
"CachePeriodInMinutes": 1440
}
]
}
One can create TimestreamWriteClient and write records in a similar way.
The documentation seems sparse and a bit misleading, to me anyhow.
This is how I got it going for a write client (assuming SDK is installed).
//Create the client
$client = new \Aws\TimestreamWrite\TimestreamWriteClient([
'version' => 'latest',
'region' => 'eu-west-1',
'credentials' => new \Aws\Credentials\Credentials('***KEY***', '***SECRET***')
]);
Note that the 'endpoint' is not specified, as I've seen in some examples. There seems to be some misleading documentation of what the endpoint should be for any given region. The SDK does some magic and creates a suitable endpoint; providing a specific endpoint didn't work for me.
$result = $client->writeRecords(
[
'DatabaseName' => 'testDB',
'TableName' => 'history',
'Records' =>
[
[
'Dimensions' => [
[
'DimensionValueType' => 'VARCHAR',
'Name' => 'Server',
'Value' => 'VM01',
],
],
'MeasureName' => 'CPU_utilization',
'MeasureValue' => '1.21',
'MeasureValueType' => 'DOUBLE',
'Time' => strval(time()),
'TimeUnit' => 'SECONDS',
]
]
]
);
This seems to be the minimum set of things needed to write a record to Timestream successfully. The code above writes one record, with one dimension, in this case, a 'Name' of a 'Server', recording its CPU utilization at time().
Note:
Time is required, although the documentation suggested it is optional.
Time has to be a String.
Laravel 5.8
I am new to this whole pusher functionality and I've been following this tutorial and trying it out,
Create Web Notifications Using Laravel and Pusher Channels.
I've followed it step-by-step and when I get to the step to manually test the event by visiting the test url, I receive the following exception:
Illuminate \ Broadcasting \ BroadcastException
No message
C:\wamp\www\ares\vendor\laravel\framework\src\Illuminate\Broadcasting\Broadcasters\PusherBroadcaster.php
Here is the code:
$response = $this->pusher->trigger(
$this->formatChannels($channels), $event, $payload, $socket, true
);
if ((is_array($response) && $response['status'] >= 200 && $response['status'] <= 299)
|| $response === true) {
return;
}
throw new BroadcastException( // <-- Exception at this line
is_bool($response) ? 'Failed to connect to Pusher.' : $response['body']
);
}
/**
* Get the Pusher SDK instance.
*
* #return \Pusher\Pusher
*/
public function getPusher()
{
return $this->pusher;
}
}
I've looked at a few other stack overflow articles which talk about changing encrypted: true to encrypted: false but that does not seem to affect anything.
I started working on Laravel 4 days ago and I came across this same problem when I was implementing a real-time chat application. After searching for many days, I discovered that this may vary depending on the version of Laravel you are running. If it is 5.8, you can fix this by adding the following code in the pusher.options array of the file config/broadcasting.php:
'curl_options' => [
CURLOPT_SSL_VERIFYHOST => 0,
CURLOPT_SSL_VERIFYPEER => 0,
],
After adding this , your pusher array in the config/broadcasting.php should look like this.
'pusher' => [
'driver' => 'pusher',
'key' => env('PUSHER_APP_KEY'),
'secret' => env('PUSHER_APP_SECRET'),
'app_id' => env('PUSHER_APP_ID'),
'options' => [
'cluster' => env('PUSHER_APP_CLUSTER'),
'encrypted' => true,
'curl_options' => [
CURLOPT_SSL_VERIFYHOST => 0,
CURLOPT_SSL_VERIFYPEER => 0,
],
],
],
You can then run php artisan config:cache(which may not be necessary in some cases) and finally run php artisan serve.You can consult your app in the pusher website and see the events you receive after sending your messages.
Hope it helps!!
If you're working on localhost try setting your .env file.
Set:
APP_URL=http://localhost
DB_HOST=localhost
And run
php artisan config:cache
Like i mentioned in a comment before this happens when the whole post goes wrong and wont deliver a response. Thats why the exception in line 116 is raised. I changed it to the domain before!
In my case i followed the code an found the method "createPusherDriver" in "vendor/laravel/framework/src/Illuminate/Broadcasting/BroadcastManager.php". At this place i inserted this
var_dump($config['key']);
var_dump($config['secret']);
var_dump( $config['app_id']);
var_dump($config['options']);
exit;
an noticed that my options still listed "host" => "localhost".
I removed those lines an cleared the config cache by executing php artisan config:cache
On next reload my event was fired an logged in the console.
Worked perfectly up to my Laravel 5.8 version. But encrypted' => true or encrypted' => false did not matter in this case for such Laravel version. But, following PUSHER suggestions, I put to broadcasting: 'useTLS' => true,.
This is the final result to me:
'options' => [
'cluster' => env('PUSHER_APP_CLUSTER'),
'encrypted' => true,
'useTLS' => true,
'curl_options' => [
CURLOPT_SSL_VERIFYHOST => 0,
CURLOPT_SSL_VERIFYPEER => 0,
],
]
Thanks to dear #Bitart
'useTLS' => true
option solved my issue.
'options' => [
'cluster' => env('PUSHER_APP_CLUSTER'),
'useTLS' => true,
]
class ProcessComment extends Job implements ShouldQueue
{
use InteractsWithQueue;
/**
* #var int
*/
public $tries = 1;
public function handle(Somedepency $someDependency) {
// method body....
// tries to connect to a database
// deliberately provide the wrong database url so that the job .
// will throw exception and hence faild
}
The problem is that when i run php artisan queue:work --daemon or php artisan queue:work --daemon --tries=1
The tries option doesn't seem to work. In my redis queue I continuously see the attempts it tries to make like. It should try only one time and if the job failed, just ignore that job and move ahead.
"EXEC"
1522044746.165780 [0 172.20.0.5:48992] "WATCH" "queues:comments:reserved"
1522044746.166110 [0 172.20.0.5:48992] "ZRANGEBYSCORE" "queues:comments:reserved" "-inf" "1522044746"
1522044746.166718 [0 172.20.0.5:48992] "UNWATCH"
1522044746.167436 [0 172.20.0.5:48992] "LPOP" "queues:comments"
1522044746.168051 [0 172.20.0.5:48992] "ZADD" "queues:comments:reserved" "1522044806" {"some serialized data here ... "attempts: 4"}
and so on
This is my configs/queue.php
'default' => env('QUEUE_DRIVER', 'redis'),
'connections' => [
'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => 'comments'
],
],
tried to google it alot but couldn't find satisfactory answer.
Thanks
I am using pusher in Laravel 5.4 but I'm getting following error:
WebSocket connection to 'ws://ws.pusherapp.com/app/731e32c5f123456298e?protocol=7&client=js&version=4.1.0&flash=false' failed: WebSocket is closed before the connection is established.
Pusher : Error : {"type":"WebSocketError","error":{"type":"PusherError","data":{"code":4001,"message":"Did you forget to specify the cluster when creating the Pusher instance? App key 731e32c5f123456298e does not exist in this cluster."}}}
It looks like you've copied your pusher app_key incorrectly. Pusher app keys typically have 20 characters, yours has 19.
'pusher' => [
'driver' => 'pusher',
'key' => env('PUSHER_KEY'),
'secret' => env('PUSHER_SECRET'),
'app_id' => env('PUSHER_APP_ID'),
'options' => [
'cluster' => <your cluster> ,
'encrypted' => true,
],
],
Specify your cluster
however if you are testing on localhost encrypted should be false
I was facing the same issue and fixed it by adding cluster in Pusher function call of javascript:
var pusher = new Pusher('PUSHER_APP_KEY', {
cluster: 'PUSHER_APP_CLUSTER'
encrypted: true
});
Amazon has announced their new FIFO SQS service and I'd like to use it in Laravel Queue to solve some concurrency issues.
I've created several new queues and changed the configurations. However, I got a MissingParameter error which says
The request must contain the parameter MessageGroupId.
So I modified the file vendor/laravel/framework/src/Illuminate/Queue/SqsQueue.php
public function pushRaw($payload, $queue = null, array $options = [])
{
$response = $this->sqs->sendMessage(['QueueUrl' => $this->getQueue($queue), 'MessageBody' => $payload,
'MessageGroupId' => env('APP_ENV', getenv('APP_ENV'))]);
return $response->get('MessageId');
}
public function later($delay, $job, $data = '', $queue = null)
{
$payload = $this->createPayload($job, $data);
$delay = $this->getSeconds($delay);
return $this->sqs->sendMessage([
'QueueUrl' => $this->getQueue($queue), 'MessageBody' => $payload, 'DelaySeconds' => $delay,
'MessageGroupId' => env('APP_ENV', getenv('APP_ENV'))
])->get('MessageId');
}
I'm using APP_ENV as the group ID (it's a single message queue so actually it doesn't matter a lot. I just want everything to be FIFO).
But I'm still getting the same error message. How could I fix it? Any help would be appreciated.
(btw, where has the SDK defined sendMessage? I can find a stub for it but I didn't find the detailed implementation)
I want to point out to others who might stumble across the same issue that, although editing SqsQueue.php works, it will easily be reset by a composer install or composer update. An alternative is to implement a new Illuminate\Queue\Connectors\ConnectorInterface for SQS FIFO then add it to Laravel's queue manager.
My approach is as follows:
Create a new SqsFifoQueue class that extends Illuminate\Queue\SqsQueue but supports SQS FIFO.
Create a new SqsFifoConnector class that extends Illuminate\Queue\Connectors\SqsConnector that would establish a connection using SqsFifoQueue.
Create a new SqsFifoServiceProvider that registers the SqsFifoConnector to Laravel's queue manager.
Add SqsFifoServiceProvider to your config/app.php.
Update config/queue.php to use the new SQS FIFO Queue driver.
Example:
Create a new SqsFifoQueue class that extends Illuminate\Queue\SqsQueue but supports SQS FIFO.
<?php
class SqsFifoQueue extends \Illuminate\Queue\SqsQueue
{
public function pushRaw($payload, $queue = null, array $options = [])
{
$response = $this->sqs->sendMessage([
'QueueUrl' => $this->getQueue($queue),
'MessageBody' => $payload,
'MessageGroupId' => uniqid(),
'MessageDeduplicationId' => uniqid(),
]);
return $response->get('MessageId');
}
}
Create a new SqsFifoConnector class that extends Illuminate\Queue\Connectors\SqsConnector that would establish a connection using SqsFifoQueue.
<?php
use Aws\Sqs\SqsClient;
use Illuminate\Support\Arr;
class SqsFifoConnector extends \Illuminate\Queue\Connectors\SqsConnector
{
public function connect(array $config)
{
$config = $this->getDefaultConfiguration($config);
if ($config['key'] && $config['secret']) {
$config['credentials'] = Arr::only($config, ['key', 'secret']);
}
return new SqsFifoQueue(
new SqsClient($config), $config['queue'], Arr::get($config, 'prefix', '')
);
}
}
Create a new SqsFifoServiceProvider that registers the SqsFifoConnector to Laravel's queue manager.
<?php
class SqsFifoServiceProvider extends \Illuminate\Support\ServiceProvider
{
public function register()
{
$this->app->afterResolving('queue', function ($manager) {
$manager->addConnector('sqsfifo', function () {
return new SqsFifoConnector;
});
});
}
}
Add SqsFifoServiceProvider to your config/app.php.
<?php
return [
'providers' => [
...
SqsFifoServiceProvider::class,
],
];
Update config/queue.php to use the new SQS FIFO Queue driver.
<?php
return [
'default' => 'sqsfifo',
'connections' => [
'sqsfifo' => [
'driver' => 'sqsfifo',
'key' => 'my_key'
'secret' => 'my_secret',
'queue' => 'my_queue_url',
'region' => 'my_sqs_region',
],
],
];
Then your queue should now support SQS FIFO Queues.
Shameless plug: While working on the steps above I've created a laravel-sqs-fifo composer package to handle this at https://github.com/maqe/laravel-sqs-fifo.
FIFO message works in a different way than standard AWS SQS queues.
You need a separate driver for handling FIFO queues.
I had to face the same situation and the below package was a lifesaver.
https://packagist.org/packages/shiftonelabs/laravel-sqs-fifo-queue
in queue.php
'sqs-fifo' => [
'driver' => 'sqs-fifo',
'key' => env('SQS_KEY'),
'secret' => env('SQS_SECRET'),
'prefix' => env('SQS_PREFIX'),
'queue' => env('SQS_QUEUE'),
'region' => env('SQS_REGION'),
'group' => 'default',
'deduplicator' => 'unique',
],
then
dispatch(new TestJob([]))->onQueue('My_Mail_Queue.fifo');
NB:
you need to specify default queue name you are going to use in your application in the .env
SQS_QUEUE=My_Default_queue.fifo
Also, you need to specify all the queue names you are going to use in your application in the listener. (if you are using the same queue name for the whole application, you don't need to specify the queue name in the listener)
php artisan queue:listen --queue=My_Default_queue.fifo,My_Mail_Queue.fifo,My_Message_Queue.fifo
Apart from the MessageGroupId, it needs a MessageDeduplicationId or enabling content-based deduplication.