Laravel email with queue 550 error (too many emails per second) - php

Our emails are failing to send using Laravel with a Redis Queue.
The code that triggers the error is this: ->onQueue('emails')
$job = (new SendNewEmail($sender, $recipients))->onQueue('emails');
$job_result = $this->dispatch($job);
In combination with this in the job:
use InteractsWithQueue;
Our error message is:
Feb 09 17:15:57 laravel: message repeated 7947 times: [ production.ERROR: exception 'Swift_TransportException' with message 'Expected response code 354 but got code "550", with message "550 5.7.0 Requested action not taken: too many emails per second "' in /home/laravel/app/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php:383 Stack trace: #0 /home/laravel/app/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php(281):
Our error only happens using Sendgrid and not Mailtrap, which spoofs emailing sending. I've talked with Sendgrid and the emails never touched their servers and their service was fully active when my error occurred. So, the error appears to be on my end.
Any thoughts?

Seems like only Mailtrap sends this error, so either open another account or upgrade to a paid plan.

I finally figured out how to set up the entire Laravel app to throttle mail based on a config.
In the boot() function of AppServiceProvider,
$throttleRate = config('mail.throttleToMessagesPerMin');
if ($throttleRate) {
$throttlerPlugin = new \Swift_Plugins_ThrottlerPlugin($throttleRate, \Swift_Plugins_ThrottlerPlugin::MESSAGES_PER_MINUTE);
Mail::getSwiftMailer()->registerPlugin($throttlerPlugin);
}
In config/mail.php, add this line:
'throttleToMessagesPerMin' => env('MAIL_THROTTLE_TO_MESSAGES_PER_MIN', null), //https://mailtrap.io has a rate limit of 2 emails/sec per inbox, but consider being even more conservative.
In your .env files, add a line like:
MAIL_THROTTLE_TO_MESSAGES_PER_MIN=50
The only problem is that it doesn't seem to affect mail sent via the later() function if QUEUE_DRIVER=sync.

For debugging only!
If you don't expect more then 5 emails and don't have the option to change mailtrap, try:
foreach ($emails as $email) {
...
Mail::send(... $email);
if(env('MAIL_HOST', false) == 'smtp.mailtrap.io'){
sleep(1); //use usleep(500000) for half a second or less
}
}
Using sleep() is a really bad practice. In theory this code should only execute in test environment or in debug mode.

Maybe you should make sure it was really sent via Sendgrid and not mailtrap. Their hard rate limit seems currently to be 3k requests per second against 3 requests per second for mailtrap on free plan :)

I used sleep(5) to wait five seconds before using mailtrap again.
foreach ($this->suscriptores as $suscriptor) {
\Mail::to($suscriptor->email)
->send(new BoletinMail($suscriptor, $sermones, $entradas));
sleep(5);
}
I used the sleep(5) inside a foreach. The foreach traverses all emails stored in the database and the sleep(5) halts the loop for five seconds before continue with the next email.

I achieved this on Laravel v5.8 by setting the Authentication routes manually. The routes are located on the file routes\web.php. Below are the routes that needs to be added to that file:
Auth::routes();
Route::get('email/verify', 'Auth\VerificationController#show')->name('verification.notice');
Route::get('email/verify/{id}', 'Auth\VerificationController#verify')->name('verification.verify');
Route::group(['middleware' => 'throttle:1,1'], function(){
Route::get('email/resend', 'Auth\VerificationController#resend')->name('verification.resend');
});
Explanation:
Do not pass any parameter to the route Auth::routes(); to let configure the authentication routes manually.
Wrap the route email/resend inside a Route::group with the middleware throttle:1,1 (the two numbers represents the max retry attempts and the time in minutes for those max retries)
I also removed a line of code in the file app\Http\Controllers\Auth\VerificationController.php in the __construct function.
I removed this:
$this->middleware('throttle:6,1')->only('verify', 'resend');

You need to rate limit emails queue.
The "official" way is to setup Redis queue driver. But it's hard and time consuming.
So I wrote custom queue worker mxl/laravel-queue-rate-limit that uses Illuminate\Cache\RateLimiter to rate limit job execution (the same one that used internally by Laravel to rate limit HTTP requests).
In config/queue.php specify rate limit for emails queue (for example 2 emails per second):
'rateLimit' => [
'emails' => [
'allows' => 2,
'every' => 1
]
]
And run worker for this queue:
$ php artisan queue:work --queue emails

I had this problem when working with mail trap. I was sending 10 mails in one second and it was treated as spam. I had to make delay between each job. Take look at solution (its Laravel - I have system_jobs table and use queue=database)
Make a static function to check last job time, then add to it self::DELAY_IN_SECONDS - how many seconds you want to have between jobs:
public static function addSecondsToQueue() {
$job = SystemJobs::orderBy('available_at', 'desc')->first();
if($job) {
$now = Carbon::now()->timestamp;
$jobTimestamp = $job->available_at + self::DELAY_IN_SECONDS;
$result = $jobTimestamp - $now;
return $result;
} else {
return 0;
}
}
Then use it to make sending messages with delay (taking in to consideration last job in queue)
Mail::to($mail)->later(SystemJobs::addSecondsToQueue(), new SendMailable($params));

Related

Laravel retryUntil job exits after 4th retry without failing

I am trying to debug some bizarre behaviour of my PHP application. It is running Laravel 6 + AWS SQS. The program downloads call recordings from a VoIP provider's API using a job. The API has a heavy rate limit of 10req/minute, so I'm throttling the requests on my side. The job is configured to try to complete within 24 hours using retryUntil method. However, the job disappears from the queue after 4 tries. It doesn't fail. The job's failed method never gets executed (I've put logging and Sentry::capture in there). It's not on the failed_jobs table. The last log says "Cannot complete job, retrying in ... seconds", which is right before the release call. However, the job simply disappears from the queue and never gets executed again.
I am logging the number of attempts, max tries, timeoutAt, etc. Everything seems to be configured properly. Here's (the essence of) my code:
public function handle()
{
/** #var Track $track */
$track = Track::idOrUuId($this->trackId);
$this->logger->info('Downloading track', [
'trackId' => $track->getId(),
'attempt' => $this->attempts(),
'retryUntil' => $this->job->timeoutAt(),
'maxTries' => $this->job->maxTries(),
]);
$throttleKey = sprintf('track.download.%s', $track->getUser()->getTeamId());
if (!$this->rateLimiter->tooManyAttempts($throttleKey, self::MAX_ALLOWED_JOBS)) {
$this->downloadTrack($track);
$this->rateLimiter->hit($throttleKey, 60);
} else {
$delay = random_int(10, 100) + $this->rateLimiter->availableIn($throttleKey);
$this->logger->info('Throttling track download.', [
'trackId' => $track->getId(),
'delay' => $delay,
]);
$this->release($delay);
}
}
public function retryUntil(): DateTimeInterface
{
return now()->addHours(24);
}
public function failed(Exception $exception)
{
$this->logger->info('Job failed', ['exception' => $exception->getMessage()];
Sentry::captureException($exception);
}
I found the problem and I'm posting it here for anyone who might struggle in the future. It all came down to a simple configuration. In AWS SQS the queue I am working with has a configured DLQ (Dead-Letter Queue) and Maximum receives set to 4. According to the SQS docs
The Maximum receives value determines when a message will be sent to the DLQ. If the ReceiveCount for a message exceeds the maximum receive count for the queue, Amazon SQS moves the message to the associated DLQ (with its original message ID).
Since this is an infra configuration, it overwrites any Laravel parameters you might pass to the job. And because the message is simply removed from the queue, the processing job does not actually fail, so the failed method is not executed.

Rate limiting PHP function

I have a php function which gets called when someone visits POST www.example.com/webhook. However, the external service which I cannot control, sometimes calls this url twice in rapid succession, messing with my logic since the webhook persists stuff in the database which takes a few ms to complete.
In other words, when the second request comes in (which can not be ignored), the first request is likely not completed yet however I need this to be completed in the order it came in.
So I've created a little hack in Laravel which should "throttle" the execution with 5 seconds in between. It seems to work most of the time. However an error in my code or some other oversight, does not make this solution work everytime.
function myWebhook() {
// Check if cache value (defaults to 0) and compare with current time.
while(Cache::get('g2a_webhook_timestamp', 0) + 5 > Carbon::now()->timestamp) {
// Postpone execution.
sleep(1);
}
// Create a cache value (file storage) that stores the current
Cache::put('g2a_webhook_timestamp', Carbon::now()->timestamp, 1);
// Execute rest of code ...
}
Anyone perhaps got a watertight solution for this issue?
You have essentially designed your own simplified queue system which is the right approach but you can make use of the native Laravel queue to have a more robust solution to your problem.
Define a job, e.g: ProcessWebhook
When a POST request is received to /webhook queue the job
The laravel queue worker will process one job at a time[1] in the order they're received, ensuring that no matter how many requests are received, they'll be processed one by one and in order.
The implementation of this would look something like this:
Create a new Job, e.g: php artisan make:job ProcessWebhook
Move your webhook processing code into the handle method of the job, e.g:
public function __construct($data)
{
$this->data = $data;
}
public function handle()
{
Model::where('x', 'y')->update([
'field' => $this->data->newValue
]);
}
Modify your Webhook controller to dispatch a new job when a POST request is received, e.g:
public function webhook(Request $request)
{
$data = $request->getContent();
ProcessWebhook::dispatch($data);
}
Start your queue worker, php artisan queue:work, which will run in the background processing jobs in the order they arrive, one at a time.
That's it, a maintainable solution to processing webhooks in order, one-by-one. You can read the Queue documentation to find out more about the functionality available, including retrying failed jobs which can be very useful.
[1] Laravel will process one job at a time per worker. You can add more workers to improve queue throughput for other use cases but in this situation you'd just want to use one worker.

Queue delay , sending an email at a certain time

What want to achieve is to send and email , at a chosen time (by the user)
so my approach was to calculate the time difference then make that the delay time on a queue.
but it seems it's not working as i expect
all emails get delivered together and not at the right time
Controller
public function notifyme($add)
{
$created = Carbon::now()->addMinutes($add);
$now = Carbon::now();
$days = 1440*$created->diff($now)->days;
$hours = 60*$created->diff($now)->h;
$minutes = $days + $hours + $created->diff($now)->i;
$user = Auth::user();
$user->notify((new notifyme($user))->delay($minutes));
return redirect('/notif');
}
route
Route::get('notifyme/{add}', 'HomeController#notifyme');
.env
QUEUE_DRIVER="database"
im using laravel 5.3 so the notifyme controller implements the ShouldQueue contract.
now when i run php artisan queue:work or php artisan queue:listen and test this
the email sending is delayed, but when i do it again, (while the first job is still delayed) both emails get sent together immediately or after sometime (not at the right time $add)
Any idea what is wrong here?
is there a better approach? like using a schedule? or what?
One of approach can be followed to achieve Queue delay for sending an email is to use Laravel inbuilt Mail function with queue facility.
You can take reference from Laravel Delay Mail Queue
EG:
Mail::later(5, 'emails.welcome', $data, function ($message) {
//
});
5 is delayed seconds.

Sending bulk emails using different credentials

I need to send hundreds of emails using different credentials from laravel.
Each customer of mine has his/hers mail list and needs to provide their own SMTP server. I process that list and send emails on customer's behalf.
This is what I have so far. It is working, but it is very slow and I don't have many emails so far. I see a problem when I get more emails.
Any suggestions on how to improve?
PS- I use cron Console Command and use Kernel to schedule the job.
public function sendMailings($allMailings) {
foreach ($allMailings as $email) {
Config::set('mail.host', $email['smtpServer']);
Config::set('mail.port', $email['smtpPort']);
Config::set('mail.username', $email['smtpUser']);
Config::set('mail.password', $email['smtpPassword']);
Config::set('mail.encryption', $email['smtpProtocol']);
Config::set('mail.frommmail', trim($email['fromEmail']));
Config::set('mail.fromuser', trim($email['fromUser']));
Config::set('mail.subject', trim($email['subject']));
Config::set('mail.toEmail', trim($email['toEmail']));
Config::set('mail.toName', trim($email['toName']));
Config::set('mail.pretend', false);
$email_body = $email['emailBody'];
Mail::send('emails.availability, compact('email_body')
, function($message) {
$message->from(config('mail.username'), config('mail.fromUser'));
$message->replyTo(config('mail.frommmail'), config('mail.fromUser'));
$message->to(config('mail.toEmail'), config('mail.toName'))->subject(config('mail.subject'));
});
Log::info('Mail was sent');
}
}
You can not change email provider configs on-the-fly, so you must make new instance of mailer in service container. I did it before, i wrote a method in my own class to get new mailer instance:
/**
* #return Mailer
*/
protected function getMailer()
{
// Changing mailer configuration
config(['mail.driver' => static::getName()]);
// Register new instance of mailer on-the-fly
(new MailServiceProvider($this->container))->register();
// Get mailer instance from service container
return $this->container->make('mailer');
}
Sending e-mail messages directly in web app can drastically slow down the responsiveness of your application. You should always queue your messages.
Instead of Mail::send You can use Mail::queue
and then from cron or manually call
php artisan queue:work
That will process the next item on the queue. This command will do nothing if the queue is empty. But if there’s an item on the queue it will fetch the item and attempt to execute it.

Laravel: Load a page and then send a mail

i am trying to load page first and then activate the $mailer to send email. because when i click on to go to next page its taking time, because its sending emails and then it is loading so,
what is the best way to do it. or any way. because i cant figure it out.
here is snippet
public function sInterest($project_id, AppMailer $mailer)
{
$project = Project::findOrFail($project_id);
if($project->investment){
$mailer->sendInterestNotificationI($user, $project);
$mailer->sendInterestNotificationD($project, $user);
$mailer->sendInterestNotificationA($project, $user);
return view('projects.offer', compact('project'));
}
}
is there a way $mailer to activate after returning a page?
In your AppMailer sendInterestNotification*() methods, replace sync email delivery with queued email delivery (see Queueing Mail in Laravel documentation)
Then page will be returned instantly, emails will be put in the corresponding queue. You will have to edit .env file to change the QUEUE driver and to start a queue listener as a separate process, detailed documentation is given on Laravel website
There is no way you can return a page to browser and then run some extra commands in your controller.

Categories