We are trying to send bulk email (100k) with PHP Laravel framework. Which way is the correct way to send bulk email with Laravel queue?
Case 1.
//controller
public function runQueue(){
dispatch(new ShootEmailJob());
}
//job
public function handle(){
$emails = EmailList::get(['email']);
foreach($emails as $email){
Mail::to($email)->send();
}
}
Case 2.
//controller
public function runQueue(){
$emailList = EmailList::get(['email']);
foreach($emailList as $emailAddress){
dispatch(new ShootEmailJob($emailAddress->email));
}
}
//job
public function handle(){
Mail::to($emailAddress)->send(new ShootMail($emailAddress));
}
Which one is the correct approach case 1 or case 2?
The first approach will first fetch all emails and then send them one by one in one "instance" of a job that is run as a background process if you queue it.
The second approach will run n "instances" of jobs, one for each email on the background process.
So performance-wise option 1 is the better approach. You could also wrap it in a try - catch block in case of exceptions so that the job does not fail if one of the emails fails, e.g.:
try {
$emails = EmailList::get(['email']);
foreach($emails as $email){
Mail::to($email)->send();
}
} catch (\Exception $e) {
// Log error
// Flag email for retry
continue;
}
Related
This is my controller that get contact emails of over 50k and iterate over it then batched it to queue
This is the job that send the email
The problem is that the email can only send to less than 500 contacts without any issue but anything above 1000 contact will result to maximum execution timeout
How can I solve this problem.
This statement is going to retrieve and store all your 50k in memory:
$contacts = Contact::where(...)->where(...)->get();
I suggest chunking the results to avoid memory exhaustion.
Back to your problem, I am thinking about chunking the results (about 500 each) and dispatch an intermediary job that will eventually send emails.
class SendEmailsInChunk implements ShouldQueue
{
public $contacts;
public $batch;
function __construct(public $batch, public $contacts) {}
public function handle()
{
foreach ($this->contacts as $contact) {
$this->batch->add(new BroadCampaignJob(..., $contact, ...));
}
}
Then you can chunk the results and dispatch the above job with each chunk:
$batch = ...;
$query = Contact::where(...)->chunk(500, function ($contacts) use ($batch) {
SendEmailsInChunk::dispatch($batch, $contacts);
})
this is my scenario: I have a Laravel queued jod, here the handle method content:
$invoiceContract = app('AsteBolaffi\Repositories\Backend\Invoice\InvoiceContract');
$error = $invoiceContract->recordInvoice($this->invoiceId);
d($error);
if (!empty($error) && !empty($this->userEmail)) {
$userEmail = $this->userEmail;
$invoice = $invoiceContract->findOrThrowException($this->invoiceId);
$invoice->load('customer');
d("Going to send mail to " . $userEmail);
d($error);
d($invoice->customer->business_name);
$data["error"] = $error;
$data["business_name"] = $invoice->customer->business_name;
$data["document_number"] = $invoice->document_number;
d($data);
\Mail::queueOn('mail', "emails.record_invoice", $data, function ($message) use ($userEmail) {
$message->from('admin#astebolaffi.it', 'AsteBolaffi');
$message->to($userEmail);
$message->subject('Errori contabilizzazione fattura');
d("HERE I AM");
});
d("Completed");
}
return;
If the recordInvoice method returns a value (e.g. "Item not found") the if clause is satisfied and it has to add a mail queue. But the mail queue is not created on db and the current job is not deleted even if the console prints the last d method value ("Completed").
If recordInvoice does not returns any error the job is deleted.
Non-sense thing (at least for me)
If I comment the recordInvoice method and set a value to error, for example:
$error = "test";
It works properly, adds the mail queue and deletes the current one.
Any tip about it?
I don't know why, but moving the project from php 5.6 to 7.0 it works correctly.
I am using Mailgun as a mail driver in my laravel application, as well as nexmo for SMS purposes.
What I am trying to achieve is to maintain the delivery status of the notifications that are sent either via Mailgun or Nexmo. Incase of Nexmo I am able achieve this, since I get the nexmo MessageId in the NotificationSent event that is fired after processing a notification.
However in the event instance for email, the response is empty.
Any idea what I am missing or, how I can retrieve the mailgun message-id?
I have found a workaround that does the job for now. Not as neat as I want it to be, but posting for future references incase anyone needs this.
I have created a custom notification channel extending Illuminate\Notifications\Channels\MailChannel
class EmailChannel extends MailChannel
{
/**
* Send the given notification.
*
* #param mixed $notifiable
* #param \Illuminate\Notifications\Notification $notification
* #return void
*/
public function send($notifiable, Notification $notification)
{
if (! $notifiable->routeNotificationFor('mail')) {
return;
}
$message = $notification->toMail($notifiable);
if ($message instanceof Mailable) {
return $message->send($this->mailer);
}
$this->mailer->send($message->view, $message->data(), function ($m) use ($notifiable, $notification, $message) {
$recipients = empty($message->to) ? $notifiable->routeNotificationFor('mail') : $message->to;
if (! empty($message->from)) {
$m->from($message->from[0], isset($message->from[1]) ? $message->from[1] : null);
}
if (is_array($recipients)) {
$m->bcc($recipients);
} else {
$m->to($recipients);
}
if ($message->cc) {
$m->cc($message->cc);
}
if (! empty($message->replyTo)) {
$m->replyTo($message->replyTo[0], isset($message->replyTo[1]) ? $message->replyTo[1] : null);
}
$m->subject($message->subject ?: Str::title(
Str::snake(class_basename($notification), ' ')
));
foreach ($message->attachments as $attachment) {
$m->attach($attachment['file'], $attachment['options']);
}
foreach ($message->rawAttachments as $attachment) {
$m->attachData($attachment['data'], $attachment['name'], $attachment['options']);
}
if (! is_null($message->priority)) {
$m->setPriority($message->priority);
}
$message = $notification->getMessage(); // I have this method in my notification class which returns an eloquent model
$message->email_id = $m->getSwiftMessage()->getId();
$message->save();
});
}
}
I am still looking for a solution to achieve this with NotificationSent event.
When looking at the code (MailgunTransport) it will do the following
$this->client->post($this->url, $this->payload($message, $to));
$this->sendPerformed($message);
return $this->numberOfRecipients($message);
Since the Laravel contract requires the implementation to send back the number of e-mails send.
Even if you would be able to get into the mail transport it doesn't store the response from for this reason it not possible to catch the message id.
What you could do is to implement your own (or look in packagist) to adapt mail client but this is not a perfect solution and will require some ugly instanceof checks.
I switched from sending my mails immediately to adding them to the queue, here is my code, the $attachments is an array of temporary paths, I've commented out what I've tried, which throws errors about files not existing.
Mail::queue($view, $data, function(\Illuminate\Mail\Message $message) use($mail,$attachments){
foreach($mail->getRecipients() as $recipient){
$message->to($recipient);
}
$message->subject($mail->getSubject());
foreach($attachments as $attachment){
$message->attach($attachment);
//this deletes the attachment before being sent
//unlink($attachment);
}
});
/* This code only works when using Mail::send() instead of Mail:queue()
foreach($attachments as $attachment){
unlink($attachment);
}
*/
Basically I want to clean up and remove my temporary attachments after the mail was sent. I am guessing this would not work with the out of the box laravel mail solutions. How can I trigger code post-queue-mail-sent?
Expanding on Deric Lima's answer a bit, you don't necessarily need a new Job class for this. You can do it with a Mailable object as well. Just override the send method.
/**
* #param MailerContract $mailer
*/
public function send(MailerContract $mailer)
{
parent::send($mailer);
//$this->clearAttachments is something you can defined in your constructor,
//making it the responsibility of whatever is applying the attachment
//to know whether it needs to remain in tact after the email is transmitted.
if ($this->clearAttachments) {
foreach ($this->attachments as $attachment) {
if (\File::exists($attachment['file'])) {
\File::delete($attachment['file']);
}
}
}
}
Personally, I'd make a BaseMailable class that all other Mailable classes extend, as opposed to the Illuminate\Mail\Mailable directly. Then you don't even have to worry about it from then on.
You have to wait until the queue is processed before removing the file.
Without knowing the implementation details of the queue it is hard to answer your question, but if your queue is processed before the script ends, you can use register_shutdown_function
http://www.php.net/manual/en/function.register-shutdown-function.php to run a cleanup function that removes the file
register_shutdown_function(function() use (filename){
if (file_exists($filename)) {
unlink($filename);
}
})
I had similar problem and I solved using Laravel Jobs. Basically, you can create a Job class to send the email:
class MailJob extends Job implements SelfHandling, ShouldQueue
{
use InteractsWithQueue, SerializesModels;
public function handle()
{
Mail::send($view, $data, function (\Illuminate\Mail\Message $message) use ($mail, $attachments) {
foreach ($mail->getRecipients() as $recipient) {
$message->to($recipient);
}
$message->subject($mail->getSubject());
foreach ($attachments as $attachment) {
$message->attach($attachment);
unlink($attachment);
}
});
foreach ($attachments as $attachment) {
unlink($attachment);
}
}
}
And then you just dispatch the Job inside the controller that you want to send the email:
$this->dispatch(new MailJob());
P.S: The job is running asynchronous on the background, so I used Mail::send instead of Mail::queue.
I need to email all my users a daily product list in cakePHP 2.
I have the following code to get all the users emails.
$users = $this->User->find('all', array('fields' => array('email')));
foreach ($users as $user) {
$this->Email->reset();
$this->Email->from = '<no-reply#test.com.au>';
$this->Email->to = $user['email'];
$this->Email->subject = "Daily Products" ;
$this->Email->sendAs = 'html';
$this->Email->send();
}
Now I understand that I could use a html template for this and parse values to it but I really need a foreach loop inside the actual view itself and send the table of products.
What would be the best practise? cakePHP code in the controller or view to get the products?
Thanks
The best practice for this would be to use a shell to send the emails out. To avoid running out of memory you should read the users and their products in chunks and not all at the same time.
Inside the foreach loop you'll need to get the data for each user and set() it as any other variable it will be available in the html template then and you can render all the products there.
Here is some (shortened) code from a shell that processes data:
public function main() {
$this->loop();
}
public function loop() {
try {
while (true) {
if (!$this->poll()) {
$this->out(__('Nothing more to process, sleeping...'));
sleep($this->sleep);
}
}
} catch (Exception $e) {
$this->out($e->getMessage());
$this->log($e->getMessage(), 'processing');
}
}
public function poll() {
$this->out('Polling...');
$result = $this->Model->getFirstUnprocessed();
if ($result === false) {
return false;
}
// do something with the result here
return true;
}
This should be enough to give you an idea. For reading your users in chunks you would need to increment the offset in your find() options. In my case I just check if there is an unprocessed record and if yes i process and and wait a moment to do the next try.
The "view" for e-mails is actually an element. It's under Views/Elements/email. There are 2 folders there html and text, both meant to hold their respective templates.
You can do your foreach in there, then make sure to set the layout in your controller with:
$this->Email->sendAs = 'html'; // Can also be 'text' or 'both' (for multipart).
$this->Email->layout = 'foo'; // Would include Views/Elements/email/html/foo.ctp
Although the Email component is deprecated since CakePHP 2.0 and you should be using the CakeEmail component instead. See the book for more details on how to use that.