PHP Sending Large Amounts of Email to Amazon SES - php

I have an online software that sends emails to Amazon SES. Currently I have a cron job that sends the emails via the SMTP with phpmailer to send the messages. Currently I have to max the send limit to around 300 every minute to make sure my server doesn't time out. We see growth and eventually I'd like to send out to 10,000 or more.
Is there a better way to send to Amazon SES, or is this what everyone else does, but with just more servers running the workload?
Thanks in advance!

You can try using the AWS SDK for PHP. You can send emails through the SES API, and the SDK allows you to send multiple emails in parallel. Here is a code sample (untested and only partially complete) to get you started.
<?php
require 'vendor/autoload.php';
use Aws\Ses\SesClient;
use Guzzle\Service\Exception\CommandTransferException;
$ses = SesClient::factory(/* ...credentials... */);
$emails = array();
// #TODO SOME SORT OF LOGIC THAT POPULATES THE ABOVE ARRAY
$emailBatch = new SplQueue();
$emailBatch->setIteratorMode(SplQueue::IT_MODE_DELETE);
while ($emails) {
// Generate SendEmail commands to batch
foreach ($emails as $email) {
$emailCommand = $ses->getCommand('SendEmail', array(
// GENERATE COMMAND PARAMS FROM THE $email DATA
));
$emailBatch->enqueue($emailCommand);
}
try {
// Send the batch
$successfulCommands = $ses->execute(iterator_to_array($emailBatch));
} catch (CommandTransferException $e) {
$successfulCommands = $e->getSuccessfulCommands();
// Requeue failed commands
foreach ($e->getFailedCommands() as $failedCommand) {
$emailBatch->enqueue($failedCommand);
}
}
foreach ($successfulCommands as $command) {
echo 'Sent message: ' . $command->getResult()->get('MessageId') . "\n";
}
}
// Also Licensed under version 2.0 of the Apache License.
You could also look into using the Guzzle BatchBuilder and friends to make it more robust.
There are a lot of things you will need to fine tune with this code, but you may be able to achieve higher throughput of emails.

If anyone is looking for this answer, its outdated and you can find the new documentation here: https://docs.aws.amazon.com/aws-sdk-php/v3/guide/guide/commands.html
use Aws\S3\S3Client;
use Aws\CommandPool;
// Create the client.
$client = new S3Client([
'region' => 'us-standard',
'version' => '2006-03-01'
]);
$bucket = 'example';
$commands = [
$client->getCommand('HeadObject', ['Bucket' => $bucket, 'Key' => 'a']),
$client->getCommand('HeadObject', ['Bucket' => $bucket, 'Key' => 'b']),
$client->getCommand('HeadObject', ['Bucket' => $bucket, 'Key' => 'c'])
];
$pool = new CommandPool($client, $commands);
// Initiate the pool transfers
$promise = $pool->promise();
// Force the pool to complete synchronously
$promise->wait();
Same thing can be done for SES commands

Thank you for your answer. It was a good starting point. #Jeremy Lindblom
My problem is now that i can't get the Error-Handling to work.
The catch()-Block works fine and inside of it
$successfulCommands
returns all the succeed Responses with Status-Codes, but only if an error occurs. For example "unverified address" in Sandbox-Mode. Like a catch() should work. :)
The $successfulCommands inside the try-Block only returns:
SplQueue Object
(
[flags:SplDoublyLinkedList:private] => 1
[dllist:SplDoublyLinkedList:private] => Array
(
)
)
I can't figure it out how to get the real Response from Amazon with Status-Codes etc.

Related

Unable to bulk send SMS with Twilio (error 52182)

Twilio tells me Error - 52182 Messaging Service not specified, so I obviously don't understand how to specificy if, even though I thought I did. The body of the Twilio debugger says Messaging service SID must be specified
I've not found anything that has helped so far on stack, so I'm chancing a question. The same goes for the Twilio docs.
$recipients = [];
foreach ($userIds as $userId) {
$user = Craft::$app->users->getUserById($userId);
$number = !empty($user->mobil);
if ($number) {
try {
$number = $twilio->lookups->v1->phoneNumbers($user->mobil)->fetch(['countryCode' => 'NO'])->phoneNumber;
$recipients[] = '{"binding_type":"sms", "address":"'.$number.'"}';
} catch (\Exception $e) {
continue;
}
}
}
$twilio = new Client('xxx', 'xxx');
$service = $twilio->notify->v1->services->create();
$twilio->notify->services($service->sid)
->notifications->create([
"toBinding" => $recipients,
"body" => $body
]);
I thought I was specifying the service sid here $twilio->notify->services($service->sid), but apparently I'm not.
Previously I would send one SMS at a time in a loop, but that times out due to a growing list of subscribers.
Thank you for shedding any light on this.
I found this guide on youtube: https://www.youtube.com/watch?v=EMOYY58jyKk which seems to have solved my issues, it's not for PHP but the steps were pretty much the same.
In any case, my final code ended up like so
$notifySid = 'ISxxxx';
// Bulk send the SMS
$notification = $twilio->notify->v1->services($notifySid)
->notifications->create([
"toBinding" => $recipients,
"body" => $body
]);

Authenticate and use Google Cloud Speech API

I am trying to make a PHP application using which I can send a .flac file to the google speech service and get text in return.
Following the official guide
This is the authentication code:
require 'vendor/autoload.php';
use Google\Cloud\Core\ServiceBuilder;
// Authenticate using keyfile data
$cloud = new ServiceBuilder([
'keyFile' => json_decode(file_get_contents('b.json'), true)
]);
Then this is the speech code:
use Google\Cloud\Speech\SpeechClient;
$speech = new SpeechClient([
'languageCode' => 'en-US'
]);
// Recognize the speech in an audio file.
$results = $speech->recognize(
fopen(__DIR__ . '/audio_sample.flac', 'r')
);
foreach ($results as $result) {
echo $result->topAlternative()['transcript'] . "\n";
}
Of course the $cloud variable is never used. Where should it go?
I ran it nonetheless got
Uncaught exception 'Google\Cloud\Core\Exception\ServiceException'
with message '{ "error": { "code": 401, "message": "Request had
invalid authentication credentials. Expected OAuth 2 access token,
login cookie or other valid authentication credential. See
https://developers.google.com/identity/sign-in/web/devconsole-project.",
"status": "UNAUTHENTICATED" } }
I just want to make a simple query. Any help would be appreciated.
Try this:
use Google\Cloud\Speech\SpeechClient;
$speech = new SpeechClient([
'languageCode' => 'en-US',
'keyFile' => json_decode(file_get_contents('b.json'), true)
]);
// Recognize the speech in an audio file.
$results = $speech->recognize(
fopen(__DIR__ . '/audio_sample.flac', 'r')
);
foreach ($results as $result) {
echo $result->topAlternative()['transcript'] . "\n";
}
You bring up a good point regarding the documentation. Changes over the last few months have caused this somewhat confusing situation, and it deserves another look to clarify things. I've created an issue to resolve this.
ServiceBuilder is a class which provides factories allowing you to configure Google Cloud PHP once and create different clients (such as Speech or Datastore) which inherit that configuration. All the options on ServiceBuilder::__construct() are also available in the clients themselves, such as SpeechClient, with a couple of exceptions.
If you want to use the ServiceBuilder, you could do something like this:
use Google\Cloud\Core\ServiceBuilder;
$cloud = new ServiceBuilder([
'keyFile' => json_decode(file_get_contents('b.json'), true)
]);
$speech = $cloud->speech([
'languageCode' => 'en-us'
]);
Alternatively, you can call putenv before before working with api:
/** Setting Up Authentication. */
$key_path = '/full/path/to/key-file.json';
putenv( 'GOOGLE_APPLICATION_CREDENTIALS=' . $key_path );

PHP AWS S3 SDK retry on network connections errors

I'm using AWS SDK V3 for PHP. Sometimes when I make calls to AWS S3 I get errors like 400 errors Bad Request RequestTimeout (client): Your socket connection to the server was not read from or written to within the timeout period due to network problem even when the object I'm trying to interact with is there. What I need to do is to implement a retry mechanism. I wonder if we can do it simply with an option in the AWS SDK to specify the number of times we want a retry after an error.
I know that I can do that with simple try catch and retry, but I'm thinking may be the SDK already provides a much cleaner way to do that.
I already found static function Middleware::retry() but I have no idea how to use it.
You can specify the number of retries when constructing a new instance of any AWS service client class:
$client = new Aws\EC2\Ec2Client([
'region' => 'eu-central-1',
'retries' => 3
]);
https://docs.aws.amazon.com/aws-sdk-php/v3/guide/guide/configuration.html#retries
You can use something like this,
public function get_s3_connection(){
$sdk = new \Aws\Sdk([
'region' => 'us-east-1',
'version' => 'latest',
'credentials' => $credentials
]);
$s3_connection = $sdk->createS3([
'retries' => 3,
'http' => [
'connect_timeout' => 15,
'timeout' => 20
],
'endpoint' => 'https://bucket.0123xyz.s3.us-east-1.amazonaws.com'
]);
return $s3_connection;
}
In the above code the retries => 3 means it will try 3 times and then stop trying.
You can also set the timeout and connect_timeout as show above.
You can also retry manually by surrounding the method with do-while loop, as shown below,
public function s3_connect_with_retry($total_retry = 3){
$retry_count = 0;
$do_retry = false;
do{
try{
$s3_connection = $this->get_s3_connection();
}catch(Exception $exception){
$retry_count++;
$do_retry = true;
}
}while($total_retry < $retry && $do_retry == true;);
}
When the connection breaks and throws exception the $do_retry value will become true and the $retry_count will increase. based on the $do_retry flag the retry happens until the $total_retry count is reached.

AWS SQS Delete Messages Using Receipt Handle

I am trying to set up SQS and after receiving the message, I need to delete it from the queue.
Creating Client -
$client = Aws\Sqs\SqsClient::factory(array(
'key' => '******',
'secret' => '******',
'region' => 'ap-southeast-1'
));
Sending Message
public static function SendMessage()
{
if(!isset(self::$queueUrl))
self::getQueueUrl();
$command = "This is a command";
$commandstring = json_encode($command);
self::$client->sendMessage(array(
'QueueUrl' => self::$queueUrl,
'MessageBody' => $commandstring,
));
}
Receiving Message
public static function RecieveMessage()
{
if(!isset(self::$queueUrl))
self::getQueueUrl();
$result = self::$client->receiveMessage(array(
'QueueUrl' => self::$queueUrl,
));
// echo "Message Recieved >> ";
print_r($result);
foreach ($result->getPath('Messages/*/Body') as $messageBody) {
// Do something with the message
echo $messageBody;
//print_r(json_decode($messageBody));
}
foreach ($result->getPath('Messages/*/ReceiptHandle') as $ReceiptHandle) {
self::$client->deleteMessage(self::$queueUrl, $ReceiptHandle);
}
}
When I try to delete the message using the Receipt Handle in the receive message code, I get error from Guzzle -
Catchable fatal error: Argument 2 passed to Guzzle\Service\Client::getCommand() must be an array, string given,
Now after searching a lot for it, I was able to find similar questions which state that they were using wrong SDK version. I am still not able to narrow it down though. I am using the zip version of the latest sdk 2.6.15
Why don't you give this a try this:
self::$client->deleteMessage(array(
'QueueUrl' => self::$queueUrl,
'ReceiptHandle' => $ReceiptHandle,
));
The Basic formatting example in the API docs for SqsClient::deleteMessage() (and other operations) should help. All of the methods that execute operations take exactly one parameter, which is an associative array of the operation's parameters. You should read through the SDK's Getting Started Guide (if you haven't already), which talks about how to perform operations in general.

Mandrill inbound emails through Laravel / PHP

I was wondering if someone could help me with a problem I have been having some trouble researching related to Laravel and inbound email processing through Mandrill.
Basically I wish to be able to receive emails through Mandrill and store them within my Laravel database. Now i'm not sure if i'm reading through the documentation with the wrong kind of eyes, but Mandrill says it deals with inbound email as well as outbound, however i'm starting to think that Mandrill deals with inbound email details as opposed to the actual inbound email, such as if the message is sent etc.
I've created a new Mandrill account, created an API key, created an inbound domain and corresponding subdomain of my site (e.g. inboundmail.myproject.co.uk), set the MX record and the MX record is showing as valid. From there i have set up a route (e.g. queries#inboundmail.myproject.co.uk), and a corresponding webhook (myproject.co.uk/inboundmail.php) and within this webhook tried a variety of the examples given in the API (https://mandrillapp.com/api/docs/inbound.php.html), such as adding a new route, checking the route and attempting to add a new domain. All of them worked and produced the correct results, so my authentication with Mandrill is not in question, but my real question is is there a specific webhook for dealing with accepting incoming mail messages?
I cant help but feel like an absolute idiot asking this question as i'm sure the answer is either staring me in the face or just not possible through Mandrill.
Thanks in advance.
Thanks to duellsy and debest for their help, in the end i found a script and expanded on it to add the mail to my own database and style / display it accordingly. Hope this helps someone who may have the same trouble:
<?php
require 'mandrill.php';
define('API_KEY', 'Your API Key');
define('TO_EMAIL', 'user#example.com');
define('TO_NAME', 'Foo Bar');
if(!isset($_POST['mandrill_events'])) {
echo 'A mandrill error occurred: Invalid mandrill_events';
exit;
}
$mail = array_pop(json_decode($_POST['mandrill_events']));
$attachments = array();
foreach ($mail->msg->attachments as $attachment) {
$attachments[] = array(
'type' => $attachment->type,
'name' => $attachment->name,
'content' => $attachment->content,
);
}
$headers = array();
// Support only Reply-to header
if(isset($mail->msg->headers->{'Reply-to'})) {
$headers[] = array('Reply-to' => $mail->msg->headers->{'Reply-to'});
}
try {
$mandrill = new Mandrill(API_KEY);
$message = array(
'html' => $mail->msg->html,
'text' => $mail->msg->text,
'subject' => $mail->msg->subject,
'from_email' => $mail->msg->from_email,
'from_name' => $mail->msg->from_name,
'to' => array(
array(
'email' => TO_EMAIL,
'name' => TO_NAME,
)
),
'attachments' => $attachments,
'headers' => $headers,
);
$async = false;
$result = $mandrill->messages->send($message, $async);
print_r($result);
} catch(Mandrill_Error $e) {
// Mandrill errors are thrown as exceptions
echo 'A mandrill error occurred: ' . get_class($e) . ' - ' . $e->getMessage();
// A mandrill error occurred: Mandrill_PaymentRequired - This feature is only available for accounts with a positive balance.
throw $e;
}
?>
Like using webhooks from other mail parsing services, you'll need to make use of
file_get_contents("php://input")
This will give you the raw data from the webhook, which you can then json_decode and work with the results.

Categories