Looping HTTP post starts to fail for Amazon SES - php

Question

I originally posted this at the Amazon SES forums here: https://forums.aws.amazon.com/thread.jspa?threadID=74561&tstart=0
But since the stackoverflow community is more active, I'll post it here :)
Basically I have a forecah loop around a cURL post (see bottom of post for script snippet). It works for a couple hundred posts, but then starts to fail for all the others. Here's the last successful post followed by the first of 100's of unsuccessful posts...
* About to connect() to email.us-east-1.amazonaws.com port 443 (#0)
* Trying 207.171.162.2... * connected
* Connected to email.us-east-1.amazonaws.com (207.171.162.2) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSL connection using RC4-MD5
* Server certificate:
* subject: /C=US/ST=Washington/L=Seattle/O=Amazon.com Inc./CN=email.us-east-1.amazonaws.com
* start date: 2010-10-08 00:00:00 GMT
* expire date: 2013-10-07 23:59:59 GMT
* subjectAltName: email.us-east-1.amazonaws.com matched
* issuer: /C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)09/CN=VeriSign Class 3 Secure Server CA - G2
* SSL certificate verify ok.
POST / HTTP/1.1
Accept: */*
Host: email.us-east-1.amazonaws.com
Content-Type: application/x-www-form-urlencoded
Date: Sat, 20 Aug 2011 18:59:56 UTC
X-Amzn-Authorization: AWS3-HTTPS AWSAccessKeyId=LKIABMH7PT8SO4YBRDQA,Algorithm=HmacSHA1,Signature=/0HFVEsTBGqUUSQGy9jvmsft2k4=
Content-Length: 5810
Expect: 100-continue
< HTTP/1.1 200 OK
< x-amzn-RequestId: 4a0f8f18-cb5f-11e0-8364-b14fdafc0888
< Content-Type: text/xml
< Content-Length: 326
< Date: Sat, 20 Aug 2011 19:04:55 GMT
<
* Connection #0 to host email.us-east-1.amazonaws.com left intact
* Closing connection #0
The the failures start....
* About to connect() to email.us-east-1.amazonaws.com port 443 (#0)
* Trying 207.171.162.2... * connected
* Connected to email.us-east-1.amazonaws.com (207.171.162.2) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSL connection using RC4-MD5
* Server certificate:
* subject: /C=US/ST=Washington/L=Seattle/O=Amazon.com Inc./CN=email.us-east-1.amazonaws.com
* start date: 2010-10-08 00:00:00 GMT
* expire date: 2013-10-07 23:59:59 GMT
* subjectAltName: email.us-east-1.amazonaws.com matched
* issuer: /C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)09/CN=VeriSign Class 3 Secure Server CA - G2
* SSL certificate verify ok.
POST / HTTP/1.1
Accept: */*
Host: email.us-east-1.amazonaws.com
Content-Type: application/x-www-form-urlencoded
Date: Sat, 20 Aug 2011 18:59:56 UTC
X-Amzn-Authorization: AWS3-HTTPS AWSAccessKeyId=LKIABMH7PT8SO4YBRDQA,Algorithm=HmacSHA1,Signature=/0HFVEsTBGqUUSQGy9jvmsft2k4=
Content-Length: 5806
Expect: 100-continue
< HTTP/1.1 400 Bad Request
< x-amzn-RequestId: 4b8f29db-cb5f-11e0-b9af-33e5c8fc863b
< Content-Type: text/xml
< Content-Length: 347
< Date: Sat, 20 Aug 2011 19:04:58 GMT
<
* Connection #0 to host email.us-east-1.amazonaws.com left intact
* Closing connection #0
Here's the script snippet
foreach($JSONarray['DATABASE'] as $E)
{
if ((array_diff($E['LISTS'], $FILTER) != $E['LISTS']) && $E['STATUS'] == "CONF")
{
$MAIL = "Action=SendEmail&Source=".$FROME."&ReturnPath=".$BOUNCE."&Destination.ToAddresses.member.1=".$TOE."&Message.Subject.Data=".$SUBE."&Message.Body.Html.Data=".$BODYE;
//curl
$aws = curl_init();
curl_setopt($aws, CURLOPT_POSTFIELDS, $MAIL);
curl_setopt($aws, CURLOPT_HTTPHEADER, $headers);
curl_setopt($aws, CURLOPT_HEADER, false);
curl_setopt($aws, CURLOPT_URL, $url);
curl_setopt($aws, CURLOPT_RETURNTRANSFER, true);
curl_setopt($aws, CURLOPT_VERBOSE, true);
curl_setopt($aws, CURLOPT_STDERR, $SESLOG);
curl_exec($aws);
curl_close($aws);
}
}
Any ideas?

Answer 1

Could it be a DOS prevention mecanism kicking in? How about putting some sleeps into your code? I would definitely put some safeguards against potential DOS attacks if I were Amazon..
Just a guess, though...

Answer 2

I figured out the problem:
<ErrorResponse xmlns="http://ses.amazonaws.com/doc/2010-12-01/">
<Error>
<Type>Sender</Type>
<Code>RequestExpired</Code>
<Message>Request timestamp: Mon, 22 Aug 2011 15:49:11 UTC expired. It must be within 300 secs/ of server time.</Message>
</Error>
<RequestId>0e3899cb-ccd7-11e0-9f09-c5d12d442026</RequestId>
</ErrorResponse>
My headers were set-up outside the forecah loop around curl. Doh! So I just moved that bit of code into the loop to solve the time-out problem.