I have an integration with Twilio's PHP library to bulk send SMS messages. This is all working fine with 5-10 numbers, however, I keep getting a 504 timeout error after 60 seconds when sending to hundreds of numbers. I have created a simple form that you can type a message into which submits to Twilio's API. My integration opens a CSV which contains all the numbers and sends the message to each number. The issue I have is that after 60 seconds I hit a 504 timeout error and therefore only 200-300 numbers receive my SMS. How can I avoid this? Is there a way to break down the nubmers into batches?
<?php
require __DIR__ . '/src/Twilio/autoload.php';
use Twilio\Rest\Client;
// Your Account SID and Auth Token from twilio.com/console
$sid = 'xxxxxxxxxxxxxxx';
$token = 'xxxxxxxxxxxxx';
$notify_sid = 'xxxxxxxxxxxx';
$client = new Client($sid, $token);
$row = 1;
if (($handle = fopen("test.csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 100, ",")) !== FALSE) {
$num = count($data);
$row++;
for ($c=0; $c < $num; $c++) {
$numbersFinal = "'".$data[$c] . "',";
if ($_SERVER["REQUEST_METHOD"] == "POST") {
// Use the client to do fun stuff like send text messages!
$message = $_POST['message'];
$phoneNumbers = array($numbersFinal);
$to = array();
foreach ($phoneNumbers as $phoneNumber) {
$to[] = '{"binding_type":"sms", "address":"'.$phoneNumber.'"}';
}
$request_data = [
"toBinding" => $to,
'body' => $message
];
// Create a notification
$notification = $client
->notify->services($notify_sid)
->notifications->create($request_data);
}
}
}
fclose($handle);
}
?>
504 is a timeout error. This error occurs because of your php.ini settings. Please check the phpinfo() output. Look for max_execution_time variable. It's probably 60 seconds. While you run the file, server terminates the process after this time exceeded.
To handle this problem, you can follow several ways:
Ask your service provider to raise this limit. Usually, they reject those requests.
You can run this file as a cron task. When you click send button, you can save it as a cronjob. You can figure it out.
When you click send, you can send sms by 100, 100, ... send_sms.php?sent=100 could send 100 sms then redirect to send_sms.php?sent=200... until they end.
Tell me if any of these solutions work for you.
Related
EDIT : Already fixed the error Creating default object from empty value . My question now is just suggestions how to make this process better.
I am making a tool that sends batch API request to a certain URL. I am using https://github.com/stil/curl-easy for batch API request That API will validate the phone numbers I am sending and return a result. That API can support maximum 20 request per second, supposed I have 50,000 API request, what is the best and fastest way to process things?
The current process is, I query to my database for the number of records I have which is the number of phone numbers. Then I make a loop with the number of phone numbers. Inside a single iteration, I will query 13 records from the database then pass it to the queue and process it, when the processing is finished I will query again to the database and update the fields for the the phone number based on the API response. My problem is that I think I have a logic error on my loop when I run this the processing will stop for sometime and gives me:
Creating default object from empty value
I guess that is because of my loop here's my code.
public function getParallelApi()
{
$numItems = DB::connection('mysql')->select("SELECT COUNT(ID) AS cnt FROM phones WHERE status IS NULL");
for($y = 0; $y < $numItems[0]->cnt; $y++)
{
$queue = new \cURL\RequestsQueue;
// Set default options for all requests in queue
$queue->getDefaultOptions()
->set(CURLOPT_TIMEOUT, 5)
->set(CURLOPT_RETURNTRANSFER, true)
->set(CURLOPT_SSL_VERIFYPEER, false);
// This is the function that is called for every completed request
$queue->addListener('complete', function (\cURL\Event $event) {
$response = $event->response;
$json = $response->getContent(); // Returns content of response
$feed = json_decode($json, true);
$phone_number = $feed['msisdn'];
if(isset($phone_number))
{
$status = "";$remarks = "";$network = ""; $country = "";$country_prefix = "";
if(isset($feed['status']))
{
$status = $feed['status'];
}
if(isset($feed['network']))
{
$network = $feed['network'];
}
if(isset($feed['country']))
{
$country = $feed['country'];
}
if(isset($feed['country_prefix']))
{
$country_prefix = $feed['country_prefix'];
}
if(isset($feed['remark']))
{
$remarks = $feed['remark'];
}
$update = Phone::find($phone_number);
$update->network = $network;
$update->country = $country;
$update->country_prefix = $country_prefix;
$update->status = $status;
$update->remarks = $remarks;
$update->save();
}
});
// Get 13 records
$phone = DB::connection('mysql')->select("SELECT * FROM phones WHERE status IS NULL LIMIT 13");
foreach ($phone as $key => $value)
{
$request = new \cURL\Request($this->createCurlUrl($value->msisdn));
// var_dump($this->createCurlUrl($value->msisdn)); exit;
// Add request to queue
$queue->attach($request);
}
// Process the queue
while ($queue->socketPerform()) {
$queue->socketSelect();
}
sleep(2);
$y += 13; // Move to the next 13 records
}
Session::flash('alert-info', 'Data Updated Successfully!');
return Redirect::to('upload');
}
Since the maximum is 20 request per seconds I'm just doing it 13 request per second just to be sure I won't clog their server. I am adding sleep(2); to pause a bit so that I can make sure that the queue is fully processed before moving on.
I have a sport betting tips website and on every tip that i publish i send an email to every user (about 700 users in total) with SendGrid. The problem comes with the delivery time. The email is delayed even half an hour from the time of send.
Does anyone know why and how could i fix it?
I am sending it with SMTP.
Here is some of my code:
$catre = array();
$subiect = $mailPronostic['subiect'];
$titlu = $mailPronostic['titlu'];
$text = $mailPronostic['text'];
$data = new DateTime($this->_dataPronostic);
foreach($users as $user){
array_push($catre, $user->_emailUser);
}
$data = urlencode($data->format("d-m-Y H:i"));
$echipe = urlencode($this->_gazdaPronostic." vs ".$this->_oaspetePronostic);
$pronostic = urlencode($predictii[$this->_textPronostic]);
$cota = $this->_cotaPronostic;
$mesaj = file_get_contents("http://plivetips.com/mailFiles/mailPronostic.php?text=".urlencode($text)."&titlu=".urlencode($titlu)."&data=$data&echipe=$echipe&pronostic=$pronostic&cota=$cota");
//return mail(null, $subiect, $mesaj, $header);
$from = array('staff#plivetips.com' => 'PLIVEtips');
$to = $catre;
$subject = "PLIVEtips Tip";
$username = 'user';
$password = 'pass';
$transport = Swift_SmtpTransport::newInstance('smtp.sendgrid.net', 587);
$transport->setUsername($username);
$transport->setPassword($password);
$swift = Swift_Mailer::newInstance($transport);
$message = new Swift_Message($subject);
$message->setFrom($from);
$message->setBody($mesaj, 'text/html');
$numSent = 0;
foreach ($to as $address => $name)
{
if (is_int($address)) {
$message->setTo($name);
} else {
$message->setTo(array($address => $name));
}
$numSent += $swift->send($message, $failures);
}
Thx.
how long does the script take to run? I have a feeling the script is taking a long time. If that's the case, I think it is because you are opening a connection for each message.
With SendGrid, you can send a message to 1000 recipients with one connection by using the X-SMTPAPI header to define the recipients. The easiest way to do this to use the official sendgrid-php library and use the setTos method.
Recipients added to the X-SMTPAPI header will each be sent a unique email. Basically SendGrid's servers will perform a mail merge. It doesn't look like your email content varies with each user, but if it does, then you may use substitution tags in the header to specify custom data per recipient.
If you don't want to use sendgrid-php, you can see how to build the header in this example.
We're using a modified version of easy-apns for sending our push messages. With the enhanced push message format the apple server responds if an error occurs, and does nothing if everything goes well.
The problem is that we have to wait for an error a certain amount of time after each message has been sent. For example if we receive no response after 1 second, we assume everything went ok.
With 20000 push messages, this takes far too long. Is there any way I can listen for errors in a faster way? For example sending to 1000 devices and then listen for errors? What happens if the connection gets closed, can I still read the error response?
Ideal would be some kind of asynchronous writing and reading, but I think that's not possible.
Here's the corresponding code:
$fp = $this->connect();
$expiry = time()+60*60;
// construct message
$msg = chr(1).pack("N",$batchid).pack("N",$expiry).pack("n",32).pack('H*',$devicetoken).pack("n",strlen($payload)).$payload;
// send message to Apple
$fwrite = fwrite($fp, $msg);
if(!$fwrite) {
// connection has been closed
$this->disconnect();
throw new Exception("Connection closed");
} else {
// read response from Apple
// Timeout. 1 million micro seconds = 1 second
$tv_sec = 1;
$tv_usec = 0;
$r = array($fp);
$we = null; // Temporaries. "Only variables can be passed as reference."
// PROBLEM: this method waits for $tv_sec seconds for a response
$numChanged = stream_select($r, $we, $we, $tv_sec, $tv_usec);
if( $numChanged === false ) {
throw new Exception("Failed selecting stream to read.");
} elseif ( $numChanged > 0 ) {
$command = ord( fread($fp, 1) );
$status = ord( fread($fp, 1) );
$identifier = implode('', unpack("N", fread($fp, 4)));
if( $status > 0 ) {
// The socket has also been closed. Cause reopening in the loop outside.
$this->disconnect();
throw new MessageException("APNS responded with status $status: {$this->statusDesc[$status]} ($devicetoken).".microtime(), $status);
} else {
// unknown response, assume ok
}
} else {
// no response, assume ok
}
}
I cant figure this out so I'm hoping you can lend a hand.
I am creating a twilio app, and I'm including this entire file in a foreach loop. But it keeps breaking my loop and wont continue after it runs.
It works great, but the foreach this is included inside of will not continue after it runs.
Any ideas?
Thanks,
Nick
<?php
//shorten the URL
$tinyurl = file_get_contents("http://tinyurl.com/api-create.php?url=".$ebay_url);
// Include the PHP TwilioRest library
require "twilio/twilio.php";
// Twilio REST API version
$ApiVersion = "2010-04-01";
// Set our AccountSid and AuthToken
$AccountSid = "removed";
$AuthToken = "removed";
// Instantiate a new Twilio Rest Client
$client = new TwilioRestClient($AccountSid, $AuthToken);
// make an associative array of server admins
$people = array(
"removed"=>"Nick",
//"4158675310"=>"Helen",
//"4158675311"=>"Virgil",
);
// Iterate over all our server admins
foreach ($people as $number => $name) {
// Send a new outgoinging SMS by POST'ing to the SMS resource */
// YYY-YYY-YYYY must be a Twilio validated phone number
$response = $client->request("/$ApiVersion/Accounts/$AccountSid/SMS/Messages",
"POST", array(
"To" => $number,
"From" => 'removed',
"Body" => 'Alert! '.$title.' found for '. $price. '. View the item here: '.$tinyurl,
));
if($response->IsError)
echo "Error: {$response->ErrorMessage}\n";
else
echo "Sent message to: {$response->ResponseXml->SMSMessage->To}\n";
}
?>
I think the problem is that you're doing a require inside the for loop. There are objects defined in that twilio library so the second time you require it, the classes get defined again and this throws an error.
If you have error_reporting(E_ALL) set then you'll see an exception to that effect in your output.
I would either change it to a require_once or move it out of the for loop.
I hope that helps.
We have a growing mailing list which we want to send our newsletter to. At the moment we are sending around 1200 per day, but this will increase quite a bit. I've written a PHP script which runs every half hour to send email from a queue. The problem is that it is very slow (for example to send 106 emails took a total of 74.37 seconds). I had to increase the max execution time to 90 seconds to accomodate this as it was timing out constantly before. I've checked that the queries aren't at fault and it seems to be specifically the sending mail part which is taking so long.
As you can see below I'm using Mail::factory('mail', $params) and the email server is ALT-N Mdaemon pro for Windows hosted on another server. Also, while doing tests I found that none were being delivered to hotmail or yahoo addresses, not even being picked up as junk.
Does anyone have an idea why this might be happening?
foreach($leads as $k=>$lead){
$t1->start();
$job_data = $jobObj->get(array('id'=>$lead['job_id']));
$email = $emailObj->get($job_data['email_id']);
$message = new Mail_mime();
//$html = file_get_contents("1032.html");
//$message->setTXTBody($text);
$recipient_name = $lead['fname'] . ' ' . $lead['lname'];
if ($debug){
$email_address = DEBUG_EXPORT_EMAIL;
} else {
$email_address = $lead['email'];
}
// Get from job
$to = "$recipient_name <$email_address>";
//echo $to . " $email_address ".$lead['email']."<br>";
$message->setHTMLBody($email['content']);
$options = array();
$options['head_encoding'] = 'quoted-printable';
$options['text_encoding'] = 'quoted-printable';
$options['html_encoding'] = 'base64';
$options['html_charset'] = 'utf-8';
$options['text_charset'] = 'utf-8';
$body = $message->get($options);
// Get from email table
$extraheaders = array(
"From" => "Sender <sender#domain.com>",
"Subject" => $email['subject']
);
$headers = $message->headers($extraheaders);
$params = array();
$params["host"] = "mail.domain.com";
$params["port"] = 25;
$params["auth"] = false;
$params["timeout"] = null;
$params["debug"] = true;
$smtp = Mail::factory('mail', $params);
$mail = $smtp->send($to, $headers, $body);
if (PEAR::isError($mail)) {
$logObj->insert(array(
'type' => 'process_email',
'message' => 'PEAR Error: '.$mail->getMessage()
));
$failed++;
} else {
$successful++;
if (DEBUG) echo("<!-- Message successfully sent! -->");
// Delete from queue
$deleted = $queueObj->deleteById($lead['eq_id']);
if ($deleted){
// Add to history
$history_res = $ehObj->create(array(
'lead_id' => $lead['lead_id'],
'job_id' => $lead['job_id']
)
);
if (!$history_res){
$logObj->insert(array(
'type' => 'process_email',
'message' => 'Error: add to history failed'
));
}
} else {
$logObj->insert(array(
'type' => 'process_email',
'message' => 'Delete from queue failed'
));
}
}
$t1->stop();
}
hard to tell. You should profile your code using xdebug to pintpoint low hanging fruit.
Also I think you might consider using a message queue to process your e-mail(redis/beanstalkd/gearmand/kestrel) asynchronous or using third-party dependency like for example google app engine which is very cheap($0.0001 per recipient/first 1000 emails a day free)/reliable. That will cost you about 10 cent a day when considering your load.
you're facing a couple of different problems.
1.) to send a lot of emails you really need a mailer queue and several mail servers to get mail from that queue and process the mail in turn (round robin style <-- this link is related but not perfectly specific to your needs. [It's enough to get you started]).
2.) your mail is likely not getting to Hotmail/yahoo for one of 2 reasons.
a.) you don't have RDNS properly configured and when the look for your IP (from the heder) it is not mapping back right to your domain in the header. or
b.), you've already been flagged as a spammer on SPAMHAUS or whatever the blacklisting service du jour is.