I was wondering if there is a way to run a PHP loop in order to send a few hundred emails to subscribers in background. My goal is to format the newsletter, click send and then close the browser or change page. Of course, the actual process of sending the e-mail will be running in background and would not be interrupted by the browser closing.
I know this can be made with a cron job reading from a queue saved in MySQL or text file, but this way, even if there is no queue for a long period, the cron will always be running, looking for the queue...
I've seen this funcionality in a script called Pommo (https://github.com/soonick/poMMo) but can't seem to understand how it's done.
Does anyone have an idea for this?
I was going to add a comment to your question, but them I didn't have enough space there to format and give the example.
Here is an idea I believe might work:
1 - Load all the emails you want to send to a database or file.
2 - From your web application click on the button to send emails. This will submit an Ajax request to the server. You can define the number of emails you want to send within a certain timeframe. Remember that most hosts have limits on number of emails you can send every hour.
3 - create a php script that will receive the Ajax request and send all the emails within the parameters you define.
4 - I believe you can kill your web browser because the PHP script will run through the whole list and will not return until it finishes sending all the emails.
The above might work, however I would never do it this way. I would use a cronjob as stated above. Your cronjob would only have to check if there are emails to send or not. This is not resource intensive.
If you decide to implement the ideas above, please make sure you let us know. I am curious if that would work.
Good luck!
I know this can be made with a cron job reading from a queue saved in
MySQL or text file, but this way, even if there is no queue for a long
period, the cron will always be running, looking for the queue...
That pretty much beats the purpose of Cron. You should create a job that runs, say, every 15 minutes and checks the queue for mails that need to be sent. If there are no mails, let the script die, it'll run again in 15 minutes.
If there are mails to be sent, update the rows to indicate that you're processing them before you start sending, so a run taking more than 15 minutes won't cause another script instance to send the same mails.
You need a queue system. There is e.g. Beanstalkd for linux, which you would feed things with php.
Related
I am developing a mass-mailing system. At a time we send 2-4K emails, the email contacts are imported using PHPexcel library at same quantity of emails.
Last night when we are sending 2k emails we get the "500 internal server" error.
I think I should develop the new process for email handling and the contacts importing, am I right? If so, how should I do it? Is there any other way to overcome such 500 errors?
The PHP script is called by web browser and browser loads it for 5-10 minutes and then 500 error occurs. I am using the PHPMailer library for sending the mail.
Calling a long-running PHP script from a web browser is not really the same as running PHP in the background. That will lock up an Apache thread, and is likely to be subject to whatever timeouts PHP has configured. My guess is the timeout is being hit before the send has completed.
It would be better to do this on a cron. Here are some general pointers:
Every ten minutes, select the next unsent set of email addresses from your data store, maybe 100 of them.
Send an email to each one, logging to a database what you have done
Pause for a few seconds. This is helpful as it makes it less likely your mail will be directed to spam bins
If your script has been running for more than five minutes, exit (it will do the next set of email addresses on the next cron call)
Otherwise, loop back to the start
That will be much more reliable. For bonus points, write a web page to show you what emails are sent and which are still waiting. Also, you may wish to use a third-party mailing system like MailChimp, to increase your delivery reliability. Make sure all of your recipients really have opted into receiving email from you.
I've suggested the script should batch in groups of 100, run for five minutes, be called every ten minutes, and pause for a few seconds after each send -- but these are just examples. If you don't mind sending more slowly (e.g. overnight) then you can change these figures to suit. Sending more slowly is generally more reliable, so do this if you can.
Here is the 'use case':
Admin goes to the newsletter.php, fills in the form for email, i.e the subject, group of users to whom email is sent, writes the message and clicks the "Send" button.
Problem:
The number of emails sent per hour should be limited to, let's say, 400. That is, one email should be sent approx. every 10 seconds. Besides, sent and not sent emails should be tracked.
Question:
Will cron job do the trick?
The code is written in Yii framework. Is it possible that cron job will be activated when the user clicks on the "Send" button or only in the command line?
If cron job can do things above, can it be activated only in specific action of specific controller? or it affects the whole script?
Thank you
1- Yes it is a cron job.
2- You can do it in Yii refer to ConsoleCommand. Don not know how much do you know about cron? but make a script using consolecomamnd which runs every 10 secs, gets a singe email address in the queue and sends email to that address + removes it from the queue.
3- Yes in an action just build a queue (hint: mysql table) with the email address you need
I think you have the concept of a cron job wrong.
Think of the process this way:
Admin presses the send button.
...That script creates a queue i.e. adds email address's etc to a database table lets call it Queue. And finishes.
You setup a cron job to run every 5 minutes for example.
...The cron starts a PHP script which processes X entries from the Queue table, sends those emails, removes the entries from the queue table, and stops.
...The cron job starts again automatically after 5 minutes and repeats the process...
All you have to do is work out how many emails to send in each execution of the cron job, and how often to run it, so you dont exceed your limits or get flagged as a spammer.
My suggestion is that if you wanted to use cron to do such a thing, you would want your newsletter.php to write out a file to disk or database which contains the list of users. You would write a simple PHP script which would be triggered by cron which would be responsible for calling sendmail to send the messages. As each recipient is mailed you script would remove them from the list.
Alternately, you might want to look into some basic mailing list software. These often support a notion of throttled message sending.
I assume you are trying to manage limits imposed by your hosting provider related to number of emails sent per hour (antispam controls)
Yes cron jobs can be used in such a scenario. I personally have developed newletter systems which use such methodology. However, I send 350 and do a cron job every hour.
You should check if there are any emails to be sent on the first line and leave the cron job running every hour. There is no need to activate the cron job when the send button is clicked
cron itself just runs programs on a schedule. It doesn't have any logic in it that can know whether somebody clicked on a button, or logic that can know how many emails you've sent this hour, or which emails have or have not been sent.
cron runs under the control of a daemon. Although you probably could enable and disable it through a controller, you probably don't want to do that. If you're using cron, you usually need it to run all the time.
Instead, put the logic and the constraints into the program that cron runs.
if you can use system or exec function you could call cron job or any thing else in terminal can do it
for example system("ps aux | grep crond");
I have a mailing script which loops through approx 900 subscribers (in database), building an individual email for each subscriber and sending via SMTP. This script is fired manually via the browser, however the number of records involved have resulted in the script starting to time out part way through. In tests, the error is almost always Fatal Error max execution time exceeded (although one time there was an error related to mail() and SMTP). I'm thinking that I should probably be running this type of script from the command line, however, the script still needs to be triggered manually (via a CMS admin user) - does anyone have any suggestions for a good way to handle this?
I think you can set the timeout for a PHP script from within the script.
However, I'd rather use CRON+PHP-CLI based solution. The PHP code remains the same whether you use CGI or CLI, however, in CLI mode there is no implicit time limit.
Tweeking the run time limit will only temporarily defer the problem.
You didn't say what OS this is running on - certainly if you've got a local MTA that may help, but the right solution would be to spawn a seperate process. Note that there are a number of pitfalls in this approach - see this post for details of how to do it correctly on Unix type systems.
set_time_limit (0); should do the trick.
There is not much to say about more efficient way, unless we see your code....
work with cronjobs and every time the cron executes the script it can send a small amount of mails.
For example the Typo3 mailer engine is invoked every 5 minutes and will check if any mails need to be sent.
So the user doesnt manually trigger the mailing script. The user only marks a newsletter which should be send by the mailer engine.
Putting aside the disdain from junk marketing I need to send around 15,000 emails to customers. My coworker has tried to send them through php mail loop but obviously it gets stuck quickly. Is there a conventional (i.e. through PHP script) to accomplish this swiftly? If not, how do you suggest I do this (maybe through exec) without too much overhead?
Thanks!
I've used PEAR's Mail_Queue to queue up 200,000+ mails at a time. Populating a database is easy and quick, even with customised content, and then a fairly simple script sends around 250 a time - if the load average isn't too high. Then it loops around and sends the next batch.
You won't send them any faster than is usually possible, but it will do it without any problems.
The tutorial gives you almost everything you need - just loop around th 'send_messages.php' script (from the command line is better) until the database queue is empty.
You could look at using something like Gearman to create a queue system as recommended here. Another option would be to look at a paid service like Amazon's Simple Email Service (SES)
No matter how you implement immediate delivery: it'll be a lengthy process that's always subject to interruptions and you can't afford restarting the delivery and sending the same message twice to 5,000 customers.
I believe that a reliable system must use queues. The main script simply adds recipients to a queue and then you have a secondary process that picks items from the queue, get them sent and finally mark them as sent. This secondary process can be launched manually (maybe from the command line) or via cron tab.
I've never used but I have this in my bookmarks: http://ledscripts.com/free/php/phpledmailer
Are you running it through CGI or as a script on the command line? It's best to run it as a script on the command line.
If you say that it gets stuck, try running set_time_limit(0); to avoid PHP quitting from the execution being too long.
I built an email alert for my users (now are only 2,000)
so every night a crontab execute a php script that query the mysql to find matches with user's saved search. it's a classified website in my case, but i would like to learn in case i had to build something for bigger clients
my concerns are:
what happen if my user grow x10 or
x100 times? is the server going to
crash? there any tip you can suggest
on manage something like that?
there is any way to protect my file
cron/nightly_script.php to be
executed form outside calling it in
the url of the browser? consider
tham im using a string in crontab
like:
lynx [absolute url/script.php]
what about the email blast? for each
query if the query has results the
script sends an email, so it means a
blast of emails...is it going to be
considered spam automatically and
then i could blacklisted?
thanks!!!
what happen if my user grow x10 or
x100 times? is the server going to
crash? there any tip you can suggest
on manage something like that?
Your server could crash/get slow as hell because of extensive memory/cpu usage. You should use a message queue like redis/beanstalkd/gearmand to throttle your email alerts. My preference goes out to redis. use the blocking pop/push with predis library which support blocking pop/push.
there is any way to protect my file
cron/nightly_script.php to be executed
form outside calling it in the url of
the browser? consider tham im using a
string in crontab like:
Don't use cron if you want to scale. Instead create couple of daemons.
1 to schedule sending messages(this part could also be cron) to message queue,
1 to process messages send to message queue.
Daemons don't need to be spawned each time and spawning processes is (relative) expensive. Second your script should not call any URL anymore but instead call the PHP scripts directly(CLI).
what about the email blast? for each
query if the query has results the
script sends an email, so it means a
blast of emails...is it going to be
considered spam automatically and then
i could blacklisted?
When using a message queue you can throttle yourself!
Well, you should probably modify your script so that you can spread the load. For example, you can have the cron run 4+ times a day and each time it does a percentage of the user base, instead of doing them all once a day.
You can take it out of the web server target path and put the cron somewhere that i not accessible externally. It could be executed like this: php /location/of/script.php
I guess it will vary depending on who you send it to, but you should consider how often you send this notice.
Number 1: Monitor the server, watch the load and the time it takes to run. It shouldn't crash it but you may find you get to the point where the load is to high and requests for web pages start to slow down.
But one thing to watch is PHP's memory garbage can be odd sometimes, so watch memory usage of the cron job. If it gets to high PHP will crash.
If it starts to get to much there are lots of solutions; there is no need to have the web server and the email sending on the same machine for instance. As long as they can access the same DB, set up a 2nd server just for email sending. This is what cloud computing is perfect for, hire a 2nd server 4 hours a night (or whatever) and turn it off the rest of the time.
That's just one suggestion ... there are many solutions and it really depends on your situation.
As for number 2, the best solution would be to move the script outside the document root so it's not accessible from a browser and call it directly
php [location/script.php]
If you can't do that, I would do a IP check and only allow it to be called from localhost IP.
You could also build in safe checks: store when the last time you sent a email to a particular user was and check that before sending another. This would protect against crontab problems as well as hackers.