I figure I can either use one mail() command with a large Bcc list or have a loop that sends many individual emails.
I was just going to use Bcc since that is easiest to program and seemed easiest for the server to handle, but then I have to choose some address for the To field. I can just send mail to the websites own address but it would look more sensible to recipients if it was addressed to them. Also, it would be nice to customize each message by saying "Hello [firstname]" at the beginning.
I'm just concerned that sending to too many people will take too long. The maximum number of recipients will be 2000. Users on the website choose a list of people to send to, type a message, and press Send. Would they be waiting forever if it was sending to 2000 people? Would the server choke?
Also what considerations are there regarding mail servers thinking of this as spam?
EDIT: Apparently my client has an SMTP server he says can throttle the outgoing emails. Still not sure though if the PHP would be slow when sending to 1000+ people...
Sending large number of emails at once can really bog down your server, or if its a shared hosting, there is a limit to the number of emails that can be sent over an hour (with bluehost its 700 per hour). So i would recommend you to send emails in chunk.
Create table email_queue with two fields email_to and email_content. Now whenever you wish to send an email, just insert a record into this table with the email address that you wish to send the email to stored in the email_to column and the raw email content in the email_content column.
Next you would create a cron job that runs every hour, that cron job would check the email_queue table to see if it has to send any email, it will pick up 100 records from the email_queue table and send those 100 emails, when the emails have been send those 100 records would be deleted.
This I think would be an ideal way to send out emails in large numbers.
It's a pretty complex subject with respect to making sure the emails don't start looking like spam. You can really do yourself some favours by hooking it up to something like MailChimp.com and letting them deal with the nasty details for you.
I would actually recommend looking at a third party that can be accessed by an API. Mailing out large numbers of e-mails can be detrimental to your server as it can end up black listed. Check out a company like www.postmark.com or something similar that will throttle your message queue, manage white listed servers, etc.
Related
I have a big amount of users that agreed to recive daily newsletter. Content of newsletter is being auto-generated, so the only thing to do is to set up a cron job which would send e-mails.
However, if there is e.g. 10.000 users, such cron job would kill my server. What can be done to solve this problem?
Is sleep(1) after sending 100 e-mails enough? (and of course setting execution time limit to 1 day)
Have a look into http://php.net/manual/en/function.mail.php
Note:
It is worth noting that the mail() function is not suitable for larger
volumes of email in a loop. This function opens and closes an SMTP
socket for each email, which is not very efficient.
For the sending of large amounts of email, see the » PEAR::Mail, and »
PEAR::Mail_Queue packages.
So simply use the Mail_Queue package... which takes every mail and then simply works through them.
I have made a system for sending emails for project few months ago so I done the following:
In database, I have 3 tables:
users
user_emails (some users have multiple emails)
email_campaign (this is table in which I store data temporary while sending campaign and on finish I truncate everything in it)
And when I start sending campaign I insert row in email_campaign table for every user that I finished sending email to.
This way if error occur before campaign finishes, I know where to continue and know to whom I sent email and to who need to send email.
Practicaly I was able to send 45.000 emails during 2 hours. Without server overload.
I use sleep() on every 100 emails like you wanted to do.
Also I send campaigns at 2 am in the morning when my server load is the lowest.
You could also configure your email server to send limited ammount of emails per hour.
This would slow sending but it will reduce server load.
I'm using a homemade PHP script with PHPmailer for sending email, currently I just do one select statement in MYSQL and then run a 100-150 email send request from PHPmailer via a for loop.
my problem is that I have to port this script to a 7000 user mailing list, and I know it's not going to hold up since sending to 150 takes about 8 secs of execution time - also sending 7000 emails might mark me as a spammer or I won't be able to handle all responces.
I thought about splitting the 7000 into groups of like 200-500 and sending an email to each group every day until sending everyone - to do that i'll have to write a PHP script that handles the limits and remembers when & who on the list hasn't received that email yet and is due to receive it on the next batch.
What are my options, what are your thoughts?
Just my 2 cents.
1) I agree with Dave re: make it take longer than it needs. Most black/grey listing is the the recipient mail server deciding there is an odd amount of traffic compared to usual. Yahoo is terrible for this and given the sheer volume of domains they deal with, it can get annoying.
2) Using multiple IP addresses doesn't hurt. Extra IPs are usually free, and if you are using exim, you can easily configure it to use each IP based on e.g. EHLO, or send to domain etc etc. This means that there is less volume coming from any one IP.
3) Keep a track of bounce backs. Some providers will keep a track of the number of attempts to deliver mail to a non-existent address, the more you try, the more suspicious it looks. Easiest way to keep a track of them in my experience is to use VERPs and get PHP to process the incoming data.
4) Set up reverse DNS. Some hosts will just ignore the mail if you have not got it set up for the sender. I believe AOL is an example.
Just a few pointers from my experience.
Your first approach is a pretty good idea. As long as you're ok with it taking about 3 minutes to send all the emails (which it should be, if you're only sending a weekly newsletter or the like). To reduce the chance of getting marked spam, lower the threshold or make it take longer. Beyond that, it might be a good idea to move into a hosted mass mailing service.
The act of sending 7000 [or more] emails is not going to get you marked as a spammer ~assuming~ that the mail you are sending is:
not spam
contains complete & accurate header information
The only problem you might get is clogging up your mail queue, but you r volume is still pretty low. I would say for scalability - throttle your delivery over about an hour or so.
and consider ceejayoz's suggestion "MailChimp" it's free up to 5000 emails/month and has an excellent API that you can literally integrate into your app in a couple of hours.
-sean
I need to send about ~20k emails to potential clients and need the best way to avoid:
server crash
being put in spammers list
I've searched a few forums and people said it's best to send mails in 1k packages
but my question is what delay between sending each 1k should I set? I mean 1k/day or 1k/hour or 1k/min etc.
I recently had to build system to do so, here is the solution we chose :
1- The management system setup emails and store them in a database.
2- We setup a no-reply account on our mail server to get the bounces back.
3- We build a small code over the great mailing library swiftmailer (swiftmailer.org).
4- We run the script we build through a cron and send mails in packages of 50 per minute (hotmail will most likely flag email as spam if too much emails are sent from the same adresse/server in the same minute). We store each swiftId i the mail database
5- At the runtime of the mail sending script we check for bounces or other errors in our no-reply mailbox and flag our messages in our mail table accordingly (status : 1 - success, 2 - invalid email, 3 - bounces)
6- We sync back our data (upon user accessing the section of the system) in the main system when they have a status and that the last change was made at least 10 minutes in the past to limit the change of states in the system. (the synchronisation script can change the status of an item in the main system, but the 10 minutes delay reduce the risk of changes)
I think your 1K thing is a red herring. There have been a number of similar questions asked with good answers, have a look though these:
How do you make sure email you send programmatically is not automatically marked as spam?
Avoid being blocked by web mail companies for mass/bulk emailing?
Sending solicited mass email
Note that the primary requirement to avoid being your mail being marked as spam is NOT TO SEND SPAM. Your description sounds a lot like spam, in which case no amount of clever tricks will help you.
Providing you are not actually sending spam, a professional organisation such as http://www.campaignmonitor.com/ or http://www.mailchimp.com/ is probably cheaper than the amount of time you will require to do this properly.
Your best bet is to farm it out to specialists.
There's a whole load of subtleties about mass mailing that are easy to get wrong.
Best case is that your server gets put on an email blacklist - meaning all your mails from now on get binned as spam. Worst case is fines and/or jail time, depending n your local laws and the laws where the recipients live.
When it comes down to being put in spammers lists, the question is, how many (very) similar mails from the same sender reach a certain host. If you flood yahoo-mail-accounts with thousands of mails within a short period of time, you can be sure to be on their list shortly.
What you can do: pay a professional service to send out the mails for you. They usually have contracts with the big providers so you don't end up on anyones blacklist (will cost you money, though).If you're sending them out from your server, make sure the "reverse DNS lookup" for your server is activated - I don't know if that's checked any more but it gave me a hard time once.
If your mails don't need to be sent out at a certain time, I'd send the mails in very small packages, e.g. not more than one mail per second, not too much in a row. Then wait for a certain amount of time and repeat until all mails have been sent.A company I used to work for would send out not more than 200 and then wait half an hour before the program proceeded. As far as I know, we never ended up on some blacklist.
The spam list has been well answered, but as far as not bombing out your server, the answer is generally the same. Use a service that does this for you.
If you absolutely must send this volume of e-mail on your own, you want to do it with some sort of background service. This can be written in anything (provided that you don't simply buy one of the many pieces of software off the shelf), but ideally it should be multi-threaded.
Your management application should not be sending these e-mails... only queuing them up in a database or something.
Again though, why reinvent the wheel? You'll save yourself a lot of time by going with something off-the-shelf.
I have created a newsletter system and my question is: How should i write my code considering that i have to send that mail to hundreds of email addresses?
I've discussed with my host administrator and he told me that i should send my emails one by one but not more than 6 per minute.
Can i use the $Timeout property? If so, how?
Thanks.
If you have to send out the mails one by one (instead of using BCC), I'd use a database-queue to respect the limit of sending out only 6 mails per minute (no matter what solution you'll finally use to actually send the mails).
E.g. you'd have a database-table containing recipient, subject, mailbody, lastsenddate, timessent and status.
Save all mails, you will send out, to the database and then set up a cronjob that will run once a minute and check if there are still mails in the queue waiting to be send (e.g. status = "unsend"). Then you'd pick a maximum of 6 (or whatever your limit is) mails from the queue, send them out, set the status to "send" (and increase "timessent" and set "lastsenddate" to the actual time, if you like) and wait for the next cronjob until all mails are sent.
This way you have several advantages:
you can respect your per-minute-limit
you have all your mails in a database and can relate to them later (e.g. to find out how many mails - and which ones - you sent last Friday or to find out whether a certain address has been processed - and when and how many times - if someone claims he never received a mail / or too many)
by tracking a mail-status you could implement a bounce-handler that'll e.g. set a mail-status to "bounced" if a mail returns, so you could start a resend of your mailing some time later to reach addresses that returned a "mailbox full"-message the first time
by saving your mails to a database, you could even setup a "deferred mail service" by adding a database-field "starttime" and make your send-script respect this date, so you could already queue your Christmas-mails in spring :)
Pear Mail will allow you to send email from PHP to allot of people.
http://pear.php.net/package/Mail/
What is the most proper way to sending email of minimal 1000 or more in PHP? Any reliable email queuing technique that is capable to handle that?
You could just insert your emails into a Mail Queue database table, and have a separate process check the queue and batch send a certain number at once.
There's a tested solution for that: PEAR Mail_Queue
Works fine for me.
as mercutio suggested, i would insert a new record into a mail queue table for each email waiting to be sent and then use a separate process (like a CRON) to check the table periodically for any queued items.
if any emails are queued (and the email is not customised for each recipient) i would then group the emails by domain and send blocks together to reduce the total number of emails that have to be sent, i.e. if you have 1000 emails queued and 250 are to gmail accounts i would send the 250 in 25 blocks of 10 (remember to Bcc recipients to avoid them seeing each other).
to actually send the mail i would use PEAR mail over php's mail() function
after sending the email either delete record(s) from the queue or change a status flag to show it was sent and loop - i would also add a counter to keep track of emails that failed to send and remove them after x failed attempts
to overcome timeout issues i would either,(depending on the situation)
- set the set_time_limit() to x seconds and keep track of the script execution time (killing the script after (x-1) seconds)
- call the script from the command line to avoid timeouts
- set a limit to the number of emails the script could send in one execution
Sure, the database table might be an idea. But what about sending 1000 e-mails with a 2mb attachment? you'd have to take that into account as well. I had the problem myself, and I eventually resorted to writing the e-mail to the database, and the files to the filesystem. The e-mail script I used then read the database records, and tried to fetch the attachments to send.
Are you sure you need do this mail queuing yourself?
Just deliver all mail to the local machine's mail transfer agent (sendmail...) and let that take care of the queuing and sending. After all, that's what was designed for.
In other words: don't worry about it!
I created Emailqueue, which is a server that allows you to add emails to a queue so your app get relieved of the stress of the mailing, and also provides useful additional options, like the ability to program emails to be sent in the future, or setting per-email sending priorities. I think this might very well be what you're searching for.
Emailqueue is available here: https://github.com/tin-cat/emailqueue
And there is also a Docker version that allows you to set up a working Emailqueue server in just a few minutes, here: https://github.com/tin-cat/emailqueue-docker
I've generally relied on a hack.
I have a database list of email addresses and then use a meta-redirect to self with an increasing 'offset' parameter that specifies which row in the database I am up to. Server redirects cause problems because browsers assume that the time taken indicates an infinite loop.