I'm using a homemade PHP script with PHPmailer for sending email, currently I just do one select statement in MYSQL and then run a 100-150 email send request from PHPmailer via a for loop.
my problem is that I have to port this script to a 7000 user mailing list, and I know it's not going to hold up since sending to 150 takes about 8 secs of execution time - also sending 7000 emails might mark me as a spammer or I won't be able to handle all responces.
I thought about splitting the 7000 into groups of like 200-500 and sending an email to each group every day until sending everyone - to do that i'll have to write a PHP script that handles the limits and remembers when & who on the list hasn't received that email yet and is due to receive it on the next batch.
What are my options, what are your thoughts?
Just my 2 cents.
1) I agree with Dave re: make it take longer than it needs. Most black/grey listing is the the recipient mail server deciding there is an odd amount of traffic compared to usual. Yahoo is terrible for this and given the sheer volume of domains they deal with, it can get annoying.
2) Using multiple IP addresses doesn't hurt. Extra IPs are usually free, and if you are using exim, you can easily configure it to use each IP based on e.g. EHLO, or send to domain etc etc. This means that there is less volume coming from any one IP.
3) Keep a track of bounce backs. Some providers will keep a track of the number of attempts to deliver mail to a non-existent address, the more you try, the more suspicious it looks. Easiest way to keep a track of them in my experience is to use VERPs and get PHP to process the incoming data.
4) Set up reverse DNS. Some hosts will just ignore the mail if you have not got it set up for the sender. I believe AOL is an example.
Just a few pointers from my experience.
Your first approach is a pretty good idea. As long as you're ok with it taking about 3 minutes to send all the emails (which it should be, if you're only sending a weekly newsletter or the like). To reduce the chance of getting marked spam, lower the threshold or make it take longer. Beyond that, it might be a good idea to move into a hosted mass mailing service.
The act of sending 7000 [or more] emails is not going to get you marked as a spammer ~assuming~ that the mail you are sending is:
not spam
contains complete & accurate header information
The only problem you might get is clogging up your mail queue, but you r volume is still pretty low. I would say for scalability - throttle your delivery over about an hour or so.
and consider ceejayoz's suggestion "MailChimp" it's free up to 5000 emails/month and has an excellent API that you can literally integrate into your app in a couple of hours.
-sean
Related
I'm working on a project for dispatching truck loads to subscribers so that they may try to get the cargo and make some money. My subscription service is going to be PHPmailer based.
Hypothetically let's say I have roughly 5000 subscribers. What would be the best way to send the emails simultaneously so that there isn't an unfair advantage as to go gets the email first vs someone who gets it 15 minutes later? I already have a working script for sending mail to different people without CC or BCC but I'm concerned with my site being marked as spam or a potential DDS attack.
What are my options? Would having a default email when a user signs up asking to be added to their address book work if the majority actually add?
I use GoDaddy as a host since that seems to matter sometimes.
I'm just concerned with being marked as spam or a potential attack site.
If I do need to create a delay, what would be the shortest delay possible while still being effective?
I have been working on this subject for a long time now and to be honest I am in need of some help if you guys have some time to answer to my problem ;)
I have a few k of emails in my database (around 2k and coming up...) and I would like to send two newsletters a week ... what would be the best way to achieve that ?
So far here is what I am doing :
- a script php (with meta tag refresh set to 600) which sends every 10 minutes an email to 50 people (using basic mail() function). It runs for 6 hours to get to the 2k emails. I am using the less expensive shared server at 1and1 (2EUR a month :D).
1- Is this way to proceed a good way ? I heard when you put people in BCC you go to SPAM directly ... Also many people do not even receive the emails ... no idea why. So far it is not really conclusive.
2- Do you think it would worth taking the dedicated server and send 1 email to each people thats to say 2k different emails ? Or is it a bad move ?
3- How do people proceed when sending newsletters ? I'm sure I look ridiculous with 2k emails and though I have difficulties in making my way through this ;)
Any help welcome.
Cheers.
G/
maximize time between connections to same mail servers
maximize number of servers you use
send to one person at a time
include images, text and html version, unsubscribe link
Check what mail servers respond (like IP blocking notice with link to unblock)
And yes, use cron
I need to send about ~20k emails to potential clients and need the best way to avoid:
server crash
being put in spammers list
I've searched a few forums and people said it's best to send mails in 1k packages
but my question is what delay between sending each 1k should I set? I mean 1k/day or 1k/hour or 1k/min etc.
I recently had to build system to do so, here is the solution we chose :
1- The management system setup emails and store them in a database.
2- We setup a no-reply account on our mail server to get the bounces back.
3- We build a small code over the great mailing library swiftmailer (swiftmailer.org).
4- We run the script we build through a cron and send mails in packages of 50 per minute (hotmail will most likely flag email as spam if too much emails are sent from the same adresse/server in the same minute). We store each swiftId i the mail database
5- At the runtime of the mail sending script we check for bounces or other errors in our no-reply mailbox and flag our messages in our mail table accordingly (status : 1 - success, 2 - invalid email, 3 - bounces)
6- We sync back our data (upon user accessing the section of the system) in the main system when they have a status and that the last change was made at least 10 minutes in the past to limit the change of states in the system. (the synchronisation script can change the status of an item in the main system, but the 10 minutes delay reduce the risk of changes)
I think your 1K thing is a red herring. There have been a number of similar questions asked with good answers, have a look though these:
How do you make sure email you send programmatically is not automatically marked as spam?
Avoid being blocked by web mail companies for mass/bulk emailing?
Sending solicited mass email
Note that the primary requirement to avoid being your mail being marked as spam is NOT TO SEND SPAM. Your description sounds a lot like spam, in which case no amount of clever tricks will help you.
Providing you are not actually sending spam, a professional organisation such as http://www.campaignmonitor.com/ or http://www.mailchimp.com/ is probably cheaper than the amount of time you will require to do this properly.
Your best bet is to farm it out to specialists.
There's a whole load of subtleties about mass mailing that are easy to get wrong.
Best case is that your server gets put on an email blacklist - meaning all your mails from now on get binned as spam. Worst case is fines and/or jail time, depending n your local laws and the laws where the recipients live.
When it comes down to being put in spammers lists, the question is, how many (very) similar mails from the same sender reach a certain host. If you flood yahoo-mail-accounts with thousands of mails within a short period of time, you can be sure to be on their list shortly.
What you can do: pay a professional service to send out the mails for you. They usually have contracts with the big providers so you don't end up on anyones blacklist (will cost you money, though).If you're sending them out from your server, make sure the "reverse DNS lookup" for your server is activated - I don't know if that's checked any more but it gave me a hard time once.
If your mails don't need to be sent out at a certain time, I'd send the mails in very small packages, e.g. not more than one mail per second, not too much in a row. Then wait for a certain amount of time and repeat until all mails have been sent.A company I used to work for would send out not more than 200 and then wait half an hour before the program proceeded. As far as I know, we never ended up on some blacklist.
The spam list has been well answered, but as far as not bombing out your server, the answer is generally the same. Use a service that does this for you.
If you absolutely must send this volume of e-mail on your own, you want to do it with some sort of background service. This can be written in anything (provided that you don't simply buy one of the many pieces of software off the shelf), but ideally it should be multi-threaded.
Your management application should not be sending these e-mails... only queuing them up in a database or something.
Again though, why reinvent the wheel? You'll save yourself a lot of time by going with something off-the-shelf.
I figure I can either use one mail() command with a large Bcc list or have a loop that sends many individual emails.
I was just going to use Bcc since that is easiest to program and seemed easiest for the server to handle, but then I have to choose some address for the To field. I can just send mail to the websites own address but it would look more sensible to recipients if it was addressed to them. Also, it would be nice to customize each message by saying "Hello [firstname]" at the beginning.
I'm just concerned that sending to too many people will take too long. The maximum number of recipients will be 2000. Users on the website choose a list of people to send to, type a message, and press Send. Would they be waiting forever if it was sending to 2000 people? Would the server choke?
Also what considerations are there regarding mail servers thinking of this as spam?
EDIT: Apparently my client has an SMTP server he says can throttle the outgoing emails. Still not sure though if the PHP would be slow when sending to 1000+ people...
Sending large number of emails at once can really bog down your server, or if its a shared hosting, there is a limit to the number of emails that can be sent over an hour (with bluehost its 700 per hour). So i would recommend you to send emails in chunk.
Create table email_queue with two fields email_to and email_content. Now whenever you wish to send an email, just insert a record into this table with the email address that you wish to send the email to stored in the email_to column and the raw email content in the email_content column.
Next you would create a cron job that runs every hour, that cron job would check the email_queue table to see if it has to send any email, it will pick up 100 records from the email_queue table and send those 100 emails, when the emails have been send those 100 records would be deleted.
This I think would be an ideal way to send out emails in large numbers.
It's a pretty complex subject with respect to making sure the emails don't start looking like spam. You can really do yourself some favours by hooking it up to something like MailChimp.com and letting them deal with the nasty details for you.
I would actually recommend looking at a third party that can be accessed by an API. Mailing out large numbers of e-mails can be detrimental to your server as it can end up black listed. Check out a company like www.postmark.com or something similar that will throttle your message queue, manage white listed servers, etc.
I'm about ready to unveil a "coming soon" page and one thing I need is a way for users to enter their email address for me to email once the site goes live. What is the best way to do this?
Should I store the emails in a DB and then run a PHP script to email them from my web host? Should I have the emails just kind of be collected and then emailed manually by me either locally or from the server? Is there another way I should do it?
Also, are there certain web-hosts that restrain the amount of emails you can send out, thus causing a problem for mass email.
The only "unknown" currently is how large of a response I'll get...only time will tell.
Thanks guys!
Store it in the database - Yes. It's no unlike storing any other piece of data.
Mass email at once - No
Mass email manually (or individually mail manually) - No
Do some web hosts limit how many you can send at once? - Yes
I don't know the "best" way but I know a really good way. We have built several mass emailing programs and the technique we incorporated was a throttling technique whereby we had a script that ran every three minutes and sent 20 emails at a time.
It keeps the server from choking and the mail queue from exceeding any of our hosts' mail limits.
Store the email addresses in a database and write a batch job that mail merges them into your message and sends them out as needed.
There are dozens of methods you can use for this kind of problem, but unfortunately there's no real way to pinpoint a solution for you, since there are a lot of variables.
If you only get 3 responses, then you might want to just manually email the users from your email client using BCC:. You've already got all the tools you need to do that, and setting up a script might be a waste of time. This isn't really a great long-term solution however.
For most moderate-sized web sites you would store the emails in a database, and use a mailing script to send them out. PHPMailer is a good tool to help with getting mail sent, and you can manage a decent amount of email addresses manually. Managing addresses manually can be a bit of a pain however, as you have to deal with unsubscribe links, script timeouts, bounced email, etc.
If you get up the tens of thousands of email addresses, you may want to start looking at a third-party mailing software or service that can do threaded sending. Looping through and sending email to 50,000 people via a PHP script can be slow, and take hours.
Basically you have to weigh the difficulty, time and cost of each method versus how much flexibility and power you think you're going to need.
To store emails in a db, and sending them out using a cronjob is a good way to solve the problem, if you have constrains regarding the amount of emails you're allowed to send within a period of time, you can handle it by keeping track of how many emails you've sent every time the cronjob runs.
Also, there are open source products to do this, such as phplist http://www.phplist.com/
Like Eric mentioned above, another option is to use a commercial service to manage your mailing list.
I use Mailchimp (http://www.mailchimp.com). They give you everything you need (signup forms, email templates, etc.), and are completely free unless your list grows to more than 500 subscribers.
Writing the emails to a database or a text/log file are both fine ways to store the emails.
Depending on how many emails you receive, you may want to write a program (PHP works) to send a separate email to each person. Don't send a mass email from your regular email client with everyone on a big To: line.
There are also commercial programs that manage mailing to lists of people (probably open source ones, too). Most of those commercial ones offer a free trial period.