I am working on a project that needs a emailable mailing list built into it with a very complex list structure. Originally I thought I could just use PHPList to do the job with some minor alterations because of its list management. But now I'm finding that I'm going to need to build this from scratch. PHPList is way to complex to make tweaks to.
So I'm looking for recommendations of scripts, classes, functions to help with the processes so I can just focus on the logic and list management.
The requirements are as follows:
User Subscription web form that captures name and email. Double opt-in that sends a confirmation email with a link back to a landing page. (Looking for a simple script to manage this aspect).
Landing page for opt-in confirmation from subscription email. This is where I need to include my list structure. The lists are very deep nested set trees and way beyond the scope of any mailing list I've come across. I'll need to build this part as it is too complex.
User login for future tweaking of their list settings. Would like a script to help with this aspect.
Admin page for constructing the email (html format). Looking for a simple script to help with this process. The site uses CKEditor for most html input so I could just end up using that here. I also then need to link it to the nested set tree list so I'll need to construct this portion myself. Looking for a simple class/script that I can use as a starting point to help with the structure.
Mail manager script. Ideally would like a cron job option to send the emails at a certain time each day/week. Also would like it to stagger the emails with a pause every 25 or so to bypass spam flags. A log file of what got sent where would be ideal. I'll need to construct the logic of what gets sent where because of the list structure. Would like an existing management script as a starting point.
Mailer script. Finally need a class/script to do the actual sending. I've used phpmailer in the past but wondered if there was anything better nowadays.
As I said at the start, PHPList is reasonably close but was far too complicated in its structure to be delving and changing the way it handles the lists. As such I'm looking for scripts that are user-friendly to tweak and hack.
My 2 cents on this:
use utf-8 for all the system
normalize your emails
create a master suppression list
(many domains wont like you sending
emails to them. with this you can
avoid send emails to people who
probably will block you)
sending emails: create a queue:
divide the process in stages: email
rendering / email sending / etc
phpmailer is enough good
abuse of the use of the
"List-unsubscribe" header and many
others
create a really good bounce handling
script (since keep sending to bad
addresses kill the reputation)
setup you rdns for all your sending
ips
use spf and DKYM
clean the email's body from bad
user input.... some users will try
to send "embed" tags on the body..
that kill the sender reputation
follow all ISP/ESP instructions
about: email per connections /
timeouts / etc ..... yahoo! has it
as many others ISP
Related
This question already has answers here:
How to send 100,000 emails weekly? [closed]
(3 answers)
Closed 9 years ago.
I have a custom CMS for my customers. That's because most of my clients are in a specific market (film industry) and need very specific tools.
But they also need to send newsletters. I did create some basic WYSIWYG HTML editor and managed to create a send out script with PHPMailer.
Everyone else seams to use MailChimp. Why? What are the reasons to use mail chimp over a custom made script?
How does it affect reader's compatibility (and does it?)
How does it affect spam blocking?
I've always thought sending e-mails through the original server was the way to go to avoid having a senders' mail not having the same reverse domain name.
Wouldn't it be better to simply create a clean inline css and send it out by connecting to the local SMTP server? No cost, no mail chimp monkey ad, etc.
Edit: This is not for the purposes of sending spam and most of my customers only have lists between 100 and 1500 recipients, they are all professional mails (film making industry). It is not about how to send 100K e-mails.
Strictly speaking there is no reason you can't do what Mailchimp (and others) are offering. However they do tend to make it a lot easier to work with.
They all have their different approaches, but they tend to speed up development of complicated mail campaigns and allow them to be managed better. This is particularly true of HTML emails, which can be an absolute time sink to get right on all browser & mail client combinations. (Gmail on IE8-11, Mail app on iOs, Outlook on Mac... etc. etc.)
I'm speaking specifically of Mailchimp here, but this should apply to many others:
Templates. They provide HTML email templates to get your designs started. This can avoid a lot of subtle errors in HTML mails.
Device preview. They provide a preview service to display your mail on a large number of devices & mail clients. So you don't have to go through them all yourself.
Opt out service built in. In Europe at least all marketing emails must have a link to unsubscribe from the mailing list. It's a legal requirement that can bite you and they handle this neatly. The link is always embedded and you can't accidentally re-add someone when you create another list.
Scale. They are built to sent lots and lots of mail. This can reduce the load on your servers and keep them from falling over. It can be particularly useful to make sure mail gets delivered quickly and doesn't sit around in a mail-queue.
Management. You can set up the campaign (as a developer) and hand over to the marketer(/someone else) to manage the campaign. => More time on SO for you!
Spam. They may come with an anti spam tool to check your mail before it's sent out. Not foolproof, but handy before you spend your money on a campaign.
This may sound like an endorsement for Mailchimp, but any of these kinds of services will do much the same. There are many of them so shop around and find the best fit for you. My main point here is don't re-invent the wheel. Emails may seem simple, but can get complicated fast.
I would like to add a form that allows my site visitors to sign up for a newsletter from my site.
I started to build the form, but apparently it's not such a good idea and instead I should use prebuilt scripts by other people to do this as they would be more secure.
Would it really be all that bad to make my own and just have it query all the signups into a database? I don't want it to be all that complex, just need something simple, but I definitely don't want to jeopardize the security of people signing up.
Thanks!
You can do it yourself if you know how to write it correctly.
The reason why people use tried and tested code is because you may be black listed by email services due to large volume of emails you would be sending. Once you are black listed then it is difficult to get yourself off the black list. The tried & tested code SHOULD send the newsletters out periodically rather then mass emailing at once.
Other reasons is due to security, if you write the code incorrectly or if you are new to forms which access databases, then it may not be sensible to write your own. It is good practice to learn it though, even if in the mean time you have to use someone else's code.
You also have to verify emails addresses by a confirmation request email and an associated form. In addition if you have a larger list (more than a few hundred members), then you must code bounce handling and unsubscriptions. For a small list you can do these manually. I did build such a web application step-by-step, but if I had to start it now, than I would consider an existing mail list software more seriously. At first I also thought that I only need a small form, and the list managers I checked was indeed difficult to integrate, but in the long run it was too much effort.
New to the forum but love the comments. I'm tech savvy but not when it comes to programming, coding etc. So - I guess that makes me tech stupid for the moment.
Basically - I want to create a mass email program that will allow us to send bulk emails to our customers who signed an email slip. Currently - we're at 100,000 or so in our database but we expect to see this rise to 200,000 at some point.
What is the best script language to use for this?
Is PHP going to be handle this many emails?
What is the most emails that I can send in any one batch for it to still be "safe" so that the emails arrive to the proper recipients?
What is the term when you send directly from the server?
We currently have 3 servers and I'm assuming they are capable of doing this no problem...but I haven't spoken to our webhosting company just yet.
Lastly - I put together a powerpoint...that gives specs - is there anyone here that would be interested in a little side work to do the project? Really interested in the feedback. Thanks.
In general and based off what you said -- do not create your own mass email program.
Use a service, such as MailChimp or iContact.
Here's a small list of some of the things they do for you:
Manage the list
Handle unsubscriptions
Handle soft bounced emails
Handle hard bounced emails
Handle white listing (You have to pay third party companies $$$s a month to be on these lists)
Give you an idea of how likely and spammy your email is
How fast your email can be sent out (AOL will put you on their blacklist if you send out too fast)
Handle Analytics (such as how many were opened, when, where, what links were clicked on, linking to Google Analytics)
Handle multiple lists and variable replacement (such as "Dear _____")
Nice, easy-to-use interface.
Again, that's a small list in the scheme of things. Those were off the top of my head without too much thought.
EDIT
A couple big advantages I didn't mention:
a) Support
b) A community
c) Constant improvements
d) No server-management
Step 1. Acquire a real email server.
Step 2. Utilize said email server by having your scripts send the mail to it for delivery, and not directly.
Step 3. Throttle the mail server as necessary
Step 4. Monitor the non-delivery reports and remove addresses as necessary...
It depends on how you do this. You can:
1) loop and send 100 000 individual mail
2) Send 1 email with 100 000 people in BCC
or mix both. php can handle it well, but it's a huge task on the mail server.
Hey guys. I need help for a project for school. Essentially, I need to write a program that sends newsletters. I can send the newsletters and stuff, but the main issue I'm having is that the newsletter needs to contain things that are contained in a database. How exactly do I draw those things and put them into a newsletter? The bulk of it needs to be in PHP, but Java and stuff can be used, if needed. Also, I cant install anything on my portion of the schools server, so I cant use any CMS's and stuff.
Unless you are writing this for fun and learning, don't do it.
You will get your IP blacklisted and have lots of problems. there are regulations (CAN SPAM etc) and best practices (SPF, domain keys etc) you need to follow if you want this to be successful. Otherwise you will not get any of your mail delivered, and possibly a call form your ISP.
How do you handle unsubscribes? How are you going to handle bounces? Mail loops? Blacklisting of problem addresses? This is not a simple topic.
You would be better off purchasing a email management system;.
Lyris makes one that you can install. There are other free/open source projects as well (majordomo, mailman etc).
Heed these words intrepid e-mailer!
Hey, do you have basic problems with querying your content from your database?
I'm about ready to unveil a "coming soon" page and one thing I need is a way for users to enter their email address for me to email once the site goes live. What is the best way to do this?
Should I store the emails in a DB and then run a PHP script to email them from my web host? Should I have the emails just kind of be collected and then emailed manually by me either locally or from the server? Is there another way I should do it?
Also, are there certain web-hosts that restrain the amount of emails you can send out, thus causing a problem for mass email.
The only "unknown" currently is how large of a response I'll get...only time will tell.
Thanks guys!
Store it in the database - Yes. It's no unlike storing any other piece of data.
Mass email at once - No
Mass email manually (or individually mail manually) - No
Do some web hosts limit how many you can send at once? - Yes
I don't know the "best" way but I know a really good way. We have built several mass emailing programs and the technique we incorporated was a throttling technique whereby we had a script that ran every three minutes and sent 20 emails at a time.
It keeps the server from choking and the mail queue from exceeding any of our hosts' mail limits.
Store the email addresses in a database and write a batch job that mail merges them into your message and sends them out as needed.
There are dozens of methods you can use for this kind of problem, but unfortunately there's no real way to pinpoint a solution for you, since there are a lot of variables.
If you only get 3 responses, then you might want to just manually email the users from your email client using BCC:. You've already got all the tools you need to do that, and setting up a script might be a waste of time. This isn't really a great long-term solution however.
For most moderate-sized web sites you would store the emails in a database, and use a mailing script to send them out. PHPMailer is a good tool to help with getting mail sent, and you can manage a decent amount of email addresses manually. Managing addresses manually can be a bit of a pain however, as you have to deal with unsubscribe links, script timeouts, bounced email, etc.
If you get up the tens of thousands of email addresses, you may want to start looking at a third-party mailing software or service that can do threaded sending. Looping through and sending email to 50,000 people via a PHP script can be slow, and take hours.
Basically you have to weigh the difficulty, time and cost of each method versus how much flexibility and power you think you're going to need.
To store emails in a db, and sending them out using a cronjob is a good way to solve the problem, if you have constrains regarding the amount of emails you're allowed to send within a period of time, you can handle it by keeping track of how many emails you've sent every time the cronjob runs.
Also, there are open source products to do this, such as phplist http://www.phplist.com/
Like Eric mentioned above, another option is to use a commercial service to manage your mailing list.
I use Mailchimp (http://www.mailchimp.com). They give you everything you need (signup forms, email templates, etc.), and are completely free unless your list grows to more than 500 subscribers.
Writing the emails to a database or a text/log file are both fine ways to store the emails.
Depending on how many emails you receive, you may want to write a program (PHP works) to send a separate email to each person. Don't send a mass email from your regular email client with everyone on a big To: line.
There are also commercial programs that manage mailing to lists of people (probably open source ones, too). Most of those commercial ones offer a free trial period.