Related
This is a tricky one and I've always relied on techniques, such as permission-based emails (i.e. only sending to people you have permission to send to) and not using blatantly spamish terminology.
Of late, some of the emails I send out programmatically have started being shuffled into people's spam folder automatically and I'm wondering what I can do about it.
This is despite the fact that these particular emails are not ones that humans would mark as spam, specifically, they are emails that contain license keys that people have paid good money for, so I don't think they're going to consider them spam
I figure this is a big topic in which I am essentially an ignorant simpleton.
Use email authentication methods, such as SPF, and DKIM to prove that your emails and your domain name belong together, and to prevent spoofing of your domain name. The SPF website includes a wizard to generate the DNS information for your site.
Check your reverse DNS to make sure the IP address of your mail server points to the domain name that you use for sending mail.
Make sure that the IP-address that you're using is not on a blacklist
Make sure that the reply-to address is a valid, existing address.
Use the full, real name of the addressee in the To field, not just the email-address (e.g. "John Smith" <john#blacksmiths-international.com> ).
Monitor your abuse accounts, such as abuse#yourdomain.example and postmaster#yourdomain.example. That means - make sure that these accounts exist, read what's sent to them, and act on complaints.
Finally, make it really easy to unsubscribe. Otherwise, your users will unsubscribe by pressing the spam button, and that will affect your reputation.
That said, getting Hotmail to accept your emails remains a black art.
Sign up for an account on as many major email providers as possible (gmail/yahoo/hotmail/aol/etc). If you make changes to your emails, either major rewording, changes to the code that sends the emails, changes to your email servers, etc, make sure to send test messages to all your accounts and verify that they are not being marked as spam.
A few bullet points from a previous answer:
Most important: Does the sender address ("From") belong to a domain that runs on the server you send the E-Mail from? If not, make it so. Never use sender addresses like xxx#gmail.com. User reply-to if you need replies to arrive at a different address.
Is your server on a blacklist (e.g. check IP on spamhaus.org)? This is a possibility when you're on shared hosting when neighbours behave badly.
Are mails filtered by a spam filter? Open an account with a freemailer that has a spam folder and find out. Also, try sending mail to an address without any spam filtering at all.
Do you possibly need the fifth parameter "-f" of mail() to add a sender address? (See mail() command in the PHP manual)
If you have access to log files, check those, of course.
Do you check the "from:" address for possible bounce mails ("Returned to sender")? You can also set up a separate "errors-to" address.
You can tell your users to add your From address to their contacts when they complete their order, which, if they do so, will help a lot.
Otherwise, I would try to get a log from some of your users. Sometimes they have details about why it was flagged as spam in the headers of the message, which you could use to tweak the text.
Other things you can try:
Put your site name or address in the subject
Keep all links in the message pointing to your domain (and not email.com)
Put an address or other contact information in the email
Confirm that you have the correct email address before sending out emails. If someone gives the wrong email address on sign-up, beat them over the head about it ASAP.
Always include clear "how to unsubscribe" information in EVERY email. Do not require the user to login to unsubscribe, it should be a unique url for 1-click unsubscribe.
This will prevent people from marking your mails as spam because "unsubscribing" is too hard.
In addition to all of the other answers, if you are sending HTML emails that contain URLs as linking text, make sure that the URL matches the linking text. I know that Thunderbird automatically flags them as being a scam if not.
The wrong way:
Go to your account now: http://www.paypal.com
The right way:
Go to your account now: http://www.yourdomain.org
Or use an unrelated linking text instead of a URL:
Click here to go to your account
You may consider a third party email service who handles delivery issues:
Exact Target
Vertical Response
Constant Contact
Campaign Monitor
Emma
Return Path
IntelliContact
SilverPop
Delivering email can be like black magic sometimes. The reverse DNS is really important.
I have found it to be very helpful to carefully track NDRs. I direct all of my NDRs to a single address and I have a windows service parsing them out (Google ListNanny). I put as much information from the NDR as I can into a database, and then I run reports on it to see if I have suddenly started getting blocked by a certain domain. Also, you should avoid sending emails to addresses that were previously marked as NDR, because that's generally a good indication of spam.
If you need to send out a bunch of customer service emails at once, it's best to put a delay in between each one, because if you send too many nearly identical emails to one domain at a time, you are sure to wind up on their blacklist.
Some domains are just impossible to deliver to sometimes. Comcast.net is the worst.
Make sure your IPs aren't listed on sites like http://www.mxtoolbox.com/blacklists.aspx.
I hate to tell you, but I and others may be using white-list defaults to control our filtering of spam.
This means that all e-mail from an unknown source is automatically spam and diverted into a spam folder. (I don't let my e-mail service delete spam, because I want to always review the arrivals for false positives, something that is pretty easy to do by a quick scan of the folder.)
I even have e-mail from myself go to the spam bucket because (1) I usually don't send e-mail to myself and (2) there are spammers that fake my return address in spam sent to me.
So to get out of the spam designation, I have to consider that your mail might be legitimate (from sender and subject information) and open it first in plaintext (my default for all incoming mail, spam or not) to see if it is legitimate. My spam folder will not use any links in e-mails so I am protected against tricky image links and other misbehavior.
If I want future arrivals from the same source to go to my in box and not be diverted for spam review, I will specify that to my e-mail client. For those organizations that use bulk-mail forwarders and unique sender addresses per mail piece, that's too bad. They never get my approval and always show up in my spam folder, and if I'm busy I will never look at them.
Finally, if an e-mail is not legible in plaintext, even when sent as HTML, I am likely to just delete it unless it is something that I know is of interest to me by virtue of the source and previous valuable experiences.
As you can see, it is ultimately under an users control and there is no automated act that will convince such a system that your mail is legitimate from its structure alone. In this case, you need to play nice, don't do anything that is similar to phishing, and make it easy for users willing to trust your mail to add you to their white list.
one of my application's emails was constantly being tagged as spam. it was html with a single link, which i sent as html in the body with a text/html content type.
my most successful resolution to this problem was to compose the email so it looked like it was generated by an email client.
i changed the email to be a multipart/alternative mime document and i now generate both text/plain and text/html parts.
the email no longer is detected as junk by outlook.
Yahoo uses a method called Sender ID, which can be configured at The SPF Setup Wizard and entered in to your DNS. Also one of the important ones for Exchange, Hotmail, AOL, Yahoo, and others is to have a Reverse DNS for your domain. Those will knock out most of the issues. However you can never prevent a person intentionally blocking your or custom rules.
You need a reverse DNS entry. You need to not send the same content to the same user twice. You need to test it with some common webmail and email clients.
Personally I ran mine through a freshly installed spam assassin, a trained spam assassin, and multiple hotmail, gmail, and aol accounts.
But have you seen that spam that doesn't seem to link to or advertise anything? That's a spammer trying to affect your Bayesian filter. If he can get a high rating and then include some words that would be in his future emails it might be automatically learned as good. So you can't really guess what a user's filter is going to be set as at the time of your mailing.
Lastly, I did not sort my list by the domains, but randomized it.
I've found that using the recipients real first and last name in the body is a sure fire way of getting through a spam filter.
In the UK it's also best practice to include a real physical address for your company and its registered number.
That way it's all open and honest and they're less likely to manually mark it as spam.
I would add :
Provide real unsubscription upon click on "Unsubscribe". I've seen real newsletters providing a dummy unsubscription link that upon click shows " has been unsubscribed successfully" but I will still receive further newsletters.
The most important thing you can do is to make sure that the people you are sending email to are not likely going to hit the "Spam" button when they receive your email. So, stick to the following rules of thumb:
Make sure you have permission from the people you are sending email to. Don't ever send email to someone who did not request it from you.
Clearly identify who you are right at the top of each message, and why the person is receiving the email.
At least once a month, send out a reminder email to people on your list (if you are running a list), forcing them to opt back in to the list in order to keep receiving communications from you. Yes, this will mean your list gets shorter over time, but the up-side is that the people on your list are "bought in" and will be less likely to flag your email.
Keep your content highly relevant and useful.
Give people an easy way to opt out of further communications.
Use an email sending service like SendGrid that works hard to maintain a good IP reputation.
Avoid using short links - these are often blacklisted.
Following these rules of thumb will go a long way.
I have had the same problem in the past on many sites I have done here at work. The only guaranteed method of making sure the user gets the email is to advise the user to add you to there safe list. Any other method is really only going to be something that can help with it and isn't guaranteed.
It could very well be the case that people who sign up for your service are entering emails with typing mistakes that you do not correct. For example: chris#gmial.com -or- james#hotnail.com.
And such domains are configured to be used as spamtraps which will automatically flag your email server's IP and/or domain and hurt its reputation.
To avoid this, do a double-check for the email address that is entered upon your product subscription. Also, send a confirmation email to really ensure that this email address is 100% validated by a human being that is entering the confirmation email, before you send them the product key or accept their subscription. The verification email should require the recipient to click a link or reply in order to really confirm that the owner of the mailbox is the person who signed up.
It sounds like you are depending on some feedback to determine what is getting stuck on the receiving end. You should be checking the outbound mail yourself for obvious "spaminess".
Buy any decent spam control system, and send your outbound mail through it. If you send any decent volume of mail, you should be doing this anyhow, because of the risk of sending outbound viruses, especially if you have desktop windows users.
Proofpoint had spam + anti-virus + some reputation services in a single deployment, for example. (I used to work there, so I happen to know this off the top of my head. I'm sure other vendors in this space have similar features.) But you get the idea. If you send your mail through a basic commerical spam control setup, and it doesn't pass, it shouldn't be going out of your network.
Also, there are some companies that can assist you with increasing delivery rates of non-spam, outbound email, like Habeas.
Google has a tool and guidelines for this. You can find them on: https://postmaster.google.com/ Register and verify your domain name and Google provides an individual scoring of that IP-address and domain.
From the bulk senders guidelines:
Authentication ensures that your messages can be correctly classified. Emails that lack authentication are likely to be rejected or placed in the spam folder, given the high likelihood that they are forged messages used for phishing scams. In addition, unauthenticated emails with attachments may be outrightly rejected, for security reasons.
To ensure that Gmail can identify you:
Use a consistent IP address to send bulk mail.
Keep valid reverse DNS records for the IP address(es) from which you send mail, pointing to your domain.
Use the same address in the 'From:' header on every bulk mail you send.
We also recommend the following:
Sign messages with DKIM. We do not authenticate messages signed with keys using fewer than 1024 bits.
Publish an SPF record.
Publish a DMARC policy.
I always use:
https://www.mail-tester.com/
It gives me feedback on the technical part of sending an e-mail. Like SPF-records, DKIM, Spamassassin score and so on. Even though I know what is required, I continuously make errors and mail-tester.com makes it easy to figure out what could be wrong.
First of all, you need to ensure the required email authentication mechanisms like SPF and DKIM are in place. These two are prominent ways of proving that you were the actual sender of an email and it's not really spoofed. This reduces the chances of emails getting filtered as spam.
Second thing is, you can check the reverse DNS output of your domain name against different DNSBLs. Use below simple command on terminal:
**dig a +short (domain-name).(blacklist-domain-name)**
ie. dig a +short example.com.dsn.rfc-clueless.org
> 127.0.0.2
In the above examples, this means your domain "example.com" is listed in blacklist but due to Domain Setting Compliance(rfc-clueless.org list domain which has compliance issue )
note: I prefer multivalley and pepipost tool for checking the domain listings.
The from address/reply-to-id should be proper, always use visible unsubscribe button within your email body (this will help your users to sign out from your email-list without killing your domain reputation)
The intend of most of the programmatically generated emails is generally transactional, triggered or alert n nature- which means these are important emails which should never land into spam.
Having said that there are multiple parameters which are been considered before flagging an email as spam. While Quality of email list is the most important parameter to be considered, but I am skipping that here from the discussion because here we are talking about important emails which are sent to either ourself or to known email addresses.
Apart from list quality, the other 3 important parameters are;
Sender Reputation
Compliance with Email Standards and Authentication (SPF, DKIM, DMARC, rDNS)
Email content
Sender Reputation = Reputation of Sending IP address + Reputation of Return Path/Envelope domain + Reputation of From Domain.
There is no straight answer to what is your Sender Reputation. This is because there are multiple authorities like SenderScore, Reputation Authority and so on who maintains the reputation score for your domain. Apart from that ISPs like Gmail, Yahoo, Outlook also maintains the reputation of each domain at their end.
But, you can use free tools like GradeMyEmail to get a 360-degree view of your reputation and potential problems with your email settings or any other compliance-related issue too.
Sometimes, if you're using a new domain for sending an email, then those are also found to land in spam. You should be checking whether your domain is listed on any of the global blocklists or not. Again GradeMyEmail and MultiRBL are useful tools to identify the list of blocklists.
Once you're pretty sure with the sender reputation score, you should check whether your email sending domain complies with all email authentications and standards.
SPF
DKIM
DMARC
Reverse DNS
For this, you can again use GradeMyEmail or MXToolbox to know the potential problems with your authentication.
Your SPF, DKIM and DMARC should always PASS to ensure, your emails are complying with the standard email authentications.
Here's an example of how these authentications should look like in Gmail:
Similarly, you can use tools like Mail-Tester which scans the complete email content and tells the potential keywords which can trigger spam filters.
To allow DMARC checks for SPF to pass and also be aligned when using sendmail, make sure you are setting the envelope sender address (-f or -r parameter) to something that matches the domain in the From: header address.
With PHP:
Using PHP's built-in mail() function without setting the 5th paramater will cause DMARC SPF checks to be unaligned if not done correctly. By default, sendmail will send the email with the webserver's user as the RFC5321.MailFrom / Return Path header.
For example, say you are hosting your website domain.com on the host.com web server. If you do not set the additional parameters parameter:
mail($to,$subject,$message,$headers); // Wrong way
The email recipient will receive an email with the following mail headers:
Return-Path: <your-website-user#server.host.com>
From: <your-website-user#domain.com>
Even though this passes SPF checks, it will be unaligned (since domain.com and host.com do not match), which means that DMARC SPF check will fail as unaligned.
Instead, you must pass the envelope sender address to sendmail by including the 5th parameter in the PHP mail() function, for example:
mail($to,$subject,$message,$headers, '-r bounce_email#domain.com'); // Right way
In this case, the email recipient will receive an email with the following mail headers:
Return-Path: <bounce_email#domain.com>
From: <your-website-user#domain.com>
Since both of these headers contain addresses from domain.com, SPF will pass and also be aligned, which means that DMARC will also pass the SPF check.
This is a tricky one and I've always relied on techniques, such as permission-based emails (i.e. only sending to people you have permission to send to) and not using blatantly spamish terminology.
Of late, some of the emails I send out programmatically have started being shuffled into people's spam folder automatically and I'm wondering what I can do about it.
This is despite the fact that these particular emails are not ones that humans would mark as spam, specifically, they are emails that contain license keys that people have paid good money for, so I don't think they're going to consider them spam
I figure this is a big topic in which I am essentially an ignorant simpleton.
Use email authentication methods, such as SPF, and DKIM to prove that your emails and your domain name belong together, and to prevent spoofing of your domain name. The SPF website includes a wizard to generate the DNS information for your site.
Check your reverse DNS to make sure the IP address of your mail server points to the domain name that you use for sending mail.
Make sure that the IP-address that you're using is not on a blacklist
Make sure that the reply-to address is a valid, existing address.
Use the full, real name of the addressee in the To field, not just the email-address (e.g. "John Smith" <john#blacksmiths-international.com> ).
Monitor your abuse accounts, such as abuse#yourdomain.example and postmaster#yourdomain.example. That means - make sure that these accounts exist, read what's sent to them, and act on complaints.
Finally, make it really easy to unsubscribe. Otherwise, your users will unsubscribe by pressing the spam button, and that will affect your reputation.
That said, getting Hotmail to accept your emails remains a black art.
Sign up for an account on as many major email providers as possible (gmail/yahoo/hotmail/aol/etc). If you make changes to your emails, either major rewording, changes to the code that sends the emails, changes to your email servers, etc, make sure to send test messages to all your accounts and verify that they are not being marked as spam.
A few bullet points from a previous answer:
Most important: Does the sender address ("From") belong to a domain that runs on the server you send the E-Mail from? If not, make it so. Never use sender addresses like xxx#gmail.com. User reply-to if you need replies to arrive at a different address.
Is your server on a blacklist (e.g. check IP on spamhaus.org)? This is a possibility when you're on shared hosting when neighbours behave badly.
Are mails filtered by a spam filter? Open an account with a freemailer that has a spam folder and find out. Also, try sending mail to an address without any spam filtering at all.
Do you possibly need the fifth parameter "-f" of mail() to add a sender address? (See mail() command in the PHP manual)
If you have access to log files, check those, of course.
Do you check the "from:" address for possible bounce mails ("Returned to sender")? You can also set up a separate "errors-to" address.
You can tell your users to add your From address to their contacts when they complete their order, which, if they do so, will help a lot.
Otherwise, I would try to get a log from some of your users. Sometimes they have details about why it was flagged as spam in the headers of the message, which you could use to tweak the text.
Other things you can try:
Put your site name or address in the subject
Keep all links in the message pointing to your domain (and not email.com)
Put an address or other contact information in the email
Confirm that you have the correct email address before sending out emails. If someone gives the wrong email address on sign-up, beat them over the head about it ASAP.
Always include clear "how to unsubscribe" information in EVERY email. Do not require the user to login to unsubscribe, it should be a unique url for 1-click unsubscribe.
This will prevent people from marking your mails as spam because "unsubscribing" is too hard.
In addition to all of the other answers, if you are sending HTML emails that contain URLs as linking text, make sure that the URL matches the linking text. I know that Thunderbird automatically flags them as being a scam if not.
The wrong way:
Go to your account now: http://www.paypal.com
The right way:
Go to your account now: http://www.yourdomain.org
Or use an unrelated linking text instead of a URL:
Click here to go to your account
You may consider a third party email service who handles delivery issues:
Exact Target
Vertical Response
Constant Contact
Campaign Monitor
Emma
Return Path
IntelliContact
SilverPop
Delivering email can be like black magic sometimes. The reverse DNS is really important.
I have found it to be very helpful to carefully track NDRs. I direct all of my NDRs to a single address and I have a windows service parsing them out (Google ListNanny). I put as much information from the NDR as I can into a database, and then I run reports on it to see if I have suddenly started getting blocked by a certain domain. Also, you should avoid sending emails to addresses that were previously marked as NDR, because that's generally a good indication of spam.
If you need to send out a bunch of customer service emails at once, it's best to put a delay in between each one, because if you send too many nearly identical emails to one domain at a time, you are sure to wind up on their blacklist.
Some domains are just impossible to deliver to sometimes. Comcast.net is the worst.
Make sure your IPs aren't listed on sites like http://www.mxtoolbox.com/blacklists.aspx.
I hate to tell you, but I and others may be using white-list defaults to control our filtering of spam.
This means that all e-mail from an unknown source is automatically spam and diverted into a spam folder. (I don't let my e-mail service delete spam, because I want to always review the arrivals for false positives, something that is pretty easy to do by a quick scan of the folder.)
I even have e-mail from myself go to the spam bucket because (1) I usually don't send e-mail to myself and (2) there are spammers that fake my return address in spam sent to me.
So to get out of the spam designation, I have to consider that your mail might be legitimate (from sender and subject information) and open it first in plaintext (my default for all incoming mail, spam or not) to see if it is legitimate. My spam folder will not use any links in e-mails so I am protected against tricky image links and other misbehavior.
If I want future arrivals from the same source to go to my in box and not be diverted for spam review, I will specify that to my e-mail client. For those organizations that use bulk-mail forwarders and unique sender addresses per mail piece, that's too bad. They never get my approval and always show up in my spam folder, and if I'm busy I will never look at them.
Finally, if an e-mail is not legible in plaintext, even when sent as HTML, I am likely to just delete it unless it is something that I know is of interest to me by virtue of the source and previous valuable experiences.
As you can see, it is ultimately under an users control and there is no automated act that will convince such a system that your mail is legitimate from its structure alone. In this case, you need to play nice, don't do anything that is similar to phishing, and make it easy for users willing to trust your mail to add you to their white list.
one of my application's emails was constantly being tagged as spam. it was html with a single link, which i sent as html in the body with a text/html content type.
my most successful resolution to this problem was to compose the email so it looked like it was generated by an email client.
i changed the email to be a multipart/alternative mime document and i now generate both text/plain and text/html parts.
the email no longer is detected as junk by outlook.
Yahoo uses a method called Sender ID, which can be configured at The SPF Setup Wizard and entered in to your DNS. Also one of the important ones for Exchange, Hotmail, AOL, Yahoo, and others is to have a Reverse DNS for your domain. Those will knock out most of the issues. However you can never prevent a person intentionally blocking your or custom rules.
You need a reverse DNS entry. You need to not send the same content to the same user twice. You need to test it with some common webmail and email clients.
Personally I ran mine through a freshly installed spam assassin, a trained spam assassin, and multiple hotmail, gmail, and aol accounts.
But have you seen that spam that doesn't seem to link to or advertise anything? That's a spammer trying to affect your Bayesian filter. If he can get a high rating and then include some words that would be in his future emails it might be automatically learned as good. So you can't really guess what a user's filter is going to be set as at the time of your mailing.
Lastly, I did not sort my list by the domains, but randomized it.
I've found that using the recipients real first and last name in the body is a sure fire way of getting through a spam filter.
In the UK it's also best practice to include a real physical address for your company and its registered number.
That way it's all open and honest and they're less likely to manually mark it as spam.
I would add :
Provide real unsubscription upon click on "Unsubscribe". I've seen real newsletters providing a dummy unsubscription link that upon click shows " has been unsubscribed successfully" but I will still receive further newsletters.
The most important thing you can do is to make sure that the people you are sending email to are not likely going to hit the "Spam" button when they receive your email. So, stick to the following rules of thumb:
Make sure you have permission from the people you are sending email to. Don't ever send email to someone who did not request it from you.
Clearly identify who you are right at the top of each message, and why the person is receiving the email.
At least once a month, send out a reminder email to people on your list (if you are running a list), forcing them to opt back in to the list in order to keep receiving communications from you. Yes, this will mean your list gets shorter over time, but the up-side is that the people on your list are "bought in" and will be less likely to flag your email.
Keep your content highly relevant and useful.
Give people an easy way to opt out of further communications.
Use an email sending service like SendGrid that works hard to maintain a good IP reputation.
Avoid using short links - these are often blacklisted.
Following these rules of thumb will go a long way.
I have had the same problem in the past on many sites I have done here at work. The only guaranteed method of making sure the user gets the email is to advise the user to add you to there safe list. Any other method is really only going to be something that can help with it and isn't guaranteed.
It could very well be the case that people who sign up for your service are entering emails with typing mistakes that you do not correct. For example: chris#gmial.com -or- james#hotnail.com.
And such domains are configured to be used as spamtraps which will automatically flag your email server's IP and/or domain and hurt its reputation.
To avoid this, do a double-check for the email address that is entered upon your product subscription. Also, send a confirmation email to really ensure that this email address is 100% validated by a human being that is entering the confirmation email, before you send them the product key or accept their subscription. The verification email should require the recipient to click a link or reply in order to really confirm that the owner of the mailbox is the person who signed up.
It sounds like you are depending on some feedback to determine what is getting stuck on the receiving end. You should be checking the outbound mail yourself for obvious "spaminess".
Buy any decent spam control system, and send your outbound mail through it. If you send any decent volume of mail, you should be doing this anyhow, because of the risk of sending outbound viruses, especially if you have desktop windows users.
Proofpoint had spam + anti-virus + some reputation services in a single deployment, for example. (I used to work there, so I happen to know this off the top of my head. I'm sure other vendors in this space have similar features.) But you get the idea. If you send your mail through a basic commerical spam control setup, and it doesn't pass, it shouldn't be going out of your network.
Also, there are some companies that can assist you with increasing delivery rates of non-spam, outbound email, like Habeas.
Google has a tool and guidelines for this. You can find them on: https://postmaster.google.com/ Register and verify your domain name and Google provides an individual scoring of that IP-address and domain.
From the bulk senders guidelines:
Authentication ensures that your messages can be correctly classified. Emails that lack authentication are likely to be rejected or placed in the spam folder, given the high likelihood that they are forged messages used for phishing scams. In addition, unauthenticated emails with attachments may be outrightly rejected, for security reasons.
To ensure that Gmail can identify you:
Use a consistent IP address to send bulk mail.
Keep valid reverse DNS records for the IP address(es) from which you send mail, pointing to your domain.
Use the same address in the 'From:' header on every bulk mail you send.
We also recommend the following:
Sign messages with DKIM. We do not authenticate messages signed with keys using fewer than 1024 bits.
Publish an SPF record.
Publish a DMARC policy.
I always use:
https://www.mail-tester.com/
It gives me feedback on the technical part of sending an e-mail. Like SPF-records, DKIM, Spamassassin score and so on. Even though I know what is required, I continuously make errors and mail-tester.com makes it easy to figure out what could be wrong.
First of all, you need to ensure the required email authentication mechanisms like SPF and DKIM are in place. These two are prominent ways of proving that you were the actual sender of an email and it's not really spoofed. This reduces the chances of emails getting filtered as spam.
Second thing is, you can check the reverse DNS output of your domain name against different DNSBLs. Use below simple command on terminal:
**dig a +short (domain-name).(blacklist-domain-name)**
ie. dig a +short example.com.dsn.rfc-clueless.org
> 127.0.0.2
In the above examples, this means your domain "example.com" is listed in blacklist but due to Domain Setting Compliance(rfc-clueless.org list domain which has compliance issue )
note: I prefer multivalley and pepipost tool for checking the domain listings.
The from address/reply-to-id should be proper, always use visible unsubscribe button within your email body (this will help your users to sign out from your email-list without killing your domain reputation)
The intend of most of the programmatically generated emails is generally transactional, triggered or alert n nature- which means these are important emails which should never land into spam.
Having said that there are multiple parameters which are been considered before flagging an email as spam. While Quality of email list is the most important parameter to be considered, but I am skipping that here from the discussion because here we are talking about important emails which are sent to either ourself or to known email addresses.
Apart from list quality, the other 3 important parameters are;
Sender Reputation
Compliance with Email Standards and Authentication (SPF, DKIM, DMARC, rDNS)
Email content
Sender Reputation = Reputation of Sending IP address + Reputation of Return Path/Envelope domain + Reputation of From Domain.
There is no straight answer to what is your Sender Reputation. This is because there are multiple authorities like SenderScore, Reputation Authority and so on who maintains the reputation score for your domain. Apart from that ISPs like Gmail, Yahoo, Outlook also maintains the reputation of each domain at their end.
But, you can use free tools like GradeMyEmail to get a 360-degree view of your reputation and potential problems with your email settings or any other compliance-related issue too.
Sometimes, if you're using a new domain for sending an email, then those are also found to land in spam. You should be checking whether your domain is listed on any of the global blocklists or not. Again GradeMyEmail and MultiRBL are useful tools to identify the list of blocklists.
Once you're pretty sure with the sender reputation score, you should check whether your email sending domain complies with all email authentications and standards.
SPF
DKIM
DMARC
Reverse DNS
For this, you can again use GradeMyEmail or MXToolbox to know the potential problems with your authentication.
Your SPF, DKIM and DMARC should always PASS to ensure, your emails are complying with the standard email authentications.
Here's an example of how these authentications should look like in Gmail:
Similarly, you can use tools like Mail-Tester which scans the complete email content and tells the potential keywords which can trigger spam filters.
To allow DMARC checks for SPF to pass and also be aligned when using sendmail, make sure you are setting the envelope sender address (-f or -r parameter) to something that matches the domain in the From: header address.
With PHP:
Using PHP's built-in mail() function without setting the 5th paramater will cause DMARC SPF checks to be unaligned if not done correctly. By default, sendmail will send the email with the webserver's user as the RFC5321.MailFrom / Return Path header.
For example, say you are hosting your website domain.com on the host.com web server. If you do not set the additional parameters parameter:
mail($to,$subject,$message,$headers); // Wrong way
The email recipient will receive an email with the following mail headers:
Return-Path: <your-website-user#server.host.com>
From: <your-website-user#domain.com>
Even though this passes SPF checks, it will be unaligned (since domain.com and host.com do not match), which means that DMARC SPF check will fail as unaligned.
Instead, you must pass the envelope sender address to sendmail by including the 5th parameter in the PHP mail() function, for example:
mail($to,$subject,$message,$headers, '-r bounce_email#domain.com'); // Right way
In this case, the email recipient will receive an email with the following mail headers:
Return-Path: <bounce_email#domain.com>
From: <your-website-user#domain.com>
Since both of these headers contain addresses from domain.com, SPF will pass and also be aligned, which means that DMARC will also pass the SPF check.
This is a tricky one and I've always relied on techniques, such as permission-based emails (i.e. only sending to people you have permission to send to) and not using blatantly spamish terminology.
Of late, some of the emails I send out programmatically have started being shuffled into people's spam folder automatically and I'm wondering what I can do about it.
This is despite the fact that these particular emails are not ones that humans would mark as spam, specifically, they are emails that contain license keys that people have paid good money for, so I don't think they're going to consider them spam
I figure this is a big topic in which I am essentially an ignorant simpleton.
Use email authentication methods, such as SPF, and DKIM to prove that your emails and your domain name belong together, and to prevent spoofing of your domain name. The SPF website includes a wizard to generate the DNS information for your site.
Check your reverse DNS to make sure the IP address of your mail server points to the domain name that you use for sending mail.
Make sure that the IP-address that you're using is not on a blacklist
Make sure that the reply-to address is a valid, existing address.
Use the full, real name of the addressee in the To field, not just the email-address (e.g. "John Smith" <john#blacksmiths-international.com> ).
Monitor your abuse accounts, such as abuse#yourdomain.example and postmaster#yourdomain.example. That means - make sure that these accounts exist, read what's sent to them, and act on complaints.
Finally, make it really easy to unsubscribe. Otherwise, your users will unsubscribe by pressing the spam button, and that will affect your reputation.
That said, getting Hotmail to accept your emails remains a black art.
Sign up for an account on as many major email providers as possible (gmail/yahoo/hotmail/aol/etc). If you make changes to your emails, either major rewording, changes to the code that sends the emails, changes to your email servers, etc, make sure to send test messages to all your accounts and verify that they are not being marked as spam.
A few bullet points from a previous answer:
Most important: Does the sender address ("From") belong to a domain that runs on the server you send the E-Mail from? If not, make it so. Never use sender addresses like xxx#gmail.com. User reply-to if you need replies to arrive at a different address.
Is your server on a blacklist (e.g. check IP on spamhaus.org)? This is a possibility when you're on shared hosting when neighbours behave badly.
Are mails filtered by a spam filter? Open an account with a freemailer that has a spam folder and find out. Also, try sending mail to an address without any spam filtering at all.
Do you possibly need the fifth parameter "-f" of mail() to add a sender address? (See mail() command in the PHP manual)
If you have access to log files, check those, of course.
Do you check the "from:" address for possible bounce mails ("Returned to sender")? You can also set up a separate "errors-to" address.
You can tell your users to add your From address to their contacts when they complete their order, which, if they do so, will help a lot.
Otherwise, I would try to get a log from some of your users. Sometimes they have details about why it was flagged as spam in the headers of the message, which you could use to tweak the text.
Other things you can try:
Put your site name or address in the subject
Keep all links in the message pointing to your domain (and not email.com)
Put an address or other contact information in the email
Confirm that you have the correct email address before sending out emails. If someone gives the wrong email address on sign-up, beat them over the head about it ASAP.
Always include clear "how to unsubscribe" information in EVERY email. Do not require the user to login to unsubscribe, it should be a unique url for 1-click unsubscribe.
This will prevent people from marking your mails as spam because "unsubscribing" is too hard.
In addition to all of the other answers, if you are sending HTML emails that contain URLs as linking text, make sure that the URL matches the linking text. I know that Thunderbird automatically flags them as being a scam if not.
The wrong way:
Go to your account now: http://www.paypal.com
The right way:
Go to your account now: http://www.yourdomain.org
Or use an unrelated linking text instead of a URL:
Click here to go to your account
You may consider a third party email service who handles delivery issues:
Exact Target
Vertical Response
Constant Contact
Campaign Monitor
Emma
Return Path
IntelliContact
SilverPop
Delivering email can be like black magic sometimes. The reverse DNS is really important.
I have found it to be very helpful to carefully track NDRs. I direct all of my NDRs to a single address and I have a windows service parsing them out (Google ListNanny). I put as much information from the NDR as I can into a database, and then I run reports on it to see if I have suddenly started getting blocked by a certain domain. Also, you should avoid sending emails to addresses that were previously marked as NDR, because that's generally a good indication of spam.
If you need to send out a bunch of customer service emails at once, it's best to put a delay in between each one, because if you send too many nearly identical emails to one domain at a time, you are sure to wind up on their blacklist.
Some domains are just impossible to deliver to sometimes. Comcast.net is the worst.
Make sure your IPs aren't listed on sites like http://www.mxtoolbox.com/blacklists.aspx.
I hate to tell you, but I and others may be using white-list defaults to control our filtering of spam.
This means that all e-mail from an unknown source is automatically spam and diverted into a spam folder. (I don't let my e-mail service delete spam, because I want to always review the arrivals for false positives, something that is pretty easy to do by a quick scan of the folder.)
I even have e-mail from myself go to the spam bucket because (1) I usually don't send e-mail to myself and (2) there are spammers that fake my return address in spam sent to me.
So to get out of the spam designation, I have to consider that your mail might be legitimate (from sender and subject information) and open it first in plaintext (my default for all incoming mail, spam or not) to see if it is legitimate. My spam folder will not use any links in e-mails so I am protected against tricky image links and other misbehavior.
If I want future arrivals from the same source to go to my in box and not be diverted for spam review, I will specify that to my e-mail client. For those organizations that use bulk-mail forwarders and unique sender addresses per mail piece, that's too bad. They never get my approval and always show up in my spam folder, and if I'm busy I will never look at them.
Finally, if an e-mail is not legible in plaintext, even when sent as HTML, I am likely to just delete it unless it is something that I know is of interest to me by virtue of the source and previous valuable experiences.
As you can see, it is ultimately under an users control and there is no automated act that will convince such a system that your mail is legitimate from its structure alone. In this case, you need to play nice, don't do anything that is similar to phishing, and make it easy for users willing to trust your mail to add you to their white list.
one of my application's emails was constantly being tagged as spam. it was html with a single link, which i sent as html in the body with a text/html content type.
my most successful resolution to this problem was to compose the email so it looked like it was generated by an email client.
i changed the email to be a multipart/alternative mime document and i now generate both text/plain and text/html parts.
the email no longer is detected as junk by outlook.
Yahoo uses a method called Sender ID, which can be configured at The SPF Setup Wizard and entered in to your DNS. Also one of the important ones for Exchange, Hotmail, AOL, Yahoo, and others is to have a Reverse DNS for your domain. Those will knock out most of the issues. However you can never prevent a person intentionally blocking your or custom rules.
You need a reverse DNS entry. You need to not send the same content to the same user twice. You need to test it with some common webmail and email clients.
Personally I ran mine through a freshly installed spam assassin, a trained spam assassin, and multiple hotmail, gmail, and aol accounts.
But have you seen that spam that doesn't seem to link to or advertise anything? That's a spammer trying to affect your Bayesian filter. If he can get a high rating and then include some words that would be in his future emails it might be automatically learned as good. So you can't really guess what a user's filter is going to be set as at the time of your mailing.
Lastly, I did not sort my list by the domains, but randomized it.
I've found that using the recipients real first and last name in the body is a sure fire way of getting through a spam filter.
In the UK it's also best practice to include a real physical address for your company and its registered number.
That way it's all open and honest and they're less likely to manually mark it as spam.
I would add :
Provide real unsubscription upon click on "Unsubscribe". I've seen real newsletters providing a dummy unsubscription link that upon click shows " has been unsubscribed successfully" but I will still receive further newsletters.
The most important thing you can do is to make sure that the people you are sending email to are not likely going to hit the "Spam" button when they receive your email. So, stick to the following rules of thumb:
Make sure you have permission from the people you are sending email to. Don't ever send email to someone who did not request it from you.
Clearly identify who you are right at the top of each message, and why the person is receiving the email.
At least once a month, send out a reminder email to people on your list (if you are running a list), forcing them to opt back in to the list in order to keep receiving communications from you. Yes, this will mean your list gets shorter over time, but the up-side is that the people on your list are "bought in" and will be less likely to flag your email.
Keep your content highly relevant and useful.
Give people an easy way to opt out of further communications.
Use an email sending service like SendGrid that works hard to maintain a good IP reputation.
Avoid using short links - these are often blacklisted.
Following these rules of thumb will go a long way.
I have had the same problem in the past on many sites I have done here at work. The only guaranteed method of making sure the user gets the email is to advise the user to add you to there safe list. Any other method is really only going to be something that can help with it and isn't guaranteed.
It could very well be the case that people who sign up for your service are entering emails with typing mistakes that you do not correct. For example: chris#gmial.com -or- james#hotnail.com.
And such domains are configured to be used as spamtraps which will automatically flag your email server's IP and/or domain and hurt its reputation.
To avoid this, do a double-check for the email address that is entered upon your product subscription. Also, send a confirmation email to really ensure that this email address is 100% validated by a human being that is entering the confirmation email, before you send them the product key or accept their subscription. The verification email should require the recipient to click a link or reply in order to really confirm that the owner of the mailbox is the person who signed up.
It sounds like you are depending on some feedback to determine what is getting stuck on the receiving end. You should be checking the outbound mail yourself for obvious "spaminess".
Buy any decent spam control system, and send your outbound mail through it. If you send any decent volume of mail, you should be doing this anyhow, because of the risk of sending outbound viruses, especially if you have desktop windows users.
Proofpoint had spam + anti-virus + some reputation services in a single deployment, for example. (I used to work there, so I happen to know this off the top of my head. I'm sure other vendors in this space have similar features.) But you get the idea. If you send your mail through a basic commerical spam control setup, and it doesn't pass, it shouldn't be going out of your network.
Also, there are some companies that can assist you with increasing delivery rates of non-spam, outbound email, like Habeas.
Google has a tool and guidelines for this. You can find them on: https://postmaster.google.com/ Register and verify your domain name and Google provides an individual scoring of that IP-address and domain.
From the bulk senders guidelines:
Authentication ensures that your messages can be correctly classified. Emails that lack authentication are likely to be rejected or placed in the spam folder, given the high likelihood that they are forged messages used for phishing scams. In addition, unauthenticated emails with attachments may be outrightly rejected, for security reasons.
To ensure that Gmail can identify you:
Use a consistent IP address to send bulk mail.
Keep valid reverse DNS records for the IP address(es) from which you send mail, pointing to your domain.
Use the same address in the 'From:' header on every bulk mail you send.
We also recommend the following:
Sign messages with DKIM. We do not authenticate messages signed with keys using fewer than 1024 bits.
Publish an SPF record.
Publish a DMARC policy.
I always use:
https://www.mail-tester.com/
It gives me feedback on the technical part of sending an e-mail. Like SPF-records, DKIM, Spamassassin score and so on. Even though I know what is required, I continuously make errors and mail-tester.com makes it easy to figure out what could be wrong.
First of all, you need to ensure the required email authentication mechanisms like SPF and DKIM are in place. These two are prominent ways of proving that you were the actual sender of an email and it's not really spoofed. This reduces the chances of emails getting filtered as spam.
Second thing is, you can check the reverse DNS output of your domain name against different DNSBLs. Use below simple command on terminal:
**dig a +short (domain-name).(blacklist-domain-name)**
ie. dig a +short example.com.dsn.rfc-clueless.org
> 127.0.0.2
In the above examples, this means your domain "example.com" is listed in blacklist but due to Domain Setting Compliance(rfc-clueless.org list domain which has compliance issue )
note: I prefer multivalley and pepipost tool for checking the domain listings.
The from address/reply-to-id should be proper, always use visible unsubscribe button within your email body (this will help your users to sign out from your email-list without killing your domain reputation)
The intend of most of the programmatically generated emails is generally transactional, triggered or alert n nature- which means these are important emails which should never land into spam.
Having said that there are multiple parameters which are been considered before flagging an email as spam. While Quality of email list is the most important parameter to be considered, but I am skipping that here from the discussion because here we are talking about important emails which are sent to either ourself or to known email addresses.
Apart from list quality, the other 3 important parameters are;
Sender Reputation
Compliance with Email Standards and Authentication (SPF, DKIM, DMARC, rDNS)
Email content
Sender Reputation = Reputation of Sending IP address + Reputation of Return Path/Envelope domain + Reputation of From Domain.
There is no straight answer to what is your Sender Reputation. This is because there are multiple authorities like SenderScore, Reputation Authority and so on who maintains the reputation score for your domain. Apart from that ISPs like Gmail, Yahoo, Outlook also maintains the reputation of each domain at their end.
But, you can use free tools like GradeMyEmail to get a 360-degree view of your reputation and potential problems with your email settings or any other compliance-related issue too.
Sometimes, if you're using a new domain for sending an email, then those are also found to land in spam. You should be checking whether your domain is listed on any of the global blocklists or not. Again GradeMyEmail and MultiRBL are useful tools to identify the list of blocklists.
Once you're pretty sure with the sender reputation score, you should check whether your email sending domain complies with all email authentications and standards.
SPF
DKIM
DMARC
Reverse DNS
For this, you can again use GradeMyEmail or MXToolbox to know the potential problems with your authentication.
Your SPF, DKIM and DMARC should always PASS to ensure, your emails are complying with the standard email authentications.
Here's an example of how these authentications should look like in Gmail:
Similarly, you can use tools like Mail-Tester which scans the complete email content and tells the potential keywords which can trigger spam filters.
To allow DMARC checks for SPF to pass and also be aligned when using sendmail, make sure you are setting the envelope sender address (-f or -r parameter) to something that matches the domain in the From: header address.
With PHP:
Using PHP's built-in mail() function without setting the 5th paramater will cause DMARC SPF checks to be unaligned if not done correctly. By default, sendmail will send the email with the webserver's user as the RFC5321.MailFrom / Return Path header.
For example, say you are hosting your website domain.com on the host.com web server. If you do not set the additional parameters parameter:
mail($to,$subject,$message,$headers); // Wrong way
The email recipient will receive an email with the following mail headers:
Return-Path: <your-website-user#server.host.com>
From: <your-website-user#domain.com>
Even though this passes SPF checks, it will be unaligned (since domain.com and host.com do not match), which means that DMARC SPF check will fail as unaligned.
Instead, you must pass the envelope sender address to sendmail by including the 5th parameter in the PHP mail() function, for example:
mail($to,$subject,$message,$headers, '-r bounce_email#domain.com'); // Right way
In this case, the email recipient will receive an email with the following mail headers:
Return-Path: <bounce_email#domain.com>
From: <your-website-user#domain.com>
Since both of these headers contain addresses from domain.com, SPF will pass and also be aligned, which means that DMARC will also pass the SPF check.
this was ask over and over and still no good solution!
When someone sends an email using php and placing another domain in "from" it will end up in spam.
Solutions normally are:
- Use your "from" and place the domain you want in the "reply-to";
- Have your domain whitelisted by main mail services.
The 1st its not really a solution and I was never able to make 2nd, because its impossible to reach hotmail.com, yahoo.com, etc..
I see lots of sites today having the option to email article to someone from the user email. How can I achieve this?
Thx,
Telmo Cardoso
When someone sends an email using php
and placing another domain in "from"
it will end up in spam.
This is not neccessarily true.
Check which mail servers identify your message as spam. Also check with your hosting company for their preferred method to send out mail.
Try to be straightforward with your message. Offer the users a short textarea (with your default masage), which they can change and customize. Also give the possibility to enter real name of the participants with their e-mail addresses.
Just be senible and your messas will go thru.
Send the email from your webserver and add a reply-to header like you have mentioned.
Make sure you have SPF setup for your server to help get yourself into the gMail, Live Mail and Yahoo accounts.
A nice and cheap alternative is to send your emails via Amazon SES to avoid having to warm your own IPs etc.
I would go with the Amazon SES (or a similar service) and leave the worrying about getting the server setup right to the experts. Make ensuring deliverability someone elses problem.
This article is a nice starting point:
http://www.codinghorror.com/blog/2010/04/so-youd-like-to-send-some-email-through-code.html
The problem is related to the header of the email. not neccessarily the 'from' address domain. Some spam filters (i.e. cox.net incoming spam filters) will perform a reverse lookup on the IP address that the email came from. If the domain name returned on the reverse lookup is not the domain name of the sending address. Then they will liekly mark it as spam.
Im using PHP's mail() function to send some emails. But all my mails land automaticly in the trash box. Is there a way of preventing this? If so, where should i read to learn more about it.
Would you recommend me using PHPmailer?
Best of regards,
Alexander
TL;DR: There's no magic bullet. Just because you can learn how to form an email in PHP, does not guarantee it is routed to someone's mailbox, or even accepted. Success is based on reputation, not any single fix.
I am (edit: was) a mail server engineer, have written SpamAssassin rules, and have deep-dived issues for customers sending or receiving email.
The recipient's mail server scans your email, looking for attributes and "historical problems" (lack of mail agent, coming from your webserver IP, etc). These get "points". The total number of points is compared, and the recipient's server may do one or more of the following:
List item
refused during SMTP,
routed to Spam folder,
routed to Inbox, but tagged "SPAM"
blackholed (accepted, then mysteriously lost).
"Points" (score) only means something to a particular anti-spam solution. There is no public test. Fix ALL the problems you can, and success goes up.
*The #1 issue is: do not send email directly to the recipient's SMTP server. This network space sends 99.9% spam. It costs money to scan email, so a good email admin will block or refuse such connections.
The "fix" for your source IP is: Use an SMTP Gateway. The gateway can be our ISP mailserver, or a commercial service. Check first with their terms of service. They may prohibit sending emails using an authenticated web form, since these are so frequently abused ("someone hacked me" is not an excuse).
If you have email hosting, do the following: create a mailbox called for example 'website-notification#websitedomain.com'. Call it what you like. Now you want your PHP script to send the email -through- that address, using Authenticated SMTP. I'll leave the process of learning how to use Authenticated SMTP from PHP as a learning exercise for you -- there are many tutorials online).
Once you send emails through your valid SMTP server, the mail is seen as "originating" from your SMTP gateway. It's not seen as coming from your script. But this isn't the end of the story
As someone else noted, Be sure you are not missing display headers such as To: From: Subject: and Date:. Strictly speaking these headers are NOT "required" in email handshaking, but in practical terms no reputable email software omits them. Also, Date must be in the standard date format, or some spam filters will flag it.
This topic is not to be confused with "envelope headers" (the hidden stuff in the SMTP handshaking), which also can also impact your score. Using an SMTP Gateway usually takes care of this (since the recipient's mailserver will handshake with your gateway host).
Your FROM address must be VALID. Do not use a fake domain. Do not use your domain name with a fake mailbox name. Some anti-spam software will do a "Sender Verify" to test if the From address is bogus or fake (oversimplified: they'll try sending a reply and see if you would accept it or not).
The #1 mistake is setting your from address as "noreply#yourdomain.com", and not creating that mailbox. When that happens, everyone's "Sender Verify" on your email fails, and you look like a spammer covering their tracks.
If your domain DNS has an SPF record, be 100% sure it lists every IP that might send email for your domain. This is a technical topic. Having a valid, correct SPF record helps your deliverability a little bit. But if you misunderstand and create a bad (incorrect) SPF record, you will be worse off. Take your time to understand before using this.
If you have a business with a real address or PO box, don't use "Domain Registration Privacy" or "Domain Proxy" services if you can avoid it. When this was written (2011) It used to be very true that anonymizing services could get your mail blocked, or "tagged spam". This is less true today, but it's still worth considering.
Know the IP address of your mailserver, and regularly check that it is not "blacklisted" at SpamCop, SpamHaus, or the Barracuda spam blacklists. Google for more. There are monitoring services, and scripts which can alert you. But if you get on these lists, it means there is something else happening you were not monitoring for...
As said, no simple answer. :)
I suppose you mean thrash box at the receiver's end. So basically the receiving email server is regarding it as spam. This can happen if:
1) The IP you are sending from is already blacklisted for spamming (happens often in shared hosting)
2) The IP and domain are relatively new and unknown.
(Note that many times, newsletters from well established sites also end up in spam).
If its your dedicated IP, then setting RDNS for the IP, to match the domain name will very likely solve the issue. Another usual practice is to alert the receiver (if she is subscribing on your website) to check their thrash/spam folder and whitelist your email address in their mail account.
regards,
JP
JP's answer is partly correct but it also could be your header's in the email i know from experience this sends stuff to the spam folder try the following;
set the emails to your domain something like no-reply or a valid reply.
$to = 'nobody#example.com';
$subject = 'the subject';
$message = 'hello';
$headers = 'From: webmaster#example.com' . "\r\n" .
'Reply-To: webmaster#example.com' . "\r\n" .
'X-Mailer: PHP/' . phpversion();
mail($to, $subject, $message, $headers);
This probably has something to do with your mail client and spam settings configuration. Try opening account on gmail.com and sending email there, if it's OK you know it is your mail server/client problem. If it's not, post your PHP code and full email headers of the email you've got.
This happens because many a times, headers are missing / if its a well known email server domain key signature is not present, or something like that. If you already have a separate email server, you should check out if you can use the PHP Pear Mail package to send email using your email server, rather than directly via mail function. That's what I find convenient, as its much more flexible.