I have been trying to get PEAR::mail to successfully deliver emails to hotmail users without being flagged as SPAM and ending up in the junk folder, i have no problems with yahoo/gmail only with hotmail.
google suggested that this is a common problem with hotmail and that possible causes can include
incorrect reverse DNS for main IP of the server
lack of SenderId/SPF records
being blacklisted
having checked all of the above i can only think of one other reason - incorrectly formatted headers ?
to test this theory i set up outlook to send email via the same address that PEAR::mail uses and sent a quick test - it delivered straight to my inbox
so i compared the headers from the email sent from PEAR::mail against the headers sent by Outlook and there are only a few differences - i have only listed the differences to save space (and peoples eyes)
PEAR::mail headers (not in outlook headers)
X-PHP-Script: www.example.com/register.php for [users ip address]
Outlook headers (not in PEAR::mail headers)
X-Mailer: Microsoft Office Outlook 11
Thread-Index: Ack6CWSQlgV8s6+6SWyifka2NNpB7g==
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3350
the only other differences that i can see are
the order of the From: and To: headers are reversed
and in the Received: section of the headers
Outlook
Received: from myhomehostname.com ([ip address] helo=simber)
by mywebhostname.com with local (Exim 4.67)
PEAR::mail
Received: from apache by mywebhostname.com with local (Exim 4.67)
could these small differences in the headers be the cause or am i looking in the wrong place ? i knew this might be problematic hence why i chose to use the PEAR::mail class rather than rolling my own but now i really have no idea where to go with this, any help would be greatly appreciated.
Update: as per changelog's suggestion i have tried adding the MS headers to the PEAR::mail class and i have tried replacing PEAR::mail with PHPMailer (with & without the extra headers) - they all end up in the junk folder.
I am starting to believe that it may not be the headers afterall.
Update 2: i should have mentioned that the emails are just a registration confirmation to validate the email address the user signed up with - no mailshots etc so our volume is extremely low.
I have considered warning users who provide a #hotmail/live email address to add us to their address book or check their junk folder - but this just seems unprofessional to me - it may be that i have to resort to this.
As for becoming Sender Score Certified - its very unlikely that i can justify the cost of this when considering the low volume and purpose of these emails.
My company does professional e-mail marketting campaigns (through strongmail servers) we send thousands of (sollicited) emails a day to all kinds of addresses.
The problem you are facing is that you have no authority. You could just be some spammer trying to send loads of spam.
The thing you need to do is:
Add unsubscribe links
Apply for hotmail's Junkmail reporting program (JMRP) and MAKE SURE people that press the 'this is junk' button do not get mailed again. This will up your 'sender score; # hotmail and allow you messages to get through.
Add SPF and other antispam solutions.
Do not send more than 50 e-mails per minute to #hotmail.com (other domains have other limits)
B.t.w we use PHPMailer to compose our messages, no problem at all with that :-)
The problem nowadays really is the restricting receiving mailservers.
Email Deliverability is closer to an art than a science. I can pretty much guarantee that it has nothing to do with your headers. Trying to spoof headers is likely the worst thing you can do. The received: header is added by the mail servers as they receive the messages: spoofing this will cause your email to get flagged as spam: one of the spam filters commonly used is to count then number of relays (ie received: headers). If there's too many you get a higher spam score.
Reverse DNS and SPF are the minimum entry barriers. For hotmail in particular, there are three other very important factors AFTER you get your SPF and DNS records in line:
IP/Domain Reputation
Volume
Being in the Address Book
Reputation isn't the same as being blacklisted. You need to build trust with hotmail. Hotmail uses Sender Score Certified as their main reputation broker -- you can check your reputation with them if you want, but it may cost you.
If you're on a shared host or an IP address that has a checkered past, you won't have much luck with hotmail.
You build reputation by having a consistent volume with low spam complaints. You can send 1M messages an hour all day long, as long as you do it every day. If you're sending less than 10,000 messages a day, you likely won't be able to build up a decent reputation. You can get a report on your volume at Sender Base.
Finally, the best way to make sure you end up in the inbox is to get your users to add the sending email address to their address book. Hotmail uses this as a safe sender list. In fact, I think there's an additional trusted sender option in Hotmail now too (it's been awhile since I've been in the delivery game and I don't use hotmail).
Here are some other best practices for sending email:
ALWAYS use the same IP address
ALWAYS use the same FROM address
if you have a large list that you send newsletters to, make sure you retire old addresses (ie, check open rates)
if you have a large list, try segmenting it and sending from different IP addresses based on risk (ie, newer addresses may mark the message as spam)
I have always used PHPMailer in my projects, and what I did to avoid Hotmail's junk folder was to call a method they had that added MS Headers to the message.
Take a look at the source, and add those headers yourself.
Also, I recommend including a text-version if you're sending HTML e-mail.
I'd suggest modifying the headers you send to match 100% what outlook sends, and see if that solves the problem. Really it's a tough one though, hotmail is known for having a super crappy spam filter, sending lots of legit email to junk, and lots of spam to your inbox.
Related
a client asked me about a little form for his website, from which it would be possible to mail the URL to someone. Something like "Hey check this out".
Since he was not happy with mailto:, I want to use PHP mail() function, but i wonder if it is smart to let users define a sender of the email. I am worried about the form being abused for spam/phishing.
Is that a reason to worry? Is it even legal?
It's legal to send e-mail. It's not legal (everywhere) to send spam. But you are just providing a share link, not a relay server, so I wouldn't worry about that. If you limit the amount of control over the content of the message, and limit the number of people to send it to, it won't be too interesting for spammers.
Letting the user choose a sender is not a very good idea. Some mail relay servers check if the originating server is allowed to send e-mails for the domain specified in the address, so the mails might never arrive. You can safely set the sender name, though.
Apart from that, if the receivers of the message consider it as spam and report it, your domain might become blacklisted, and your mails will be sent to junk mail in many cases, so you want to make sure no (or little) spam is sent through your form.
Those bots try every form automatically just to see what happens, so you'll need to make some effort. You could add a captcha, which is an obstacle for humans too, although Google is going to put an end to that. Or you could protect it through other means, like a honeypot. Maybe you can just generate the form through JavaScript, which is a big obstacle for most spam bots.
Setting the From on an email in php mail isn't the cause for concern. The problem is that you'll be sending emails from your server. The mail headers will have your server information embedded - so any issues will tie back to you.
As long as you can safeguard your own server from allowing these spam/phishing attacks, then there's nothing wrong with it.
Just limit the number of people this mail function can send to - and make sure it can't be called multiple times in succession -- like with a script.
This way, the spammers wouldn't benefit from using your page to try to send spam. They'll go elsewhere.
There's much more to do to work with sending email, but this will at least get you started.
"Is it legal" depends upon the country you are in.
I don't think you need to worry about spam if you set up a login.
Or you could limit the number of emails by IP address. This can be spoofed, however, so it may not be the best option.
There are other control options you could do; limit number of emails by User Agent/IP combination, etc.
Aside from the reasons pointed out by others who have answered this question, I would advise against doing this because these messages will likely be marked as spam by spam filters, due to SPF and DMARC records.
For example, is someone sends a message through your system from a yahoo.com address, most spam filters will treat the message as spam, because os Yahoo's DMARC record, which basically says, 'any message sent from a yahoo.com email address that did not originate from a mail server on yahoo's network is spam'. See https://help.yahoo.com/kb/mail/SLN24016.html?impressions=true for more info.
I'm using PHPMailer to send mail from my server.
I have dkim=pass and SPF=pass on my outgoing emails. I am not listed on spamhaus blacklist or via the mxtoolbox blacklist. My emails usually don't go to spam.
However, today I noticed this line in the header of my outgoing mail:
X-OutGoing-Spam-Status: No, score=-2.8
I'm concerned that this negative number will lead to my emails going to spam. (I'm assuming the negative number is bad, but I couldn't find any information online.)
Can someone shed some light on this? Is it a concern? If so, how do I improve this reading?
Thanks!!
The X-spam-status scores are put into the header of any email that passes through a mail server that is running Spamassassin (and some other anti-spam programs). Here's an article regarding tests Spamassassin does in version 3.3.
The higher the score the more likely the email is spam. It uses a range of things to decide on what is and isn't spam, blacklists are only part of its algorithms.
As listed here, an email can have a negative score if it is whitelisted, which is a good thing. If an email is whitelisted, this usually means (not always) that the recipient has received an email from the sender before, and the user has acted on that email (by acted on, I mean you have responded to it, or clicked "always show images from this sender" kind of thing) or you have that email address saved in your address book.
Hope this helps.
EDIT: In short, the negative number is good and you shouldn't worry about it.
im developing a site who sends A LOT of emails notifications to my users from a php script who is running almost all the time, I will like to format the email mensage in order to make it non-spamm-like so my users can read the info non in the spam folder.
What suggestions do you can offer me? Do i need to use non html mensages? Is there some rules in the naming of the email address? (like no-reply#myhost.com)
I know there is not a 100% secure method, but i will like to make it the best possible. Thanks!
To do that you probably have to configure DKIM/SPF.
DKIM
DomainKeys Identified Mail (DKIM) is a method for associating a domain
name to an email
To setup DKIM in PHP you could have a look at this Stackoverflow topic
SPF
Sender Policy Framework (SPF), as defined in RFC 4408, is an e-mail
validation system designed to prevent e-mail spam by tackling source
address spoofing, a common vulnerability.
This link might help you setup DKIM and SPF.
List
Have a look at this list(more complete?) from Sendgrid:
Sengrid(outsource)
I don't think this is the most simple task. Luckily services like Sendgrid help you sent out emails without them being flagged as spam. They also provide a free plan(200 messages per day).
Also you have to keep in mind that when you sent a lot of emails you should be queuing your emails to sent out emails in a controlled manner(not overloading your server). This is also taking care of by third-party services to sent emails. Also if you are using shared hosting I don't think they will appreciate if you sent a lot emails because your resources are shared with all other users.
Alternatives
Some other alternatives to sendgrid are:
http://postmarkapp.com/
http://www.cloudsmtp.com/
http://www.critsend.com/
When searching Google you can find a lot more.
The email name won't be your issue with spam filters, most of them will be environmental.
The simplest way of making sure that your emails don't get spammed is to ensure the reverse lookups are correct for your MX records and from where the email is sent from. (Important for filters used by google and hotmail et al)
The second is to ensure that the HTML in the email is correctly formatted. Doddgy HTML formatting will cause some Spam filters to instantly class it as spam.
Other than that I would suggest also telling users to add the email address to their address book (hotmail et al) so that the spam filters see it as a not spam email.
Other than that I there are no other rules for emails that I can think of.
Hope that helps
this is my first question on StackOverflow, but I think that we'll both come to the happy end. :) The problem is: I've got newsletter script written in PHP and when I send those emails to the various accounts they are put in the spam folder. Here is what I get in mail headers:
X-Spam-status: Yes, score=5.01 tagged_above=1 required=4.5
tests=[HTML_FONT_SIZE_LARGE=0.001, HTML_MESSAGE=0.001,
HTML_TAG_BALANCE_BODY=0.712, MIME_HTML_ONLY=1.105, MISSING_DATE=1.396,
MISSING_MID=0.14, RCVD_IN_BRBL_LASTEXT=1.644,TO_EQ_FM_DOM_HTML_ONLY=0.001,
T_FRT_CONTACT=0.01] autolearn=no
And here are my questions:
What is that and how to fix status RCVD_IN_BRBL_LASTEXT? I haven't found anything informative googling that phrase.
How to fix MISSING_DATE status? I've put in the code generating whole email:
$mailHeaders[] = "Date: ".date('Y-m-d H:i:s', time());
but with no success. "Date" fields comes only as H:i (13:45, for example) - Thunderbird 3.1.3 FYI. Searching on SO didn't help me. I have also tried adding Delivery-Date status - still nothing.
RCVD_IN_BRBL_LASTEXT indicates that your email has been flagged by the Baracuda RBL, which is a service which tracks IP addresses that have been known to send spam.
Possibly your newsletter has been flagged up as spam in the past? or possibly its the ISP you're using to route your email which has been responsible for some spam. Either way, this particular point isn't an issue with your mail headers.
The missing date is the important one which will bring your points below the threshold.
The date format you need looks like this: Date: Wed, 15 SEP 2010 14:12:27 +0100
Most of that is self explanatory (the last bit is the time zone), and it looks like you know your way around the PHP date function, so hopefully that should sort you out. But I found this page helpful as a walk-through of a legitimate email header format.
The challenge with bulk email sending is that there's so many different factors that could throw you off and get you blocked as spam. Headers tell you what's going on, but in the grand scheme of things they're not one of the biggest challenges.
My company sends 50000+ emails per week, sometimes that many per day. Here's what we've learned:
1) If your server hasn't established "reputation" with email hosts, you're more likely to get flagged. There's no great way to establish it, though sites like Socket Labs simply throttle down new clients in the beginning and after 60-90 days release that throttle to allow more email to go through. As many emails as Socket Labs processes, it tells me it's a valid practice.
1a) Monitor the RBL list to ensure you're not on it. If you do get flagged (happens to just about everyone at some time or another) aggressively work to get yourself off ASAP. Contact the RBL in question and work with them to quickly right the situation.
2) The "big guys" including Gmail, Yahoo, AOL, and MSN are sensitive to being rapidly hit by the same host in succession. My company has chosen to overcome this by keeping track of who our email processes are sending to via a "log" If the next email has the same domain as the previous sent, we wait a period of time. If not, we fire at will. It prevents our system from sending more than 1 email per X seconds to the same host, and has meant our emails are getting through at a very high rate.
3) AOL mail is borderline worthless. I saw a stat once that someone had proven something like 20% of email sent to AOL just "disappears" I'm not sure if it's that high, but I know we have nothing but problems with getting AOL email through...it's the nature of the beast. The good news is that AOL is on its way out, so we shouldn't have to deal with it on this level too much longer.
4) The obvious step is to ensure that you're doing the best you can to stay CAN-Spam compliant. Include a real-time opt-out, company information in the footer, and don't try to deceive with your message.
5) Finally, don't send email to people who haven't requested it. It seems like a silly easy step, but it's abused SO much. You won't be flagged as spam if you send to people who want your email...it's that easy. If you get a bounce, process it out of your list immediately so that you're not trying to resend to a bad account.
Good luck.
The X-Spam-Status header is being added by a Barracuda spam filter, and what you're seeing is diagnostic information it attaches to explain why it marked the message as spam.
Barracuda is a rules-based engine, and as you guessed, the score you're getting (5.01 in the example above, though you say you fixed the date so your score may be lower) is above the allowed threshold.
Since the highest-weighted signal is the RCVD_IN_BRBL_LASTEXT field, you're likely getting dinged primarily because your IP address is in their Real-time Block List (RBL). It's possible you can find the data source that hates your IP -- check http://www.spamhaus.org/lookup.lasso to see if it's Spamhaus that has tagged you -- and convince them to remove you, but more likely the best path is to use an IP that's already been groomed to have a good reputation. Two providers you may wish to check out are http://sendgrid.com and http://authsmtp.com, both of whom will allow you to proxy your traffic through their servers (assuming your content isn't likely to be voted as spam).
Hope that is helpful.
This question already has answers here:
Closed 13 years ago.
Possible Duplicate:
Avoid being blocked by web mail companies for mass/bulk emailing ?
Hi all,
if I send many emails per day (100s or 1000s) to users, style "someone commented on your picture" etc..., what are the best practices to avoid that these emails are labeled spam and blocked? I already have an opt-in (ie. users check a checkbox when signing up to get emails, and confirm their email.)
I'm looking for tips on email headers, and other ways to avoid being labeled as spam - from the basics to advanced techniques. Thanks!!
In terms of tracking how successful your anti-spam measures are, you will not have a great deal of luck with this unfortunately as a lot of spam filtering software simply drops mail that it believes to be bad. However, well-behaved anti-spam systems will reject (at SMTP time) the mail: so, you could capture the rejections if you have a valid return-path for your outgoing mail. Make sure the Sender: header is set to an address of yours which will receive mail so that the bounces end up there.
Some of the big ISPs support providing sites with copies of mail they have identified as spam ("feedback loops"). E.g. AOL does, see http://postmaster.aol.com/fbl/fblinfo.html . This would mean you would receive a sanitized copy of mail that they had received from you and had been marked as spam.
You will find some users are very lazy and rather than follow even very clear or easy unsubscribe instructions, will just click "spam". Even if they opted in explicitly in the first place. Feedback loops can help you identify those users although it's not trivial because the loops are sanitized precicely to stop you identifying the users!
Finally keep an eye on popular RBL lists to ensure your mail servers are not listed. Some of the popular ones are listed at the wikipedia page "Comparison_of_DNS_blacklists" (no direct link, sorry, only one URL per answer for me!)