Related
I use a very basic and simple mail() php function to automatically send email to the new users so that they can activate their account, which looks like this:
$body = "Thank you for your registering at ". BASE_URL. ". To activate your account, please click on this link:\n\n";
$body .= BASE_URL . 'activate.php?e=' . urlencode($e) . "&a=$a";
mail($trimmed['email'], 'Registration Confirmation', $body, 'From: admin#hiteachers.com');
In which the $trimmed['email'] is the email the new user uses to enter the website registration form after validation, and the admin#hiteachers.com is my REAL email address, which I use with Google apps (I followed the link https://support.google.com/a/answer/33353?hl=en (#2,3 and 4) to edit the MX records in my domain manager).
Note :
- I bought the domain and a shared linux server at Godaddy.
I also consulted PHP mail() function cannot send to hotmail?, and some other related links, but it didn't help.
I also followed the link https://mail.live.com/mail/troubleshooting.aspx to troubleshoot the SMTP Non-Delivery Report by adding hiteachers.com IN TXT "v=spf1 -all" into the TXT records and waited for 48 hours, but no luck!
I also added the TXT value v=spf1 a mx -all into my domain TXT records as per other stackoverflow users' suggestions, and waited for 48 hours, but no luck!
I also added the from email address to the safe contact list of the recipient addresses at #hotmail.com, and #outlook.com, which i used for testing, but no luck!
No email has been sent to #hotmail or #outlook inbox, even in the junk folder, and no bouce-back email either (because the recipient accounts are real).
It works well with real gmail and yahoo email recipient addresses.
I tried sending email directly from admin#hiteachers.com to the real recipient email address at outlook.com, it stayed there in the inbox (means that it accepts my admin#hiteachers.com address). However, when I viewed the message source, it says This sender failed our fraud detection checks and may not be who they appear to be.
Do you know what was wrong with it, or have you ever been in the same situation as mine? And the important part of my question is what I should do to eliminate the whole issue, please?
UPDATED
After adding the SPF and other required records, which you can take a look at here http://mxtoolbox.com/ for the domain hiteachers.com, I received the report from <email>noreply-dmarc-support#google.com</email> like this, and all my testing auto-email to #yahoo.com and #gmail.com are going to spam folders, but not the inbox before:
<?xml version="1.0" encoding="UTF-8" ?>
<feedback>
<report_metadata>
<org_name>google.com</org_name>
<email>noreply-dmarc-support#google.com</email>
<extra_contact_info>https://support.google.com/a/answer/2466580</extra_contact_info>
<report_id>10844434555482221094</report_id>
<date_range>
<begin>1458172800</begin>
<end>1458259199</end>
</date_range>
</report_metadata>
<policy_published>
<domain>hiteachers.com</domain>
<adkim>r</adkim>
<aspf>r</aspf>
<p>quarantine</p>
<sp>quarantine</sp>
<pct>100</pct>
</policy_published>
<record>
<row>
<count>1</count>
<policy_evaluated>
<disposition>quarantine</disposition>
<dkim>fail</dkim>
<spf>fail</spf>
</policy_evaluated>
</row>
<identifiers>
<header_from>hiteachers.com</header_from>
</identifiers>
<auth_results>
<spf>
<domain>xxxxxxxxx.xxx.xxx.xxxx.secureserver.net</domain>
<result>none</result>
</spf>
</auth_results>
</record>
</feedback>
I want to setup my mail server in a way so that if someone sends email to user#example.com it'll be directed to user's inbox. This is not mail box. Rather its a chat platform. So communication should be real time. The workflow is,
Someone sends an email to user#example.com.
user gets a chat window with that message.
I can solve it by writing a program that polls the mail server each min and check for new messages. If found it just send a chat message. But that not real-time.
Another option could be adding some sort of plugin to the mail server that does the work. I haven't setup any mail server yet. I'll setup only that mail server that helps me to do this.
I am using Python, PHP. So any solution using those two language is welcomed. If all else fails I guess I have to write plugin in C.
Lamson could work, it's written in Python. It sits in front of your SMTP server and filters out the emails you define in it's routes file. The main rationale appears to be ease of developer use, and it's designed to be integrated into other software.
http://lamsonproject.org/
That's how you can do it:
1) Set up your DNS so that a MX record for the domain points to your server.
2) Configure a postfix virtual alias /etc/postfix/virtual:
#example.com django-mail-in
3) and /etc/aliases:
django-mail-in: "|/usr/local/bin/mta2django.py http://127.0.0.1:8000/mail-inbound"
4) The /usr/local/bin/mta2django.py is called by postscript and sends the mail to the mail-inbound django view. This mta2django.py should work:
#!/usr/bin/python
import sys, urllib
import os
def post_message(url, recipient, message_txt):
""" post an email message to the given url
"""
if not url:
print "Invalid url."
print "usage: mta2django.py url <recipient>"
sys.exit(64)
data = {'mail': message_txt}
if recipient and len(recipient) > 0:
data ['recipient'] = recipient
try:
result = urllib.urlopen(url, urllib.urlencode(data)).read()
except (IOError,EOFError),e:
print "error: could not connect to server",e
sys.exit(73)
try:
exitcode, errormsg = result.split(':')
if exitcode != '0':
print 'Error %s: %s' % (exitcode, errormsg)
sys.exit(int(exitcode))
except ValueError:
print 'Unknown error.'
sys.exit(69)
sys.exit(0)
if __name__ == '__main__':
# This gets called by the MTA when a new message arrives.
# The mail message file gets passed in on the stdin
# Get the raw mail
message_txt = sys.stdin.read()
url = ''
if len(sys.argv)>1:
url = sys.argv[1]
recipient = ''
# If mta2django is executed as external command by the MTA, the
# environment variable ORIGINAL_RECIPIENT contains the entire
# recipient address, before any address rewriting or aliasing
recipient = os.environ.get('ORIGINAL_RECIPIENT')
if len(sys.argv)>2:
recipient = sys.argv[2]
post_message(url, recipient, message_txt)
5) Write a django view /mail-inbound which receives the mail and does the things you need it to do. In the request you have:
mail - the full email message
recipient - the original recipient (useful when you do not catch a specific email address but the whole domain / subdomain)
You can parse the email using the python email module:
import email
msg = email.message_from_string(request.get('mail'))
As I'm no postfix expert, I'm not sure if editing /etc/postfix/virtual and /etc/aliases is sufficient. Please consult the postfix documentation for details.
TL;DR: Mails sent from shared hosting (such as a cheap domain from Unoeuro or One.com) end up in spam. How to fix?
I made a mail-system, that first generated a PDF-file (using FPDF), whereafter it sent the PDF-file as an attachment with PHP's Swiftmailer. This email was sent out to 130 people (as a 'one-of' invoice). But it landed in the spam-filter for almost everybody. I tried adjusting SwiftMailers header-settings, but without any luck. Even mails that I haven't sent to before (thoroughly tested). This was my initial setup:
function sendMailEt($toEmail, $toName, $invoiceNumber){
require_once('includes/lib/swift_required.php');
$transport = Swift_SmtpTransport::newInstance('mailout.one.com', 25)
->setUsername('EMAIL-ACCOUNT1#THE-DOMAIN.DK')
->setPassword('THE-PASSWORD')
;
$mailer = Swift_Mailer::newInstance($transport);
$message = Swift_Message::newInstance('FROM COMPANY')
->setSubject('Thanks for signing up - COMPANY')
->setFrom(array('EMAIL-ACCOUNT1#THE-DOMAIN.DK' => 'Company name'))
->setTo(array($toEmail => $toName))
->setBody('A brief body, that explains that this is an invoice and that it has to be paid within 5 days. (written in danish)')
->addPart('A brief body, that explains that this is an invoice and that it has to be paid within 5 days. (written in danish)', 'text/html')
->attach(Swift_Attachment::fromPath('/URL-TO-THE-PDF-FILE.pdf'))
;
$result = $mailer->send($message);
}
I also tried sending out the emails with PHP's native mail()-function, and then simply link to the invoice ( http://www.company-domain-name.dk/invoice/base64_encoded-name.pdf )... Same result (spam).
I tried writing the entire header myself. I've read a numerous amount of forums about what headers should include, but they all wrote different things. So I tried a few different things (both emails I had sent to previously and emails I hadn't)... Same result (spam).
Then I tried writing the header exactly as MailChimps does, in their header. That led me to this:
$headers = "Reply-To: Company name <UNUSED-EMAIL-ACCOUNT-FROM-DOMAIN#DOMAIN-NAME.DK>\r\n";
$headers .= "Return-Path: Company name <UNUSED-EMAIL-ACCOUNT-FROM-DOMAIN#DOMAIN-NAME.DK>\r\n";
$headers .= "From: Message from Company name <UNUSED-EMAIL-ACCOUNT-FROM-DOMAIN#DOMAIN-NAME.DK>\r\n";
$headers .= "MIME-Version: 1.0\r\n";
$headers .= "Sender: Message from Company name <UNUSED-EMAIL-ACCOUNT-FROM-DOMAIN#DOMAIN-NAME.DK>\r\n";
$headers .= "Content-type: text/plain; charset=\"utf-8\"; \r\n";
$headers .= "X-Mailer: PHP". phpversion() ."\r\n";
And then I send the mail like this:
mail($toName . '<'.$toEmail.'>', utf8_decode('Faktura på depositumet'), utf8_decode($someMessage), $headers);
... Same result (spam).
The webspace is with One.com, so I can't use PHPmailer (since that has to be installed, and that can't be done on one.com's servers). And I can't define a SPF with One.com.
All I want, is to be able to send emails that doesn't go to spam.
Here are my questions:
Is it because my header is off, or is it something 'deeper down'?
Does the Gmail-spam filter ban single email accounts (such as
this#example.com) or does it ban entire domains (such as
#example.com)?
Can one get a blacklisted email whitelisted
somehow?
* Addition 1 *
Ok... I have now tried a number of things:
I tried adding LoneWolfPR's returnpath, like recommended, and it didn't help.
I contacted One.com (the hosting company), and confirmed with them, that it isn't possible to set a SPF-record or a DKIM-record. It still isn't.
I considered setting up an 'unsubscribe'-link, with a link to a website with a form, but I didn't believe that approach. I mean - invoices are sent all the time, with e-mails. And why should you be able to unsubscribe an invoice?! Since that made so far from sense in my head, then I only tried it for about 20 minutes (obviously, without results).
Here is my current email header (gotten from Gmail, by clicking the 'View original'):
Delivered-To: NEWLY-CREATED-GMAIL-ACCOUNT#gmail.com
Received: by 10.76.75.104 with SMTP id b8csp48728oaw;
Sat, 16 Mar 2013 17:32:56 -0700 (PDT)
X-Received: by 10.152.116.45 with SMTP id jt13mr7897860lab.0.1363480376067;
Sat, 16 Mar 2013 17:32:56 -0700 (PDT)
Return-Path: <XXX111#DOMAIN-NAME.dk>
Received: from mail-out2.b-one.net (mail-out2.one.com. [91.198.169.19])
by mx.google.com with ESMTP id p10si4637427lbb.120.2013.03.16.17.32.55;
Sat, 16 Mar 2013 17:32:55 -0700 (PDT)
Received-SPF: neutral (google.com: 91.198.169.19 is neither permitted nor denied by best guess record for domain of XXX111#DOMAIN-NAME.dk) client-ip=91.198.169.19;
Authentication-Results: mx.google.com;
spf=neutral (google.com: 91.198.169.19 is neither permitted nor denied by best guess record for domain of XXX111#DOMAIN-NAME.dk) smtp.mail=XXX111#DOMAIN-NAME.dk
Date: Sat, 16 Mar 2013 17:32:55 -0700 (PDT)
Message-Id: <51450f37.6a0b700a.6239.5dbcSMTPIN_ADDED_MISSING#mx.google.com>
Received: from localhost.localdomain (srv18.one.com [193.202.110.18])
by mail-out2.b-one.net (Postfix) with ESMTP id F3D0B10365
for <NEWLY-CREATED-GMAIL-ACCOUNT#gmail.com>; Sun, 17 Mar 2013 01:32:53 +0100 (CET)
Received: from 85.218.159.219 by www.DOMAIN-NAME.dk via URL_TO_THE_SCRIPT.php with HTTP; Sun, 17 Mar 2013 00:32:53 +0000
To: RECIEVERS_NAME <NEWLY-CREATED-GMAIL-ACCOUNT#gmail.com>
Subject: EMAIL-SUBJECT
X-PHP-Originating-Script: 87486:NAME-OF-THE-SCRIPT-THE-E-MAIL-WAS-SENT-FROM.php
Reply-To: COMPANY NAME <XXX111#DOMAIN-NAME.dk>
From: Besked fra COMPANY NAME <XXX111#DOMAIN-NAME.dk>
MIME-Version: 1.0
Sender: Besked fra COMPANY NAME <XXX111#DOMAIN-NAME.dk>
Content-type: text/plain; charset="utf-8";
X-Mailer: PHP5.3.21
1) Normally an email address won't go easily into a blacklist, it takes time and/or a lot of people to tag you as spammer to actually get that address into a blacklist.
2) Yes. A whole domain name can be blacklisted, because spammers normally generate random email addresses like f4j3ifl#something.com.
3) It doesn't matter how many times it went to the spambox, basically, the spam filters nowadays are strong, because spammers try to improve their ways to get around day by day, so the filters gets more strict every day. If it goes into spam folder first time, and the user didn't actually put it into the spam box, it will continue going unless users unmark it, or you fix the troubles.
How to avoid spambox?
Basically you need some signatures, and a lot of access to your DNS records, because there is where we are going to do most of the setups.
Reverse DNS Lookup: On dedicated servers or even on some VPS you are able to set up a reverse dns record, sometimes you just open a ticket and the IT's set it up for you. If you can't have it, change your hosting or keep being tagged as spammer xD. This is to preven header forgeries, as you could set on your headers that your email comes from gmail.com but it doesn't this is the way the email servers check it.
SPF is a must have as well, if you can't set a SPF then don't even try any further, consider changing your hosting, and you can almost stop reading by here xD.
DKIM/Domain Key: preferably DKIM, is a encrypted signature, you set the public key on the DNS, and store a private key in your email server, when a server receives an email, it has the private key attached in the headers (you need a mailserver software which manages DKIM, for windows for example it worked for me hmailserver) and the mail service (gmail for example) will check your dns record to see if the public key matches. This is almost a must have as well
Those three were the basics, if you Set up DMARC and ADSP it will get you better score for the SpamAssassins. To get a even better score search for some spam keywords lists on google and try to avoid them, some stuff like starting an email with "Dear xxx" are harmful for your score, Set up the unsuscribe system(even if it's crappy, as long as you provide a clear link) will help you a bit as well.
Also:
Avoid sloppy html and white text over (any) backgrounds, some spammers use it to fit in hidden text, those filters are smarter than you think.
Read the specific recommendations. Most email services have a FAQ or something in their website with some tips to help you sending emails and not going into the junk. on Some of them you can even apply for getting into a white list ( at least some years ago, on some services like gmail they don't do it anymore)
If you are sending in bulk, watch the time! If you have X emails per second sent into somewhere, you are likely to get into blacklist, set up a script or something to get a 1sec delay or so, the delay might depend on the destinatary to get into the blacklist or not.
Hope those tips help you, I had to deal with some spam filters recently and it's a pain in the ass, that's why I know all that info, that's all my research xD Even after all the signatures and things I have set up, some of the emails are still going into spambox(a smaller percentage but it still hurts me) The only reliable way is to get the users adding you to the contacts list (while having the signatures and headers correctly), so remind them to do so if possible.
One thing to bear in mind, I had trouble with emails being blocked by Gmail and Yahoo! mail from php because the Return-Path header didn't match the from. On a lot of servers if you explicitly set the Return-Path in the headers PHP Mail will ignore that and set the return path to the machine name. You have to force it in the 'additional parameters' section of the mail function using the '-f' flag. Now I've never used Swift Mailer so I don't know the equivalent to PHP's native mail() function, but here's what it would look like using mail();
mail($to,$subject,$message,$headers,'-f returnpath#example.com')
If you can find out the equivalent to this in swift mailer it might solve your problem.
Edit:
It looks like you're not actually setting the Return-Path at all. I know GMail really doesn't like that to be left out. Try setting it explicitly to your Swift_Mailer message (and make sure it matches your From):
$message->setReturnPath('from#example.com');
Solution: Use Mailgun (not tested) or Sendgrid (tested and works wonders!). There is a price-difference between the two, - but in short: Mailgun is good if you're small; Sendgrid is good if you're big.
Either that, - or send mails using MailChimps API or something. It's can't be fixed on shared hosts (most likely). The reason is below.
Explanation: I've later learned more about how shared hosts work. Imagine that several different sites are located on the same server (such as domain-1.org, domain-2.org and domain-3.org). That means that if domain-3.org sends a bunch of crap-emails, then Gmail (and other spam-filters) mark that IP-address as spam. So if domain-2.org then send out stuff, then that'll (probably) come from the some IP-address and therefore end up in spam. Shared hosts can't really do anything about it (and don't care, since so few people have this problem). And that is why it's so cheap.
Sendgrid and Mailguns IP-addresses are marked as 'fine' by all the spam-filters, and that's the service that you're paying for with them. They keep it that way, by monitoring how many emails you send out are being marked as 'spam'. If it's something like 5%-10% or something crazy low, then Sendgrid/Mailgun will block your account until you fix it (going through a long process, where you have to contact their customer service and do 1.000 hail-Mary's and all kinds of wierd stuff).
I heard that if you get your own server (which is way more expensive), and set up your own mail-server, then you have to be really careful, not to be marked as spam. Cause spam-filter are really tough nowadays...
Make sure the email address you are using as the FROM is actually an email address. I have had the same issue been resolved by going into my account management from the host (one.com for you) and adding the account that I want the email to be from. I added an account called "mailer" and through the panel I was able to setup an auto-responder that said, "Sorry. This email address is reserved for server functions".
in the from header you would then use (mailer#yourdomain.com)
having that actual email address and auto-reponder did the trick. I think gmail is just smart enough to know that an email adress that has never been used before is spam. Also, the email address must come from the domain that the script lives on so that when it says it is FROM there it is not lying.
here is the code that I use when I want to send email from my shared host (justhost.com) , It does not go to spam (this is using post data from a web form):
<?php
// Contact subject
$subject = $_POST["subject"];
// Details
$message=$_POST["detail"];
// Email of sender
$mail_from=$_POST["customer_mail"];
//Name of sender
$name=$_POST["name"];
putenv("TZ=America/Phoenix");
$now = date("F j, Y, g:i a T");
$header="Reply-To: $name <$mail_from>";
$header .= "From: MyDomainName.com <mailer#mydomainname.com>";
$header .= "\r\n";
$header .= "Reply-To: $name <$mail_from>";
$introMSG= "Message From:".$name." <".$mail_from.">"."\r\n"
."Sent On:".$now."\r\n"."From a web form on MyDomaiNname.com"."\r\n"."-----------
-----------------------"."\r\n"."\r\n";
$to ='me#mydomainname.com'; // Domain Owners Email Address
$send_contact=mail($to,$subject,$introMSG.$message,$header);
$send_copy=mail($mail_from,"Copy Of:".$subject,$introMSG.$message,$header);
// Check if message sent
if($send_contact){
echo "<strong>Thanks! Your message has been sent to me.</strong>";
}
else {
echo "<strong>There was an error sending your message </strong>";
}
if($send_copy){
echo "<strong><br>A copy of this message was sent to your email.<br>If you do not
receive a copy please check your spam folder</strong>";
}
else{
echo "<strong> There was an error sending a copy of this message to your email
</strong>";
}
$send_reminder=mail("5555555555#txt.att.net","","You Have a new contact message from
".$name.", remember to check your spam folder.",$header);
if($send_reminder){
echo ".";
}
else {
echo "<br><strong>TXT Error</strong>";
}
?>
There are at least two "spammy" looking things that jump out of your email headers:
Message-Id: <51450f37.6a0b700a.6239.5dbcSMTPIN_ADDED_MISSING#mx.google.com>
Notice the SMTPIN_ADDED_MISSING section? You aren't behaving like a proper mailer and generating a unique Message-ID. You might find reading RFC 5322 to be instructional.
Received: from localhost.localdomain (srv18.one.com [193.202.110.18])
by mail-out2.b-one.net (Postfix) with ESMTP id F3D0B10365
for <NEWLY-CREATED-GMAIL-ACCOUNT#gmail.com>; Sun, 17 Mar 2013 01:32:53 +0100 (CET)
That initial received header has an illegal HELO hostname (localhost.localdomain). Your mailer app should provide a way for you to set that to a valid value. It might even be as easy as configuring the hostname of the machine running PHP. See RFC 1035 (hostname validity), RFC 2821 (SMTP) and RFC 5321 (SMTP).
I'm using CakePHP to send automated emails to clients. It's been working great, but it seems some recipients aren't receiving our emails. So I decided to use the SMTP option for sending emails, and route emails through our email provider at Media Temple.
However, when trying to send email from a Media Temple account, I get the error "550- relay not permitted".
That sounds like the Media Temple server is just plain not allowing me to send mail through it.
That's odd because I've confirmed the username and password I'm using is correct and I can send mail via SMTP through it from my macmail client and iPhone mail client. I've also confirmed my cakephp email settings are correct, because I can send emails via SMTP with a gmail account with the exact same configuration in cakephp.
Any idea why I'm getting this error and how to resolve it?
Thanks
Here's the code that handles sending an email. I use this class just like the regular EmailComponent from within many different controllers.
class CanadafindsEmailerComponent extends EmailComponent
{
...
function send($content = null, $template = null, $layout = null) {
if(!in_array(TECHY_MONITOR_EMAIL,$this->bcc) && is_array($this->bcc))
$this->bcc[]=TECHY_MONITOR_EMAIL;
else if (!in_array(TECHY_MONITOR_EMAIL,$this->bcc) && !is_array($this->bcc))
$this->bcc=array(TECHY_MONITOR_EMAIL);
if(DEVSITE){//commented-out code are settings for smtp with gmail, which works fine
$this->delivery = 'smtp';
$this->smtpOptions = array(
'port'=>'465',//'465',
'timeout'=>'30',//'30',
'auth' => true,
'host' => 'ssl://mail.thenumber.biz',//'ssl://smtp.gmail.com',
'username'=>USERNAME,//'USERNAME#gmail.com',
'password'=>SMTP_PASSWORD//,
);
$this->to=$this->correctFormatOn($this->to);
$this->bcc=$this->correctFormatOn($this->bcc);
$this->cc=$this->correctFormatOn($this->cc);
$this->replyTo=$this->correctFormatOn($this->replyTo);
$this->from=$this->correctFormatOn($this->from);
}
return parent::send($content,$template,$layout);
}
function correctFormatOn(&$email){
if(is_array($email)){
$copiedEmail=array();
foreach($email as $singleEmail){
$copiedEmail[]=$this->correctFormatOnSingle($singleEmail);
}
$email=$copiedEmail;
}else{
$email=$this->correctFormatOnSingle($email);
}
return $email;
}
function correctFormatOnSingle(&$email){
$subEmails=explode(",",$email);
$fixedSubEmails=array();
foreach($subEmails as $subEmail){
$fixedSubEmails[]=preg_replace('/<?([^< ]+)#([^>,]+)[>,]?/i', '<$1#$2>', trim($subEmail));
}
$email=implode(",",$fixedSubEmails);
return $email;
}
}
The main problem I was having was that clients weren't receiving emails from our server, (and so I wanted to use an SMTP server to see if that would fix it, instead of the server's default email server).
But I managed to get those clients to receive emails from the server by making some other changes, thus removing the need to use SMTP and the Media Temple email server.
(As an FYI, I found that we were getting bouncebacks from client email servers stating Diagnostic-Code: smtp; 550 Access denied - Invalid HELO name (See RFC2821
4.1.1.1), but they were being sent directly back to the server, and going into the linux user account "www-data". (I read them in /var/mail/www-data, just using tail and vim). I found that postfix, which was handling the sending of emails, was marking the email sender's hostname (ie, "HELO name") as canadafinds3, the name I gave the server in Rackspace, not the domain name: canadafinds.com. So I changed that in /etc/postfix/main.cf, restarted postfix, et voila! No more bouncebacks from those particular clients, and everyone's happy again.)
I ended up writing my own PHP mail() script based on https://web.archive.org/web/20180401094709/http://www.dreamincode.net/forums/topic/36108-send-emails-using-php-smtp-direct/ in order to circumvent this error.
Here is my scenario:
I have 2 email accounts: admin#domain.com and bounce#domain.com.
I want to send email to all my users with admin#domain.com but then "reply to" bounce#domain.com (until here, my PHP script can handle it).
When, the email can't be sent, it's sent to bounce#domain.com, the error message could be 553 (non existent email ...) etc.
My question is: How do I direct all those bounce emails (couldn't-sent emails) to bounce#domain.com through a handling script to check for the bounce error codes?
What programming language should I be using for the "handling script"?
What would the "handling script" look like? Can you give a sample?
in other words:
What are the procedures I should follow to handle the bounce email ?
The best scenario is be able to classify the type of bounce: soft, hard...
what we use is BounceStudio. You need to compile it and add the php libraries... not hard at all. You have the free and paid version of that product
once we detect the kind of bounce we use PEAR::MAIL::MIME to search for custom headers that we added previously to the email, lets say:
X-user-id: XXXXX
X-campaign-id: YYYYYY
X-recipient-id: SSSSSSSSS
in this way we can know the real recipient that we sent the email to.
hope this help you! so you can help me to get to the 500 points :P
Why not create a bounce#domain.com and use php to read those emails and do what ever you want?
Edit After your comment : Please chec my link whcih has a php script which will teach you how to open and email box using php and read the emails. You can use this scrip to check the error messages.
Let the emails bounce to an address that is really an emailadress (with login details etc.).
Make a php script which runs ever x minutes (for example with a cron job). This php script must do the following.
- Retrieve all email from the box (use for example Zend Mail)
- Check for the error in the message (e.g. by searching it with regular expressions)
- Do what ever is necessary.
If you want to know specifically who has bounced back you can use user specific bounce addresses. (See for example this site)
Maybe it's a little late for the answer, but you can always try something new.
I had the last week a task like this, and used BOUNCE HANDLER Class, by Chris Fortune, which chops up the bounce into associative arrays - http://www.phpclasses.org/browse/file/11665.html
This will be used after you connect to the POP3 with some mailer to get the bounces from it, then parse it into pieces with this, and if has the status you searched for, do the necessary actions.
Cheers.
If you've got a POP3 mailbox set up for bounce#domain.com, you could use a POP3 client script written in PHP to retrieve the messages and check for undeliverable messages.
You can use imap_open to access your mails from PHP.
This functions also works for POP3 but not every function may work here. However I guess in 2018 most email-clients should support IMAP.
This function can also be used to open streams to POP3 and NNTP
servers, but some functions and features are only available on IMAP
servers.
Here is a little example, how to iterate through your emails:
/* connect to server */
$hostname = "{$your-server:$your-port}INBOX";
$username = 'my-username';
$password = '123';
/* try to connect */
$inbox = imap_open($hostname,$username,$password) or die('Cannot connect to mailbox: ' . imap_last_error());
/* grab emails */
$emails = imap_search($inbox,'ALL');
/* if emails are returned, cycle through each... */
if($emails) {
/* for every email... */
foreach($emails as $email_number) {
$message = imap_body($inbox,$email_number,2);
$head = imap_headerinfo($inbox, $email_number,2);
// Here you can handle your emails
// ...
// ...
}
}
In my case, I know that I always get my mail delivery failed from Mailer-Daemon#myserver.com. So I could identify bounces like that:
if($head->from[0]->mailbox == 'Mailer-Daemon')
{
// We have a bounce mail here!
}
You said:
When, the email can't be sent, it's sent to bounce#domain.com, the
error message could be 553 (non existent email ...) etc.
So if your bounce emails have the subject "Mail delivery failed: Error 553" then you could identify them by the subject like this:
if($head->subject == 'Mail delivery failed: Error 553')
{
// We have a bounce mail here!
}
The failed email address is not in the header, so you need to parse it from the $message variable with some smart code.
You could always use something like http://cloudmailin.com to forward the bounced emails on to your php server via http however you may be better with a service dedicated to sending emails and using their api to retrieve the bounce details.
i have had pretty bad luck looking for a PHP solution for this, but i ran across this product that does just what i needed.
it runs as a native app mac/win but it does the job.
http://www.maxprog.com/site/software/internet-marketing/email-bounce-handler_sheet_us.php
I was searching for the answer to the same question. There are more parts of the question, and more options.
For handling the bounced e-mail, I found a PHP class, purely in PHP, no compile or additional software installation needed if you have a PHP powered site. It is very easy to use.
If you are using cPanel, or InterWorx/SiteWorx, you can configure some of the addresses to handle the received e-mails with a script, for example a PHP script, so you can write your own handling with the aid of the mentioned class. Or of course still you can create ordinary e-mail accounts and retrieve the mails via POP3 or IMAP, and then interpret them. I think the first one is better, because it's direct, you don't have to use additional channels, like IMAP. Of course if you can't configure your mail server, or don't know how to do it, then the former is better for you.
Good luck! :)
In the php mail command http://php.net/mail
you use the fifth parameter and add "-f" to it.
So, you use "-f mybouncebox#mydomain.com" as the parameter
the phpList newsletter manager uses this to manage bounces.
Once the bounces fill up in the mailbox, you can POP them, and process them. That's the easiest way to deal with them, as opposed to handling them when they arrive.
Here is a canned solution to process bounces using IMAP.
I changed the Return-Path header of my Mail instance to a dedicated bounce#xxxxxx.us
The only method easy enough for me to consider viable is the following, which checks via POP3 the dedicated inbox and can handle each email based on the message received.
$inst=pop3_login('mail.xxxxxx.us','110','bounce#xxxxxx.us','pass');
$stat=pop3_stat($inst);
//print_r($stat);
if($stat['Unread']>0){
echo "begin process<br><br>";
$list=pop3_list($inst);
//print_r($list);
foreach($list as $row){
if(strpos($row['from'],'MAILER-DAEMON')!==FALSE){
$msg=imap_fetchbody($inst,$row['msgno'],'1');
if(strpos($msg,'550')!==FALSE){
echo "handle hard bounce".$msg."<br><br>";
//WHATEVER HERE TO PROCESS BOUNCE
}
}
else{
$msg=imap_fetchbody($inst,$row['msgno'],'1');
echo "not from my server. could be spam, etc.".$msg."<br><br>";
//PROBABLY NO ACTION IS NEEDED
}
//AFTER PROCESSING
//imap_delete ( resource $imap_stream , int $msg_number [, int $options = 0 ] )
//commented out because I havent implemented yet. see IMAP documentation
}
}
else{
echo "no unread messages";
}
//imap_close ( resource $imap_stream [, int $flag = 0 ] )
//commented out because I havent implemented yet. see IMAP documentation.
//flag: If set to CL_EXPUNGE, the function will silently expunge the mailbox before closing, removing all messages marked for deletion. You can achieve the same thing by using imap_expunge()
function pop3_login($host,$port,$user,$pass,$folder="INBOX",$ssl=false)
{
$ssl=($ssl==false)?"/novalidate-cert":"";
return (imap_open("{"."$host:$port/pop3$ssl"."}$folder",$user,$pass));
}
function pop3_stat($connection)
{
$check = imap_mailboxmsginfo($connection);
return ((array)$check);
}
function pop3_list($connection,$message="")
{
if ($message)
{
$range=$message;
} else {
$MC = imap_check($connection);
$range = "1:".$MC->Nmsgs;
}
$response = imap_fetch_overview($connection,$range);
foreach ($response as $msg) $result[$msg->msgno]=(array)$msg;
return $result;
}
function pop3_retr($connection,$message)
{
return(imap_fetchheader($connection,$message,FT_PREFETCHTEXT));
}
function pop3_dele($connection,$message)
{
return(imap_delete($connection,$message));
}
We are using Procmail to filter these kind of mails. After examining some of the solutions already mentioned here, we ended up with a simple Procmail recipe to detect bounce messages. Depending on the accuracy you need, this might be applicable to your situation.
For details, check this blog entry.
I had the same problem, exact situation. By default my mail server, is sending all my returned mails to the same account that it was originally sent from, with automatic msg "Mail delivery failed: returning message to sender".
I dont really want to know why it was returned, had so many mails transactions that I just want to remove the bad ones. Dont have time to check specific rule such as Doestn Exist, Unavailable, etc ,,, Just want to flag for deletion and go on.
Bounce mails are so trivial as you need to deal with a lot of different servers and responses types. Each anti spam software or operating system scenario can send a different error code with the bounced email.
I recomend you to read and download this fixed debugged version of Handling Bounced Email - USING PHPMAILER-BMH AND AUTHSMTP from KIDMOSES here http://www.kidmoses.com/blog-article.php?bid=40 if you want to setup IMAP and and send your own custom headers, send them to your bounce#domain.com and then cross your fingers to see if the script catches the headers you sent written in the bounced mail. I tried it, works.
But if you want to follow my quick and easy fix that resolved my problem, here is the secret.
1 - Download the better version from KIDMOSES site or from my site, just in case KIDMOSES want to move somewhere else http://chasqui.market/downloads/KIDMOSES-phpmailer-bmh.zip
2 - The variable that contains the text of your returned mail is $body and itself contains the bad returned email (SO ITS AN MULTIDIMENSIONAL ARRAY ). (Also contains your servers mail and other DNS mails stuff, but we are looking for the BAD MAIL BOUNCED.
3 - Since your OWN SERVICE is sending you back the bounced email, then its not likely to change its format and own headers, sending back bounced emails, so you are safe to pick the order of bounced email array returned. In my case was always the same format template. (Unless you change systems or providers)
4 - We look into that $body and search for all email string variables and extract them positioning them into a two dimensional array called $matches
5 - We select the array position, by printing the array using print_r( array_values( $matches ));
6 - This is the code that you need to modify. Its around line 500 from class.phpmailer-bmh.php file
// process bounces by rules
$result = bmhDSNRules($dsn_msg,$dsn_report,$this->debug_dsn_rule);
} elseif ($type == 'BODY') {
$structure = imap_fetchstructure($this->_mailbox_link,$pos);
switch ($structure->type) {
case 0: // Content-type = text
$body = imap_fetchbody($this->_mailbox_link,$pos,"1");
$result = bmhBodyRules($body,$structure,$this->debug_body_rule);
//MY RULE IT WORKS at least on my return mail system..
$pattern = '/[a-z0-9_\-\+]+#[a-z0-9\-]+\.([a-z]{2,3})(?:\.[a-z]{2})?/i';
preg_match_all($pattern, $body, $matches);
//print_r( array_values( $matches )); //To select array number of bad returned mail desired, usually is 1st array $matches[0][0]
echo "<font color = red>".$matches[0][0]."</font><br>";
break;
So we forget about returned headers and concentrate on the bad emails. You can excel them, you can MySQL them, or process to whatever you want to do.
IMPORTANT
Comment the echos in callback_echo.php in the samples directory otherwise you get all the junk before printed.
function callbackAction ($msgnum, $bounce_type, $email, $subject, $xheader, $cheader, $remove, $rule_no=false, $rule_cat=false, $rule_msg='', $totalFetched=0) {
$displayData = prepData($email, $bounce_type, $remove);
$bounce_type = $displayData['bounce_type'];
$emailName = $displayData['emailName'];
$emailAddy = $displayData['emailAddy'];
$remove = $displayData['remove'];
//echo "<br>".$msgnum . ': ' . $rule_no . ' | ' . $rule_cat . ' | ' . $bounce_type . ' | ' . $remove . ' | ' . $email . ' | ' . $subject . ' | ';
//echo 'Custom Header: ' . $cheader . " | ";
//echo 'Bounce Message: ' . $rule_msg . " | ";
return true;
}
MY OUTPUT
Connected to: mail.chasqui.market (bounce#chasqui.market)
Total: 271 messages
Running in disable_delete mode, not deleting messages from mailbox
kty2001us#starmedia.com
...
entv#nuevoface.com
Closing mailbox, and purging messages
Read: 271 messages
0 action taken
271 no action taken
0 messages deleted
0 messages moved
You should look at SwiftMailer. It's completely written in PHP and has support for "bounce" emails.
http://swiftmailer.org/