I am trying to process the bounced emails in phplist using a gmail's email as the bounce back address. When I tried to process bounces, I got stucked in the similar scenario as mentioned in this Post - There are 250 bounces to process.
Phplist was able to fetch only 250 emails from my gmail's account. On further investigating phplists' code I came across this line of code that seems like the culprit.
$num = imap_num_msg($link); // get only count of 250
Skipping more details. I wrote few lines of code to get the mail count using imap and pop. The pop version returns the wrong count whereas that returned by imap version is correct
$username = 'bounceemail#mydomain.com';
$password = 'password';
$hostname = '{imap.gmail.com:993/imap/ssl}INBOX';
$inbox = imap_open($hostname,$username,$password);
$num = imap_num_msg($inbox);
echo $num; // prints 65,051 ( correct one)
$hostname = '{pop.gmail.com:995/pop3/ssl}INBOX';
$inbox = imap_open($hostname,$username,$password);
$num = imap_num_msg($inbox);
echo $num; // prints 250 as count ( wrong one)
I actually need to know why the counts are different for same email with different protocols. Also , All the help I found on internet related to phplist bounce processing explicitly ask to use the {pop.gmail.com:995/pop3/ssl}INBOX version. So I can't risk using the other version to process bounces.
Gmail has a non standard POP implementation that only exposes 250-300 messages at a time until you download and delete them. Or if you use recent:username as your username, it will show you the last 30 days instead.
Either way, if you want full access to your Gmail account you need to use IMAP.
For my project i need to read Emails using POP
I am using the IMAP extension for PHP.
I do not need to fetch all the emails , but get emails by filtering TO and FROM .
it works with FROM not with TO
What i tried is
(TO fetch all email FROM user#domain.com)
$mbox = imap_open ("{pop.domain.net:995/pop3/ssl/novalidate-cert}", "username", "password")or die(imap_last_error());
$some = imap_search($mbox, 'FROM "user#domain.com"',SE_UID);
imap_close($mbox);
var_dump($some);
it works and i am getting an array .
(To fetch all email that which send to user#domain.com using TO)
$mbox = imap_open ("{pop.domain.net:995/pop3/ssl/novalidate-cert}", "username", "password")or die(imap_last_error());
$some = imap_search($mbox, 'TO "user#domain.com"',SE_UID);
imap_close($mbox);
var_dump($some);
Whats wrong with me ? Please note that i am not specifying any mailbox name here ,for IMAP i need to mention the Mailbox name.
Please help me.
You may be searching the inbox for sent items, I think search only works within the root folder often referred to in the RFC spec as a mailbox...
The easiest way would be to switch mailboxes to search sent items (as most of the to's in your inbox should be your email unless you get CC's a lot and that opens up a whole other can of worms)
Sources
https://www.rfc-editor.org/rfc/rfc3501#page-49
I am doing a project that involves processing data from my Gmail history.
Specifically, I want to generate a multi-page styled PDF that has a custom page for each of 100 or so people - showing data such as number of e-mails sent in the past year, number of e-mails received in the past year, average word length of e-mail, most used terms in e-mail, date of oldest e-mail sent or received, maybe even average number of exclamation points or expletives per e-mail, etc.
I saw this question, which had a helpful link to IMAP functions in the PHP manual, but can someone help me out with what the architecture and difficulty of such a project would be?
I am envisioning:
write a php script to run some IMAP functions on my Gmail data and write it to a MySQL database.
write another script to run a loop of MySQL queries on the database and print to a PDF based on the results
First of all, you need php imap library.
Then, just use this simple step-by-step tutorial:
$email = "email#gmail.com";//or alamatemail#nama_domain_hosted
$password = "ini password anda";
$imap_host = "{imap.gmail.com:993/imap/ssl}";
$imap_folder = "INBOX"; //it's what is called label in Gmail
$mailbox = imap_open($imap_host . $imap_folder,$email,$password) or die('Failed to open connection with Gmail: ' . imap_last_error());
With code above, you've already created connection to Gmail.
Now, if you want to search particular message, use this:
$emails = imap_search( $mailbox, 'ALL');
Read RFC 1176 for more detailed options. Search for string "tag SEARCH search_criteria" or read on PHP's imap_search documentation.
This code will process retrieved messages (you can then process it to MySQL as you please):
if( $emails )
{
foreach( $emails as $email_id)
{
$email_info = imap_fetch_overview($mailbox,$email_id,0);
$message = imap_fetchbody($mailbox,$email_id,2);
echo "Subject: " . $email_info[0]->subject . "\n";
echo "Message: " . $message . "\n";
}
}
Answering your additional question:
It's possible to process email on your local server or even from your own laptop / desktop. It work just the way desktop email client work.
It's not that hard once you grasp the basic flow.
Here is my scenario:
I have 2 email accounts: admin#domain.com and bounce#domain.com.
I want to send email to all my users with admin#domain.com but then "reply to" bounce#domain.com (until here, my PHP script can handle it).
When, the email can't be sent, it's sent to bounce#domain.com, the error message could be 553 (non existent email ...) etc.
My question is: How do I direct all those bounce emails (couldn't-sent emails) to bounce#domain.com through a handling script to check for the bounce error codes?
What programming language should I be using for the "handling script"?
What would the "handling script" look like? Can you give a sample?
in other words:
What are the procedures I should follow to handle the bounce email ?
The best scenario is be able to classify the type of bounce: soft, hard...
what we use is BounceStudio. You need to compile it and add the php libraries... not hard at all. You have the free and paid version of that product
once we detect the kind of bounce we use PEAR::MAIL::MIME to search for custom headers that we added previously to the email, lets say:
X-user-id: XXXXX
X-campaign-id: YYYYYY
X-recipient-id: SSSSSSSSS
in this way we can know the real recipient that we sent the email to.
hope this help you! so you can help me to get to the 500 points :P
Why not create a bounce#domain.com and use php to read those emails and do what ever you want?
Edit After your comment : Please chec my link whcih has a php script which will teach you how to open and email box using php and read the emails. You can use this scrip to check the error messages.
Let the emails bounce to an address that is really an emailadress (with login details etc.).
Make a php script which runs ever x minutes (for example with a cron job). This php script must do the following.
- Retrieve all email from the box (use for example Zend Mail)
- Check for the error in the message (e.g. by searching it with regular expressions)
- Do what ever is necessary.
If you want to know specifically who has bounced back you can use user specific bounce addresses. (See for example this site)
Maybe it's a little late for the answer, but you can always try something new.
I had the last week a task like this, and used BOUNCE HANDLER Class, by Chris Fortune, which chops up the bounce into associative arrays - http://www.phpclasses.org/browse/file/11665.html
This will be used after you connect to the POP3 with some mailer to get the bounces from it, then parse it into pieces with this, and if has the status you searched for, do the necessary actions.
Cheers.
If you've got a POP3 mailbox set up for bounce#domain.com, you could use a POP3 client script written in PHP to retrieve the messages and check for undeliverable messages.
You can use imap_open to access your mails from PHP.
This functions also works for POP3 but not every function may work here. However I guess in 2018 most email-clients should support IMAP.
This function can also be used to open streams to POP3 and NNTP
servers, but some functions and features are only available on IMAP
servers.
Here is a little example, how to iterate through your emails:
/* connect to server */
$hostname = "{$your-server:$your-port}INBOX";
$username = 'my-username';
$password = '123';
/* try to connect */
$inbox = imap_open($hostname,$username,$password) or die('Cannot connect to mailbox: ' . imap_last_error());
/* grab emails */
$emails = imap_search($inbox,'ALL');
/* if emails are returned, cycle through each... */
if($emails) {
/* for every email... */
foreach($emails as $email_number) {
$message = imap_body($inbox,$email_number,2);
$head = imap_headerinfo($inbox, $email_number,2);
// Here you can handle your emails
// ...
// ...
}
}
In my case, I know that I always get my mail delivery failed from Mailer-Daemon#myserver.com. So I could identify bounces like that:
if($head->from[0]->mailbox == 'Mailer-Daemon')
{
// We have a bounce mail here!
}
You said:
When, the email can't be sent, it's sent to bounce#domain.com, the
error message could be 553 (non existent email ...) etc.
So if your bounce emails have the subject "Mail delivery failed: Error 553" then you could identify them by the subject like this:
if($head->subject == 'Mail delivery failed: Error 553')
{
// We have a bounce mail here!
}
The failed email address is not in the header, so you need to parse it from the $message variable with some smart code.
You could always use something like http://cloudmailin.com to forward the bounced emails on to your php server via http however you may be better with a service dedicated to sending emails and using their api to retrieve the bounce details.
i have had pretty bad luck looking for a PHP solution for this, but i ran across this product that does just what i needed.
it runs as a native app mac/win but it does the job.
http://www.maxprog.com/site/software/internet-marketing/email-bounce-handler_sheet_us.php
I was searching for the answer to the same question. There are more parts of the question, and more options.
For handling the bounced e-mail, I found a PHP class, purely in PHP, no compile or additional software installation needed if you have a PHP powered site. It is very easy to use.
If you are using cPanel, or InterWorx/SiteWorx, you can configure some of the addresses to handle the received e-mails with a script, for example a PHP script, so you can write your own handling with the aid of the mentioned class. Or of course still you can create ordinary e-mail accounts and retrieve the mails via POP3 or IMAP, and then interpret them. I think the first one is better, because it's direct, you don't have to use additional channels, like IMAP. Of course if you can't configure your mail server, or don't know how to do it, then the former is better for you.
Good luck! :)
In the php mail command http://php.net/mail
you use the fifth parameter and add "-f" to it.
So, you use "-f mybouncebox#mydomain.com" as the parameter
the phpList newsletter manager uses this to manage bounces.
Once the bounces fill up in the mailbox, you can POP them, and process them. That's the easiest way to deal with them, as opposed to handling them when they arrive.
Here is a canned solution to process bounces using IMAP.
I changed the Return-Path header of my Mail instance to a dedicated bounce#xxxxxx.us
The only method easy enough for me to consider viable is the following, which checks via POP3 the dedicated inbox and can handle each email based on the message received.
$inst=pop3_login('mail.xxxxxx.us','110','bounce#xxxxxx.us','pass');
$stat=pop3_stat($inst);
//print_r($stat);
if($stat['Unread']>0){
echo "begin process<br><br>";
$list=pop3_list($inst);
//print_r($list);
foreach($list as $row){
if(strpos($row['from'],'MAILER-DAEMON')!==FALSE){
$msg=imap_fetchbody($inst,$row['msgno'],'1');
if(strpos($msg,'550')!==FALSE){
echo "handle hard bounce".$msg."<br><br>";
//WHATEVER HERE TO PROCESS BOUNCE
}
}
else{
$msg=imap_fetchbody($inst,$row['msgno'],'1');
echo "not from my server. could be spam, etc.".$msg."<br><br>";
//PROBABLY NO ACTION IS NEEDED
}
//AFTER PROCESSING
//imap_delete ( resource $imap_stream , int $msg_number [, int $options = 0 ] )
//commented out because I havent implemented yet. see IMAP documentation
}
}
else{
echo "no unread messages";
}
//imap_close ( resource $imap_stream [, int $flag = 0 ] )
//commented out because I havent implemented yet. see IMAP documentation.
//flag: If set to CL_EXPUNGE, the function will silently expunge the mailbox before closing, removing all messages marked for deletion. You can achieve the same thing by using imap_expunge()
function pop3_login($host,$port,$user,$pass,$folder="INBOX",$ssl=false)
{
$ssl=($ssl==false)?"/novalidate-cert":"";
return (imap_open("{"."$host:$port/pop3$ssl"."}$folder",$user,$pass));
}
function pop3_stat($connection)
{
$check = imap_mailboxmsginfo($connection);
return ((array)$check);
}
function pop3_list($connection,$message="")
{
if ($message)
{
$range=$message;
} else {
$MC = imap_check($connection);
$range = "1:".$MC->Nmsgs;
}
$response = imap_fetch_overview($connection,$range);
foreach ($response as $msg) $result[$msg->msgno]=(array)$msg;
return $result;
}
function pop3_retr($connection,$message)
{
return(imap_fetchheader($connection,$message,FT_PREFETCHTEXT));
}
function pop3_dele($connection,$message)
{
return(imap_delete($connection,$message));
}
We are using Procmail to filter these kind of mails. After examining some of the solutions already mentioned here, we ended up with a simple Procmail recipe to detect bounce messages. Depending on the accuracy you need, this might be applicable to your situation.
For details, check this blog entry.
I had the same problem, exact situation. By default my mail server, is sending all my returned mails to the same account that it was originally sent from, with automatic msg "Mail delivery failed: returning message to sender".
I dont really want to know why it was returned, had so many mails transactions that I just want to remove the bad ones. Dont have time to check specific rule such as Doestn Exist, Unavailable, etc ,,, Just want to flag for deletion and go on.
Bounce mails are so trivial as you need to deal with a lot of different servers and responses types. Each anti spam software or operating system scenario can send a different error code with the bounced email.
I recomend you to read and download this fixed debugged version of Handling Bounced Email - USING PHPMAILER-BMH AND AUTHSMTP from KIDMOSES here http://www.kidmoses.com/blog-article.php?bid=40 if you want to setup IMAP and and send your own custom headers, send them to your bounce#domain.com and then cross your fingers to see if the script catches the headers you sent written in the bounced mail. I tried it, works.
But if you want to follow my quick and easy fix that resolved my problem, here is the secret.
1 - Download the better version from KIDMOSES site or from my site, just in case KIDMOSES want to move somewhere else http://chasqui.market/downloads/KIDMOSES-phpmailer-bmh.zip
2 - The variable that contains the text of your returned mail is $body and itself contains the bad returned email (SO ITS AN MULTIDIMENSIONAL ARRAY ). (Also contains your servers mail and other DNS mails stuff, but we are looking for the BAD MAIL BOUNCED.
3 - Since your OWN SERVICE is sending you back the bounced email, then its not likely to change its format and own headers, sending back bounced emails, so you are safe to pick the order of bounced email array returned. In my case was always the same format template. (Unless you change systems or providers)
4 - We look into that $body and search for all email string variables and extract them positioning them into a two dimensional array called $matches
5 - We select the array position, by printing the array using print_r( array_values( $matches ));
6 - This is the code that you need to modify. Its around line 500 from class.phpmailer-bmh.php file
// process bounces by rules
$result = bmhDSNRules($dsn_msg,$dsn_report,$this->debug_dsn_rule);
} elseif ($type == 'BODY') {
$structure = imap_fetchstructure($this->_mailbox_link,$pos);
switch ($structure->type) {
case 0: // Content-type = text
$body = imap_fetchbody($this->_mailbox_link,$pos,"1");
$result = bmhBodyRules($body,$structure,$this->debug_body_rule);
//MY RULE IT WORKS at least on my return mail system..
$pattern = '/[a-z0-9_\-\+]+#[a-z0-9\-]+\.([a-z]{2,3})(?:\.[a-z]{2})?/i';
preg_match_all($pattern, $body, $matches);
//print_r( array_values( $matches )); //To select array number of bad returned mail desired, usually is 1st array $matches[0][0]
echo "<font color = red>".$matches[0][0]."</font><br>";
break;
So we forget about returned headers and concentrate on the bad emails. You can excel them, you can MySQL them, or process to whatever you want to do.
IMPORTANT
Comment the echos in callback_echo.php in the samples directory otherwise you get all the junk before printed.
function callbackAction ($msgnum, $bounce_type, $email, $subject, $xheader, $cheader, $remove, $rule_no=false, $rule_cat=false, $rule_msg='', $totalFetched=0) {
$displayData = prepData($email, $bounce_type, $remove);
$bounce_type = $displayData['bounce_type'];
$emailName = $displayData['emailName'];
$emailAddy = $displayData['emailAddy'];
$remove = $displayData['remove'];
//echo "<br>".$msgnum . ': ' . $rule_no . ' | ' . $rule_cat . ' | ' . $bounce_type . ' | ' . $remove . ' | ' . $email . ' | ' . $subject . ' | ';
//echo 'Custom Header: ' . $cheader . " | ";
//echo 'Bounce Message: ' . $rule_msg . " | ";
return true;
}
MY OUTPUT
Connected to: mail.chasqui.market (bounce#chasqui.market)
Total: 271 messages
Running in disable_delete mode, not deleting messages from mailbox
kty2001us#starmedia.com
...
entv#nuevoface.com
Closing mailbox, and purging messages
Read: 271 messages
0 action taken
271 no action taken
0 messages deleted
0 messages moved
You should look at SwiftMailer. It's completely written in PHP and has support for "bounce" emails.
http://swiftmailer.org/