Is their a way to either code server side or client side to check new incoming replies to emails and register them in some way? I read upon using something called Reverse Ajax is this possibly the solution I should look into to code what I want.
Also another question is how do you keep track of email conversations? I took at a look at the hidden headers for emails sent and received and it seems that they carry a unique message id which could possibly be used to solve the problem I have.
Thanks :)
Split that into two tasks:
Checking for e-mails on the server
Communication between client and server
For 1: messages have a Message-ID header and optionally a References and an In-Reply-To header that allow you to put together conversations. Note that you also have to keep track of sent messages for that to work (because a reply that you receive has the Message-ID of a message that you sent in the In-Reply-To header).
For 2: the umbrella term seems to be Comet
http://www.phpclasses.org/package/2-PHP-Access-to-e-mail-mailboxes-using-the-POP3-protocol.html
Here solution for poping mail, keeping track of new ones, each message has unique id we can use that to track new one.
Related
I notice that a lot of websites like Facebook and GitHub allow you to reply to an email notification which then posts the reply back to the application using a unique Reply-To address. How does this work on the back-end? I searched a lot but, only found that people were using unique Reply-To addresses. That's understandable but, what do they use to receive these emails? Do these companies set up email servers? I am trying to build this using PHP.
A common method is to set unique message ID's in the outgoing email and then looking at the In-Reply-To header to see which email this was in reply to. That lets you match up your server application's messaging system to emails. For example, if you send an email that represents a help desk ticket with ID 1234, the Message-ID could be something like
ticket-1234#myserver
Then, in email clients that work nicely (I'm looking at you Android), when you hit Reply, the reply has the header In-Reply-To: ticket-1234#mysever. You can go further by adding security tokens to ensure people can't forge messages (e.g. ticket-1234-hf29e9f2gf2e9fh29f#myserver). That security token is generated on the server when you send your first message and is then confirmed in the reply.
This normally works but can cause real headaches when email clients don't behave as they should!
This is how GitLab manages emails by using an identifier in the email address itself and catching it when the user replies. Although it is not relevant to any scripting language like PHP and uses server configs, this is what backend servers do to support reply by email functionality.
You have to set up a mail server to receive mail answers.
SIEVE filters on a IMAP server could be use to make some actions when a certain filter is detected.
But you can't only use PHP to make that type of system.
Im building an email management software using PHP. Im a bit stuck on something, and thought SO might provide some insight. The user retrieves messages. Messages get replied. I was thinking that I could create some sort of custom hash for an incoming message, store the data and hash in the database, and then for replies, inject the custom hash into the message header to denote that this message being sent is apart of that particular incoming message.
After I reply to a message, and then the user re-replies, will the injected custom hash be in the message? If not, it would be treated as a new incoming message.
Should I be looking into injecting custom hashes into messages? Is that even possible? Is it a good method?
Is this a good theory to use? Any suggestions or comments? I really have no experience in this and Im just trying to figure out the best method for implementation.
** NOTE: If there are any open-source PHP email management software that I could reverse-engineer, that would also be something I would be interested in looking at.
Injecting custom headers into the message is possible but it's pretty rare that they would be included in the reply. Sometimes, clients will include a In-Reply-To header, that quotes the original message id and you can use that.
However, the easiest and most common method of doing this is by using a customised from address. If you send the email from message-12345#yourdomain.com then any bounces or replies will come back to that email address. If the next message uses message-12346#yourdomain.com then you can easily tell which reply is for which original message.
There are a few options when receiving the emails:
Poll using POP3 or IMAP
Have the mail server init a script as messages are received
Use a system that converts the emails from SMTP and forwards them as HTTP
I wrote a blog post outlining the methods to receive the messages, it was for Ruby but the same principals all apply. Unfortunately I don't know of any PHP software for this.
I am not sure if it is in the headers or not, but I am looking for a way to tell if an email I receive is a response to an email I sent, and if so, to only grab the new text, not "quoted text"
A little background: I am creating a script that will send out emails automatically. I am creating a cron job to run at periodic intervals to check to see if there were any replies. If there were replies, I only want to grab the new stuff, and not the old stuff.
In the past, I would send out emails with the id in the subject (You have a new response [1234]), and would then check the subject for the stuff in between the [ and ]. Then I would grab all the message and store it since every web browser/email uses a different character or style for quoted-text. Some do ">" some do a horizontal rule, some do absolutely nothing.
Anyways, I am just looking for something in the email header that would indicate they are replying and what the new text might be. If it's not possible, I will just keep on doing what I am doing.
You can know if an email is the reply of another email or not by using the combination of In-Reply-To and References.
Every email has a unique ID in its header called Message-ID, according to this RFC 1, you can track the ancestors of any email.
I have checked it and it is working in all clients (Outlook, Thunderbird)
I will give an example to use.
1- In the header of the email you send for the first time, you (your mail server or you in code) send an ID (Message-ID), if you open source of the email you will see it like this in top section:
... // You (your code) send:
Message-ID: <1#your-domain-mandatory.com>
...
You just need to keep this Message-ID in your program. any subsequent reply will refer to this ID.
2- Client will reply email 1 to you. Client will send a crucial header for you to tell you for which email this reply is in addition to its own Message-ID.
... // Client(Thunderbird) send:
Message-ID: <2#your-domain-mandatory.com>
In-Reply-To: <1#your-domain-mandatory.com>
...
When you receive the second email, it will be easy for you to keep track of the previous email you have sent because the ID of mail(1) is in the In-Reply-To header of the mail(2).
3- if you want to reply back this email again inside your code, you just need to put the Message-ID of the mail(2) in In-Reply-To header and Message-ID of mail(1) and mail(2) in References header. So the client will understand the chain correctly.
... // You (your code) send:
Message-ID: <3#your-domain-mandatory.com>
In-Reply-To: <2#your-domain-mandatory.com>
References: <1#your-domain-mandatory.com> <2#your-domain-mandatory.com>
...
By this header, you are telling the client that this email is a reply to the mail(2) and the ancestors are mail(1) and mail(2).
I have worked with them and read about them and it is working, my problem now is to just get the text of the last email and not the quoted text from the replies. (we are running our own Ticketing system, we create a comment for each email)
Unfortunately, e-mail clients can essentially do whatever they want with your message, and there is no reliable standard for determining how a received message originated at the client. In addition, IMAP doesn't really have anything to do with it. E-mails can be sent a number of different ways, including webmail.
The best you can do is look for an ID number in the subject line (assuming folks don't change it, which they rarely do). You can also do what Google does... fuzzy match the reply text to e-mail you sent to that address. If it matches, consider it part of the reply. This takes great effort though.
I would like to know what's the best way to manage answers to email notifications.
eg. The user receive an email notification, and instead of having to click on a link to go on the website and answer it he could directly reply to the email.
There's no best way. Depending on what you want to do with the responses, you could as well handle them with thunderbirds message filters ;) .
Put a unique identifier into the subject or the body of the notification that you're sending so that you can identify & classify the email correctly when a user replies by email.
If you want to stick with php: pick up the php pop3 client class from: http://www.phpclasses.org/browse/package/2.html or find a client elsewhere and then just parse your message responses and classify / handle them however you want to.
Use the Message-ID header field to uniquely identify each message. If there is a reply on that message, the client should put that message ID in the In-Reply-To header field in the reply message. See RFC 2822 for further details.
Stick an id and authentication token in the subject line.
Pipe it into a script that can process it with Procmail
As a learning exercise you could read the php code and Documentation to the drupal module http://drupal.org/project/mailhandler mail handler. This accepts emails and looks for script like instructions.
The answer above about checking the message id is also very valid.
I am building a symfony module for sending newsletters. the idea is to build a queue list of adreeses to receice the mail and then ,with a cron job, send let's say 50 at a time.
What i don't now how to do is to read the Mail Delivery reports that are sent back by the server when an email adress doesn't exist or the mail is full. The idea is to store these error reports an clean the adress list.
Any ideea how to implement that?
When reading the "bounced inbox", you can use a class like this to actually parse the mail and see what status was returned (e.g. permanent or temporary error):
http://www.phpclasses.org/browse/package/2691.html
To really parse a mail accurately will give you a hard time, as not all mailservers are alike and some will send you a "mailbox full"-error marked with a "permanent" flag while others may tell you that the error "user doesn't exist" is "temporary".
I tried a solution for this once and ended up setting up my own parser connected to a huge database containing possible server replies (and their "real" meaning :).
You can use a reply to address while sending. So bouned emails will be sent to this id. You can also create another PHP script which will read this "reply to" email inbox and get the id from it. You can then remove this id from the list you have.