Say I have my own server, and I want to send an email that's reads "hello World" to my site, submit#example.com
Is there a way to automatically extract the text "hello world" and post the content to my website?
I believe I can read a file with PHP, and the file can be created by apache for every incoming email... but I'm missing some knowledge here
can anybody help me?
You need to have PHP act as an email client. Take a look at PHP IMAP (which actually works for POP too) http://php.net/manual/en/intro.imap.php
While emails are almost always stored on disk as files, accessing those files directly would not be advisable unless you were writing a new email server (I've contemplated that actually, but it is definitely NOT a trivial task) and your web server (e.g., PHP running in Apache) wouldn't have that level of access. The basic steps the right way are:
Add the email account, set the password, etc. Decide whether you want to access it as IMAP (emails stick around unless you delete and then empty the trash, which lets you archive on the server and/or access from other devices) or POP (typically "read the emails and delete from the server immediately").
Write PHP code to read the mail account. The specifics will vary slightly depending on the email server/hosting configuration. You may need to work at it a bit until you can successfully read a list of messages.
Once you are able to read the messages, you will likely need to handle multiple formats. In particular, messages may come in as plain text or as mime parts. There can be quite a bit of variation depending on the software used to write the original message, so to test you may want to send in messages using Outlook, Thunderbird and other programs.
You will also likely need to filter out HTML tags and other extraneous stuff from legitimate messages AND you will inevitably have to come up with a way to block spam. If you only accept messages from registered users then you can verify based on the From: address (at least as a start, because that can be spoofed too...) but if the support email address is "open" then filtering the junk can be quite a challenge.
This is a non-trivial project, but I have successfully set up a few systems of this type, though typically with controls to help block spam (e.g., required codes in the Subject line; limited "From" addresses).
Related
I need to handle some mail. I already have a script built that can parse through a mailbox and perform several actions like save attachments, move email to a folders and other administrative tasks. A few of the emails are identified as rogue during this process and need to be forwarded. The messages may or may not have one or more attachments and are dumped into their own folder labeled fwd.
I can create and send new email messages but am having trouble finding information on forwarding or replying to existing email. One solution would be to save the parts (body, subject, attachments) to a database and construct a new message with MIME::Lite but this seems inefficient at best.
I am handling the email with Net::IMAP::Simple::SSL and MIME::Parser.
Since the email is dumped into a temporary folder for holding I am not totally against using a PHP script to handle the messages, but prefer something in line with my current Perl handler to execute the task.
Looking for some helpful info to help complete this task.
You might want to look into CPAN at Mail::Box, a rich (and a bit complex) module handling mail messages, including primitives such as message->copy and message->reply.
For documentation and examples, author's website is at http://perl.overmeer.net/mailbox/
Our web-based PHP software currently sends out a newsletter to anywhere between 1-2000 recipients. Often the newsletter has a PDF attachment (15KB-5MB). The newsletter does not need to be customized to the individual recipients.
Question: Is it better to send one e-mail that has each recipient blind carbon copied (BCC) or to generate a unique e-mail message for each recipient?
Considerations:
- Which option puts less stress on the mail transfer agent?
- Which option is more efficient programmatically?
- Which option is less resource intensive?
- Are there any limitations to either option? (e.g. BCC having a maximum number)
I've tried Google and I just can't find anyone that has a definitive opinion based on empirical evidence. It's actually hard to find anyone that has an opinion at all.
THANKS: To everyone who contributed to answering this question. Greatly appreciate the feedback from people to ensure we're doing things properly!
Generate a single email per recipient. Use the To field instead of BCC to make it personal.
Advantages
The mail queue will accurately reflect what is happening.
You can distribute the load to multiple email servers.
You can personalize the "To" "Subject" "Body" etc.
You can use tracking URL's.
Mail servers often have a BCC limit per message. You will not hit a limit if you send a single message at a time.
BCC emails typically remain in the queue until all deliveries are complete. It is rare, but we have experienced (with the latest qmail) that sometimes a single recipient will respond with an error that confuses the mail server to send it again, fail, again, fail...until we remove it from the queue. This gets people very upset.
Disadvantages
PHP script has to work harder to generate the individual requests.
There are surely other advantages and disadvantages, but that is the list I follow.
UPDATE: Regarding the PDF attachment, I would recommend providing a download link unless it is crucial to include it with the email.
PDF attachments make an email look more suspicious to spam/virus scanners, because spam is known to try to exploit vulnerable versions of Acrobat. Those PDF attachments might make your newsletter more likely to end up in the recipient's Spam folder.
Large PDF's (1+mb) are not friendly to people checking their email with slow connections or constrained devices such as smartphones.
A link is much smaller than the attachment. You will save upwards to 13GB of bandwidth if you leave off that 5MB attachment!
It depends on MTA infrastructure at your site. If the box that is running your web app is set up to forward all e-mails to some e-mail hub at your ISP then BCC is definitely the advantage. Otherwise, it may save some bandwidth for you but not necessarily (it depends on the actual addresses you send to) Also, I would recommend you not to attach the pdf to the message but place it on the web server and include hyperlink in e-mail. As I got your message is a bulk message. I believe that many people do not read your messages even they opted in to receive.
Instead of attaching such a large file (which might also be rejected by some MTAs because of the size) upload it somewhere on a publicly accessible place (i.e. a web server) and send a simple link to all of the email recipients which they can use to view the PDF.
The good thing about this approach is that you save loads of bandwidth and even if you need different PDFs for every recipient you could still use it.
I've got a web application with approx. 30k users and growing fast. We want to enable some functionality over email, which means assigning each user a unique email address (for security reasons). Emails sent to these unique addresses should be forwarded to a PHP script. The PHP script needs access to all parts of the email - the headers, body and any attachments. I'm not sure what the right way is to set this up. We're currently using sendmail and would prefer to stick with that (this is linux, if it matters). I've been able to get an individual email forwarding to a script, but it is not scalable (in my mind) to create 30k aliases and then a new one each time a new user signs up. I'd be much happier with some type of catch-all / RegEx-based solution that just tells the PHP script the unique email address that was the recipient, which would let us look-up the user.
30k accounts does not give us any information about the traffic you expect to handle. But let's assume it is another question.
You could create catch all type of account, which would simply accept any email in domain you create your users' addresses in (basically *#domain). Then you could launch your processing script on each incoming mail (you usually would be able to fetch whole mail body from stdin stream (in PHP accessible via php://stdin), and process it as you like. But this is bad if you want to use PHP (or almost any scripting language), as each mail would need to spawn new PHP process, init all the extensions and so on. It all this takes time and resources, and one bigger spam flood and your box is almost toasted.
Alternatively, you could collect your mail in mailbox as any other mail and then have your script getting it from there (either directly, digging into maildir or via mail protocols like IMAP or POP). This looks better from administration standpoint and as long as your mail server can accept incoming mails, your whole system still works. You can even try to "partition" your system by setting some filter rules (on MTA level) and deliver incoming mails not to single mailbox but more. Then you could process these mails in parallel with less hassle (no need to bother about processing same mail with more than one instance etc). "Partition" schema could be quite anything, i.e. depending on your full address naming scheme, if letter based it'd be separate mailbox for a*#, b*# or [a-d]*#] (depends on traffic), does not really matter.
Alternatively, you could try to put bodies of incoming mails into database and then process as described above, without any need of POP/IMAP handling.
Whatever you choose I recommend NOT putting your processing script of any form directly in mail receiving process chain. It usually is not a problem to have 1 minute delay (so you can have any sort of daemon running and fetching mails for processing, cronjob etc), users can wait that much w/o complaining - it looks like instant processing is most likely not mandatory in your case as well. But if it is, then PHP might not be the best tool choice I am afraid.
EDIT: (to answer comment question and not hit comment size limit)
I am not familiar with Sendmail enough to give you precise configuration directives here (I am on customized qmail), but some general hints would apply I hope. First, I believe Sendmail supports virtual users (which means mail boxes for users not present in /etc/passwd). If so, then MTA usually should let you either query DB (to find out if the target email address is valid and where is that user's home dir to put mail into) with some built-in providers. Or (which is even better) invoke external program to do that. The latter is a winner here because there are no users really, so you could write small script or (again, better for performance) C app that would do the trick. Since all you need is to "separate" incoming mails into several physical mailboxes based on some simple predefined rules, couple of if()s + substr()s and you are done. Couple of lines, no DB needed, pretty quick. Or, even better, you could even try the dumbest roundrobin approach here as well - it is in fact irrelevant if mail hits accountA or accountB as long as 50% of mails ends in A and other 50% in B (or 25% if you go for 4 target accounts etc) and that's the only condition you should care I think. You can try to use procmail for that, but I think it is pointless and is not best for performance.
As for putting mails into database. It depends on MTA - by default nobody does so as it slower than putting mails into maildirs (as DB is usually other host), but there still are couple of solutions. Either look for 3rd party patches for Sendmail that do that or add another script/app that would "copy" mails from physical mailboxes into DB. You may say - but it would be slower. Sure, but again - I do not know if non-realtime processing really matters in your case (my blind guess - it does not), so why really bother. If you save couple of milliseconds here but loose minutes on further maintenance of the whole solution, then choice is quite obvious.
BTW: just in case - whenever I wrote mailbox I definitely meant container for user's mails, NOT mailbox file format. As for file format maildir (or something derrivative) is much, much better choice.
Do you really want to fork off a PHP for every incoming email? That could get expensive if you're dealing with decent mail volume. Why not have a custom callback in your mail server that processes the incoming mail and inserts into a database, and then have a worker script that continually polls for new messages to process?
The mail-to-db part should be much lighter weight than doing all the processing, and this gives you a free queue so that if you get a bunch of requests at nearly the same time, your system doesn't fall over trying to process them all at once.
did a similar implementation of this, though aon a smaller scale... should scale to meet your purposes though.
two seperate problems
(1) mail - catch-all as mentioned is a good solution, though you will have to deal with spam and malformed email addresses (which may be a good thing or a bad thing). If you own the server, simply aliasing the "ficticious" accounts and directing the mail server to drop them all in the same central mailbox will accomplish the task as you don't need seperate actual accounts, and the header information will remain intact. the specifics depend on your mail solution, but any mail server should be more than equipped to handle this task and the volume, and you should be able to script the alias creation... otherwise back to catch-all
(2) PHP parsing and processing - PHP's builtin pop3 functions can easily handle checking the (now) single email account and processing the messages; Some examples of functions I used to go through this are;
function pop3_login($host,$port,$user,$pass,$folder="INBOX",$ssl=false){
$ssl=($ssl==false)?"/novalidate-cert":"";
return (imap_open("{"."$host:$port/pop3$ssl"."}$folder",$user,$pass,OP_SILENT));
}
function pop3_stat($connection){
$check = imap_mailboxmsginfo($connection); # changed from $imap
return ((array)$check);
}
function pop3_retr($connection,$message,$section='1'){
return(quoted_printable_decode(imap_fetchbody($connection,$message,$section)));
}
function pop3_dele($connection,$message){
return(imap_delete($connection,$message));
}
function pop3_close($connection){
return(imap_close($connection,CL_EXPUNGE));
}
all very generic, and there are probably classes out there if you don't need the control. Bottom line, PHP has all you need to quickly parse and process mail messages.
What would be the best way to manage the emails bounces/errors if you are building a web-mass-mailing software ? I plan to use PHP for that.
Before someone start screaming, yes, the lists will be from valid customers who have opt-in.
I know there are desktop software or third party website that can manage such things. I would like to display it with a design similar to the current CMS's visual and be accessible in there.
If the only solution is to connect to a mail server and read the bouncing back messages, then I will head that way. Then, how would you parse the data to flag that email as "invalid"?
Thank you
One possible thing you could do aside from having a PHP script read mail from a pop/imap server would be to pipe incoming mail for a certain address to a php script. See Google
You would then read the entire contents of the message in by doing something like $email = file_get_contents('php://stdin'); I've installed the php extension mailparse to assist in parsing RFC emails, but there are other options available. You don't even necessarily have to use anything to parse the message.
Once you have the message, there are a number of indicators you can use to try to flag a message as a bounch. First, see the Wikipedia article about Non Delivery Reports, specifically Format and RFC 6522 - The Multipart/Report Media Type for the Reporting of Mail System Administrative Messages. You can also check for common headers in the message such as X-Failed-Recipients or Diagnostic-Code.
Once you've determined a message as a bounce in the PHP mail processor, you can take appropriate action and set a flag in the database related to that email. Mind you, some errors may not mean the address is no good. For example, if a mail server is down for a few days, your MTA may give up, but it doesn't mean the address is no good. Also a user's mailbox could be full.
It wouldn't be a bad idea to log a copy of the bounced message so it could be checked by a person if necessary to diagnose an issue or reverse the flagging or a particular email address.
Here are some additional references:
http://forums.theplanet.com/lofiversion/index.php/t89873.html (note Improvement possibility 2)
https://stackoverflow.com/questions/5700034/how-do-i-process-a-bounce-email-to-find-the-error-code
Bounce Email handling with PHP?
Hope that helps.
we build newsletter module,
and send email to members.
The environment is LAMP.
Are there any way to know whether member open the mail ?
i hear about put image if 'php' source ,
what is the best way?
Ultimately there is obviously no fool-proof way to get notifications, because there is no guaranteed way of getting the email client to respond back in some fashion; the email client can be set up to just interpret the incoming email as ASCII text and nothing more, and there is nothing you can do about that.
However; in most cases if you are emailing to users that are expecting an email, odds are that HTML rendering and inline images are probably turned on for your source address, so using an inline IMG tag and monitoring access to the referenced file (obviously using some per-user unique ID in the reference) should give you the best you are going to get.
Regardless you'll never be able to make the assumption that if you do not get a notification back that that means the user has not seen the email.
There's no foolproof way to do it since you're not the one in control of the email client. Many people take their privacy seriously enough to prevent read-receipts, web beacons and all the other tricks which can be used to detect the reading (people can turn off read receipts, block images, prevent unsolicited outgoing connections and so on).
This is my opinion of course but I believe you're approaching the problem the wrong way. Instead of trying to force the user to let you know if they've read the email, just make it worth their while. It's obviously of some benefit to you to have this information (otherwise why do it?) so you share that benefit around and make sure it's the user's decision.
That way, you turn the relationship from a battleground into a partnership (win/win).
Yes, there is a standard mechanism (RFC 3798) called read receipts. It is implemented by all modern mail clients, and the user can choose to send (or not) the reciept as they choose.
There are also various non-standard subterfuges for doing this without the user's consent, which I won't detail.
EDIT:
It should be like the below (using built-in PHP mail function):
mail("foo#foo.com", "Let me know if you get this", "Important message", "Disposition-Notification-To: sender#sender.com\r\n");
A common way to check if an email has been read is a web beacon, which is usually a small 1x1px invisible image that is loaded from your server, which can track when the image has been loaded and therefore the email has been read.
This is not guaranteed to work, however, since many email clients block images in their emails or your readers could be using text-only email clients.
Each email has a uniquely named image in it corresponding to the users account (or db row), when that image is loaded or accessed, you can see which user has opened the email. This relies on the user receiving HTML emails though.