I assume I will need to point MX records at my server (LAMP), -- what processes the incoming e-mail message?
Are there any existing PHP libraries to do this?
You don't want to use PHP as a mail server. You've got two options:
Set up a classic email server (postfix, sendmail, exim, etc) that delivers new messages to a local mailbox. Use IMAP or POP to access that mailbox from PHP, and pull messages out of it. Alternatively, this same method may be used with (virtually) any remote mail service as well, thus relieving you of the duty of administering a mail server. (Which you'll likely find to be not worth it for the sake of one mailbox.) This method would usually be run via cron every few minutes, so you're not going to get "instant" activation if that's a requirement.
Set up a classic email server (postfix, sendmail, exim, etc) and use procmail or similar to intercept messages at delivery time, and pipe them to a PHP script. This method will fire the script the instant the email arrives, so you'll have no lag time like in #1. However, it's more difficult to configure (especially if you haven't maintained a mail server before) and won't work with most external hosted email services.
Use a pipe alias to receive the emails.
I would recommend you to do processing in Perl (python is also ok, but Perl has very similar syntax to PHP), which is much more suitable for the task. You can also find a lot of libraries through CPAN there.
http://search.cpan.org/~rjbs/Email-Simple-2.100/lib/Email/Simple.pm
Related
I have a system that sends lots of emails reports ( about 500+ emails per day). I am not a spammer :) It's not really big amount of e-mails, but they are send in a loop and I often get this error "PHP Warning: mail(): Could not execute mail delivery program". I know there is PEAR::Mail_Queue package for this issue. But can you please let me know if that package is really useful thing, or do I need to look for something else. Thanks a lot
You can use PEAR's Mail_Queue package to send mails through an SMTP server directly - also it will work in the background and so won't delay or increase your script execution time.
Another advantage of using the Mail_Queue package is that you can retrieve the esmtp id of each mail sent for logging purposes - I don't know if this is possible with the Zeta Mail component or any other one.
I've developed a number of public-facing, and also intranet only, solutions that use this component and haven't had any major problems with it.
"PHP Warning: mail(): Could not execute mail delivery program"
...means exactly what it says on the tin. PHP will just hand the email off to an external program on a Linux/Unix/POSIX system (as defined by sendmail_path in php.ini). And in some cases that is returning an error. It's not a fault within the PHP code.
Unless you've got a really badly configured MTA, then the problem will not be resolved by using an SMTP connection instead.
You need to look at the logs from your MTA to see why its failing to send the mail - or wrap the mail executable in a logging script.
You gotta use sleep() between mails, that solves this.
An alternative could also be to send mails through an SMTP server directly. This saves you the round trip of PHP calling sendmail (or whatever MTA is used) "through the shell" in order to deliver the mail. For example the Zeta Mail component allows you to send mails directly through an SMTP server without needing a special extension for this.
Want to improve this post? Provide detailed answers to this question, including citations and an explanation of why your answer is correct. Answers without enough detail may be edited or deleted.
Why does the PHP mail() function require a mail program like sendmail/postfix/etc. for sending emails?
I ask because sending email is a client action and not something that needs to run a server.
What is so complicated about creating a native PHP mail function without the need to install a sendmail/postfix/etc. program that has excessive functionality as a server that can get requests and not just send email as a client?
Most likely: sendmail existed before PHP's mail(), so in the true spirit of *nix:
Why re-create functionality when it already exists on the CLI?
The reason you want to hand the mail off to something else is that Mail is a complicated beast that is best deferred to something better designed to handle that.
A simple case is when you mail something to a server that is offline. It's "OK" for a mail server to go offline, there's built in logic to retry message sends in the operations of the mail servers. But if you simply open up an SMTP socket and start barking protocol, and it fails, then you've lost that queueing capability that you get "for free" with the mail servers.
Mail is just one of those things best delegated to those ancient systems that have all of the lore and details coded in to them over years of painful interoperability testing and implementation.
PHP doesn't send mail itself, but delegates to another program to send mail for it.
You can get around this restriction by using the PEAR Mail package instead, which has SMTP support.
The reason is because the mail() function is not an MTA, only a "encoder" for the actual MTA.
Why not? I believe it would be unpractical to implement a decent and secure MTA setup with PHP alone.
EDIT: What exactly does an MTA do?
When a mail server forwards (routes) email to another server, it acts like a client and uses SMTP to forward the email. The client part is called an Mail Delivery Agent (MDA) and is often a distinct piece of software.
This part of the sendmail tutorial is helpful.
It's called "sendmail" for a reason. Networks' used it for decades for inter-communication. Whether you know it or not, that IS how mail gets sent. -- By a program confined to an RFC protocol and designed to format, package and deliver data to ... other servers. It can run as a daemon sitting on a port or from an instance invoked at the cmdline. It's a simple tool, and an efficient one.
-Gama Xul
(*nix guy)
Trying to send mail with something other than a mail-sending-program (like a web-browser client or a PHP language mod) would be like trying to cut something with a gun. There's a lot more going on there and too many conflicting conditions could hang it up. It just wasn't designed for it and wouldn't function efficiently.
I'm thinking about how to handle sending large amounts of email from my web applications, and whether there are any best practices for doing so. StackOverflow is already labeling this as 'subjective', which it may be to an extent, but I need to know the most successful way to implement this system, or whether any standardized approach exists.
In my webapp, there are users who are heads of groups of 1 to 10,000 users. This user must be able to email send a message to all of these users through my system. Therefore, my system is responsible for sending up to 10,000 emails to individual users for each group head.
As far as I can tell, there is no rate limit in GMail for sending messages to individuals (although there is a 500 recipient max).
Right now, my current setup is:
When a message is sent through the system, it enters an email queue.
A cron script grabs messages from the queue every few minutes, and sends out those emails.
All email is taking place through GMail's SMTP server.
The actual application doing the mailing is PHPMailer.
This setup, as the user base grows, will probably not suffice. The questions I have are:
Should I be using a local SMTP server instead?
Should I use a mail binary on the local machine instead? I this case, I could probably skip the queue altogether?
Is there an accepted way to do this?
Thanks!
Google App Engine
I would write this in Google app engine (python) because:
It scales well.
It has a good email api.
It has a taskqueue with a good api to access it.
Because python is a real nice language.
It is (relatively) cheap.
PHP
If I would implement it in PHP I would
Find yourself a good SMTP server which allows you to sent this volume of mails because Gmail will not allow you to sent this kind of volume. I am for sure that this will cost you some money.
Find yourself a decent PHP email library to sent messages with like for example PHPMailer like you said.
Use a message queue like for example beanstalkd to put email messages in queue and sent email asynchronously. First because with this the user will have snappier page load. Second with a message queue like beanstalkd you can regulate speed of sending better which will prevent from overloading your pc with work. You will need to have ssh access to the server to compile(install) beanstalkd. You can find beanstalkd at beanstalkd
You would also need ssh access to run a PHP script in the background which will process the message queue. You can find a beanstalkd-client at php beanstalkd-client
from php/apache/webpage
This is the page from which you will sent messages out to user. From this page you will sent message to beanstalkd by coding something in the lines of this:
// register Pheanstalk class loader
require_once('pheanstalk_init.php');
$pheanstalk = new Pheanstalk('127.0.0.1');
$message = ""; // This would contain your message
$pheanstalk->put(json_encode($message);
You have to put messages in message queue using put command
From long running PHP script in background:
The code would look something like this:
// register Pheanstalk class loader
require_once('pheanstalk_init.php');
$pheanstalk = new Pheanstalk('127.0.0.1');
while(true) {
$job = $pheanstalk->reserve();
$email = json_decode($job->getData());
// Sent email using PHP mailer.
$pheanstalk->delete($job);
}
Like I am saying it is possible with both PHP and Google app engine but I would go for app engine because it is easier to implement.
With an email count as "high" as 10.000 a day, I wouldn't rely on GMail (or any other) SMTP. Not that they can't handle it, obviously they can handle A LOT more. But they possibly don't want to.
Having a local SMTP server is IMO the way to go :
It's pretty easy to set up (just DON'T let people use it without a strong authent scheme)
Most modern MTA handle sending queues very well
You won't have to deal with GMail (or others) deciding to block your account someday for quota reasons
Gmail and Google Apps limits you to around 500 emails a day. I'm not sure how that combines with the 500 recipient max, but if you want to send 10 000 emails you'll probably want to find another mail server. I personally use a local server or the ISP or datacenter's SMTP.
If you are sending that many emails, I would recommend using the queue so the user's isn't sitting there waiting for the email to be sent.
Be very careful that your domain doesn't get blacklisted as a spam domain. If it does, you can expect most of your emails to be blocked, support, sales, etc. Which could in turn be very expensive.
You may instead want to use a service like AWeber. Not only are they setup to handle these amounts of emails, but they can probably give you more metrics than you can implement on your own.
I'm not sure if it's publishe anywhere, but from experience I can tell you that Gmail will put a fifteen minute or so freeze on your account if you start sending hundreds of messages at a time. This happened to me last week. I think you should host your own SMTP server. Using the mail() function often will put your mail in someone's spam folder.
Just install Postfix on the local machine, or a machine on the same LAN for maximum access speeds. Make sure it is well secured from the outside, and quickly accessible from the inside.
Then code your PHP script to directly inject the emails to the Postfix queue. That shall dramatically increase the processing speed of mail delivery.
Is there a way to configure a server or a script to execute a php script when an email is received?
In theory this could be extended to other protocols, such as XMPP or SMS, etc.
What I have in mind is a user could send a message to checking-in#example.org and this would trigger a script which would then do whatever needed to be done, either irrelevant to the message (an automated message that gets sent whenever some other even occurs, like a server having issues) or related to the message (like it could store the subject in a database that other users could view as an RSS feed).
I know that most list-serve software have a means of sending commands (like unsubscribe), but I'm not sure how complicated the process is and if it is feasible to have something like this on a server-script level.
Would this need to occur at the IMAP/SMTP level, or could it be done closer to the script or HTTP server?
Just to give some context, one of the things I'm considering is a message-based clock in server for one of my work sites. Users could IM/email/text that they are at their scheduled location and this could trigger a script that would update a DB, rather then send the managers a direct message they need to log. This would just be one option (a web-based method is already in the works), but we want to make it as platform/protocol independent as possible. But if we can achieve that, we can look at implementing it for many other day-to-day needs.
Another way of asking might be: is there a way to have "discovery" of users from a server-script app or does something need to be doing a constant check to relevant sources to see such changes?
If you control your own machine you can configure it to pipe the e-mail through a filter. Simplest would be to setup a new account and setup a .forward or alias
The following examples are for Sendmail, but (all?) Unix e-mail programs offer a similar service.
Setting up an alias (option 1)
Look in the directory /etc on your server for your alias file. Add the line:
script: "|/path/toyourscript/pipe.php"
Using a .forward file (option 2)
Create a .forward file in your main home directory.
"|/path/toyourscript/pipe.php"
or:
myemail#example.com,"|/path/toyourscript/pipe.php"
If you are on shared hosting then most hosting providers also provide the possiblity to "pipe" e-mails received to a particular account through a script instead of storing them in a mailbox. Check the CPanel setup.
Use a cron job to check for emails through a pop3 interface. If an "interesting" mail is found, run your php script.
Note that the cron script can be in PHP too.
As jldupont said, it is easy to do ith php itself, simple reading the smtp continuosly with a cronjob.
Otherwise, you could use a daemon, in python for example.
Both ways allow you to do an important thing: check if the sender of the email is a your user (you have the users in a db, right?), or a spambot (nomatter what kind of anti-spam you use, soon or later, checking-in#example.org will be full of spam)
We've built a web service that needs to check constantly for email messages. Basically, an user sends us an email, and the server should make actions based on that email. We could use a crontab PHP script that checks for new messages every minute, with POP. But that's kind of offensive to the popserver and not very efficient (1min is too long).
But, I've read about PUSH email using IMAP around mobile devices. In my case is not a mobile device but a webserver.
Can I push an email to my webserver and have it execute a PHP script? We're using GMail as POP/SMTP/IMAP server.
EDIT 1 from the answers, we figured out:
there must be a 24/7 running process (daemon) on my webserver checking for emails
this daemon may communicate with Gmail using: i) POP with NOOP coomand or ii) IMAP with IDLE command
What's the best? POP or IMAP? Google seems to invite more the use of IMAP.
I don't want to overuse gmail (what's their 'fair use' for checking email? every 10secs?
If you want to use postfix as your MTA you can alias an email address to any executable like this. It really isn't that difficult to setup on a *nix box...
I've no experience with IMAP, but had to do the same thing. If I were you, I'd install Postfix and use the pipe command (basically lets you run an arbitrary script) to invoke a small http client that parses the email and sends it to your server.
You can replace postfix with whatever MTA you'd like, of course. The point is: don't implement your own email server. Use an existing one and use its hooks to send the mail off to where ever it is you want, however it is you want.
email_scheduler.c
int main()
{
while(1){
sleep(5); // every 5 secs the email_parser.php would get executed.
system("/usr/bin/php -q /var/www/html/email_parser.php");
}
}
email_parser.php :
/* Use any email message parser to
parse the email messages like MIME
message parser based on your purpose
: Refer :
http://www.phpclasses.org/browse/package/3169.html
*/
Also check relevant answers here for more info
There's an old (but great) article on Evolt about piping emails to a php script, which therefore gets the full content of the email and can act on it.
http://www.evolt.org/article/Incoming_Mail_and_PHP/18/27914/index.html