I have developed a CRM that is used with a marketing website similar to manta.com. The CRM has a built-in email client. Basically it matches names and emails to addresses on file, and if the email is from a customer it loads their account information next to the email.
The entire system works, but is in some cases slow. Because each "region" of the country is managed by a different company, the software connects to several different mail servers, all of which behave differently.
In some cases (mostly with godaddy servers) it takes a very long time to send an email. It also takes a long time to load the inbox if they have a lot of messages (500+), even if it is paged.
I am using pear Mail::Factory smtp to send, and the built in php imap functions to receive.
I am thinking for outgoing messages I will just store the emails into a db and then have a cron script send it out, then they can send their email and move on right away.
Does anyone have any ideas/concepts for handling large inboxes via IMAP and php? I will have no control over the mail servers themselves, but have full control over the server that the CRM is on.
I am not looking for code, just concepts if you have them.
It is Linux (ubuntu), apache, mysql, php. I can use Zend if needed.
It looks like a combination of caching the emails and using the imap_fetch_overview function which omits the body of the email. This seems to eliminate any issues with the lag. I can also use it to page the results so I am only loading a small number (25-50) at a time. Thanks for the help.
Related
I have special e-mail accounts on my web server (#mydomain.com). I want to run a php script automatically when one of these accounts get a new e-mail. For example: when info#mydomain.com address receives a mail, I want to run "receivedMail.php" file and read this new e-mail. I don't know where I will start or how can I do this.
You're wanting to pipe email received for a specific email address to a PHP script.
If your webhost has cPanel, this makes it very easy to setup. See:
http://kb.siteground.com/how_to_pipe_an_email_to_a_php_script/
Alternatively, if you don't have cPanel but have Exim mailserver, this will guide you on how to do this:
http://www.phpshare.org/articles/Piping-Incoming-Mail-with-PHP
Hope this helps!
You have to create a cron job calling your php file every 1 minute for exemple.
The script have to the derver and read the email.
This has nothing to do with web servers directly, Since web servers do not receive emails. So no .htaccess style files come into play. The email is received by a mail server, so that is where you have to get active. Two strategies are possible:
you use the possibilities to trigger an action as offered by your
mail server (typically the smtp server you operate). That obviously
depends on which mail server you operate, different software offers
different features.
you poll those email accounts on a regular base, using a standard protocol like POP3 or IMAP4. You can do this using any suitable client. When a new message is found whilst polling the account, then you trigger the action you wish.
Option 2. is probably easier to start with. So give it a try: create a php file which polls your email server. You can use the imap php extion for this, it supports all important email protocols. The extensions allows to easily detect and retrieve new messages. For each such message you can implement whatever action you wish. All that is left is to run this script on a regular base, say every 3 minutes, which is typically done using a cron system. There are many examples for this out there on google...
Have fun!
I am about to start a project that requires an email to be somehow ran through a PHP script. I have full control of the server that the email will be sent to and wondered if people could give me some thoughts or pointers as to the most elegant way parsing it with PHP. I am not editing the email and then forwarding it on.
The server is CentOS with Exim email.
Thanks in advance.
You implement a client for the mail box (php imap modules can work with pop3, imap4 and local mailboxes). You poll the mailbox for new arrived messages, retrieve new ones and parse it. All using the php imap module.
One of the best examples of this comes with Wordpress... there is a file wp-mail.php which is set up as a cron task to retrieve and parse emails... I have hacked it up several times to do such things!
To make life easier for you, rather that stress yourself with parsing emails (if you don't mind the cost) you could decide to use Postmark
I've been using their services for quite some time and i love them. They have an Incoming email API service now. Enough talk, simply check it out because I believe it will help with what you're trying to do.
There are three main approaches to this:
Run a cron task to poll an IMAP/Pop3 server every x minutes
Make exim run a script whenever it receives an email
Use a third party service to receive the email and send it on to your site.
I wrote a Blog Post detailing the options, although it's for Rails the main concept applies to any language including PHP.
I have a website that receives incoming emails via cloudmailin, adds the content to the database, and then sends a confirmation email back to the person who sent the email. Currently I'm using a php mail() function to send the reply, but it seems to take about 1.5 min to actually get to the user's inbox. Is there a way to make this more instantaneous? I'm currenly hosting the application on FatCow, and I'm sure it has to do with their mail server, but I don't really know much about how that works and am wondering if I have to change hosts to accomplish faster delivery times or if I can do anything about it without switching.
mail() is a black box. You're telling PHP to try sending a mail however the host has configured PHP to do so.
It's very likely that it's just calling sendmail in your case. It's also very likely that the mail queue on that machine isn't the fastest in the world. Shared hosting machines are often overloaded.
The very first thing you should do is ask your host about the mail delay. Perhaps something is wrong, and they can fix it. Or they might just tell you that a minute and a half is not a long time to wait.
There are lots of other options, mostly in the form of companies that provide an SMTP service to you. I can't recommend one, but I can recommend that you pop over to your favorite search engine and look for "smtp service." I've recently starting working with Amazon Simple Email Service. It's supposed to be fast and well-maintained, and it's certainly inexpensive.
All of those options are likely going to require some configuration changes on your end. For example, you'll want to set up custom DNS records (for DKIM and SPF) to ensure that mail from a third party provider isn't automatically flagged as spam.
Using SMTP with PHP is dead easy. There are plenty of mail generating options out there. My personal favorite is SwiftMailer. It even has a transport option for Amazon SES.
The final option, of course, is grabbing your own virtual private server or dedicated server, which will let you configure outgoing mail as you desire, at the cost of needing to know how to maintain that server.
Edit: Obligatory link to relevant Coding Horror post.
Rich sites such as facebook send you an email for every event that takes place, notification, friend request and so on, I would like to know how costly is it for the web server to perform this action? that also 100s of time a day for active and popular users.
Usually the site do not send emails itself. It just redirects email to server MTA, which then saves it in it's queue and sends when it thinks best.
To answer your question we need the definition of "costly". To send an email you typically need to execute some DNS queries, connect to remote server, exchange data with it. Is it costly in your opinion?
It's a good idea group destinatarys by BCC header to reduce the email server calls, if you is using an external SMTP that it's the slower phase.
PHP only uses one thread that means that next email will be send once the current mail was sent to the SMTP server. In Java for example you can send emails in a pool of threads at same time, and course, it's faster.
I have a client that I'm wanting to move to a virtual private server. I'm trying to keep his server as simple as possible, not even having an email server on there (or at least that is the plan). The reason being I've read that email servers are complex to setup and take up alot of resources. I'm not sure what to do here.
He has a newsletter that he sends out to almost 75,000 people. Currently on his shared hosting there is a cron job that calls a script that sends out a 100 emails at a time (to keep within the limits of the host). I'm curious to know if the same thing is possible to do but with using his his Google Apps email account? Effectively logging in and sending the email from there? I can't seem to contact Google about this and it doesn't seem to be in their help section.
Is it possible? Would it be wrong to do? My client has looked at many of the mail out things out there (MailChimp, etc) but they are very expensive for what they are. Especially as what he has now does work. Thing is, I'm implementing the whole thing new on a VPS and so am looking at options.
UPDATE: It seems that the Google option just isn't an option (I'm not surprised by this at all). May I ask which minimal email server could be set up on Ubuntu 10.04 LTS which would allow me to continue the cron approach I had before on the shared server that will allow sending the emails out? (I will still continue to use MX records to Google for the actual emails the client will be using)
UPDATE: A friend of mine went to a mini Django convension in London the other week and there was guy who runs Mixcloud. He said he uses Google App Engine to send emails, so it's totally scalable. There is Google App Engine code up on Github here: http://github.com/tow/appengine-mailer and this seems to make sending email very easy from Django, from a command-line (so could use with PHP) etc. Looks like a very good option and cheap too! What do you guys reckon?
UPDATE: I've started a new question to make it more refined and clear to hopefully help others who maybe having this problem: Effectively using Google App Engine to send lots of emails using PHP?
There is a limit of how many emails can be sent per day through the google apps accounts. The limit is waaay below 75k I think it is 2k per mailbox if he has the premier account.
Two options here. Sign up with a company that does this for him: Constant Contact, Mail Chimp, ...etc. Or install an SMTP server and have it go out how it did before. The performance hit the server will depend on what mail server you choose and how much load is being placed on it.
UPDATE: I primarily work with windows servers so my experience is there (setting up secure SMTP in windows is pretty easy). Googled around and found this page on Ubuntu's site. It has a bunch of options for doing this: https://help.ubuntu.com/community/MailServer . From that page you should only need the Mail Transfer Agent and possibly the Mailing Lists components. Also make sure to set up your SPF records correctly and identify this server's IP as an outgoing only server. It can also help to set up the server with an MX record with a priority higher than your other MX records. If google goes down incomming mail will try to go to your VPS but as long as you don't set it up to receive mail, just send it, it will bounce the request. Which would happen anyway if your google mail servers go down so nothing lost.
Lastly you can try to register your server's IP with yahoo as a bulk mailer.
UPDATE2: Also make sure to have the "FROM" address in the newsletter be a valid email address from a valid domain. This will help immensely with deliverability.
UPDATE3: Can you get a static IP with a APP Engine Account? If you can't get an IP (preferrably dedicated) assigned to you then your deliverability will be horrible and you will get blacklisted. IF you can get a static IP assigned to your App Engine account you can set it up in your SPF records and that solution should work.
I did this a while ago for a commercial website. To describe it as an utter ball-ache would be an understatement. The amount of man-hours we sent working with our mails, enabling the various headers to avoid it being flagged as spam, to ensure that it looked to ISPs as valid mail etc was far greater than anyone envisaged at the start of the project. Then there are issues with mail throttling and throughput - at one point we were sending a weekly mailout that took 8 days to go through the entire mailing list.....with obvious results.
Anyways, upshot is, outsource it - there are many companies out there who have already solved this problem for you - granted you have to pay for them, but its cheaper in the long run.
You should use a third party site to fulfill this clients requirements. There are lots of technical, legal, and spam issues that need to be overcome to ensure that your client is not blacklisted or fined.
A company like Industry Mailout, MailChimp, or Campaign Monitor will send your mail for you, and is often a lot cheaper than trying to do it all yourself. :D
Plus these companies offer useful analytics to make your campaign's better.