I'm looking for a way to programmatically insert messages into a specific email account - created and maintained with CPanel.
I have a website that provides a webmail interface for a video game's inner messsaging (using its API) and I would like to take this service one step further and make the messages available on a POP3 server.
I have looked for a solution multiple ways so far:
Insert mails using CPanel's API: I could not find an api call for this, only for creating new accounts (with https://documentation.cpanel.net/display/SDK/UAPI+Functions+-+Email%3A%3Aadd_pop) This is the preferred method for me as I'd rather leave alone the filesystem.
Manually insert the mails to the filesystem: I found out, that Cpanel stores email data in root/mail/domain/user/ but I have no knowledge of the exact file structure. I recognize the maildirsize file and the other directories containing the mails, but I have no knowledge of the maildirsize file's structure (seems to contain 2 integers divided by a space per line) and also the mail file's filenames are as well not obvious. This is an example of a filename:
1422094110.H186037P182351.hosting-server-domain.com,S=15645
I'd rather use this method as a backup method, because there is very little (or I didn't find any useful) documentation available.
Simply E-mail all messages with custom headers: This would be far the easiest method, however the hosting provider has a very strict anti-spam policy and the outgoing message number is limited.
Implement an own POP3 server: I played around with #cleong 's PHP implementation (https://stackoverflow.com/a/11973533/1030464) and while it worked on localhost, I am not sure it'd be a joyride on the live page. I found a Perl implementation as well which might be worth a try, but I have never ever worked with Perl and I'd rather look at another solution before learning how to implement and integrate that module.
Thank you for reading all this,
Bálint
It sounds like the email is being stored using the Maildir format. This makes inserting mails fairly straightforward. For full details see the Maildir specification, but in summary:
The hardest part: create a guaranteed-unique name for the mail, using the techniques suggested on that page (e.g. local hostname plus high-resolution timestamp plus high-quality random number).
Create a file with this name under .../Maildir/tmp/
Rename the file to be under .../Maildir/new. This renaming step ensures that only fully-formed mails are seen by the mail software - i.e. no other process will attempt to look at the file when you're half way through writing it.
That's it!
Related
I tried to search for this problem, but there are alway things like "i don't recomment to use E-Mail for that", and then there are other solutions. So, I'm here and ask u:
I need a way to upload/generate files(.html/.php) on my Website via E-Mail. And it realy need to be E-Mail, because E-Mail will be the only possible way for me to communicate(google: winlink) with the internet in the next month/year.
But I have no idea how i could do this. I'm looking here for some ideas/ways/tipps for what i should look, or better a good and ready solution.
It should be possible to send a E-Mail with a text, and this needs to be placed in a .html or .php file on my webspace. And good would be, if I'm able to give the name of the file ... and then replace or generate it.
I hope there is a solution in php ... but if sql is necessary it ok too(I'm going to learn it next, too).
MfG Torben
If you have the opportunity to setup a mail server, you can handle stuff like this with some effort with sieve (depending on subject, content or from address) and dovecot. Here's a tutorial in german for setting up a mail server: Mail-Server
Once setup, you can write a program which is called by sieve and is running the code required for what you desire. Please note carefully that this will never be
a) save
b) failure prove
If you can live with those side effects, feel free to go with it!
SIDENOTE:
Better solution might be: Wherever you have access to the internet, you can access the internet and your site/server to.
For the WinLink and Sailmail community, SailBlog offers a blog web space, which is editable via E-Mail. This feature is integrated to the GUI of Airmail (WinLink client for PACTOR, Telnet, Packet and also WINMOR using BPQ32 TCP/IP virtual serial port emulation). Sadly, the Post By Email feature requires an Premium Basic account.
But what about setting up a new E-Mail account, looking for a Perl or PHP library that can handle E-Mail and MIME attachments and running a script on your server, which checks for new E-Mails (new Upload files, directory listing request, Upload files etc.)
I need to create an email account in PhP. I found a ton of code samples online that do this, however I have a few problems:
I'm on a shared server host with the default php.ini, and my host won't change it
I need to create it via a URL, not with forms (something like: mywebsite.com/email.php?username=user)
The password and quota created needs to be the same every time, so the only thing after the ? I need is the username
I have 0 experience with PhP so I can't just edit one I find online, so sorry if this is easy. If someone could point me in the direction of a script someone made or something that'd be great! Thanks.
From your reply, it turns out that you are using cPanel. This is quite helpful, as cPanel provides its own XML API for some administrative tasks, including creating email accounts.
You can find a ready made script that uses the cPanel XML API and does what you require in this forum post:
http://forums.glowhost.com/programming-talk/using-cpanel-api-creating-email-account-forwarders-your-own-php-scripts-2798.html
This SO question contains information on how to use the cPanel XML API yourself.
If for some reason you can't get the XML API to work, there's another script that makes regular HTTP calls to cPanel itself over here:
http://www.zubrag.com/scripts/cpanel-create-email-account.php
I have set up an email id my PHP web application. Users will send emails to this id.
I want to process these emails in the application. Ho do I go about doing this?
Thanks in advance.
I recently worked on a project that required parsing of email from gmail and updating database with certain values based on the contents of the email. I used the ezcMail (now) Zeta Components library to connect to the mail server and parse the emails.
The strategy I adopted was to filter all interesting incoming mail with a label "unprocessed". Run the PHP script via a crontab every 15 minutes. The script would connect to the mail server and open the IMAP unprocessed folder and parse each email. After inserting the interesting values into the database, the script moves the files to another IMAP folder "Proccessed".
I also found IMAP to be better than POP for this sort of processing.
Recently I wanted to be able to receive emails immediately in something I was making so I did some research (I came looking on this question here too actually) and I ended up finding Google App Engine to be pretty helpful. It has an api you can use to receive and process emails sent to ____#yourapp.appspotmail.com. I know that it doesn't really seem helpful since you probably don't want your app on App Engine and you want to receive emails at yourdomain.tld, but with a little setup you can get what you want.
My basic setup is like this:
User sends email to user_id#mydomain.tld (an email address that doesn't actually exist)
mydomain.tld has a catchall email address that forwards to inbox#GAEapp.appspotmail.com
GAEapp (a tiny app on app engine) receives the email, processes it out, and sends a post request with relevant stuff to mydomain.tld
So basically you can make a little GAE app that works like a go between to grab the emails. Even with the redirect it'll work out ok, the email will be fine.
Also I decided to learn me some django and I made a free app called Emailization that will basically do that for you. You create a recipient like ___#emailization.com and give a URL to POST to. Anything sent to that address gets POSTed to you URL. You can make a catchall on your domain that forwards to that emailization recipient and you'll get email through the catchall too!
or you can see a small GAE app I made that you can setup yourself that does the same thing.
Hope that helps somebody!
Use procmail if it is installed on your system. Put these lines in a .procmailrc file in the home directory of the user who receives the e-mail.
:0
| /path/to/your/script.php
Or you can also use a .forward file containing
"|/path/to/your/script.php"
Procmail has the advantage that it allows you to deal with more complicated filtering if your application ever requires it.
Your script.php file will read the headers and body of the e-mail from stdin.
Check out fMailbox. It does not require any non-standard extensions (such as imap) and has been tested with various servers, attachments, multipart messages, SSL, and more.
I suggest using Zend_Mail component of Zend Framework.
There is a great library: Try this: http://code.google.com/p/php-imap
You need to implement an email client in Php. This is probably going to be a POP client.
This code would query the POP server containing your email, download it, and then you could parse it as needed.
A quick google search of "POP client php" has revealed a vast array of different options. Its hard to tell if there's really "The One True PHP POP Library", otherwise I'd include it here. If you are using a preexisting framework, you may wish to check to see its level of POP support, otherwise check the google results above and take your pick. Or it may just be easiest (and most educational :) ) to roll your own.
There are a number of hosted solutions that will accept email for your domain and then post it a script on your website. Most of these will handle the parsing of the messages for you (separating the attachments, "to" "from" and other addresses, etc).
You just create a script that receives a FORM POST and does whatever you need with it.
Mailgun
CloudMailin
You can also look at Mandrill (by MailChimp), SendGrid, and PostMarkApp.
Hosted solutions as Travis Austin suggested work well.
If you are looking for a self-hosted one, you can have a look at the Mailin module allows you to receive emails, parse them and post them to a webhook of your choice.It also checks the dkim and spf, computes a spamassassin score and determines the message language.
I don't know if it will suit your needs since it is written in node.js, but the more options you have, the better. (Disclaimer: I am the maintainer of Mailin)
There is a great tutorial for this here:
http://www.evolt.org/incoming_mail_and_php
which covers how to have the emails forwarded directly to your script, which your script reads via stdin (fopen, fread, etc.) The tutorial code even does basic parsing of the header/body for you.
If you want to avoid reaching out over POP or IMAP to another server to pull-down the email, you can add a 'hook' into the email receive process on some SMTP server you set up (possibly the same php server). Then just have the destination email handled by this server.
Here is an example with postfix, but similar things are possible with sendmail as well.
http://www.adkap.com/autoresponder.html
I've seen on site like flickr or brightkite, a personnal email is provided to the users.
If the user mail somethin to this adresse, the content is posted on his public profile.
How can I do that on a web application ?
There are two ways to do this, as I see it:
First, you can use an existing SMTP server/email box system and, on an interval, pull the messages from that mail box using POP3 or IMAP to insert stuff into your database/system.
Alternatively, you can write an implementation of SMTP that will accept email messages coming in and perform your custom logic to put data into your database/system instead of into a mailbox. This is, ultimately, a cleaner design that will have much less overhead. In fact, there may be an SMTP server implementation out there somewhere already that will allow you to inject this kind of custom logic (I'll edit if I can find one).
Personally, I'd go with the second option. This will give you much more control over what's going on in your system and it will have an overall cleaner design.
Good luck!
Edit: It's not PHP, but JAMES from Apache is a Java mail server that allows you to inject custom mail processing units (called mailets) to handle mail processing. You could write such a mailet that will process email messages and put the updates in your database instead of a mailbox. There may be other projects that implement this kind of design, so it's worth a look.
Edit again: Ooo... here's an open source php SMTP server on SourceForge. I don't know that you can inject custom logic, but you can always edit the source and make it do what you want! (If you insist on PHP anyway)
You write a lot of code between your app and the php imagp/pop3 functions.
There are several free mail servers available that support using MySQL or any other database as a storage backend and requires only configuration to do so. If you're not comfortable customizing an existing mail server or writing your own, I'd go with that solution. It's several orders of magnitude faster than using POP3 or IMAP to communicate with the mail server.
Flickr has published their methods for doing exactly this in the book Building Scalable Websites. The whole of chapter 6 is dedicated to the topic. You don't need a non-standard MTA, as mentioned above. The default MTAs will work fine (sendmail, qmail, postfix, exim, etc.). All you have to do is edit /etc/aliases. /etc/aliases can be used to set a mailbox to pass all email to a script.
I strongly recommend reading through this chapter, as it goes on to outline a lot of the common issues you'll run into doing exactly this -- parsing attachments, coping with email from mobile devices (which frequently includes bad/quirky headers), doing authorization correctly, etc.
A new feature I wish to add to our local network is the ability to retrieve email from free email services such as Gmail, Yahoo and Hotmail using PHP. There are services we can pay for but I would rather hack it up myself!
I find that Google only has an API but the rest do not. What are the problems associated then with me just retrieving email using CURL?
I have even implemented the GMail part using CURL and PHP.
It almost certainly violates their terms of service to screen-scrape their websites for that purpose. If they redesign your site, the scripts you're using to parse out the e-mail contents etc. will probably break catastrophically, as well.
Yahoo, Gmail, and Hotmail all support POP3, a standard protocol for retrieving e-mails. Why not use that instead?
When someone gives you an API, they're promising you that "if you run code X, Y will happen. When you screen scrape, there's no such promise from the provider, and many providers have items in their terms of service that explicitly forbid screen scraping. From a technical standpoint, this means their page/application may undergo changes that will break your screen scraping, wither accidently or purposefully by the provider. This is why CAPTCHA's exist.
Also, increasingly, these applications are using more and more "AJAX" style architectures, which means you're committing yourself to reverse engineering how their application works, as well as keeping up with the changes each application makes.
Finally, well, you're doing it wrong. Email is a set of protocols in and of itself. Most providers have a way to access email via POP3 and IMAP. I'd look into hacking PHP code to interact with the POP/IMAP servers which, like an API, are a promised set of behaviors. You also have the advantage that code written for one provider will likely work (with minor tweaks) for another.
I assume you have a reason for not using the pop protocol which is the supported standard way to retrieve email. To do it like you want it is something that is not supported and maybe also not be covered by the terms of use of the providers.
But if no captcha solving gets in your way it is technically possible. You will have to write a different application for each provider. In case they change something you will have to adopt your application.
To make it work with curl be sure to collect all the cookies they give you in all the pages and to return them in every request.
In case of any problems (and also for development) you could analyze the http requests and answers with some tool (e.g. proxomitron on windows) and make the curl requests more and more look exactly like the browser requests until you succeed. In the end there is nothing they can do to distinguish your curl requests from human requests through a browser. Except captcha like I said before.
Another thing is the intervals between your requests, you could get blocked for requesting to often or when there is no pause between 2 requests (which a human cannot do). Try inserting randomly modified pauses between requests if you suspect this.
I can imagine they block your accounts or IPs during development, in this case it would be necessary to change the IP and/or the account you work on.