I have an inbox with 5000+ emails saying that somebody has signed up. Within the body of the email is their email address and name which I want to store in a database.
Q) How can I get this into a database? Ideally mysql.
Ideally I would like to do this in a PHP which I am most familiar with.
I am using iRedMail with: OpenLDAP
UPDATE: since posting the question I have already written the actual email parser using Plan Cake Email Parser, literally took 5 mins once I found this.
UPDATE 2:
To make things easier I have moved all of the emails I want in the database into a separate IMAP folder.
But when I do a search it returns nothing: find /var/vmail -name 'Subscribers' -type d –
UPDATE 3:
Another alternative is that I already have the emails downloaded in my email client locally using Thunderbird Windows 7. When I check my Profiles I can see 3 files related to:
Subscribers.msf
Subscribers-1
Subscribers-1.msf
The real tricky part isn't parsing the emails, but getting the emails from your server. There are three ways I can think of doing this:
Use php's imap library http://php.net/manual/en/book.imap.php. I personally haven't worked with it very much but since you're just fetching all emails, it might be useful. The downside is that you'll end up fetching all of your other mails as well.
Use a 3rd-party service to handle the imap and expose your email via a simpler REST interface. Ideally you'd want to search for a key term (maybe "user signed up") and then parse it.
Use offlineimap to download all the emails as a Maildir and store it localy. Now you can use PHP to search all the emails and it'll be faster and easier to test the version here.
As for parsing, I personally recommend using grep to find the mails that contain the information you want to extract, and awk to convert this information into a CSV file. You can do text processing using PHP but it's far simpler and cleaner to do it with these tooks.
You can now easily push the data into MySQL.
Related
I would like to read a data file (such as csv or txt) and mail it using PHP. I would like to be able to have at 5 minute pause in between each send of data. I would also like to use the email field as the senders email. Any help would be appreciated.
Each data chunk will contain:
name
email
address
city
state
zip code
phone
comments
I'm guessing you want to write a newsletter application, so proceeding under that assumption:
You are probably not going to find this answer very helpful, but you should really consider using a newsletter service instead of writing your own code. The reason for this is that sending newsletters requires a LOT more effort and different skills than writing a simple script.
For example, but not exclusively:
Setting up a mailserver correctly.
Taking care about the reverse DNS.
Managing digital signatures (DKIM) for your mail.
Subscribing and tracking sender feedback loops.
Tracking and avoiding DNSBL blocking lists for your server.
etc.
If you really want to write your own code, you have been warned. Take a look at str_getcsv() and SwiftMailer and don't be surprised if you get headaches from this project.
I built support system (web) where my helpdesk can open new tickets/issues.
I want my clients to be able to send email with the issue text and file attache, to specific email address, and new ticket will be opened in my system.
For that I need to know how to scan the folder and how to add the email data to my DB.
what do I need to look for, in the internet, for that? what is the subject?
(I'm using PHP)
As the other's have commented, it's likely a larger task than you're ready to handle. But it doesn't hurt to try.
If I was tasked with the job, I'd take advantage of Gmail (for it's spam reducing features and large storage) to accept incoming email. From there, you simply need to setup a script that connects to your email account and processes the email for storage in your database.
Normally I recommend a solid library for making the job easier and cleaner, but I have a suspicion that you may not be familiar with OOP. If you at least know how to utilize classes, then check out Github:
https://github.com/search?l=PHP&q=imap&ref=searchresults&type=Repositories
Otherwise, if you're new and don't mind writing something "messy" then the following should at least point you in a good direction:
Connecting & retrieving emails for IMAP:
http://www.php.net/imap
http://www.php.net/manual/en/function.imap-open.php
Fetching/processing attachments:
(note that attachments are part of the email body)
http://www.php.net/manual/en/function.imap-fetchstructure.php
Storing attachments (in the filesystem):
http://www.php.net/manual/en/function.mkdir.php
http://www.php.net/manual/en/function.file-put-contents.php
There's plenty of Googling left for you to do. So go forth and make a lot of mistakes. Read the manual. Kick yourself for not having read it sooner, then go make more mistakes. Isn't that how most of us learn?
The first thing that comes to mind is to pop the most recent emails if you have pop3 set up or use imap functions. I did something similar to this using c# using openpop.net. So that could be a starting point.
You can use this method, using the cURL to fetch the emails from Gmail server through feed atom. XML response will return and we can convert it to HTML.
http://www.code4share.net/items/get-unread-email-in-gmail-by-php/XRGXVVh.html
I want to make it so that I can email a url and a php script or something at that url will take the contents I emailed such as message body and write them into a DB. I know how to write something into a DB but I dont know what to do in order to EMAIL to a url or person and have it pull out the message.
The reason I am thinking of emailing to a url if possible is because this saves me the massive blob of trouble of setting up IMAP and using all that mess. I know this can easily be done with php I just have no idea how.
You cannot send email to a URL, you must use a valid email address.
If you want to read your email from within php to import into a DB, you'll need to communicate to your IMAP/SMTP/POP3 server via a socket connection with the fsockopen() function.
I suggest you download any open source web based mail client such as RoundCube, Squirrel Mail, etc... and examine the source code.
There are quite a lot of options when it comes to receiving email. You'll still need to use an email address to send to but you can forward that email on via an HTTP POST using a couple of options such as cURL scripts or using a third party like CloudMailin.
I wrote a blog post about this in Rails (although the principles apply directly to PHP too without much modification. This is a dup of a couple of other questions but it was asked in a fairly different way hence the answer.
What is the best way to save a record of outgoing mail through the PHP mail function?
Basically, I want to archive the messages like the Sent folder in Thunderbird/Outlook.
The only 2 options I can think of is create a DB field, or save them at txt files on the server.
I was wondering if anybody had any suggestions, or methods I might be over looking
Either make a copy of the mail and store it yourself in your mailing script, or... and this one's quite handy: BCC a copy of the email to whatever account you want to be the archive of the sent messages. You can set up a mail folder rule to redirect those BCC'd copies into a specific folder if need be.
I suggest saving them in database since it makes searching them a lot easier than plain text files
If there is a massive amount of mail look into MongoDB. It is a NoSQL DB and many sites use it for archival purposes.
I would save them in a database. When the emailer runs to send out messages, a copy is added to the database. You can also keep track when the email was sent and all that fun information. It will make searching the database easier later. Text files would be a pain, imo.
I want someone to be able to email something (image, video, text) and then have that data be saved on my server in a special directory (the one representing the email address) and if possible take the text with the email and put the data into a database (tags, title, etc).
What is the best way to do this?
The best way to do this is to run a mail server on the server, and have it output email to a particular address to a command. This command will then parse the email and handle saving the attachment and writing to the database.
Try the Postfix mail server and a | foo.sh alias. The script can be written in whatever language you like. Choose something which will make decoding the email easy.
What is the best way to do this?
If you've no idea where to start then I'd recommend posting a question somewhere like Stack Overflow, providing the relevant information needed for people to suggest a solution, including:
the OS where the mail server resides
whether php can be run on this box or if it needs to be located elsewhere
the software in use for the email service - and whether this could be changed easily
the relevance of timing between delivering an email and content being available
Peering into my crystal ball, the best solution would be a php script triggered via procmail
C.