I have come across this PHP code to check email address using SMTP without sending an email.
Has anyone tried anything similar or does it work for you? Can you tell if an email customer / user enters is correct & exists?
There are two methods you can sometimes use to determine if a recipient actually exists:
You can connect to the server, and issue a VRFY command. Very few servers support this command, but it is intended for exactly this. If the server responds with a 2.0.0 DSN, the user exists.
VRFY user
You can issue a RCPT, and see if the mail is rejected.
MAIL FROM:<>
RCPT TO:<user#domain>
If the user doesn't exist, you'll get a 5.1.1 DSN. However, just because the email is not rejected, does not mean the user exists. Some server will silently discard requests like this to prevent enumeration of their users. Other servers cannot verify the user and have to accept the message regardless.
There is also an antispam technique called greylisting, which will cause the server to reject the address initially, expecting a real SMTP server would attempt a re-delivery some time later. This will mess up attempts to validate the address.
Honestly, if you're attempting to validate an address the best approach is to use a simple regex to block obviously invalid addresses, and then send an actual email with a link back to your system that will validate the email was received. This also ensures that they user entered their actual email, not a slight typo that happens to belong to somebody else.
Other answers here discuss the various problems with trying to do this. I thought I'd show how you might try this in case you wanted to learn by doing it yourself.
You can connect to an mail server via telnet to ask whether an email address exists. Here's an example of testing an email address for stackoverflow.com:
C:\>nslookup -q=mx stackoverflow.com
Non-authoritative answer:
stackoverflow.com MX preference = 40, mail exchanger = STACKOVERFLOW.COM.S9B2.PSMTP.com
stackoverflow.com MX preference = 10, mail exchanger = STACKOVERFLOW.COM.S9A1.PSMTP.com
stackoverflow.com MX preference = 20, mail exchanger = STACKOVERFLOW.COM.S9A2.PSMTP.com
stackoverflow.com MX preference = 30, mail exchanger = STACKOVERFLOW.COM.S9B1.PSMTP.com
C:\>telnet STACKOVERFLOW.COM.S9A1.PSMTP.com 25
220 Postini ESMTP 213 y6_35_0c4 ready. CA Business and Professions Code Section 17538.45 forbids use of this system for unsolicited electronic mail advertisements.
helo hi
250 Postini says hello back
mail from: <me#myhost.com>
250 Ok
rcpt to: <fake#stackoverflow.com>
550-5.1.1 The email account that you tried to reach does not exist. Please try
550-5.1.1 double-checking the recipient's email address for typos or
550-5.1.1 unnecessary spaces. Learn more at
550 5.1.1 http://mail.google.com/support/bin/answer.py?answer=6596 w41si3198459wfd.71
Lines prefixed with numeric codes are responses from the SMTP server. I added some blank lines to make it more readable.
Many mail servers will not return this information as a means to prevent against email address harvesting by spammers, so you cannot rely on this technique. However you may have some success at cleaning out some obviously bad email addresses by detecting invalid mail servers, or having recipient addresses rejected as above.
Note too that mail servers may blacklist you if you make too many requests of them.
In PHP I believe you can use fsockopen, fwrite and fread to perform the above steps programmatically:
$smtp_server = fsockopen("STACKOVERFLOW.COM.S9A1.PSMTP.com", 25, $errno, $errstr, 30);
fwrite($smtp_server, "helo hi\r\n");
fwrite($smtp_server, "mail from: <me#myhost.com>\r\n");
fwrite($smtp_server, "rcpt to: <fake#stackoverflow.com>\r\n");
The general answer is that you can not check if an email address exists event if you send an email to it: it could just go into a black hole.
That being said the method described there is quite effective. It is used in production code in ZoneCheck except that it uses RSET instead of QUIT.
Where user interaction with his mailbox is not overcostly many sites actually test that the mail arrive somewhere by sending a secret number that must be sent back to the emitter (either by going to a secret URL or sending back this secret number by email). Most mailing lists work like that.
This will fail (amongst other cases) when the target mailserver uses greylisting.
Greylisting: SMTP server refuses delivery the first time a previously unknown client connects, allows next time(s); this keeps some percentage of spambots out, while allowing legitimate use - as it is expected that a legitimate mail sender will retry, which is what normal mail transfer agents will do.
However, if your code only checks on the server once, a server with greylisting will deny delivery (as your client is connecting for the first time); unless you check again in a little while, you may be incorrectly rejecting valid e-mail addresses.
I can confirm Joseph's and Drew's answers to use RCPT TO: <address_to_check>. I would like to add some little addenda on top of those answers.
Catch-all providers
Some mail providers implement a catch-all policy, meaning that *#mydomain.com will return positive to the RCPT TO: command. But this doesn't necessarily mean that the mailbox "exists", as in "belongs to a human". Nothing much can be done here, just be aware.
IP Greylisting/Blacklisting
Greylisting: 1st connection from unknown IP is blocked. Solution: retry at least 2 times.
Blacklisting: if you send too many requests from the same IP, this IP is blocked. Solution: use IP rotation.
HTTP requests on sign-up forms
This is very provider-specific, but you sometimes can use well-crafted HTTP requests, and parse the responses of these requests to see if a username already signed up or not with this provider.
Here is the relevant function from an open-source library I wrote to check *#yahoo.com addresses using HTTP requests: check-if-email-exists. I know my code is Rust and this thread is tagged PHP, but the same ideas apply.
Full Inbox
This might be an edge case, but when the user has a full inbox, RCTP TO: will return a 5.1.1 DSN error message saying it's full. This means that the account actually exists!
Disclosure
I run [Reacher][1], a real-time email verification API. My code is written in Rust, and is 100% open-source. Check it out if you want a more robust solution:
Github: https://github.com/reacherhq/check-if-email-exists
With a combination of various techniques to jump through hoops, I manage to verify around 80% of the emails my customers check.
Not really.....Some server may not check the "rcpt to:"
http://www.freesoft.org/CIE/RFC/1123/92.htm
Doing so is security risk.....
If the server do, you can write a bot to discovery every address on the server....
Some issues:
I'm sure some SMTP servers will let you know immediately if an address you give them does not exist, but some won't as a privacy measure. They'll just accept whatever addresses you give them and silently ignore the ones that don't exist.
As the article says, if you do this too often with some servers, they will blacklist you.
For some SMTP servers (like gmail), you need to use SSL in order to do anything. This is only true when using gmail's SMTP server to send email.
About all you can do is search DNS and ensure the domain that is in the email address has an MX record, other than that there is no reliable way of dealing with this.
Some servers may work with the rcpt-to method where you talk to the SMTP server, but it depends entirely on the configuration of the server. Another issue may be an overloaded server may return a 550 code saying user is unknown, but this is a temporary error, there is a permanent error (451 i think?) that can be returned. This depends entirely on the configuration of the server.
I personally would check for the DNS MX record, then send an email verification if the MX record exists.
function EmailValidation($email)
{
$email = htmlspecialchars(stripslashes(strip_tags($email))); //parse unnecessary characters to prevent exploits
if (eregi('[a-z||0-9]#[a-z||0-9].[a-z]', $email)) {
//checks to make sure the email address is in a valid format
$domain = explode( "#", $email ); //get the domain name
if (#fsockopen ($domain[1],80,$errno,$errstr,3)) {
//if the connection can be established, the email address is probably valid
echo "Domain Name is valid ";
return true;
} else {
echo "Con not a email domian";
return false; //if a connection cannot be established return false
}
return false; //if email address is an invalid format return false
}
}
Although this question is a bit old, this service tip might help users searching for a similar solution checking email addresses beyond syntax validation prior to sending.
I have been using this open sourced service for a more in depth validating of emails (checking for mx records on the e-mail address domain etc.) for a few projects with good results. It also checks for common typos witch is quite useful. Demo here.
"Can you tell if an email customer / user enters is correct & exists?"
Actually these are two separate things. It might exist but might not be correct.
Sometimes you have to take the user inputs at the face value. There are many ways to defeat the system otherwise.
Assuming it's the user's address, some mail servers do allow the SMTP VRFY command to actually verify the email address against its mailboxes. Most of the major site won't give you much information; the gmail response is "if you try to mail it, we'll try to deliver it" or something clever like that.
I think you cannot, there are so many scenarios where even sending an e-mail can fail. Eg. mail server on the user side is temporarily down, mailbox exists but is full so message cannot be delivered, etc.
That's probably why so many sites validate a registration after the user confirmed they have received the confirmation e-mail.
You have many simple online tools like https://mail7.net
This service check the email address format, then make sure the domain name is valid and extracts the MX records.
So in 90% you can be sure if it valid. 90% because some mail servers are not involved in the process.
<?php
$email = "someone#exa mple.com";
if(!filter_var($email, FILTER_VALIDATE_EMAIL))
echo "E-mail is not valid";
else
echo "E-mail is valid";
?>
Related
a client asked me about a little form for his website, from which it would be possible to mail the URL to someone. Something like "Hey check this out".
Since he was not happy with mailto:, I want to use PHP mail() function, but i wonder if it is smart to let users define a sender of the email. I am worried about the form being abused for spam/phishing.
Is that a reason to worry? Is it even legal?
It's legal to send e-mail. It's not legal (everywhere) to send spam. But you are just providing a share link, not a relay server, so I wouldn't worry about that. If you limit the amount of control over the content of the message, and limit the number of people to send it to, it won't be too interesting for spammers.
Letting the user choose a sender is not a very good idea. Some mail relay servers check if the originating server is allowed to send e-mails for the domain specified in the address, so the mails might never arrive. You can safely set the sender name, though.
Apart from that, if the receivers of the message consider it as spam and report it, your domain might become blacklisted, and your mails will be sent to junk mail in many cases, so you want to make sure no (or little) spam is sent through your form.
Those bots try every form automatically just to see what happens, so you'll need to make some effort. You could add a captcha, which is an obstacle for humans too, although Google is going to put an end to that. Or you could protect it through other means, like a honeypot. Maybe you can just generate the form through JavaScript, which is a big obstacle for most spam bots.
Setting the From on an email in php mail isn't the cause for concern. The problem is that you'll be sending emails from your server. The mail headers will have your server information embedded - so any issues will tie back to you.
As long as you can safeguard your own server from allowing these spam/phishing attacks, then there's nothing wrong with it.
Just limit the number of people this mail function can send to - and make sure it can't be called multiple times in succession -- like with a script.
This way, the spammers wouldn't benefit from using your page to try to send spam. They'll go elsewhere.
There's much more to do to work with sending email, but this will at least get you started.
"Is it legal" depends upon the country you are in.
I don't think you need to worry about spam if you set up a login.
Or you could limit the number of emails by IP address. This can be spoofed, however, so it may not be the best option.
There are other control options you could do; limit number of emails by User Agent/IP combination, etc.
Aside from the reasons pointed out by others who have answered this question, I would advise against doing this because these messages will likely be marked as spam by spam filters, due to SPF and DMARC records.
For example, is someone sends a message through your system from a yahoo.com address, most spam filters will treat the message as spam, because os Yahoo's DMARC record, which basically says, 'any message sent from a yahoo.com email address that did not originate from a mail server on yahoo's network is spam'. See https://help.yahoo.com/kb/mail/SLN24016.html?impressions=true for more info.
I'm using PHPMailer to send mail from my server.
I have dkim=pass and SPF=pass on my outgoing emails. I am not listed on spamhaus blacklist or via the mxtoolbox blacklist. My emails usually don't go to spam.
However, today I noticed this line in the header of my outgoing mail:
X-OutGoing-Spam-Status: No, score=-2.8
I'm concerned that this negative number will lead to my emails going to spam. (I'm assuming the negative number is bad, but I couldn't find any information online.)
Can someone shed some light on this? Is it a concern? If so, how do I improve this reading?
Thanks!!
The X-spam-status scores are put into the header of any email that passes through a mail server that is running Spamassassin (and some other anti-spam programs). Here's an article regarding tests Spamassassin does in version 3.3.
The higher the score the more likely the email is spam. It uses a range of things to decide on what is and isn't spam, blacklists are only part of its algorithms.
As listed here, an email can have a negative score if it is whitelisted, which is a good thing. If an email is whitelisted, this usually means (not always) that the recipient has received an email from the sender before, and the user has acted on that email (by acted on, I mean you have responded to it, or clicked "always show images from this sender" kind of thing) or you have that email address saved in your address book.
Hope this helps.
EDIT: In short, the negative number is good and you shouldn't worry about it.
Scenario:
I have a contact form on my web app, it gets alot of spam.
I am validating the format of email addresses loosely i.e. ^.+#.+\..+$
I am using a spam filtering service (defensio) but the spam scores returned are overlapping with valid messages. At a threshold of 0.4 some spam gets through and some customer's questions are wrongly thrown in a log and an error displayed.
All of the spam messages use fake email addresses e.g. zxmzxm#ywduasm.com
Dedicated PHP5 Linux server in US, mysql, logging spam only, emailing the non spam messages (not stored).
Proposal:
Use php's checkdnsrr(preg_replace(/^.+?#/, '', $_POST['email']), 'MX') to check the email domain resolves to a valid address, log to file, then redirect with an error for messages that don't resolve, proceed to the spam filter service as before for addresses that do resolve according to checkdnsrr().
I have read (and i am sceptical about this myself) that you should never leave this type of validation up to remote lookups, but why?
Aside from connectivity issues, where i will have bigger problems than a contact form anyway, is checkdnsrr going to encounter false positives/negatives?
Would there be some address types that wont resolve? gov addresses? ip email addresses?
Do i need to escape the hostname i pass to checkdnsrr()?
Solution:
A combination of all three answers (wish i could accept more than one as a compound answer).
I am using:
$email_domain = preg_replace('/^.+?#/', '', $email).'.';
if(!checkdnsrr($email_domain, 'MX') && !checkdnsrr($email_domain, 'A')){
//validation error
}
All spam is being logged and rotated.
With a view to upgrading to a job queue at a later date.
Some comments were made about asking the mail server for the user to verify, i felt this would be too much traffic and might get my server banned or into trouble in some way, and this is only to cut out most of the emails that were being bounced back due to invalid server addresses.
http://en.wikipedia.org/wiki/Fqdn
and
RFC2821
The lookup first attempts to locate an MX record associated with the name.
If a CNAME record is found instead, the resulting name is processed as if
it were the initial name.
If no MX records are found, but an A RR is found, the A RR is treated as
if it was associated with an implicit MX RR, with a preference of 0,
pointing to that host. If one or more MX RRs are found for a given
name, SMTP systems MUST NOT utilize any A RRs associated with that
name unless they are located using the MX RRs; the "implicit MX" rule
above applies only if there are no MX records present. If MX records
are present, but none of them are usable, this situation MUST be
reported as an error.
Many thanks to all (especially ZoogieZork for the A record fallback tip)
I see no harm doing a MX lookup with checkdnsrr() and I also don't see how false positives may appear. You don't need to escape the hostname, in fact you can use this technique and take it a little further by talking to the MTA and testing if the user exists at a given host (however this technique may and probably will get you some false positives in some hosts).
DNS lookups can be slow at times, depending on network traffic & congestion, so that's something to be aware of.
If I were in your shoes, I'd test it out and see how it goes. For a week or so, log all emails to a database or log file and include a field to indicate if it would be marked as spam or legitimate email. After the week is over, take a look at the results and see if it's performing as you would expect.
Taking this logging/testing approach gives you the flexibility to test it out and not worry about loosing customer emails.
I've gotten into the habit of adding an extra field to my forms that is hidden with CSS, if it's filled in I assume it's being submitted by a spam bot. I also make sure to use a name like "url" or "website_url" something that looks like a legitimate field name to a spam bot. Add a label for the field that says something like "Don't fill out this field" so if someone's browser doesn't render it correctly, they will know not to fill out the spam field. So far it's working very well for me.
function mxrecordValidate($email){
list($user, $domain) = explode('#', $email);
$arr= dns_get_record($domain,DNS_MX);
if($arr[0]['host']==$domain&&!empty($arr[0]['target'])){
return $arr[0]['target'];
}
}
$email= 'user#radiffmail.com';
if(mxrecordValidate($email)) {
echo('This MX records exists; I will accept this email as valid.');
}
else {
echo('No MX record exists; Invalid email.');
}
//The Code *https://davidwalsh.name/php-email-validator*
function domain_exists($email, $record = 'MX'){
list($user, $domain) = explode('#', $email);
return checkdnsrr($domain, $record);
}
if(domain_exists('user#davidwalsh.name')) {
echo('This MX records exists; I will accept this email as valid.');
} else {
echo('No MX record exists; Invalid email.');
}
An MX Lookup is only part of the picture, if you want to ensure the email address is itself valid, then you need to attempt to send an email to that account.
The other possible scenario is, someone can be simply using hijacked email accounts from a compromised machine anyway. Of course, that is probably a little bit less likely to occur, but it still does.
There are email address validation libraries out there that do this, simply search for email validation.
All of this can be done asynchronously. I have this setup on my site in which case the email is saved in the database (for auditing purposes), a job queued, then when the job comes time to execute, any additional validation is performed at that point in time. It offloads the heavy lifting to another thread.
To the user, it appears as if the email was sent already, it was (it's in the database), and can be viewed internally, but the actual email won't get mailed out until that job executes which can be immediately or some set amount of time depending on the server load.
Walter
I have a list of email addresses, which I tested that they are invalid, not exists. But my mail server still keep trying to send email to them.
Is there any way to stop sending email to the non-exists, invalid email addresses?
I am using php though..
I'm not sure exactly what you are asking. If you want a way to validate email addresses before sending, I'm afraid you're going to be disappointed. It's essentially impossible to define a valid email address - even if you go to extreme lengths to pattern match email address syntax and keep up to date on all the valid top-level domain names, you still haven't accounted for email address which are in a valid format but do not have a mailbox assigned to them (or even are just not monitored).
If you're looking for how to remove bouncing-but-retrying emails from your mail servers queue, you should specify what mailserver you are using.
If you want to know more about the PHP mail command, as per Chell's answer, the documentation is a good place to start
I have been trying to get PEAR::mail to successfully deliver emails to hotmail users without being flagged as SPAM and ending up in the junk folder, i have no problems with yahoo/gmail only with hotmail.
google suggested that this is a common problem with hotmail and that possible causes can include
incorrect reverse DNS for main IP of the server
lack of SenderId/SPF records
being blacklisted
having checked all of the above i can only think of one other reason - incorrectly formatted headers ?
to test this theory i set up outlook to send email via the same address that PEAR::mail uses and sent a quick test - it delivered straight to my inbox
so i compared the headers from the email sent from PEAR::mail against the headers sent by Outlook and there are only a few differences - i have only listed the differences to save space (and peoples eyes)
PEAR::mail headers (not in outlook headers)
X-PHP-Script: www.example.com/register.php for [users ip address]
Outlook headers (not in PEAR::mail headers)
X-Mailer: Microsoft Office Outlook 11
Thread-Index: Ack6CWSQlgV8s6+6SWyifka2NNpB7g==
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3350
the only other differences that i can see are
the order of the From: and To: headers are reversed
and in the Received: section of the headers
Outlook
Received: from myhomehostname.com ([ip address] helo=simber)
by mywebhostname.com with local (Exim 4.67)
PEAR::mail
Received: from apache by mywebhostname.com with local (Exim 4.67)
could these small differences in the headers be the cause or am i looking in the wrong place ? i knew this might be problematic hence why i chose to use the PEAR::mail class rather than rolling my own but now i really have no idea where to go with this, any help would be greatly appreciated.
Update: as per changelog's suggestion i have tried adding the MS headers to the PEAR::mail class and i have tried replacing PEAR::mail with PHPMailer (with & without the extra headers) - they all end up in the junk folder.
I am starting to believe that it may not be the headers afterall.
Update 2: i should have mentioned that the emails are just a registration confirmation to validate the email address the user signed up with - no mailshots etc so our volume is extremely low.
I have considered warning users who provide a #hotmail/live email address to add us to their address book or check their junk folder - but this just seems unprofessional to me - it may be that i have to resort to this.
As for becoming Sender Score Certified - its very unlikely that i can justify the cost of this when considering the low volume and purpose of these emails.
My company does professional e-mail marketting campaigns (through strongmail servers) we send thousands of (sollicited) emails a day to all kinds of addresses.
The problem you are facing is that you have no authority. You could just be some spammer trying to send loads of spam.
The thing you need to do is:
Add unsubscribe links
Apply for hotmail's Junkmail reporting program (JMRP) and MAKE SURE people that press the 'this is junk' button do not get mailed again. This will up your 'sender score; # hotmail and allow you messages to get through.
Add SPF and other antispam solutions.
Do not send more than 50 e-mails per minute to #hotmail.com (other domains have other limits)
B.t.w we use PHPMailer to compose our messages, no problem at all with that :-)
The problem nowadays really is the restricting receiving mailservers.
Email Deliverability is closer to an art than a science. I can pretty much guarantee that it has nothing to do with your headers. Trying to spoof headers is likely the worst thing you can do. The received: header is added by the mail servers as they receive the messages: spoofing this will cause your email to get flagged as spam: one of the spam filters commonly used is to count then number of relays (ie received: headers). If there's too many you get a higher spam score.
Reverse DNS and SPF are the minimum entry barriers. For hotmail in particular, there are three other very important factors AFTER you get your SPF and DNS records in line:
IP/Domain Reputation
Volume
Being in the Address Book
Reputation isn't the same as being blacklisted. You need to build trust with hotmail. Hotmail uses Sender Score Certified as their main reputation broker -- you can check your reputation with them if you want, but it may cost you.
If you're on a shared host or an IP address that has a checkered past, you won't have much luck with hotmail.
You build reputation by having a consistent volume with low spam complaints. You can send 1M messages an hour all day long, as long as you do it every day. If you're sending less than 10,000 messages a day, you likely won't be able to build up a decent reputation. You can get a report on your volume at Sender Base.
Finally, the best way to make sure you end up in the inbox is to get your users to add the sending email address to their address book. Hotmail uses this as a safe sender list. In fact, I think there's an additional trusted sender option in Hotmail now too (it's been awhile since I've been in the delivery game and I don't use hotmail).
Here are some other best practices for sending email:
ALWAYS use the same IP address
ALWAYS use the same FROM address
if you have a large list that you send newsletters to, make sure you retire old addresses (ie, check open rates)
if you have a large list, try segmenting it and sending from different IP addresses based on risk (ie, newer addresses may mark the message as spam)
I have always used PHPMailer in my projects, and what I did to avoid Hotmail's junk folder was to call a method they had that added MS Headers to the message.
Take a look at the source, and add those headers yourself.
Also, I recommend including a text-version if you're sending HTML e-mail.
I'd suggest modifying the headers you send to match 100% what outlook sends, and see if that solves the problem. Really it's a tough one though, hotmail is known for having a super crappy spam filter, sending lots of legit email to junk, and lots of spam to your inbox.