The Project
We have a competition coded in PHP, with CodeIgniter. The form has validation on email addresses and mobile numbers. The page itself is hosted inside an iframe on a different domain (it's an agency-client relationship).
The Problem
We get users with 1000s of entries. We know they are fake because:
They use the same mobile number - assumedly they figure out a mobile number that passes the validation and then use that every time.
The email addresses are all on weird domains, with some of the domains repeated multiple times.
However, the IP addresses are unique, the entries are spread over a few days, the domains themselves have MX records, the user-agents look normal.
The client doesn't want to do anything which could result in fewer entries.
The Question
What are the pros and cons of methods like Captcha? What UI and code patterns have you used that worked?
One method I read is to allow entries that are suspicious, so that spammers entries are accepted, but their data has a 'suspicious' flag against it, which is then checked manually. What data can I check to see whether it is suspicious?
Some methods you could use:
Captcha: Stops bots submitting the form
Email Validation: Send them an email with a unique link to activate their competition entry. Stops invalid email addresses.
Mobile Number Validation: Send them a text message with an activation code. Stops invalid phone numbers.
In my opinion your approach should not be to prevent submission of entries but to require a level of validation on the details entered.
CONS of CAPTCHA:
Users hate it, and it can be frustrating when implemented poorly (failed captcha resets other form fields for instance).
Can be difficult for legit users to complete when the letters are hard to read.
Doesn't always work. Someone just scammed Ticketmaster by beating ReCAPTCHA a few months ago for instance*.
Ugly, more code to implement, and it passes the burden or responsibility from you to the users. PROVE YOU ARE HUMAN is not what I want to see when sending a form, very insulting.
#Nick's got the right idea, use text/email validation. IP checking can be OK sometimes, but as you said, you're getting unique IPs with the same mobile number, so it's not reliable.
There are lots of great posts here regarding CAPTCHA alternatives, definitely worth a read if you plan on employing it. You'll probably have to find a balance between making it easy for the user (encouraging submissions) and front end security techniques.
Why though, can't you simply disregard duplicate mobile numbers or phome number + IP combination? Just because they can can submit multiple times doesn't mean you have to accept it. If it is a human, let them think they are sending in multiple votes :)
*Ticketmaster used various means
to try to thwart Wiseguy’s operation,
at one point switching to a service
called reCaptcha, which is also used
by Facebook. It’s a third-party
Captcha that feeds a Captcha challenge
to a site’s visitors. When a customer
tries to purchase tickets,
Ticketmaster’s network sends a unique
code to reCaptcha, which then
transmits a Captcha challenge to the
customer.
But the defendants allegedly were
able to thwart this, as well. They
wrote a script that impersonated users
trying to access Facebook, and
downloaded hundreds of thousands of
possible Captcha challenges from
reCaptcha, prosecutors maintained.
They identified the file ID of each
Captcha challenge and created a
database of Captcha “answers” to
correspond to each ID. The bot would
then identify the file ID of a
challenge at Ticketmaster and feed
back the corresponding answer. The bot
also mimicked human behavior by
occasionally making mistakes in typing
the answer, authorities said.
Captcha is perfect in spam protection while confusing people very often.
But there is a workaround - You can use JavaScript to hide the captcha for real users (using browsers with JavaScript turned ON) while it will always be "visible" for spam bots (that do not have JS). It's quite simple - just by using of JS You set the div where the captcha is held to display:none, and create a hidden input with value containing that from captcha image...
Strongest approach may be the email validation - but then it means sometimes the rwritting of application. If user submit his reply You register it as not active and send him a validation email to the email address provided. If it is valid, after clicking on the link he will validate his email answer and You can turn his reply to status active...
Also a good workaround for users to prevent the re-submitting of forms on refresh is to redirect users to that same page after the form is submitted and processed... Yes, it takes a second or two longer to view the result, but it's much safer...
Related
I want to hide part of a page (a div) for bots (such as Google Bot), but it should be visible to Humans.
I did quite a bit of research on the above topic but could not find a proper solution. Using the knowledge obtain from searching I came up with the following work around.
Users are offerd to solve a Google-reCaptcha
User solve the captcha and submit a request to view content
A Persistant-Cookie and a new Session-Variable is created
Above details are recorded in a database with IP and User-Agent
As long as Session is valid user can view the content
If Session is absent, but a valid cookie is present
Create a new Session-Variable and update records in the database
If cookie or session invalid
Offer reCaptcha again
So the users do not have to solve a Captcha on each visit to the site and a valid cookie can be used in different pages for the same purpose.
Any comments on how to limit the number of times a user has to solve a Captcha or any weakness of the propose method will be highly appreciated.
May i suggest you reversing the approach by making an input text box then use css to make it invisible to the human eye then during validation you check if this invincible text box has been filled ? If it has then you end validation as only a bot could have seen it, this is called Honeypot Validation.
Never depend on data from cookies all user inputs are dangerous and can be manipulated, so step 6 is wrong.
About IP address it is against the law in some countries to store IP addresses of your visitors with out their consent, make sure your users know that their IP addresses will be stored, should they decide to view and continue to use the site.
So I have a form that can create an account and right now the process to create the account is by calling a javascript REST API. I was thinking that it might be really easy to hack that programmatically since all they would need to do it look at the javascript to find out the url to spam and that it might be safer to do the processing in a PHP script. Then I though well, they could just look at the form to find the URL just as easy if I don't do it through javascript. The form is going to be processing only POST data but not sure if that is enough and if it matters if i process it through javascript or PHP.
What it the best way to prevent someone from spamming a form programmatically (ie prevent them from writing server, like PHP, or client, like javascript, code to spams the processing script).
One way is to use Captcha to filter the bots out reCaptcha but its not 100% protection
Using Captcha is probably the first method:
Google's Version
Secondly I would do data checking on the server side and possibly email verification, if the E-Mail is not verified I would have a cron to clean out the rows in your table which don't have e-mail verification.
With these two methods you should avoid a good majority of it.
Go for reCAPTCHA. It's pretty easy.
You can obtain a key pair there by registering your website URL. Use that key to generate the reCAPTCHA image/textbox in your form. Your form's data will be posted and added to database only if entry matches the text displayed in the image, otherwise not (that's aserverside check that you have to keep). You'll get plenty of related code in Google :)
Another technique, as most of the websites now a days follow, is to send an account activation link to the user via email. An account will get created only when that activation link is clicked upon. You can also set an expiration time (say, 24 hours) for this purpose.
Hope this helps.
Most of the examples I see on the web create user accounts in this sequence: user comes to the site, they choose a username and password and enter their email. A confirmation email to sent to this email and if they click the link, the account gets "verified". If they don't verify, the account gets deleted after a while.
I was told about another way: get the user to verify the email first, and when they click the verification link in their email they can start to create a username and password.
Does anyone see any problems with the second way, whether a security concern or anything else? It's not common and I personally cannot find a totally obvious problem with it, but I'd prefer to use it only after many people confirm they don't see problems or loopholes with it either.
Personally I do see an issue that can be inconveniencing for the user:
When most people register with a web site, they expect that they will have to answer quite a few questions, spend some time reading the FAQ and the terms of service and then spend some more time setting up some preliminary aspects of their profile.
The traditional flow allows the user to choose the time to go through that process. Afterwards, the user only receives a verification link, which normally is a 3-second process to use and can be done at practically any time.
Your proposed flow forces the potential user of your site to spend time reading your documentation, then wait until they receive the message and then find some more time, potentially after a few days, to fill in the forms. I, for one, would find that at least slightly annoying - if not outright discouraging - especially if the mail takes its sweet time to arrive, as it's often bound to do.
I also don't like the inherent implication of such a scheme:
Traditional flow: "Oh nice, you filled in our forms, just give us an address to send you a proper verification". The user here is merely waiting to complete what is essentially a done deal.
Mail-first flow: "Oh it's you. Well, wait for a while and we will send you an invitation if we want you". Here, on the other hand, the user is left in a limbo of subconscious uncertainty until they receive your message.
I believe that the first approach is far more open and friendly to the user. It's also the current standard flow for these cases, which should be enough of an incentive to use on its own - you should avoid forcing your users into processes they are not used to, unless there is no other way.
Getting an email from a friend with an invite link to access a site is exciting - it feels exclusive and new and fun. I'm being given something - so I gladly sign up.
Being required to enter an email address in order to start using a site feels draconian and restrictive and annoying. I'm being asked to give something up as the first step then possibly (maybe?) get something of value down the road.
It's not logical - in both cases, my email address is must be verified before an account can be created. In fact, the first case requires my friend to actively SPAM me with an offer I never requested.
Do you know why I first created this StackOverflow account? Because when I wanted to contribute an answer I could click on the Google logo on the login page and start using the site immediately. No username, password, first name, last name, DOB, or other B.S.
Do you know why I never created an Experts Exchange account? Because the first time I tried to access an answer I was prompted to enter a credit card number, billing address and phone number. Before I could even sample what the site had to offer, I had to give something up.
The point is this: barriers to entry make your site suck. Account creation should be as seamless and painless as possible. Being able to access a site immediately after filling out a single-page signup form and a CAPTCHA is awesome, even if access to other features is restricted until email verification is completed. Maybe I'll even tell you my DOB and favorite color if it unlocks more features.
Personally I don't see a problem with it - its a matter of choice. I think the key point though is making it clear to the user that they must
1) enter their email address
2) wait for a confirmation email before they can get to step 3
3) sign up for the account.
It potentially removes the amount of data held and time invested by the user if they only have to enter a single piece of information (their email address) before filling in the rest of the information you require.
Personally, I'd keep it standard so users don't get confused. The amount of work is the same - get a username/password/email address - wait for users to click the link before they can login to your site.
So how many times would you allow to use link send in email?
If only once, user can't create an account if he close browser before selecting username.
If multiple times, a lot of people can create accounts using same link. Publishing this link and using password recovery feature can be nice phishing trick.
And if you check for this email in your database and allow it only once, user would not be able to create two legitimate accounts.
I could see this method being slightly simpler - when the user clicks the verify link in their email, you send them to a form with a hidden pre-generated id number inserted, and then assign a username and password to it afterwards. Blank accounts, with just and id and no other information, are easy to periodically filter out and you're not storing any details whatsoever until the account is successfully created.
However, there's probably a reason why most sites collect username and password before email - you're getting a user invested before you ask for a more personal bit of information. The account is created - now just verify your email. The other way around ask for an email address first and an account second - even though functionally it's the same, perceptually it's not. Also, the advantage of the standard "flow" is that users know what to expect - following conventions mean users feel like they know what's happening and don't get confused or lose interest.
I want to share some thoughts about second approach.
First of all, it is very similar to invite system, but IT IS NOT the same.
You have to allow to send more than one registration request for a single e-mail address. If you don't - potential user might get it accidentally deleted and there will be no way to repeat the procedure. If you do allow that some angry dudes might use this as spam tool (send as many mails as possible to one(maybe even more) e-mail address. Imagine how would you company/site look for a person who got 10k registration requests...
Standard way has one serious advantage: it allows user name reservation without confirming e-mail (user might want to register, but don't want or has no access to the e-mail server/account).
You MUST consider that your server might delay email sending for pretty long time. Possible reasons: out of memory, DoS attack, email server failure and etc. If you choose mail first approach and user don't receive that mail in 5 minutes (for ANY reason), 3 of 4 potential users will course you company/site and never complete registration.
There is a reason why it is called a standard way, as a lot of small details are considered.
Both approaches are OK - but if you're going defer creation of the account, then you're going to have to embed all the required details into the URL - expiry date, username, password and email address and then encrypt it all to prevent tampering - which makes it rather large.
Actually - you couldn't allow people to pick their own usernames - since you'd have no way of checking whether the username had already previously been requested and not verified. And if you're going to publish usernames, then you'd therefore be publishing email addresses.....not such a good idea?
Here would be my concerns with this approach.
Email delivery is not guaranteed and can be slow. If the user doesn't get the email right away, they may not complete the registration process. What if they mistype their email address or if the email gets marked as SPAM?
In my experience, it is always better to keep record of the users that try to register to a site.
The problem is that more then often the users do not get the confirmation e-mail.
When that happens they often forget the site and do not come back.
What I do is to retry sending the confirmation e-mail after a while, say one week. Often they receive the second e-mail and you end up recovering a registered user that otherwise would be lost.
As a matter of fact, I retry sending the confirmation e-mail once every week until the user confirms or it passed 30 days since the registration attempt.
Even if the user does not confirm after 30 days, I do not delete the account. Often the user comes back trying to register again. Then I just send him again the account confirmation once again and encourage the user to contact the site if he does not get it again.
All this is to maximize the chances of recovering a registered user that otherwise could be lost.
I would suggest the second option. Let the users verify themselves by clicking the link in their email. Then they can choose their preferred username and password. I hope the usernames are unique in the site.
It would be helpful in the situation where some users forget to verify the link in their emails for a long time and so their usernames are locked. Others cannot choose those usernames (until that record is deleted later). Also this can eradicate spammers from picking their own usernames and locking them for use by others.
Hence i would suggest to go with the second option. Let the user first verify his email and his existence before he picks a username and locks it for use by others.
There are actually some sites that do that.
You enter your mail
you get a
verification mail with an initial
password and verification link
once you click the link your account is
active you're directed to a form with
additional details (full name, etc.)
but you may skip them and fill them
any time in the future.
This minimal registration process will help you avoid the loss of potential customers who don't want to bother with filling to many forms and supplying data before they really need to.
What it comes down to is convenience for the user. If the only reason for them the check their email is to verify the account then it may seem like an inconvenience. Instead have the system generate a password for them, email it to them, and instruct them to check their email to get their password. You can allow them to change the password after they log in if they want. This method also help to make sure "strong" passwords are out there initially.
I have a classifieds website, where users must fill in a form to post a classified (offcourse).
I wonder, do you think I need some form of captcha on the form?
Also, I am working on a "register" page where users may get their own username/pass and login to post classifieds more easily.
Also, I have a mail-server setup...
So now, users may click on a classified and at the bottom fill in just three fields in a form (name, email, message) and send an email to the poster of the classified. No captcha there. Is this safe?
My firewall is setup so that it is preventing any outside access to the mail-server except from the websites IP.
Can I set up the firewall to some settings so that I don't need captcha?
I have my own Virtual Private Server btw, and it is running Ubuntu.
A follow-up Q is, if a spam-bot or whatever, gets the hold of my forms and fills alot of them out, or use my mail-server to send emails, what would happen then?
Would my site and mail-server get blacklisted? Is this reverssible or do I have to create a new mail-server then?
Thanks
I go by the design principle of "least barrier to entry". You want people to use your site, so you want to make it as easy as possible for them. Anything at all -- including a captcha -- might turn them away. So my standard line would be to definitely not include a captcha anywhere until you've actually seen a problem with spam. And even then, see if the problem can be solved without a captcha first.
Regarding your question of "well what if a spam bot starts spamming". Simple solution to this is to pre-implement a rate limit. Make it so that someone at a given IP address cannot initiate the sending of an email more frequently than say once every minute. This will not actually cause a problem for real people, but will stop a spam bot in its tracks. You can even try to detect situations where you see a high rate and temporarily block that IP address for 24 hours. That will prevent even the once-a-minute spam.
I can see two situations where you might be thinking of using Captcha's.
Registration
Interaction/Contact Form
However, captcha does tend to upset and drive away users if overused.
So, in these instances:
Registration
The easiest way to stop, or at least inconvenience, anyone trying to create bulk accounts would be to use an email confirmation. So, the user fills out the form, including their email address. The form then sends them an email, which contains a link they must click to activate the account and allow them to login.
Interaction/Contact Form
There will be two kinds of people who will interact through this form. There will be registered, and logged-in users and there will be casual visitors. As Captcha is a method to test for whether a person is a person or not, any registered/logged-in users have already proven themselves, there is no need to use Captcha for them.
However, for a casual, non-logged-in users, then you can use the Captcha for them.
So, the short version, as you can tell, I hate Captcha, and only use it when absolutely necessary.
captch is mainly used for security purpose to stop automation. Like if u have a signup column, a geek can easily make a 1000 users within minutes if he automates it.. and u lose so much space.. At the same time, if u use captcha, its much more safer and you can be sure that all the users are human and none are computer generated usernames.
So if u take your website seriously, i would suggest to go for captch. But keep it simple like they have at megaupload. I dont know which service that is but still its simple.
Definitely YES.
Take a look at reCAPTCHA
You have a PHP plugin here
Can I set up the firewall to some settings so that I don't need captcha?
No. Captcha and firewall are totally different things and cannot be used as the alternative of each other.
A follow-up Q is, if a spam-bot or whatever, gets the hold of my forms
and fills alot of them out, or use my
mail-server to send emails, what would
happen then?
One can write a script which will automate the registration process on your website. It can create so many accounts on your server and/or can post some content which is not good for your website or can use your website for some other persons ads.
Beter try Sblam, its transparent for the visitors, there is no need for user input, it works good for me on numerous sites.
I am using a custom captcha php script along with news letter scripts to let users subscribe using an email id. The method of registration is based on jQuery.post command.
My question is that am I really safe If I remove the captcha validation from my subscription script. The subscription is simple. For example
mydomain/subscribe.php?email=myemail#somedomian.com
The subscribe.php is called along with email as parameter using jQuery.post command. I am new to web programming stuff and don't have much idea about spammers in conjunction with above scenario. Any advise would greatly be appreciated.
Thanks
Spambots fill out every form they can get their hands on.
I receive subscriptions from bots everyday and I'm slowly adding captchas to all forms.
A bot does not know if the form he is filling out is for a subscription, a contact form or a mass mailing opportunity.
If you don't protect yourself by either using captchas or moving some of the logic into javascript functions, you will send emails to all these addresses.
This increases your traffic, decreases your control over your subscribers, makes a mess out of your stats and sometimes you might send an email to users that didn't want to subscribe because a bot has subscribed thru a trick (i.e. realuser+buy-viagra-now#gmail.com also reaches the user realuser#gmail.com).
So my advise: stay with the captcha
Or: modify your form to make more use of javascript (i.e. load the form via ajax), because bots are not normal users with a browser, they only simulate them
From the answers that I have read only the one from favo is really true. A spambot is a computer program that most likely goes one by one in google and searches your page for any input box and submit button.
I have experience with spam and my subscriptions but adding a capcha can get you less subscriptions. Take it off for now and if you can moderate the emails before adding them to to the mailing list. If you see you are getting to much spam then go ahead and put up the capcha. But really one or two spam emails a week is normal.
So, I recommend moderation until it gets ridiculous.
Depdends. You'd probably safe against general malicious subscription attempts, but not against those that are specifically targeting your signup. I personally wouldn't chance it. Did a majority of subscribers complain about a difficult signup process?
What's the motivation behind your thinking of removing the captcha?
I've never heard of spamming subscription lists.
Spammers are known for sending their own spam, not subscribing to someone's else one %)
By removing the captcha, you would basically allow bots to subscribe.
Why they would want to - depends on your content. If the news letter contains no links, or only links to other non-interactive (or captcha protected) resources, you're good.
If your news letter contains link to resources where a bot can go and spam other people, chances are that they will.
Something simple I have done in the past is add a few hidden fields. Most bots will fill these in, a normal user couldn't so you simply put in some logic that says if the hidden fields contain data simply do nothing.
It is not fail safe but it certainly has stopped large amounts of bot sign ups on sites where I have implemented it.
I am using code like this:
jQuery.post("php/varify_captcha.php", jQuery("#contact_form").serialize(),function(result, status){
if(status == "success"){
jQuery.post("php/send_mail.php", jQuery("#contact_form").serialize(),function(result, status){
alert(status);
}
}
}
First, captcha is validated then mail is sent. My question that should I move the captcha validation inside "send_mail.php"? I have no idea about spams and their ability to read and execute the code. If yes, then they can directly call the "send_mail.php". Let me if above method is safe or shall I move captcha validation inside the "send_mail.php" to make code bit more safer.