I'm writing an Adsense style adserver now. Want to know which are the best methods to fight against repetitive clicks. Now i'm storing the clickers IP address in an other table, and allow 1 click in every 24hr for an ad. This solution is not the best, and it still can be screwed. How does Google and the others does?
Thanks for the suggestions.
One click per 24h will hurt your business big time. Just think of big organizations with just one public IP for their "staff computers".
As far as I know, there is no proper solution to this, but a combination of these things might do the trick for you:
Check in your session if a click has already occurred.
Check against a cookie.
If using flash, check against a shared object.
IP-based limit (but probably much more lax than what you've got now).
Also up-voted Michael J.V's comment. Collecting data about the specific client (and store in database) will make much sense.
Of course, people can spoof that data, but I don't think you can do anything about that (except having a lax IP-based limit).
My solution: Store a cookie + an IP etc.
Maybe even sessions are helpful.
Related
Is there a way to make a reasonably secure system to vote without having to login. I now use cookies to set if the person has voted yet and also insert the users ip in the database.
If that user removes his cookies, he will be able to vote again. That's why I do a check if the user's ip exists in the database and if that IP has voted in the last 30 seconds. That way he'll have to remove his cookies and change his IP address to vote again.
I know there's no 100% failproof solution to this, but
is there a more secure way to do this?
There are two ways that could improve your results, but read and judge for yourself, if you need them:
More persistent cookies
There is the Evercookie project, which stores cookie-like information in a lot of places. It is much harder to delete than just normal cookies.
I personally think that this project should be considered a proof of concept and actually using it would be unethical
Better user recognition
Instead of just looking at the IP address in order to identify a returning visitor, you could use Browser fingerprinting. The EFF has shown with their Panopticlick project, that the combination of Browser version, OS version, installed add-ons etc. is often unique. The Piwik web analytics tool also uses this kind of user heuristics to tell visitors apart. I don't know the implementation, but it's FOSS and in PHP, so you should be able to find that part.
You can run with both of those solutions in unison - but it's still not very secure. You could go as far as blocking a subnet from voting (192.168.1.xxx) to prevent against dynamic IP changes, but then you're also blocking up to 254 people from voting - and it won't prevent against a proxy.
One method I've seen used quite a bit is making it look like you allow duplicate votes; i.e: show it on the end user's end that their duplicate vote has been counted, but don't actually count it in your own database.
But realistically, a login system is about the only relatively "secure" way of doing this - but if someone is determined enough, that can obviously be gamed too.
Hope this helps.
Eoghan
You could ad the
User agent (on short periods there's often little chance that 2 surfers have exactly the same : https://panopticlick.eff.org/index.php?action=log&js=yes)
But again ' if someone is determined enough, that can obviously be gamed too.'
Let's say I want to open a simple, simple poll which could be hand-made via PHP. There are only two options. Option A or Option B. Without telling people to register, what is the most secure way to deter cheating?
If possible, I would like to use MySQL to store the data such as the votes. I am not asking for code, I am asking for ways on what I should do.
I say no registering because it puts people off just for a simple vote...
Thank you and have a good day.
There is no "good" way, let alone a "secure" way. Store their IP address, and don't allow additional votes from that IP. People behind NAT get screwed, but it's really the only thing you can do.
Alternatives include sending them a cookie which prevents them from voting twice, but that is trivially circumvented by even the most tech-unsavvy user.
Give each voter an ever cookie. It's sneaky as all hell, and some people take issue with them, but if you want a fairly good guarantee this is probably it.
Defence in depth. Store as much identifying information as you can.
Store IP address, browser agent, host address, host name ... everything you like the look of, in your MySQL table. If ALL of these match, then it's someone trying to dupe.
Set a cookie to stop them voting. If this exists, they're trying to dupe.
Set a flash cookie to stop them voting (entirely different to normal cookies, get Googling :P) If this exists, they're trying to dupe.
Plus anything else you can think of. There will always be ways to get around it, of course, as it's always extremely hard to say "yes, that almost untraceable request came from that person", but it's more about making it a MASSIVE pain in the ass to beat the system with something like this.
IP is the way to go without registration. You could also get and check against the useragent in addition to the IP address, this might allow for a few more people who are using different computer configurations from the same IP address. Good luck.
Of course, I store all players' ip addresses in mysql and I can check if there is a person with the same ip address before he registers, but then, he can register to my page at school or wherever he wants. So, any suggestions?
The only way that proves particularly effective is to make people pay for accessing your game.
Looking behind the question:
Why do you want to stop the same person registering and playing twice?
What advantage will they have if they do?
If there's no (or only a minimal) advantage then don't waste your time and effort trying to solve a non-problem. Also putting up barriers to something will make some people more determined to break or circumvent them. This could make your problem worse.
If there is an advantage then you need to think of other, more creative, solutions to that problem.
You can't. There is no way to uniquely identify users over the internet. Don't use ip addresses because there could be many people using the same ip, or people using dynamic ip's.
Even if somehow you made them give you a piece of legal identification, you still wouldn't be absolutely sure that they were not registered on the site twice as two different accounts.
I would check the user's IP every time they log onto the game, then log users who come from the same IP and how much they interact. You may find that you get some users from the same IP (ie, roomates, spouses, who play together and are not actually the same person). You may just have to flag these users and monitor their interactions - for example, is there a chat service in the game? If they don't ever talk to each other, they're more than likely the same person, and review on an individual basis.
If its in a webrowser you could bring the information like OS or browser but this even makes it not save but still safer.
It would take the hackers only a little more time and You have to look for the possibility that some people could play on systems with the same OS and browser
The safest thing would be that people on the same IP cannot do things with each other like trading or like in the game PKR (poker game) that you cannot sit on the same table.
An other thing would be wise to do is to use captcha's, its very user unfriendly but it keeps a lot bots out
If it is a browser-based game, Flash cookies are a relatively resilient way to identify a computer. Or have them pay a minimal amount, and identify them by credit card number - that way, it still won't be hard to make multiple account (friends' & family members' cards), but it will be hard to make a lot of them. Depending on your target demographic, it might prohibit potential players from registering, though.
The best approach is probably not worrying much about it and setting the game balance in such a way that progress is proportional to time spent playing (and use a strong captcha to keep bots away). That way, using multiple accounts will offer no advantage.
There are far too many ways to circumvent any restrictions to limit to a single player. FAR too many.
Unless the additional player is causing some sort of problem it is not worth the attempt. You will spend most of your time chasing 'ghosts' instead of concentrating on improving the game and making more money.
IP bans do not work nor flash cookies as a control mechanism either.
Browser fingerprinting does not work either. People can easily use a second browser.
Even UUID's will not work as those too can be spoofed.
And if you actually did manage to discover and implement a working method, the user could simply use a second computer or laptop and what then?
People can also sandbox a browser so as to use the same browser twice thus defeating browser identification.
And then there are virtual machines....
We have an extreme amount of control freaks out there wanting to control every aspect of computing. And the losers are the people who do the computing.
Every tracking issue I ever had I can circumvent easily. Be it UUID's, mac addresses, ip addresses, fingerprinting, etc. And it is very easy to do too.
Best suggestion is to simply watch for any TOU violations and address the problem accordingly.
Currently we are running a competition which proceeds very well. Unfortunately we have all those cheaters back in business who are running scripts which automatically vote for their entries. We already saw some cheaters by looking at the database entries by hand - 5 Star ratings with same browser exactly all 70 minutes for example. Now as the userbase grows up it gets harder and harder to identify them.
What we do until now:
We store the IP and the browser and block that combination to a one hour timeframe. Cookies won't help against these guys.
We are also using a Captcha, which has been broken
Does anyone know how we could find patterns in our database with a PHP script or how we could block them more efficiently?
Any help would be very appreciated...
Direct feedback elimination
This is more of a general strategy that can be combined with many of the other methods. Don't let the spammer know if he succeeds.
You can either hide the current results altogether, only show percentages without absolute number of votes or delay the display of the votes.
Pro: good against all methods
Con: if the fraud is massive, percentage display and delay won't be effective
Vote flagging
Also a general strategy. If you have some reason to assume that the vote is by a spammer, count their vote and mark it as invalid and delete the invalid votes at the end.
Pro: good against all detectable spam attacks
Con: skews the vote, harder to set up, false positives
Captcha
Use a CAPTCHA. If your Captcha is broken, use a better one.
Pro: good against all automated scripts.
Con: useless against pharygulation
IP checking
Limit the number of votes an IP address can cast in a timespan.
Pro: Good against random dudes who constantly hit F5 in their browser
Pro: Easy to implement
Con: Useless against Pharyngulation and elaborate scripts which use proxy servers.
Con: An IP address sometimes maps to many different users
Referrer checking
If you assume that one user maps one IP address, you can limit the number if votes by that IP address. However this assumption usually only holds true for private households.
Pro: Easy to implement
Pro: Good against simple pharyngulation to some extent
Con: Very easy to circumvent by automated scripts
Email Confirmation
Use Email confirmation and only allow one vote per Email. Check your database manually to see if they are using throwaway-emails.
Note that you can add +foo to your username in an email address. username#example.com and username+foo#example.com will both deliver the mail to the same account, so remember that when checking if somebody has already voted.
Pro: good against simple spam scripts
Con: harder to implement
Con: Some users won't like it
HTML Form Randomization
Randomize the order of choices. This might take a while for them to find out.
Pro: nice to have anyways
Con: once detected, very easy to circumvent
HTTPS
One method of vote faking is to capture the http request from a valid browser like Firefox and mimic it with a script, this doesn't work as easy when you use encryption.
Pro: nice to have anyway
Pro: good against very simple scripts
Con: more difficult to set up
Proxy checking
If the spammer votes via proxy, you can check for the X-Forwarded-For header.
Pro: good against more advanced scripts that use proxies
Con: some legitimate users can be affected
Cache checking
Try to see if the client loads all the uncached resources.
Many spambots don't do this. I never tried this, I just know that this isn't checked usually by voting sites.
An example would be embedding <img src="a.gif" /> in your html, with a.gif being some 1x1 pixel image. Then you have to set the http header for the request GET /a.gif with Cache-Control "no-cache, must-revalidate". You can set the http headers in Apache with your .htaccess file like this. (thanks Jacco)
Pro: uncommon method as far as I know
Con: slightly harder to set up
[Edit 2010-09-22]
Evercookie
A so-called evercookie can be useful to track browser-based spammers
Have you tried to do browser fingerprinting?
Check this open source from EFF:
https://panopticlick.eff.org/
Could be used to identify one person similar to 500-1500 in the world (!).
You may add captcha to voting form. Also requiring e-mail confirmation will be useful
If you're really worried about it then you have to do something like email verification, which might be sufficient to block most cheaters.
Also it depends whether multiple people behind a NAT are likely to want to vote for the same option (e.g. favourite school).
Any scheme you create can be gamed.
EDIT: As everyone else has suggested, you can use a CAPTCHA such as reCAPTCHA to block automated bots, and make humans less likely to repeat vote. At the cost of making humans less likely to vote at all.
The Vote to Promote pattern (you may be aware of it) has a section on how to mitigate against gaming - but it is a tricky one to avoid altogether. Given your actions to date I would consider using weighting, for example consider a reasonable level of voting over a time period, say 10 votes per ting per hour (just an example not a guide) and for surplus votes weight the next 10 at 90% (ie only count 9), the next 10 at 80% and so on. This is Yahoo's advice on gaming within this pattern:
Community voting systems do present a
number of challenges. Particularly the
possibility that members of the
community may try to game the system,
out of any number of motivations:
malice - perhaps against another member of the community and that
member's contributions.
gain - to realize some reward, monetary or otherwise, from
influencing the placement of certain
items in the pool)
or an overarching agenda - always promoting certain viewpoints or
political statements, with little
regard for the actual quality of the
content being voted for.
There are a number of ways to attempt
to safeguard against this type of
abuse. Though nothing can stop gaming
altogether. Here are some ways to
minimize or hinder abusers in their
efforts:
Vote for things, not people. In keeping with Yahoo's general strategy,
don't offer users the ability to
directly vote on another user: their
looks, their likeability,
intelligence, or anything else. It's
OK for the community to vote on a
person's contributions, but not on the
quality of their character.
Consider rate-limiting of votes.
o Only allow the user a certain number of votes within a given
time-period.
o Limit the number of times (or the rate at which) a user votes
down a particular user's content. (To
prevent ad-hominem attacks.)
Weigh other factors besides just the number of votes. Digg, for
instance, does not calculate their
Digg-score solely on the number of
votes a submission receives. Their
algorithm also considers: "story
source (is it a blog repost, or the
original story), user history, traffic
levels of the category the story falls
under, and user reports." They update
this algorithm frequently. Consider
keeping the exact algorithm a secret
from the community, or only discuss
the factored inputs in general terms.
If relationship information is available consider weighting user
votes accordingly. Perhaps prohibit
users with formal relationships from
voting for each other's submissions.
While this is currently a popular
pattern on the Web, it is important to
consider the contexts in which we use
it. Very active and popular
communities (Digg is an excellent
example) that enable community-voting
can also engender a certain negativity
of spirit (mean comments, opinionated
cliques, group attacks on 'outlier'
viewpoints).
Check out Asirra: http://research.microsoft.com/en-us/um/redmond/projects/asirra/
It's still in beta, but it's pretty cool.
To prevent the bots from voting you can use CAPTCHA.
The only thing that comes to mind is using a Captcha. Either an elaborate one with pictures and noise like the ReCaptcha service, or a very simple and unobtrusive one like "What is seven plus three?" or (If you're located in the US), "What is the last name of our President", simple common sense questions everybody can answer. If you change them often enough, this could even be more effective than a classic image-based CAPTCHA.
CAPTCHA's aren't a silver bullet, the user could have their script display the CAPTCHA to them and solve them manually for at least several votes per minute.
You need to use them in combination with other techniques mentioned here.
You could add a honeypot field like in Django. Most likely, this will not protect you from cheaters who deliberately want to change your competition, but at least you will have lesser 'drive-by' spammers to additionally take care of.
Sorry for the double post, but I wasn't allowed to post two URLs in the same post...
If you're looking at building your own tracking, maybe this link might provide some inspiration: https://panopticlick.eff.org/
Turns out that a lot of browsers can be uniquely identified, even without any form of tracking cookies. I'm guessing a vote-bot might give a very specific fingerprint?
So if everyone ever wants to make a competition where people can win something and wanna use a community driven rating system... here i share some experiences:
The bad:
1) First it cant be made secure for 100%
2) to reach a mass of users which filters out all the nonsense ratings is very hard
3) Forget about star ratings in that case... their is always either 5 Stars or 1 Star
The good
1) Dont give them orientation about where they stand... We replaced the "Order by place" view with a random presentation of the TOP 100 (only the top 30 wll win a price)... This really helped because a lot of users lost their interest as soon as they didnt see where they stood.
2) Don't allow votings like: 1x5_Stars 40x1_Star... Just allow users which vote in a fair way...
3) Most of them act a little bit stupid... You'll see them in your logs and can trace down who votes fair and who unfair... Search for patterns...
**GOOD LUCK ;-) **
CAPTCHA is always good, might be "disturbing" for some users though.
reCAPTCHA is a fairly used service
How about only allow users who logged in with openid and with reCaptcha before submitting the vote, and monitering the submitter list with same ip address.
We use a combination of CAPTCHA and email. The user receive a link with a GUID by mail.
This one must be unique for each user that try to vote.
www.votesite.com/vote.aspx?guid=.....
By using this link the vote is confirmed or not. In database we check the combination of email address and GUID to be unique.
I use a combination of CAPTCHA, IP verification and LSO (Flash Local Shared Objects, hard to find and delete for common people).
1.Use recaptcha
2. Yes randomize your voting options but not like this:
-> from vote_id_1 to asdsasd_1, grdsgsdg_2,
Instead use session variables to set a mask from vote_id_1 to asgjdas87th2ad in the vote form.
What about some post hoc stochastic analysis, like time series analysis - looking for periodicity in events of particular (ip, browser, vote)? You could then assign probability to each such group of events that it belongs to 1 person and either discard all such groups of events beyond some probability level, or use some kind of weighting to lower the weight according to the probability.
Look in R, it contains A LOT of useful analysis packages.
Check the domain details of the email they are using. I had the same problem and found that all of them were registered to the same registrant. I wrote it up here: http://tincan.co.uk/659/news/competition-spammers.html
Now, I filter on the DNS information for the email used in the registration.
I was thinking of implementing a thumbs system, but mine would require a registration thus ruling out the possibility of people voting more than once unless they create a new account to do so. So I was wondering about Urban Dictionary's thumb system. How does it work? I would imagine that my IP would be stored in a database, so people would not be able to vote more than once however IPs do change pretty often and especially when you're on an iPhone. Probably a combination of cookies and IP checking. Can anyone give me a better insight? What would they check for to ensure you don't vote more than once?
The reason I ask is because I may want to make my a public system instead. Maybe even a hybrid, similar to SO where you can ask a question before creating an account and then have the two linked together. I am using PHP and MySQL.
Almost always it's done with cookies. As you say, IPs can't be used (naively) as they change, or cover too many people (i.e. everyone in a given office, etc).
But online polls not reliable anyway, so don't get too concerned about solving a problem no-one cares about. You can implement more 'intelligent' rules but then you need to ask what benefit you are getting for all your work.
Personally, I would go with:
Cookies
Forced signup voting
Some sort of analysis of voting patterns
Because it goes without saying that people can just sign up constantly, to submit more votes. It really depends on what benefit people get from voting, and how much you care (in terms of time, which is, obviously, money).
I know urban dictionary allows for more than one vote per day. Once every six hours to be exact.