Well, I have a completely AJAX driven site. I inserted a jQuery code that affects all forms and queries site-wide. It's to the point, even though I want to change it, I fathom and accept the idea of a website that utitlizes a single function to process all queries (search, links, & profile, etc....)
How do you accomodate speed and security to such a platform? My php files can be accessed directly from their location's link. That is a threat. Help me; as well as AJAX, I need validation and '777' protection.
Before you read my answer read also this (as answer to a comment on your question) : Possible to view PHP code of a website?
Don't put speed and security in the same box. A website can be secure and fast at the same time.
I would secure a folder with 777 access (why not 755?) with an empty 0Kb index.html file (yes, even if you have inside a bunch of .php files!) ad put an .htaccess with restrictions deny from all that allows a folder to be accessed internally but not from 'outside'.
Than I would NEVER send sensitive data through the requests, but rather a client-side transformed Hashing algorithm like MD5 or SHA1 to compare data and validate on server-side. So don't ever ever send sensitive data in it's pure state over the yellow wire.
Need more security? https
Regarding a "single function" that drives your JS client-end of the site, well, if well formatted the browser doesn't care if it's one or hundreds, a good code is a readable code. Performance wise there's lots of suggestions on the www how to speed up your code.
To add to a really good #Tim's comment/tip, you can still open your console in Firebug (Net) and inspect every single piece of information that is sent from your Page to the server (and vice-versa!!) and act accordingly.
Related
As most programmers I try to program my applications in the safest way possible but this we know that does not guarantee security at 100%. Therefore I think it is also appropriate to have methods to monitor if we may be being attacked. So this is my question.
(My websites are made with PHP and MySQL)
In the case of SQL injection I think this can be done in two ways, but if there are other ways I would also like to know them.
Parsing access/error logs. Does anyone have or know a script that adequately analyzes the access logs (apache) to detect possible attacks? And notify to the administrator automatically with all details.
Analyze HTTP params at real time. It would be a script that analyzes in real time the content passed by GET / POSt and notify (e.g. via email) to the administrator of the website
For example, I do not know much about SQLi attacks but I think it's common for the 'SELECT', 'UINON',...(Others?) strings to appear in query strings and params.
In this way we can analyze the attack and see if it succeeds or not, and then take the consequent actions.
Thanks for your attention!
Edited: Simple bash script
I have made a simple system for analyzing the Apache access_log files and communicate results by email. Which is detailed in this question:
Linux bash to iterate over apache access_log files and send mail
In addition, another one using AWK. The only one resource I've found related about that:
https://www.unix.com/shell-programming-and-scripting/248420-sql-injection-detection.html
(But I have not been able to make it runs in my case)
Oh boy.
Alright, where to start?
For starters, remember that bad hackers are usually financially motivated. You know your website has been injected if you wake up one morning to a red error message from Chrome or Firefox, and you open it anyway to find that your website is now among the more popular places to find free cruises and viagra online.
Sites that score well with SEO are more likely to be hacked than sites that do not. More users means more exposure. Password protected sites don't get hacked as often, but the password protection itself does not necessarily mean any added security. If you're vulnerable, you're vulnerable, and you need to be on top of it.
First and foremost, remember to filter your variables. Never trust anything that comes in from a browser. IT'S ALL SUSPECT. That's means filtering anything that counts as a super global, GET POST, REQUEST, etc. I wouldn't even trust sessions, honestly. Filter it all. More on this can be found here: http://php.net/manual/en/function.filter-var.php
Something else to think about is file uploading. Bad guys love uploading files, and taking over your server. Most common method is exploit files disguised as images. You're going to want to resample every image that comes in. GD Works, but I like Imagick better, personally, more options. More on that here: http://php.net/manual/en/book.imagick.php You're also going to want to make sure that your site can't upload images or any other type of file from pages that you don't explicitly designate as form or upload pages. You would be shocked how often I see sites that can upload from the index, it's insane.
Another method you can deploy for this, is use your php ini to set a global include, and open up any file in a $_FILES array that comes in. Open up the first million spaces in the file, and scan it for php reserved words, and unix shell scripting. If you find one, kill the upload, exit or die, whatever you like to do there.
Apache has a setting for forensic logs. Forensic logs will capture all GET and POST stuff, but the issue with it, and the reason it's not exposed by default is that your log get big, and quickly. You can read up on it here: https://httpd.apache.org/docs/2.4/mod/mod_log_forensic.html
Lastly, you're going to want to evaluate your site for injection vulnerabilities and cross site scripting. Cross site scripting isn't the issue it once was, given the way browsers are constructed these days. All those little details that make life harder for us as a developers actually make us more secure.
But you do want to check for SQL vulnerabilities, especially if you're writing code from scratch. There are a couple reasonably solid plugins for Chrome that make pen testing a little easier.
Hackbar: https://chrome.google.com/webstore/detail/hackbar/ejljggkpbkchhfcplgpaegmbfhenekdc?utm_source=chrome-ntp-icon
HackTab:
https://chrome.google.com/webstore/detail/hack-tab-web-security-tes/nipgnhajbnocidffkedmkbclbihbalag?utm_source=chrome-ntp-icon
For Firefox, there's scrippy
https://addons.mozilla.org/en-US/firefox/addon/scrippy/?src=search
Hope that helps.
Good luck.
Therefore I think it is also appropriate to have methods to monitor if we may be being attacked.
The biggest waste of time ever.
ANY site gets "attacked" 100% of time. There are freely avalable scripts that allow any stupid schoolboy to scan whole internet, probing sites just by chance. You'll grow bored the very next day after scouring these logs of your detection system.
In your place I would invest in the protection. Other vectors than you could think of. For examle, all recent breakings I was a vitness of were performed by means of stealing ftp passwords stored on the webmaster's PC. And I can assure you that there are much more attack vectors than a blunt SQL injection. Which is a simplest thing to protect from, with only two simple rules to follow:
Any variable data literal (i.e. a string or a number) should be substituted with a parameter, whereas actual value should be sent to the query separately, through bind/execute process.
All other query parts that happen to be added through a variable, should be explicitly filtered through a hardcoded list of allowed values.
I am using PHP to write my server side code for my website. What is the best way to prevent someone from scraping my data?
Like in PHP if someone uses file_get_contents() or someone fetches my login form in an iframe element or the data entered in the login form -
how can I prevent such things?
I am using PHP 5.47, MySQL, HTML and CSS.
I think that being a web-developer these days is terrifying and that maybe there is a temptation to go into "overkill" when it comes to web security. As the other answers have mentioned, it is impossible to stop automated scraping and it shouldn't worry you if you follow these guidelines:
It is great that you are considering website security. Never change.
Never send anything from the server you don't want the user to see. If the user is not authorised to see it, don't send it. Don't "hide" important bits and pieces in jQuery.data() or data-attributes. Don't squirrel things away in obfuscated JavaScript. Don't use techniques to hide data on the page until the user logs in, etc, etc.
Everything - everything - is visible if it leaves the server.
If you have content you want to protect from "content farm" scraping use email verified user registration (including some form of GOOD reCaptcha to confound - most of - the bots).
Protect your server!!! As best you can, make sure you don't leave any common exploits. Read this -> http://owasp.org/index.php/Category:How_To <- Yes. All of it ;)
Prevent direct access to your files. The more traditional approach is defined('_SOMECONSTANT') or die('No peeking, hacker!'); at the top of your PHP document. If the file is not accessed through the proper channels, nothing important will be sent from the server.
You can also meddle with your .htaccess or go large and in charge.
Are you perhaps worried about cross site scripting (XSS)?
If you are worried about data being intercepted when the user enters login information, you can implement double verification (like Facebook) or use SSL
It really all boils down to what your site will do. If it is a run of the mill site, cover the basics in the bullet points and hope for the best ;) If it is something sensitive like a banking site... well... don't do a banking site just yet :P
Just as an aside: I never touch credit card numbers and such. Any website I develop will politely API on to a company with insurance and fleets of staff dedicated to security (not just little old me and my shattered nerves).
No there is no way to make this sure. You can implement some Javascript functions which try to prevent this, but if the client just deactivate JS (or a server just ignores it), you can't prevent this.
It is really hard to prevent this. I have found a similar discussion here. This will answer most of your queries but if you want even more perfect protection then sophisticated programs and services like Scrapesentry and Distil would be needed.
Using JavaScript or php, you just decrease the data scraping, but you can't stop the data scraping.
Browser can read the html data so user can view your page source and get that. You can disable key events but can't stop the scraping.
This question already has answers here:
How do I hide javascript code in a webpage?
(12 answers)
Closed 9 years ago.
is it possible to hide codes written in java script (j query)?
i have written a program and i have used two much load() function .
every one can see my pages address is it a risk?
something Like this:
load('account/module/message/index.php');
load('account/module/ads/index.php');
load('account/module/stat/index.html');
No.
JavaScript is client side therefore all code written is, in some fashion, directly visible to the client (end user). You can obfuscate it and make it more difficult to decipher, but in the end is still accessible.
If security is of concern you can keep "business logic" within php and access it using JavaScript (e.g. Ajax calls) but the end points would still be visible.
On every site that uses Javascript, that javascript code is visible to the end user. Not only that, but the end user is able to debug it, and change the either the variable contents or even the code itself at any moment.
Despite this, millions of sites use Javascript, and many of those sites are considered secure. The point is that while JS code may be visible to the end user, it doesn't necessarily mean your system is insecure. You just have to write your system with the understanding of how it works.
Here are some pointers:
If you put secrets (eg passwords or business logic that must be kept private) into your JS code, then those secrets are not secure. Don't do this; keep those details on the server.
If your JS code does any kind of validation, then that validation could be bypassed, so your server-side code must also do the same validation.
If your JS code makes calls that run code on the server (eg your load(...) calls, then the server must verify that the user has permission to do so; don't rely on the JS code to do that check.
You can't "hide" the client-side code, the most you could hope to do is obfuscate it, which to me is largely pointless in the context of the web - code that is delivered to the client should be exposable without being dangerous - and you can hardly obfsucate URLs, anyway.
For parts that shouldn't be exposed, don't expose them. Do server-side generation and output only what is needed, what is "safe"; some trouble can come when mixing the two (say, wanting to hide away logic by doing it on the server, but still deliver it dynamically using AJAX), because your logic is indirectly exposed (that is, although it can't be seen, the results can be gathered, perhaps from a different domain to use your content, etc.)
You can try using an Obfuscation Tool like YUI Compressor http://yui.github.io/yuicompressor/
So your code will not be readable for end user... but hidding it it's impossible
Hidding values and stuff
If you want to keep your values private, so user can't read them obfuscation won't be really your choice, but of course your source will be minified, it will be a mess if you want to read it, but it's still there...
So your choice here is use some kind of encryption which will be decrypted when page loads, but it is a hard work, you can use base64, sha1 or whatever you want only the strings or values you want. But anyone can decrypt it if they really want to.
Definately not, because javascript executed client side so either if possible you do all the operation on server side scripting ( jsp/php/asp) or minify/compress your javascript code after moving it to a sepatate file.
Unfortunately not.
Javascript runs on the client machine in the web browser and cannot be hidden from someone looking at the source code.
However this does not pose a security risk for your application provided nothing internal is visible should you visit those pages in your browser.
process all your "secret" code on the server, where the user doesn't have access to the code. Send only "non secret" things to the client, like for the UI. If you can't avoid sending secret code to the client, obfuscate it to make it more difficult to read.
Put your Javascript code in external file.
And then minified your javscript code, may this helps you.
To Convert Normal Javascript into Minified Javascript Refer this http://jscompress.com/
I am designing a website, and I really want it to be as secure as possible.
I have a private folder that cannot be accessed (.htaccess) which contains all my php classes (and similar structures), and a public folder that has my Javascript, CSS and a PHP file for the Javascript(via AJAX) to interface with, which in turn accesses the classes in the private folder.
Now here is my issue, and for the life of me I just cannot seem to get my head around this one:
If someone was to look at the js code they would see the commands / data being sent to the publicly available PHP Script (as described above), therefore getting an idea of what commands to use interface with that script and potentially gain access to stored data etc.
now I know that ajax wont work remotely etc but as long as you got the commands from the ajax script you could interface directly with it, so i thought i would do a referrer check on the interface script and that worked perfectly until I realized how easy it was to spoof your referrer header!
does anyone have any ideas on how to secure this. if this just sounds like complete garbage tell me and I'll try and break it down further.
AJAX and JS are client-based - everything they do, any user can do. If you expose an API method to AJAX, you expose it to the user - there's nothing you can do about that. That's your design choice. You could of course obfuscate your API calls, but that doesn't really do anything other than make it less user-friendly.
The bottom line: don't trust any user input, regardless of whether it came from your AJAX code or somewhere else.
Well, someone scripting your site directly would only be able to access the same stuff he already can in UI, right?
If you have an script function doAdminStuff(), you would check server side if the user is logged in AND is an admin, before taking any actions
Relax, dude.
This is SPARTA! WEB.
Every site in the world is "exposed" like this. That's the way the web works, Ajax or non-ajax based.
You can't help it, yet there is no harm in this. There is nothing to secure.
Here are my recommendations:
Use SSL if you are not already.
Use a (software) token for all requests that you want to protect.
To discourage others from reading your javascript files, you can obfuscate them. Dean Edward's packer is a famous one.
Write a script that sniffs logs and data for potentially bad activity. If you are not logging all the activity you need to (like if the apache logs are not enough) consider writing activity to your own log.
don't be paranoid, just filter input params, maybe you should switch on SSL so you ajax requests content will be hard to sniff, etc.
Are you using the ajax-thing only for security-reasons or for any other reason? Because you can build up an architecture like this (a PHP-file as "gateway" and all other PHP-files in access-restricted folder) without using ajax as well. If you want to check out, you could take a look at the default folder structure of Zend Framework. This structure has the advantage that there is no logic visible for your users at all.
Also important is that IE (at least IE 6 & 7 I think) does not send a referrer at all by default so this probably wouldn't work anyway.
I am setting up a site using PHP and MySQL that is essentially just a web front-end to an existing database. Understandably my client is very keen to prevent anyone from being able to make a copy of the data in the database yet at the same time wants everything publicly available and even a "view all" link to display every record in the db.
Whilst I have put everything in place to prevent attacks such as SQL injection attacks, there is nothing to prevent anyone from viewing all the records as html and running some sort of script to parse this data back into another database. Even if I was to remove the "view all" link, someone could still, in theory, use an automated process to go through each record one by one and compile these into a new database, essentially pinching all the information.
Does anyone have any good tactics for preventing or even just detering this that they could share.
While there's nothing to stop a determined person from scraping publically available content, you can do a few basic things to mitigate the client's concerns:
Rate limit by user account, IP address, user agent, etc... - this means you restrict the amount of data a particular user group can download in a certain period of time. If you detect a large amount of data being transferred, you shut down the account or IP address.
Require JavaScript - to ensure the client has some resemblance of an interactive browser, rather than a barebones spider...
RIA - make your data available through a Rich Internet Application interface. JavaScript-based grids include ExtJs, YUI, Dojo, etc. Richer environments include Flash and Silverlight as 1kevgriff mentions.
Encode data as images. This is pretty intrusive to regular users, but you could encode some of your data tables or values as images instead of text, which would defeat most text parsers, but isn't foolproof of course.
robots.txt - to deny obvious web spiders, known robot user agents.
User-agent: *
Disallow: /
Use robot metatags. This would stop conforming spiders. This will prevent Google from indexing you for instance:
<meta name="robots" content="noindex,follow,noarchive">
There are different levels of deterrence and the first option is probably the least intrusive.
If the data is published, it's visible and accessible to everyone on the Internet. This includes the people you want to see it and the people you don't.
You can't have it both ways. You can make it so that data can only be visible with an account, and people will make accounts to slurp the data. You can make it so that the data can only be visible from approved IP addresses, and people will go through the steps to acquire approval before slurping it.
Yes, you can make it hard to get, but if you want it to be convenient for typical users you need to make it convenient for malicious ones as well.
There are few ways you can do it, although none are ideal.
Present the data as an image instead of HTML. This requires extra processing on the server side, but wouldn't be hard with the graphics libs in PHP. Alternatively, you could do this just for requests over a certain size (i.e. all).
Load a page shell, then retrieve the data through an AJAX call and insert it into the DOM. Use sessions to set a hash that must be passed back with the AJAX call as verification. The hash would only be valid for a certain length of time (i.e. 10 seconds). This is really just adding an extra step someone would have to jump through to get the data, but would prevent simple page scraping.
Try using Flash or Silverlight for your frontend.
While this can't stop someone if they're really determined, it would be more difficult. If you're loading your data through services, you can always use a secure connection to prevent middleman scraping.
force a reCAPTCHA every 10 page loads for each unique IP
There is really nothing you can do. You can try to look for an automated process going through your site, but they will win in the end.
Rule of thumb: If you want to keep something to yourself, keep it off the Internet.
Take your hands away from the keyboard and ask your client the reason why he wants the data to be visible but not be able to be scraped?
He's asking for two incongruent things and maybe having a discussion as to his reasoning will yield some fruit.
It may be that he really doesn't want it publicly accessible and you need to add authentication / authorization. Or he may decide that there is value in actually opening up an API. But you won't know until you ask.
I don't know why you'd deter this. The customer's offering the data.
Presumably they create value in some unique way that's not trivially reflected in the data.
Anyway.
You can check the browser, screen resolution and IP address to see if it's likely some kind of automated scraper.
Most things like cURL and wget -- unless carefully configured -- are pretty obviously not browsers.
Using something like Adobe Flex - a Flash application front end - would fix this.
Other than that, if you want it to be easy for users to access, it's easy for users to copy.
There's no easy solution for this. If the data is available publicly, then it can be scraped. The only thing you can do is make life more difficult for the scraper by making each entry slightly unique by adding/changing the HTML without affecting the layout. This would possibly make it more difficult for someone to harvest the data using regular expressions but it's still not a real solution and I would say that anyone determined enough would find a way to deal with it.
I would suggest telling your client that this is an unachievable task and getting on with the important parts of your work.
What about creating something akin to the bulletin board's troll protection... If a scrape is detected (perhaps a certain amount of accesses per minute from one IP, or a directed crawl that looks like a sitemap crawl), you can then start to present garbage data, like changing a couple of digits of the phone number or adding silly names to name fields.
Turn this off for google IPs!
Normally to screen-scrape a decent amount one has to make hundreds, thousands (and more) requests to your server. I suggest you read this related Stack Overflow question:
How do you stop scripters from slamming your website hundreds of times a second?
Use the fact that scrapers tend to load many pages in quick succession to detect scraping behaviours. Display a CAPTCHA for every n page loads over x seconds, and/or include an exponentially growing delay for each page load that becomes quite long when say tens of pages are being loaded each minute.
This way normal users will probably never see your CAPTCHA but scrapers will quickly hit the limit that forces them to solve CAPTCHAs.
My suggestion would be that this is illegal anyways so at least you have legal recourse if someone does scrape the website. So maybe the best thing to do would just to include a link to the original site and let people scrape away. The more they scrape the more of your links will appear around the Internet building up your pagerank more and more.
People who scrape usually aren't opposed to including a link to the original site since it builds a sort of rapport with the original author.
So my advice is to ask your boss whether this could actually be the best thing possible for the website's health.