I've got a simple question:
When is it best to sanitize user input?
And which one of these is considered the best practice:
Sanitize data before writing to database.
Save raw data and sanitize it in the view.
For example use HTML::entities() and save result to database.
Or by using HTML methods in the views because in this case laravel by default uses HTML::entities().
Or maybe by using the both.
EDIT: I found interesting example http://forums.laravel.com/viewtopic.php?id=1789. Are there other ways to solve this?
I would say you need both locations but for different reasons. When data comes in you should validate the data according to the domain, and reject requests that do not comply. As an example, there is no point in allowing a tag (or text for that matter) if you expect a number. For a parameter representing.a year, you may even want to check that it is within some range.
Sanitization kicks in for free text fields. You can still do simple validation for unexpected characters like 0-bytes. IMHO it's best to store raw through safe sql (parameterized queries) and then correctly encode for output. There are two reasons. The first is that if your sanitizer has a bug, what do you do with all the data in your database? Resanitizing can have unwanted consequences. Secondly you want to do contextual escaping, for whichever output you are using (JSON, HTML, HTML attributes etc.)
I have a full article on input filtering in Laravel, you might find it useful http://usman.it/xss-filter-laravel/, here is the excerpt from this article:
You can do a global XSS clean yourself, if you don’t have a library to write common methods you may need frequently then I ask you to create a new library Common in application/library. Put this two methods in your Common library:
/*
* Method to strip tags globally.
*/
public static function global_xss_clean()
{
// Recursive cleaning for array [] inputs, not just strings.
$sanitized = static::array_strip_tags(Input::get());
Input::merge($sanitized);
}
public static function array_strip_tags($array)
{
$result = array();
foreach ($array as $key => $value) {
// Don't allow tags on key either, maybe useful for dynamic forms.
$key = strip_tags($key);
// If the value is an array, we will just recurse back into the
// function to keep stripping the tags out of the array,
// otherwise we will set the stripped value.
if (is_array($value)) {
$result[$key] = static::array_strip_tags($value);
} else {
// I am using strip_tags(), you may use htmlentities(),
// also I am doing trim() here, you may remove it, if you wish.
$result[$key] = trim(strip_tags($value));
}
}
return $result;
}
Then put this code in the beginning of your before filter (in application/routes.php):
//Our own method to defend XSS attacks globally.
Common::global_xss_clean();
I just found this question. Another way to do it is to enclose dynamic output in triple brackets like this {{{ $var }}} and blade will escape the string for you. That way you can keep the potentially dangerous characters in case they are important somewhere else in the code and display them as escaped strings.
i'd found this because i was worried about xss in laravel, so this is the packages gvlatko
it is easy:
To Clear Inputs = $cleaned = Xss::clean(Input::get('comment');
To Use in views = $cleaned = Xss::clean(Input::file('profile'), TRUE);
It depends on the user input. If you're generally going to be outputting code they may provide (for example maybe it's a site that provides code snippets), then you'd sanitize on output. It depends on the context. If you're asking for a username, and they're entering HTML tags, your validation should be picking this up and going "no, this is not cool, man!"
If it's like the example I stated earlier (code snippets), then let it through as RAW (but be sure to make sure your database doesn't break), and sanitize on output. When using PHP, you can use htmlentities($string).
Related
I'm trying to find the best way to sanitize requests in PHP.
From what I've read I learned that GET variables should be sanitized only when they're being displayed, not at the beginning of the "request flow". Post variables (which don't come from the database) either.
I can see several problems here:
Of course I can create functions sanitizing these variables, and by calling something like Class::post('name'), or Class::get('name') everything will be safe. But what if a person who will use my code in the future will forget about it and use casual $_POST['name'] instead of my function? Can I provide, or should I provide a bit of security here?
There is never a one-size-fits-all sanitization. "Sanitization" means you manipulate a value to conform to certain properties. For example, you cast something that's supposed to be a number to a number. Or you strip <script> tags out of supposed HTML. What and how exactly to sanitize depends on what the value is supposed to be and whether you need to sanitize at all. Sanitizing HTML for whitelisted tags is really complex, for instance.
Therefore, there's no magic Class::sanitize which fits everything at once. Anybody using your code needs to think about what they're trying to do anyway. If they just blindly use $_POST values as is, they have already failed and need to turn in their programmer card.
What you always need to do is to escape based on the context. But since that depends on the context, you only do it where necessary. You don't blindly escape all all $_POST values, because you have no idea what you're escaping for. See The Great Escapism (Or: What You Need To Know To Work With Text Within Text) for more background information on the whole topic.
The variables are basically "sanitized" when PHP reads them. Meaning if I were to submit
"; exec("some evil command"); $blah="
Then it won't be a problem as far as PHP is concerned - you will get that literal string.
However, when passing it on from PHP to something else, it's important to make sure that "something else" won't misinterpret the string. So, if it's going into a MySQL database then you need to escape it according to MySQL rules (or use prepared statements, which will do this for you). If it's going into HTML, you need to encode < as < as a minimum. If it's going into JavaScript, then you need to JSON-encode it, and so on.
You can do something like this... Not foolproof, but it works..
foreach($_POST as $key => $val)
{
//do sanitization
$val = Class::sanitize($val);
$_POST[$key] = $val;
}
Edit: You'd want to put this as close to the header as you can get. I usually put mine in the controller so it's executed from the __construct() automagically.
Replace the $_POST array with a sanitizer object which is beheaving like an array.
I'm using $_POST and aware about mysql exploit, I decided to use this function on the top of my page, therefore all POST will be safe:
Can you tell me if I miss something and this function will really do the job as I think it will?
function clean_post(){
if ( $_POST){
foreach ($_POST as $k => $v) {
$_POST[$k]=stripslashes($v);
$_POST[$k]=mysql_real_escape_string($v);
$_POST[$k]=preg_replace('/<.*>/', "", "$v");
}
}
if ( $_COOKIE){
foreach ($_COOKIE as $k => $v) {
$_COOKIE[$k]=stripslashes($v);
$_COOKIE[$k]=mysql_real_escape_string($v);
$_COOKIE[$k]=preg_replace('/<.*>/', "", "$v");
}
}
}
It will also remove all html tag, a safest option to output the result might be to use:
<pre>
$foo
</pre>
Cheers!
Cheers!
I think it's a bad idea to do this. It will corrupt the data your users enter even before it hits the database. This approach will also encourage you to use lazy coding where you consistently don't escape data because you believe that all your data is already "clean". This will come back to bite you one day when you do need to output some unsafe characters and you either forget to escape them or you aren't really sure which function you need to call so you just try something and hope that it works.
To do it properly you should ensure that magic quotes is disabled and only escape data when necessary, using precisely the correct escaping method - no more, no less.
There are some problems with it.
First you apply functions on types that doesn't need them, your integers for example needs only a (int) cast to be secure.
Second you do not secure lenght, when you're requesting a '12 chars string' it would be a good idea to ensure you've got only 12 chars, and not 2048. Limiting size is really something your attackers will not like.
Third in your foreach loop you have a $v variable, you assign 3 times a function on $v to $_POST[$k]. So the 1st two assignements are lost when the 3rd occurs...
Then all the things previous people said are right :-)
The rule is apply the filter at the right moment for the right output. HTML output need an html filter (htmlspecialchars), but the database doesn't need it, it need a database escaping. Let's say you want to extract data from your database to build a CSV or a PDF, HTML escaping will make you life harder. You'll need CSV escaping at this time, or PDF escaping.
Finally it is effectively hard to remember if you are manipulating a data which is already well escaped for your output. And I recommend you an excellent read on Joel on Software about Apps Hungarian. The text is quite long, but very good, and the web escaping sequence is used as an example on why Apps Hungarian is good (even if System Hungarain is bad).
Hi this is my first answer for any question asked on web so please review it.
Put this code in top of your script and no need to assign these posted values to any variables for doing the same job of making the input data safe for database. Just use $_POST values as it is in your query statements.
foreach ($_POST as $k => $v) {
if(!is_array($_POST[$k]) ) { //checks for a checkbox array & so if present do not escape it to protect data from being corrupted.
if (ini_get('magic_quotes_gpc')) {
$v = stripslashes($v);
}
$v = preg_replace('/<.*>/', "", "$v"); //replaces html chars
$_POST[$k]= mysql_real_escape_string(trim($v));
}
}
Don't forget $_GET[]
if ($_POST OR $_GET)
Also you can add strip_tags()
I don't know whether your function is correct or not, but the principle is certainly incorrect. You want to escape only where you need to, i.e. just before you pass things into MySQL (in fact you don't even want to do that, ideally; use bound parameters).
There are plenty of situations where you might want the raw data as passed in over the HTTP request. With your approach, there's no ability to do so.
In general, I don't think it's that good of an idea.
Not all post data necessarily goes into MySQL, so there is no need to escape it if it doesn't. That said, using something like PDO and prepared statements is a better way, mysql_* functions are deprecated.
The regular expression could destroy a lot of potentially valid text. You should worry about things like HTML when outputting, not inputting. Furthermore, use a function like strip_tags or htmlspecilchars to handle this.
stripslashes is only necessary if magic quotes are enabled (which they shouldn't be, but always is possible)
When working with stripslashes I'd use get_magic_quotes_gpc():
if (get_magic_quotes_gpc()) {
$_POST[$k]=stripslashes($v);
}
Otherwise you'll over-strip.
I'm putting together a site, (we're already using javascript to prevalidate on the client side). However after getting tired of writing mysql_real_escape_string every other line. I wrote this class that only has two functions mainly focused on sanitising data in user-input/sql. My question is, what are ways to achieve easier input-sanitizing and while improving code readability?
<?php
class Safe {
function userinput($string){
$string = strip_tags($string);
$string = htmlspecialchars($string);
return $string;
}
function sql ($string){
$sqlstuff = Array("union", "select", "update", "delete", "outfile", "create");
$string = Safe::str($string);
$string = mysql_escape_string($string);
$string = str_ireplace($sqlstuff, "", $string);
return $string;
}
}
?>
Sorry, this is going to sound harsh, but your class is completely broken.
You should not be using htmlspecialchars for sanitizing input, it is only useful for escaping output. You do not need to encode HTML for insertion to the database nor should you. Only using htmlspecialchars when sending output to the browser
You should not be stripping tags from your input, you should be leaving them alone and again using htmlspecialchars when you output that data later to insure HTML tags are escaped and not interpreted by the browser
You should not be using mysql_escape_string or mysql_real_escape_string, you should be using PDO. If you are writing a new site there is absolutely no reason not to start out correctly and use PDO. Do it.
You should not be filtering out "union", "select", etc, that's dumb. Those words can appear in regular English language, and they're harmless if you're properly escaping quotes which PDO will handle for you.
Again, sorry for the harsh tone of this answer, but scrap the entire thing and use PDO. There is literally nothing salvageable here.
It's a good idea to use a class like that, particularily if it simplifies input handling. There's however a few points I'd like to comment on:
You should use mysql_real_escape_string instead of the PHP3 mysql_escape_string.
The first function should be called html or something. userinput sounds to vague and misrepresentative.
HTML escaping needs more parameters htmlspecialchars($str, ENT_QUOTES, "UTF-8") to be perfectly safe
The blacklisting of dangerous SQL keywords is not a good idea. It hints at a wrong approach to using SQL queries (if you receive queries via HTTP requests, that's your problem).
Also you should not attempt to filter them. Instead detected them, write to the error/security log, and die() immediately. If there is an attempt to circumvent security, there's no point in attempting any "cleaning" of the request.
You can also use filter_* functions that are bundled with PHP and provide you with the mechanism to filter request parameters according to specific filtering rules.
With few extra tricks, you could even filter arrays of different types of data (thanks to erisco!).
class sanitizer {
public function sanitizeValues($values, $filters) {
$defaultOptions=FILTER_FLAG_NO_ENCODE_QUOTES | FILTER_FLAG_STRIP_LOW | FILTER_NULL_ON_FAILURE;
$filters=(array)$filters;
$values=(array)$values;
foreach ($filters as $key => $filter) {
if($parts=explode('/', $key)){
$v=&$values;
foreach ($parts as $part){
$v=&$v[$part];
}
$filter=(array)$filter;
$filter[1]=isset($filter[1])?$filter[1]:$defaultOptions;
$v=filter_var($v, $filter[0], $filter[1]);
// consider if you really need this here instead of PDO
// $v=mysql_real_escape_string($v);
}
else{
$values[$key]=isset($values[$key]) ? filter_var($values[$key], $filter[0], $filter[1]) : null;
}
}
return $values;
}
}
$manager=sanitizer::sanitizeValues($_GET['manager'], array(
'manager/managerID'=>FILTER_VALIDATE_INT,
'manager/username'=>FILTER_SANITIZE_STRING,
'manager/name'=>FILTER_SANITIZE_STRING,
'manager/email'=>FILTER_SANITIZE_STRING,
'manager/phone'=>FILTER_SANITIZE_STRING,
'manager/bio'=>FILTER_SANITIZE_STRING,
'manager/enabled'=>FILTER_VALIDATE_BOOLEAN,
'manager/password'=>FILTER_SANITIZE_STRING));
This will produce an array complete with all the needed fields based on the 'manager' parameter in _GET, with all values filtered and, optionally, escaped.
I have a lot of user inputs from $_GET and $_POST... At the moment I always write mysql_real_escape_string($_GET['var'])..
I would like to know whether you could make a function that secures, escapes and cleans the $_GET/$_POST arrays right away, so you won't have to deal with it each time you are working with user inputs and such.
I was thinking of an function, e.g cleanMe($input), and inside it, it should do mysql_real_escape_string, htmlspecialchars, strip_tags, stripslashes (I think that would be all to make it clean & secure) and then return the $input.
So is this possible? Making a function that works for all $_GET and $_POST, so you would do only this:
$_GET = cleanMe($_GET);
$_POST = cleanMe($_POST);
So in your code later, when you work with e.g $_GET['blabla'] or $_POST['haha'] , they are secured, stripped and so on?
Tried myself a little:
function cleanMe($input) {
$input = mysql_real_escape_string($input);
$input = htmlspecialchars($input, ENT_IGNORE, 'utf-8');
$input = strip_tags($input);
$input = stripslashes($input);
return $input;
}
The idea of a generic sanitation function is a broken concept.
There is one right sanitation method for every purpose. Running them all indiscriminately on a string will often break it - escaping a piece of HTML code for a SQL query will break it for use in a web page, and vice versa. Sanitation should be applied right before using the data:
before running a database query. The right sanitation method depends on the library you use; they are listed in How can I prevent SQL injection in PHP?
htmlspecialchars() for safe HTML output
preg_quote() for use in a regular expression
escapeshellarg() / escapeshellcmd() for use in an external command
etc. etc.
Using a "one size fits all" sanitation function is like using five kinds of highly toxic insecticide on a plant that can by definition only contain one kind of bug - only to find out that your plants are infested by a sixth kind, on which none of the insecticides work.
Always use that one right method, ideally straight before passing the data to the function. Never mix methods unless you need to.
There is no point in simply passing the input through all these functions. All these functions have different meanings. Data doesn't get "cleaner" by calling more escape-functions.
If you want to store user input in MySQL you need to use only mysql_real_escape_string. It is then fully escaped to store safely in the database.
EDIT
Also note the problems that arise with using the other functions. If the client sends for instance a username to the server, and the username contains an ampersand (&), you don;t want to have called htmlentities before storing it in the database because then the username in the database will contain &.
You're looking for filter_input_array().
However, I suggest only using that for business-style validation/sanitisation and not SQL input filtering.
For protection against SQL injection, use parametrised queries with mysqli or PDO.
The problem is, something clean or secure for one use, won't be for another : cleaning for part of a path, for part of a mysql query, for html output (as html, or in javascript or in an input's value), for xml may require different things which contradicts.
But, some global things can be done.
Try to use filter_input to get your user's input. And use prepared statements for your SQL queries.
Although, instead of a do-it-all function, you can create some class which manages your inputs. Something like that :
class inputManager{
static function toHTML($field){
$data = filter_input(INPUT_GET, $field, FILTER_SANITIZE_SPECIAL_CHARS);
return $data;
}
static function toSQL($field, $dbType = 'mysql'){
$data = filter_input(INPUT_GET, $field);
if($dbType == 'mysql'){
return mysql_real_escape_string($data);
}
}
}
With this kind of things, if you see any $_POST, $GET, $_REQUEST or $_COOKIE in your code, you know you have to change it. And if one day you have to change how you filter your inputs, just change the class you've made.
May I suggest to install "mod_security" if you're using apache and have full access to server?!
It did solve most of my problems. However don't rely in just one or two solutions, always write secure code ;)
UPDATE
Found this PHP IDS (http://php-ids.org/); seems nice :)
I'm wondering if there is a quick and easy function to clean get variables in my url, before I work with them.( or $_POST come to think of it... )
I suppose I could use a regex to replace non-permitted characters, but I'm interested to hear what people use for this sort of thing?
The concept of cleaning input never made much sense to me. It's based on the assumption that some kinds of input are dangerous, but in reality there is no such thing as dangerous input; Just code that handles input wrongly.
The culprit of it is that if you embed a variable inside some kind of string (code), which is then evaluated by any kind of interpreter, you must ensure that the variable is properly escaped. For example, if you embed a string in a SQL-statement, then you must quote and escape certain characters in this string. If you embed values in a URL, then you must escape it with urlencode. If you embed a string within a HTML document, then you must escape with htmlspecialchars. And so on and so forth.
Trying to "clean" data up front is a doomed strategy, because you can't know - at that point - which context the data is going to be used in. The infamous magic_quotes anti-feature of PHP, is a prime example of this misguided idea.
I use the PHP input filters and the function urlencode.
Regular expressions can be helpful, and also PHP 5.2.0 introduced a whole filter extension devoted to filtering input variables in different ways.
It's hard to recommend a single solution, because the nature of input variables is so... variable. :-)
I use the below method to sanitize input for MYSQL database use. To summarize, iterate through the $_POST or $_GET array via foreach, and pass each $_POST or $_GET through the DBSafe function to clean it up. The DBSafe could easily be modified for other uses of the data variables (e.g. HTML output etc..).
// Iterate POST array, pass each to DBSafe function to clean up data
foreach ($_POST as $key => $PostVal) {
// Convert POST Vars into regular vars
$$key=DBSafe($PostVal);
// Use above statement to leave POST or GET array intact, and use new individual vars
// OR, use below to update POST or GET array vars
// Update POST vars
$_POST[$key]=DBSafe($PostVal);
}
function DBSafe($InputVal) {
// Returns MySQL safe values for DB update. unquoted numeric values; NULL for empty input; escaped, 'single-quoted' string-values;
if (is_numeric($InputVal)) {
return $InputVal;
} else {
// escape_string may not be necessary depending on server PHP and MySQL (i.e. magic_quotes) setup. Uncomment below if needed.
// $InputVal=mysql_escape_string($InputVal);
$InputVal=(!$InputVal?'NULL':"'$InputVal'");
return $InputVal;
}
}