My partner on a PHP project objects my practice of always sanitizing integer values in dynamic SQL. We do use parameterized queries when possible. But for UPDATE and DELETE conditions Zend_Db_Adapter requires a non-parameterized SQL string. That's why I, even without thinking, always write something like:
$db->delete('table_foo', 'id = ' . intval($obj->get_id()));
Which is equivalent, but is a shorter version of (I've checked the ZF source code):
$db->delete('table_foo', $db->qouteInto('id = ?', $obj->get_id(), 'INTEGER'));
My partner strongly objects this intval(), saying that if $obj ID is null (the object is not yet saved to DB), I will not notice an error, and the DB operation will just silently execute. That's what has actually happened to him.
He says that if we sanitize all the HTML forms input, there's no way an integer ID can possibly get into '; DROP TABLE ...', or ' OR 1 = 1', or another nasty value, and get inserted into our SQL queries. Thus, I'm just paranoid, and am making our lives unnecessarily more complicated. "Stop trusting the $_SESSION values then" he says.
However, for string value conditions he does agree with:
$db->update->(
'table_foo',
$columns,
'string_column_bar = ' . $db->qoute($string_value))
);
I failed to prove him wrong, and he failed to prove me wrong. Can you do either?
Frankly, your partner is off his rocker: sanitizing is cheap, there's no good reason not to do it. Even if you are sanitizing what is in the HTML forms, if those checks somehow break on production, you'll be happy that you have a backup in other places. Also, it promotes a good practice.
You should sanitize—always
Which do you consider more trouble:
Having to track down a bug that didn't cause a failed SQL query.
Having to restore data after you make a mistake in sanitizing the form input and someone exploits that.
Whichever you pick, there's your answer. Personally, I tend to lean towards the paranoid side of things as well.
If anything, you could do both: create your own function that first checks for null and then calls intval(), and use that instead. Then you get the best of both worlds.
I think your partner is wrong - he is not considering separation of concerns between data sanatisation in the model (where your DB code lives) and data validation of your forms.
Normally your form validation logic will be in a separate area of the application to your model. I.e. when adding validators to form elements, and so this is often done in the form class itself. The purpose of this layer of validation code is to validate the input of the form and return the appropriate messages if there is anything wrong.
So I think data sanitisation in the model should be considered separately to this, as the Model is really a standalone class - and thus should be responsible for it's own data sanitisation. Since in theory you should be able to re-use this model elsewhere in your application, the model should not then assume that the sanitisation has been done elsewhere, i.e. as part of the form validation layer.
Your partners main point about not noticing failed SQL queries is not really an issue in practice - it is better to code defensively.
You should of course always sanitize and not rely on HTML forms. What if you change your code and reuse that part with some other data, that came not from HTML form but from webservice or email or any other source that you decide to add a year later? Using intval() here seems to be fine.
form input should idd always be sanitized, but not every variable that goes into a query should be sanitized imo...
The origin of the variable does play a significant role here.
You just have to know if the data used can be trusted...
If you look deeper inside the Zend framework code, you'll see that $db->quoteInto() turns into $db->quote which returns (string) intval($value) with INTEGER as type.
If type is undefined, $db->_quote() is called. Its code is following:
protected function _quote($value)
{
if (is_int($value) || is_float($value)) {
return $value;
}
return "'" . addcslashes($value, "\000\n\r\\'\"\032") . "'";
}
Whatever the calling method used (with or without specifying type), $db->delete is totally safe.
You should first check that the ID is not null. If it is, you know not to do a useless query and continue from there. It does not make sense to track down issues by reading failed SQL queries or so.
All data retrieved from a form should be sanitized. No exceptions. All data retrieved from your system should have already been sanitized before it got into your system, and therefore shouldn't be sanitized when retrieved from the system again.
So, the question is - where is this integer coming from?
Related
Long time since I've been on but I've hit kind of a head scratcher of a question and I was wondering if anyone could suggest a solution.
The Scenario:
I have number of HTML forms feeding back data via AJAX to a set of PHP PDO query functions.
Although the input will be validated on the form before it is sent I don't want to rely solely on this.
What I'm asking:
What I'm after is some means of checking that the type of the PHP parameter passed is compatible with the field type its passing data to.
At the moment I'm having to use a huge set of if else statements to check each value in turn and throw Exceptions if there not, so I was wondering if anyone knows of an easier way to check whether a given PHP type is compatible with a given MySQL field type.
I'm sure this must be a problem that's been faced before and if someone could suggest a solution it would save me a great deal of coding and be much appreciated.
Thanks in advance
You're putting it wrong.
It is not table structure that dictates the data format, but your application logic.
Say, there is a string that should contain only a-z letters. You cannot tell it from the field type.
Yes, there are some formats that can be easily validated, like data or integers, but they are mere exceptions and can't be of much help.
Most of time form validation is indeed a long code with custom validations and you cannot avoid it. However, I wouldn't throw exceptions in case of failed validation as it is considered more user friendly to show all the errors occurred, not just one.
If you still want some automated checking - look for some ORM, which will indeed do some validations... for the price of writing much more code in the model, than you writing right now with your validations :)
I need to pass special symbols through the URL for my MySQL query. For example, I need to have a URL that is something like:
www.example.com/index.php?q=AND name LIKE '%hi%'
When I tried it at first, I got a 406 error. I looked it up and apparently I have to use urlencode() and urldecode(). I put those in, and the 406 error went away, but then I got a MySQL error:
mysql_fetch_array(): supplied argument is not a valid MySQL result resource
Usually when I get these it means the query isn't written properly. So I echoed MySQL query, and everything looked fine. I even removed the urldecode() and hard-coded into a variable what I wanted to be passed to the page, and the MySQL error went away. However, both queries from using urldecode() and not using that are EXACTLY the same, so I'm kind of confused.
I went onto the php.net documentation page for urldecode(), and there was a warning that said something like using _GET and urldecode() together can result in unexpected things, and that _GET already functions as a decoder (or at least that's how I interpreted the wording), so I removed urldecode() but still left in the _GET, and that resulted in the text not being decoded, so I guess I didn't interpret the documentation correctly.
Is urldecode() not compatible with MySQL queries? I'm fairly certain it's an issue with the encode/decode, since I already tested my code with hard-coded info that bypassed the encode/decode, and it worked fine. Maybe the urldecode() is somehow turning the characters into special ones that look the same but are internally different so MySQL can't read them?
Don't do this. It's wrong. Anyone can of course just end the query using a ; and start a new one, deleting everything or reading out users and passwords.
One easy and much better way to do this is to use www.example.com/index.php?q=hi as your URL, then on the server (let's assume PHP)
$queryString = mysql_real_escape_string($_GET['q']);
$query = "AND name LIKE '%$queryString%'";
// Then replace $query for whatever you were using $_GET['q'] for before.
// Feel free to rename my variables what you like!
This way, the user can't mess things up, and the URL looks cleaner. the mysql_real_escape_string function makes the string safe to use in the query by escaping things. Read http://php.net/manual/en/function.mysql-real-escape-string.php for more on that.
If you still aren't convinced this is useful, consider what happens if someone publishes a link that will drop your tables, and then Google crawls it once a week. Your data will be removed any time that Google happens to swing by.
Once you are happy with this technique, read up on mod_rewrite (if using Apache) to clean the URL up even more, mysqli and how it is an improved version of the mysql functions, and finally PHP Data Objects (PDO) which helps to clean the PHP up even more.
Rich has the right general idea, but even better than quoting is to use database parameters. It essentially places a "variable token" into your SQL query, and then passes the variable separately. The query ends up looking something like this:
SELECT ID, VALUE1, VALUE2
FROM MY_TABLE
WHERE VALUE3 = :param
And then in your code, you add a value to substitute in for :param, and then send that to the database alongside your SQL. (On some DB libraries, you'd use a ? instead of a :parametername.) It works better than quoting for three reasons:
First, you can keep the query string constant instead of having to rebuild it every time, which improves performance in your server.
Second, if you send the same constant query to the database multiple times with multiple different parameters, the database engine can cache the query plan, which improves performance on the DB.
Third, when you get used to writing your queries in parameterized style, it becomes natural, especially if you use a query function that takes a parameter list as an argument, so it's hard to get wrong. By contrast, it's easy to forget to quote something, and then you've accidentally opened a security hole in your program.
Exactly how parameterization works depends on the database and the DB access library you're using, but every SQL database supports them. You should look at how it's done for the system you're using. It really is the best way to work with SQL, especially when you have input coming from untrusted sources such as the Internet.
At some point after calling mysql_query() and before calling mysql_fetch_array() you should check that the query didn't return false, and print mysql_error() if it did. e.g.:
$result = mysql_query($query);
if (!$result) {
die(htmlenitites(mysql_error())." when running <pre>".htmlenitites($query)."</pre>");
}
But:
You shouldn't be using php's mysql functions, you should use PDO, and you should use prepared / parameterized queries.
You shuoldn't be letting SQL be passed from the client unless you really trust anything the client might type - e.g. this is for a back-end admin interface, and even then, it's probably bad practice unless you really need it (like you are writing phpMySQLAdmin)
I have a PHP code stored in the database, I need to execute it when retrieved.
But my code is a mix of HTML and PHP, mainly used in echo "";
A sample that looks like my code:
echo "Some Text " . $var['something'] . " more text " . $anotherVar['something2'];
How can I execute a code like the either if I add the data to the DB with echo""; or without it.
Any ideas?
UPDATE:
I forgot to mention, I'm using this on a website that will be used on intranet and security will be enforced on the server to ensure data safety.
I have a PHP code stored in the database
STOP now.
Move the code out of the database.
And never mix your code with data again.
It's not only a bad idea but also invitation to several type of hacking attempts.
You can do with eval(). but never use it . The eval() is very dangerous because it allows execution of arbitrary PHP code. Its use thus is discouraged. If you have carefully verified that there is no other option than to use this construct, pay special attention not to pass any user provided data into it without properly validating it beforehand.
See eval. It lets you pass a string containing PHP and run it as if you'd written it directly into your file.
It's not a common practice to store executable PHP in a database; is the code you store really that different that it makes more sense to maintain many copies of it rather than adapting it to do the same thing to static data in the database? The use of eval is often considered bad practice as it can lead to problems with maintenance, if there's a way of avoiding it, it's normally worth it.
You can execute code with eval():
$code_str = "echo 'Im executed'";
eval($code_str );
BUT PAY ATTENTION that this is not safe: if someone will get access on your database he will be able to execute any code on your server
use the eval() function.
heres some info
http://www.php.net/manual/en/function.eval.php
something along the lines of:
eval($yourcode);
If that is the last resort, you want it to be secure as it will evaluate anything and hackers love that. Look into Suhosin or other paths to secure this in production.
As everyone'd indicated using eval() is a bad approach for your need. But you can have almost the same result by using whitelist approach.
Make a php file , db_driven_functions.php for instance. get your data from db. and map them in an array as below
//$sql_fn_parameters[0] = function name
//$sql_fn_parameters[1,2,3.....] = function parameters
Then define functions those include your php code blocks.for instance
my_echo($sql_fn_parameters){
echo $sql_fn_parameters[1];//numbered or assoc..
}
then pull the data which contains function name
after controlling if that function is defined
function_exists("$sql_fn_parameters[0]")
call function
call_user_func_array() or call_user_func()
( any you may also filter parameters array $sql_sourced_parameters_array does not contain any risky syntaxes for more security.)
And have your code controlled from db without a risk.
seems a little bit long way but after implementing it's really a joy to use an admin panel driven php flow.
BUT building a structure like this with OOP is better in long term. (Autoloading of classes etc. )
Eval is not safe obviously.
The best route IMO
Save your data in a table
Run a stored procedure when you are ready to grab and process that data
You should not abuse the database this way. And in general, dynamic code execution is a bad idea. You could employ a more elegant solution to this problem using template engines like Smarty or XSLT.
There are a few way to achieve this:
1) By using evil
eval($data);
That's not a typo, eval is usually considered evil and for good reasons. If you think you have fully validated user data to safely use eval, you are likely wrong, and have given a hacker full access to your system. Even if you only use eval for your own data, hacking the database is now enough to gain full access to everything else. It's also a nightmare to debug code used in eval.
2) Save the data to a file, then include it
file_put_contents($path, $data); include $path;
There are still the same security concerns as eval but at least this time the code is easier to debug. You can even test the code before executing it, eg:
if (strpos(exec('php -l '.$path), 'No syntax errors detected') === false))
{
include $path;
}
The downside to this method, is the extra overhead involved in saving the code.
3) Execute the code straight from the database.
You'd need to use database software that allows this. As far as I am aware, this is only includes database software that stores the content as text files. Having database software with "php eval" built in would not be a good thing. You could try txt-db-api. Alternatively, you could write your own. It would like become very difficult to maintain if you do though but is something to consider if you know exactly how you want your data to be structured and are unlikely to change your mind later.
This could save a lot of overhead and have many speed benefits. It likely won't though. Many types of queries run way faster using a traditional database because they are specifically designed for that purpose. If there's a possibility of trying to write to a file more than once at the same time, then you have to create a locking method to handle that.
4) Store php code as text files outside of the database
If your database contains a lot of data that isn't php code, why even store the php code in the database? This could save a lot of overhead, and if you're database is hacked, then it may no longer be enough to gain full access to your system.
Some of the security considerations
Probably more than 99% of the time, you shouldn't even be attempting to do what you are doing. Maybe you have found an exception though, but just being an intranet, isn't enough, and certainly doesn't mean it's safe to ignore security practices. Unless everyone on the intranet needs full admin access, they shouldn't be able to get it. It's best for everyone to have the minimum privileges necessary. If one machine does get hacked, you don't want the hacker to have easy access to everything on the entire intranet. It's likely the hacker will hide what they are doing and will introduce exploits to later bypass your server security.
I certainly need to do this for the CMS I am developing. I'm designing it mainly to produce dynamic content, not static content. The data itself is mostly code. I started off with simple text files, however it slowly evolved into a complicated text file database. It's very fast and efficient, as the only queries I need are very simply and use indexing. I am now focusing on hiding the complexity from myself and making it easy to maintain with greater automation. Directly writing php code or performing admin tasks requires a separate environment with Superuser access for only myself. This is only out of necessity though, as I manage my server from within, and I have produced my own debugging tools and made an environment for code structured a specific way that hides complexity. Using a traditional code editor, then uploading via ssh would now be too complicated to be efficient. Clients will only be able to write php code indirectly though and I have to go to extreme lengths to make that possible, just to avoid the obvious security risks. There are not so obvious ones too. I've had to create an entire framework called Jhp and every piece of code, is then parsed into php. Every function has to pass a whitelist, is renamed or throws an error, and every variable is renamed, and more. Without writing my own parser and with just a simple blacklist, it would never be even a tiny bit secure. Nothing whatsoever client-side can be trusted, unless I can confirm on every request that it has come entirely from myself, and even then my code error checks before saving so I don't accidentally break my system, and just in case I still do, I have another identical environment to fix it with, and detailed error information in the console that even works for fatal errors, whilst always been hidden from the public.
Conclusion
Unless you go to the same lengths I have (at minimum), then you will probably just get hacked. If you are sure that it is worth going to those lengths, then maybe you have found an exception. If your aim is to produce code with code, then the data is always going to be code and it cannot be separated. Just remember, there are a lot more security considerations other than what I have put in this answer and unless the entire purpose of what you are doing makes this a necessity, then why bother at all mix data with code?
The following are ways to XSS-clean data in Codeigniter:
set global_xss_filtering in config to TRUE
use xss_clean()
use xss_clean as a validation rule
set the second parameter to TRUE in $this->input->post('something', TRUE)
Is it okay to use all or more than one of them on one piece of data?
For example, would it be okay if I still used $this->input->post('something', TRUE) even if the data has already been cleaned by global_xss_filtering and xss_clean validation rule?
It's not going to hurt you, but it is definitely is pointless.
There's a very good chance that eventually, you will reach a point where the global XSS filter is going to be cumbersome. Since it can't be disabled per controller without extensive hacks, and access to the raw $_REQUEST data will be impossible, you will need to disable it globally. This will happen the moment you want to process a single piece of trusted data, or data that isn't HTML output and must remain intact.
Using it as a form validation rule is pointless and potentially destructive as well. Imagine what this site would be like if every time you typed <script> it was replaced with [removed], with no way to revert it in the future. For another example, what if a user users some "XSS" content in his password? Your application will end up altering the input silently.
Just use the XSS filter where you need it: on your HTML output, places where javascript can be executed.
Yes. Assume, your input is 'A'. Then, lets say you run an xss_clean to get XSS-safe content:
B = xss_clean(A)
Now, lets say I do it again to get C:
C = css_clean(B)
Now, if B and C differ, then it must mean that B had some xss-unsafe content. Which clearly means that xss_clean is broken as it did not clean A properly. So as long as you assume that the function returns xss-safe content, you are good to go.
One argument that can be made is what if the function modifies even xss-safe content? Well, that would suck and it would still mean that the function is broken, but that is not the case (saying just out of my experience, as in haven't seen it behave like this ever).
The only drawback I see is the additional processing overhead, but doing it twice is fine (once with global filtering, and once doing it explicitly, just in case global filtering is turned off sometime by someone), and is a pretty ok overhead cost for the security assurance.
Also, if I may add, codeigniters xss clean doesn't really parse the HTML and drop the tags and stuff. It just simply converts the < and > to < and >. So with that in mind, I don't see anything that could go wrong.
Using xss_clean even once is bad as far as I am concerned. This routine attempts to sanitise your data by removing parts or replacing parts. It is lossy and not guaranteed to return the same content when run multiple times. It is also hard to predict and will not always act appropriately. Given the amount of things it does to try to sanitise a string there is a massive performance hit for using this on input. Even the tiniest bit of input such as a=b will cause a flurry of activity for xss_clean.
I would like to say that you should never use xss_clean but realistically I can't say that. This system is made for inexperienced developers who do not know how to safely manage user content. I'm an experienced developer so I can say that no project I am working on should ever use xss_clean. The fact is though, the corruption issues will be less problematic for inexperience developers with simpler usage and ultimately it probably will make their code more secure even if they should be making their code more secure themselves rather than relying on quick dirty and cheap hacks. On the otherhand, xss_clean isn't guaranteed to make your code completely secure and can ultmimately make things worse by giving a false sense of security. You are recommended to really study instead to make sure you understand exactly everything your code does so you can make it truly secure. xss_clean does not compensate for code errors, it compensates for coder errors.
Ideally xss_clean wants to be done only on output (and wants to be replaced with htmlentities, etc) but most people wont bother with this as it's simpler for them to violate data purity by just filtering all input rather than filtering output (something can be input once but output ten times). Again, an undisciplined developer may not put xss_clean for one out of those ten cases of output.
Realistically however, the only real decent way is to properly encode everything in the view the moment it is to be displayed on a page. The problem with pre-emptive encoding is that you store data that might be incorrectly encoded and you can double encode data if it is input, then output into a form, then inputted again. If you think of something like an edit box you can have some serious problems with data growth. Not all sanitation removes content. For example, if you addslashes this will add content. If you have a slash in your content every time you run addslashes a new slash is added causing it to grow. Although there is a good chance your data will end up embedded in HTML you also can't always really know where data will end up. Suddenly you get a new requirement that applies to previous data and that's it, you're screwed because you applied and lossy filter to incoming data prior to storage. By lossy, in this case, that might mean your job after corrupting all the user data in your database. Your data is usually the most valuable thing for a web application. This is a big problem with pre-emptive encoding. It is easier to work with if you always know your data is pure and can escape it according to the situation at had but if your data could be in any condition down the line this can be very problematic. The filtering can also cause some occasional logical breakages. As the sanitisation can remove content for example, two strings that don't match can be made to match.
Many of the problems with xss_clean on input are the same or similar to those for magic_quotes:
http://en.wikipedia.org/wiki/Magic_quotes
Summary: You should not use it but instead block bad data on user input and escape properly on output. If you want to sanitise user data, it should happen in the client (browser, form validation) so that the user can see it. You should never have invisible data alteration. If you must run xss_clean. You should only run it once, on output. If you're going to use it for validation of input, have $posted_data !== xss_clean($posted_data) then reject.
I am pretty new at php, quick note this is an android application that i am calling this script, I am not having the users make up the script lol. There is a series of checkboxs and when they check off certain ones it appends a script to the string builder. i am trying to run a query based on the value of a variable that is being passed in. Usually i do something like this,
mssql_query("UPDATE Userdata SET BrowseScript = '".$_REQUEST['sqlscript']."'
WHERE Userdata.Username = '".$_REQUEST['Username']."'");
and where it says .$_REQUEST[''] I can grab values that i pass in.
But this time the .$_REQUEST[''] is the whole script, so i want something like this
mssql_query($_REQUEST['sqlscript']);
and thats it i want it to run the query thats in that value, The query is correct, but it just will not return any value, I think i may have some type of syntax error or something, like i said i am new to php. Thanks for any help. Also I am not posting the whole code because everything is running ok, i just cant get that query to run. so all i need assistance with is the mssql_query part thanks again.
First of all there is huge security flaw in what you are doing. You should always sanitalize and escape the variables you use in your queries for example using mysql_real_escape_string or prepared statements.
Since you are importing whole script for your query, it could be that quotes are not escaped. You need to put these functions like this before your variables:
mysql_real_escape_string($_REQUEST['your_var']);
Using $_REQUEST in itself instead of proper $_GET or $_POST is vulnerable.
Tell your script to output the contents of $_REQUEST['sqlscript']
echo $REQUEST['sqlscript'];
Check that the output is correct. You might well find that there are some parsing errors, or just that your script is incomplete. Check for quotes (") and make sure the query is valid.
Also, you should never run a script from a request. Imagine if someone browsed to your site and typed
youtsite.com?sqlscript='drop * from tables';
Your entire database would be wiped.
For all that's holy, do not do this!
You do not just use request parameters and put them in SQL queries. You at the very least use mysql_real_escape_string (or whatever the MSSQL equivalent is, never used it). And you most certainly don't accept whole queries in the request and execute them sight unseen.
You're opening huge gaping security holes. Especially if you're new, stop that now before it becomes a habit!
example.com/foo.php?sqlscript=DROP TABLE users
Minimum required lecture: http://www.php.net/manual/en/security.database.sql-injection.php