I've heard people say (in relation to C#/SQL Server, but also in relation to PHP/MySql): Don't escape strings manually - use stored procedures instead.
Ok, I can accept this suggestion, but why? Many say (including on SO) mysql_real_escape_string() is quite enough, mysql_real_escape_string() is good, mysql_real_escape_string() is the first way of protection.
Why? Is there a case where mysql_real_escape_string() can fail? At least one... I don't need many :)
When mysql_real_escape_string FAIL:
$sql = "SELECT * FROM users WHERE id=" + mysql_real_escape_string($_GET['id']);
If $_GET['user_id'] is set to 1 OR 1=1, there are no special chars and it's not filtered.
The result: All rows are returned.
It gets worse. How about this... what if $_GET['user_id'] is set to 1 OR is_admin = 1?
The function is only designed to be used when inside single quotes.
There are two things that can go wrong with mysql_real_escape_string:
You can forget to use it
If you are using some specific multibyte connection encodings, and you have set these encodings with SET NAMES instead of mysql_set_charset as is proper, it can still leave you vulnerable
Update:
You are safe with UTF-8 (although this does not mean that you should continue using SET NAMES!)
For an explanation, see here
Just for info:
mysql_real_escape_string() does not escape % and _. These are wildcards in MySQL if combined with LIKE, GRANT, or REVOKE.
mysql_real_escape_string() can fail to clean the input.
Since mysql_real_esacpe_string() takes character set into account while cleaning strings.
There's the problem. You can change character via mysql_query function sending the query to change connection's character set. However, mysql_real_escape_string() is oblivious to the set you're using and it will escape some characters improperly.
The other thing is constantly invoking it manually. Even wrapping it in a function is a P.I.T.A. because it implies you have to create some sort of database wrapper / database abstraction layer of your own in order to be able to automate calls to mysql_real_escape_string().
That's why we have PDO in PHP which helps us alleviate all of the above and you get to use parametrized prepared statements which are preferable way of dealing with repeating queries that alter the data.
Prepare the statement, bind input variables and it will clean the input according to the database driver being used and connection's character set. It's less code and completely safe.
There is only one thing about mysql_real_escape_string() and SQL Injection -
The former does not have a slightest relation to the latter.
Although it sounds paradoxical, nothing can be more true.
Here are 2 statements proving it
You have to escape quoted strings only, as this function helps nothing else.
In fact, you have to escape every string you are adding to the query, even safiest one. just because it may contain some special character and thus break the query (just as an accident, not malicious plot).
Thus, when applicable, this function have to be used anyway, despite of whatever dangers or fears. And in any other case it will help nothing.
The only thing I have to add is that prepared statements do not provide full protection either. Here is the explanation and recommendations: https://stackoverflow.com/a/8255054/285587
Related
So I was busy with PHP, inserting data into mysql when I wanted to know: I have come accross some posts that say it's bad practice to use single quotes to insert data into a database. one of the examples: Why are VALUES written between quotes when sent to a database?
The post is about why they're written between quotes, but one thing was clear: It's bad practice to insert it like:
$sql = INSERT INTO example (example1, example2, example3) VALUES
('$example1', '$example2', '$example3');
Why is this be bad practice? Apparently it is vurnerable to injection as stated in the above link given. the OP his question was related to the comment was: We use mysqli_real_escape_string for this. The respons given was:
#XX To a large extent, yes, it is an alternative solution to the problem. It doesn't disable anything, but it escapes things so that for instance ' becomes '' or \' in the SQL string, keeping the attacker from ending the string. There are awkward cases where such escaping is hard, and it's easy to miss one escape call among many, which is why parametrised queries are considered the most reliable approach.
First of all: How does a script want to fool mysqli_real_escape_string into NOT escaping certain stuff? I found something that said the following and correct me if i'm wrong: mysqli_real_escape_string - example for 100% safety. As you can see he refers to another page, that has an answer. However He then makes a claim that should make his data 100% safe and someone else responds with:
Yes, that is generally safe. Mysql and mysqli are perfectly safe when used right (specific bugs in very specific encodings notwithstanding). The advantage of prepared statements is that it's more difficult to do things the wrong way.
I have the following example to make it clear for myself: I have 2 doors, 1 door is open, but behind a closed door. How would you attack an open door with a closed door in front of it?
There is an answer here: SQL injection that gets around mysql_real_escape_string(), but he says as an safe example:
mysql_query('SET NAMES utf8');
$var = mysql_real_escape_string("\xbf\x27 OR 1=1 /*");
mysql_query("SELECT * FROM test WHERE name = '$var' LIMIT 1");`
Isn't mysqli_real_escape_string already doing the same? he's just specifying what characters should be mysqli_real_escaped_string. So how can this all of the sudden become safe? Since it is doing the exact same thing as when you would say:
$example = mysqli_real_escape_string ($conn, $_POST['exampleVariable']);
So how does this:
mysql_query('SET NAMES utf8');
$var = mysql_real_escape_string("\xbf\x27 OR 1=1 /*");
mysql_query("SELECT * FROM test WHERE name = '$var' LIMIT 1");
become safe and this:
$example = mysqli_real_escape_string ($conn, $_POST['exampleVariable']);
not? Isn't he just narrowing down what mysqli_real_escape_string would escape, thus making it more vulnerable?
The thing is, mysqli_real_escape_string() or other proper escaping is actually technically secure, in the end, that's about what parameterized queries do too. However, there is always a however. For example it is only secure if you have quotes around variables. Without quotes it is not. When it's a 1000 line project with one developer, it's probably ok in the first 3 months. Then eventually even that one developer will forget the quotes, because they are not always needed syntactically. Imagine what will happen in a 2 million LOC project with 200 devs, after 5 years.
Similarly, you must not ever forget using your manual escape function. It may be easy first. Then there is a variable that is validated and can only hold a number, so you are sure it's ok to just put it in without escaping. Then you change your mind and change it into a string, because the query is ok anyway. Or somebody else does it after 2 years. And so on. It's not a technical issue. It's a management-ish issue, in the long run, your code will be vulnerable somehow. Hence it is bad practice.
Another point, it's much harder to automatically scan for SQL injection vulnerabilities if manual escaping is in place. Static code scanners can easily find all instances of concatenated queries, but it's very hard for them to correlate previous escaping if there is any. If you use something like parameterized queries, it is straightforward to find sql injections, because all concatenations are potential candidates.
Setting the character set for the connection does not change what mysqli_real_escape_string() escapes. But it avoids the multi-byte character bug because it controls how the string is interpreted after the backslash escape characters have been inserted.
Of course, you can avoid any uncertainty by using query parameters instead.
I'm developing my personal web site using php. everything is ok but I just read mysql_real_escape_string manual in php.net and found two things:
This function must always (with few exceptions) be used to make data safe before sending a query to MySQL.
mysql_real_escape_string() does not escape % and _. These are wildcards in MySQL if combined with LIKE, GRANT, or REVOKE.
I have two questions:
1-what are these exceptions?
2- how to escape those characters?
This function must always (with few exceptions) be used to make data safe before sending a query to MySQL.
To my great disappointment, the manual page says complete rubbish, and they refuse to make it correct.
So, quite contrary, there are only few cases when you need this function. So to say ONLY ONE: when you are adding a string into SQL query.
mysql_real_escape_string() does not escape % and _. These are wildcards in MySQL if combined with LIKE, GRANT, or REVOKE.
It doesn't matter too much. As long as you are using LIKE operator on purpose, these characters won't do any harm.
But if you want to escape the string going to LIKE statement, you can use this code
$like = addCslashes($like,'\%_');
(note the slash - it is also required to be escaped as manual stating it. also note the C letter in the function name).
After this procedure you may use the resulting $like variable whatever way you are using to build your queries - either quote and escape them or use in the prepared statement.
I 'm not sure what exceptions the manual is referring to when talking about making data safe. You could say that the exception is when the data is already known to be safe. For example, here are a few cases that come to mind:
the data is typed as a number (this is really a specialization of the next item)
you already know it does not contain any characters that need to be escaped (e.g. it comes from looking up something in a "whitelist" array that contains a few options you hardcoded)
For example, if you have $id = intval($_GET['id']) then you do not need to escape $id before injecting it into a query.
However! It can never hurt you to escape all input, and doing so eliminates the chance that you introduce vulnerabilities in your code (e.g. if you forget to escape, if the requirements change, or anything really). So I recommend getting into the habit of escaping everything and forgetting about "exceptions".
As for the % and _ characters as part of the input, these do not need to be escaped unless you are going to feed this input to a command that recognizes them. So for example, if you have a query like this:
$term = $_GET['term'];
$sql = sprintf("SELECT FROM table WHERE column LIKE '%%s%'",
mysql_real_escape_string($term));
In this case, if the user types a % as part of $term it's reasonable to assume that they want to actually search for a literal %. Therefore in such cases you should escape % by replacing it with \% (\ is the default escape character). str_replace or strtr are two good options for this.
You may write your own function ;) See this thread for more information.
Else you may use the PDO library or any other such libraries.
So I know about MySQL injection and always escape all my user input before putting it in my database. However I was wondering, imagine a user tries to submit a query to inject, and I escape it. What if I then at a later moment take this value from the database, and use it in a query. Do I have to escape it again?
So: (sql::escape() contains my escape function)
$userinput = "'); DROP `table` --";
mysql_query("INSERT INTO `table`
(`foo`,`bar`)
VALUES
('foobar','".sql::escape($userinput)."')");
// insert php/mysql to fetch `table`.`bar` into $output here
mysql_query("INSERT INTO `table2`
(`foo`,`bar`)
VALUES
('foobar','".$output."')");
Does MySQL automatically escape their output or something like that, or should I escape in the second query as well?
This is a testcase but this occurs in some other ways within my program and I'm wondering how tight the security has to be for cases like this.
EDIT
My escape function
static function escape($string){
if(get_magic_quotes_gpc())
$string = stripslashes($string);
return mysql_real_escape_string($string);
}
Does MySQL automatically escape their output or something like that, or should I escape in the second query as well?
You need to escape in the second query as well. MySQL does not do any escaping on its output.
Long answer: MySQL string escaping does not modify the string that is being inserted, it just makes sure it doesn't do any harm in the current query. Any SQL injection attempt still remains in the data.
Yes, you have to escape the string in the second query too.
Escaping the string sounds magical to many people, something like shield against some mysterious danger, but in fact it is nothing magical. It is just the way to enable special characters being processed by the query.
The best would be just to have a look what escaping really does. Say the input string is:
'); DROP `table` --
after escaping:
\'); DROP `table` --
in fact it escaped only the single slash. That's the only thing you need to assure - that when you insert the string in the query, the syntax will be OK!
insert into table set column = '\'); DROP `table` --'
It's nothing magical like danger shield or something, it is just to ensure that the resultant query has the right syntax! (of course if it doesn't, it can be exploited)
The query parser then looks at the \' sequence and knows that it is still the variable, not ending of its value. It will remove the backslash and the following will be stored in the database:
'); DROP `table` --
which is exactly the same value as user entered. And which is exactly what you wanted to have in the database!!
So this means that the if you fetch that string from the database and want to use it in the query again, you need to escape it again to be sure that the resultant query has the right syntax.
But, in your example, very important thing to mention is the magic_quotes_gpc directive!
This feature escapes all the user input automatically (gpc - _GET, _POST and _COOKIE). This is an evil feature made for people not aware of sql injection. It is evil for two reasons. First reason is that then you have to distinguish the case of your first and second query - in the first you don't escape and in the second you do. What most people do is to either switch the "feature" off (I prefer this solution) or unescape the user input at first and then escape it again when needed. The unescape code could look like:
function stripslashes_deep($value)
{
return is_array($value) ?
array_map('stripslashes_deep', $value) :
stripslashes($value);
}
if (get_magic_quotes_gpc()) {
$_POST = stripslashes_deep($_POST);
$_GET = stripslashes_deep($_GET);
$_COOKIE = stripslashes_deep($_COOKIE);
}
The second reason why this is evil is because there is nothing like "universal quoting".
When quoting, you always quote text for some particular output, like:
string value for mysql query
like expression for mysql query
html code
json
mysql regular expression
php regular expression
For each case, you need different quoting, because each usage is present within different syntax context. This also implies that the quoting shouldn't be made at the input into PHP, but at the particular output! Which is the reason why features like magic_quotes_gpc are broken (never forget to handle it, or better, assure it is switched off!!!).
So, what methods would one use for quoting in these particular cases? (Feel free to correct me, there might be more modern methods, but these are working for me)
mysql_real_escape_string($str)
mysql_real_escape_string(addcslashes($str, "%_"))
htmlspecialchars($str)
json_encode() - only for utf8! I use my function for iso-8859-2
mysql_real_escape_string(addcslashes($str, '^.[]$()|*+?{}')) - you cannot use preg_quote in this case because backslash would be escaped two times!
preg_quote()
I'd say that whole idea of this question is wrong.
You're taking this problem absolutely wrong way.
One doesn't have to count his queries, if it's first or second or 100th.
Same goes for the the user input: it doesn't matter, where the data come from!
Data destination, not source should be your concern. Is this string going to database? Escape it! With no questions. This rule is plain and simple and require no query counting or anything.
But that's not only fault in your question.
One:
Does MySQL automatically escape their output or something like that?
That's a very bad idea. Funny part, you're fighting with a consequence of the same idea in your code, by applying get_magic_quotes_gpc(). What are these magic quotes if not such automatic escaping?
Two:
moreover, using get_magic_quotes_gpc() in your escaping function is a very bad idea again :)
imagine you have magic quotes on and using your function to protect your "second query". And there is some blob that contain \' sequence in the data. Your function will strip the slash and spoil the data. In fact, stripslashes has absolutely nothing to do with any escaping function. do it separately, on the data where it belongs - on the user input.
Three:
mysql_real_escape_string() is not some magic function that "makes everything safe". In fact, to create dynamic mysql query, one have to escape four kinds of data:
strings
numbers
identifiers
operators
while mysql_real_escape_string() escaping only one of them. And your query stand absolutely naked in all three other cases. Funny, eh?
Most disappointing part:
I know that all this ultimate knowledge is in vain and would be read scarcely by few noobs and never change either overall knowledge level of PHP community in general, nor answers quality on SO in particular. :(
Try to use PHP's PDO for database access if you can. There are two important reasons for this:
You can use PDO's prepare function to compile your query. This is efficient if you need to issue the same query with different input (as is often the case). So, compile once and execute multiple times.
Compiling the query with prepare has other nice effects. Once the query is compiled, the database engine knows the exact syntactic structure of the query, and does not allow any input that changes this syntactic structure. This is good because in SQL injection, the injected input changes the syntax of the query.
Warning: This doesn't prevent all kinds of SQL injection, but it prevents the most common kind.
References:
Are PDO prepared statements sufficient to prevent SQL injection?
http://php.net/manual/en/pdo.prepare.php
I'm new to PHP, and not yet familiar with how this works.
If I use mysqli_real_escape_string() and parameterize every variable in the SQL query, can I spare myself doing the validation with is_numeric(), etc.?
What is the best way to create a injection detection system? Simply validate the user's input with regex stuff and save it in the database?
Parametrizing your query is enough, you don't need anything else. You'll input the stuff they "inject" als a string, and there is nothing especially wrong with sql in a database... I suspect SO's database is full of SQL ;)
Escaping a value for MySQL simply adds a backslash in front of of the following characters: NULL \n \r \ ' " and 0x1A. It does nothing else.
An "escaped" value isn't magically safe for use in SQL. Escaping prevents special characters (such as quotes) from being misinterpreted. But if you insert an escaped value into an SQL query without surrounding it in quotes, then you've completely missed the point, and you are no more safe than you would have otherwise been.
Remember, the security procedure is quote and escape. You should always use those two together, since neither is safe without the other.
Also, if you can absolutely guarantee that your input value contains none of the above characters, then you also can be certain that your escaped string will always be identical to the unescaped one, and therefore escaping serves no additional purpose under those conditions.
But More Importantly:
Finally we have, in retrospect, realized that SQL was poorly designed from a security perspective, and relying on users properly quote and escape their data is just a really bad idea.
Parameterized queries are the solution, since it safely separates the data from the control structure. Parameterized queries are possible with mysqli by using prepare() followed by bind_param(). No escaping is necessary when using this method, and you can be confident that you are absolutely immune from SQL injection attacks.
Even if you have protected your variables against injection by using a parameterized query or mysql_real_escape_string() (not mysql_escape_string()), you should still validate them on the server side to ensure that they match the expected type of input. That way, if they do not, you can return an error message to your user with a request to retry those form fields.
If you use a parameterized query, such as offered by MySQLi as prepared statements, you needn't also escape the strings. However, if you don't use a parameterized query, it is essential to call mysql_real_escape_string() on every input parameter received by PHP.
This is a good question, and I think one of the best ways to avoid SQL injection is to learn about proper use of prepared statements
These will really help out the security of your application.
The issue with sql injection is the user inserting SQL data into a command and not validating that the data meets your business rules.
Making sure all user data is passed through mysqli_real_escape_string or used prepared statements is the best way to avoid problems.
On the PDO::Prepare page it states,
"and helps to prevent SQL injection attacks by eliminating the need to manually quote the parameters"
Knowing this, is there a PHP function like mysql_real_escape_string() that takes care of escaping stings for PDO? Or does PDO take care of all escaping for me?
EDIT
I realize now that I asked the wrong question. My question really was, "What all does PDO take care of for me?" Which I realize now with these answers that it really only removes the need to escape the quotes. But I would still need to do any other PHP sanitize calls on the values that I pass to the execute function. Such as htmlentities(), strip_tags()...etc...
PDO does not escape the variables. The variables and the SQL command are transferred independently over the MySQL connection. And the SQL tokenizer (parser) never looks at the values. Values are just copied verbatim into the database storage without the possibility of ever causing any harm. That's why there is no need to marshall the data with prepared statements.
Note that this is mostly a speed advantage. With mysql_real_escape_string() you first marshall your variables in PHP, then send an inefficient SQL command to the server, which has to costly segregate the actual SQL command from the values again. That's why it's often said that the security advantage is only implicit, not the primary reason for using PDO.
If you concat the SQL command and don't actually use prepared statments (not good!), then yes, there still is an escape function for PDO: $pdo->quote($string)
Very few people here understand what escaping is and when to use it.
Escaping itself does not make any data "safe". It just escapes delimiters, to distinguish a delimiter from a part of data. field = 'it's me' will cause an error, while field = 'it\'s me' will not. That's the only purpose of escaping. So, it works only when you use quotes. If you don't - escaping becomes useless.
Do you use quotes with placeholders? No. Thus, no escaping would be sensible.
When you are binding your variables, it works a very different way: it does not send the whole query to the server, but sends your prepared query separated from the bound data. So it cannot interfere. And thus makes no injection possible.
Yes and no:
Literals which you embed into the statement string need to be escaped as normal.
Values which you bind to the prepared statement are handled by the library.
If you prepare a statement and use bindParam or bindValue to supply variables, you do not need to escape the variables. Note that these functions assume that the variable contains a string, so use the third parameter to bindValue if you want to use booleans or floats.
You don't have to worry about it. PDO does not require you to escape your data before passing it along to the database.
Edit: Just to be clear, I mean to say that as long as you are passing variables into your parameters (for example, the value of a form field), you don't have to worry about it. However, if you're passing variables that you've defined as strings, for example, then obviously you need to escape anything that needs escaping in that string in order to avoid breaking syntax. However, this wouldn't even make much sense since one of the main advantages of PDO is that you're passing information from the user to the database without having to sanitize it yourself, and there aren't many times (if any?) that you would be passing strings that you yourself had defined.
Also, make sure you still sanitize your data for type. For example, make sure it's an integer if you expect it to be, make sure it's less than or greater than x if you expect it to be, etc.