I'm developing my personal web site using php. everything is ok but I just read mysql_real_escape_string manual in php.net and found two things:
This function must always (with few exceptions) be used to make data safe before sending a query to MySQL.
mysql_real_escape_string() does not escape % and _. These are wildcards in MySQL if combined with LIKE, GRANT, or REVOKE.
I have two questions:
1-what are these exceptions?
2- how to escape those characters?
This function must always (with few exceptions) be used to make data safe before sending a query to MySQL.
To my great disappointment, the manual page says complete rubbish, and they refuse to make it correct.
So, quite contrary, there are only few cases when you need this function. So to say ONLY ONE: when you are adding a string into SQL query.
mysql_real_escape_string() does not escape % and _. These are wildcards in MySQL if combined with LIKE, GRANT, or REVOKE.
It doesn't matter too much. As long as you are using LIKE operator on purpose, these characters won't do any harm.
But if you want to escape the string going to LIKE statement, you can use this code
$like = addCslashes($like,'\%_');
(note the slash - it is also required to be escaped as manual stating it. also note the C letter in the function name).
After this procedure you may use the resulting $like variable whatever way you are using to build your queries - either quote and escape them or use in the prepared statement.
I 'm not sure what exceptions the manual is referring to when talking about making data safe. You could say that the exception is when the data is already known to be safe. For example, here are a few cases that come to mind:
the data is typed as a number (this is really a specialization of the next item)
you already know it does not contain any characters that need to be escaped (e.g. it comes from looking up something in a "whitelist" array that contains a few options you hardcoded)
For example, if you have $id = intval($_GET['id']) then you do not need to escape $id before injecting it into a query.
However! It can never hurt you to escape all input, and doing so eliminates the chance that you introduce vulnerabilities in your code (e.g. if you forget to escape, if the requirements change, or anything really). So I recommend getting into the habit of escaping everything and forgetting about "exceptions".
As for the % and _ characters as part of the input, these do not need to be escaped unless you are going to feed this input to a command that recognizes them. So for example, if you have a query like this:
$term = $_GET['term'];
$sql = sprintf("SELECT FROM table WHERE column LIKE '%%s%'",
mysql_real_escape_string($term));
In this case, if the user types a % as part of $term it's reasonable to assume that they want to actually search for a literal %. Therefore in such cases you should escape % by replacing it with \% (\ is the default escape character). str_replace or strtr are two good options for this.
You may write your own function ;) See this thread for more information.
Else you may use the PDO library or any other such libraries.
Related
There is a (ugly)project I was asked to help with by doing a specific task.
The thing is, I don't have any control over the other files in the site much less over the server configuration. Some data I'm going to use comes from a query like this:
'SELECT * FROM table where value like "'.$unsafe.'"';
$unsafe is an unescaped value coming from $_POST or $_GET.
I checked the server, is PHP5.1.6 and has magic_quotes_gpc On so the data is being auto escaped.
Is this query breakable? Being $unsafe between colons gives me the impression It cant be broken but maybe I'm missing something.
I know magic_quotes_gpc is deprecated because of its insecurity so I'm concerned about it, not because of the application security which fails every where but for my own knowledge.
EDIT: I'm aware of the security implications of *magic_quotes_gpc* and I never use it in my own projects. I always use parameterized queries to avoid injection but this time I was asked to add a very specific pice of code in a friend/client project, so I cant change what is already done. I'd like to know if there is a specific value I can use to create an injection so I can illustrate my friend why he should change it.
if the DB is mysql use mysqli_real_escape_string() instead, if the PHP version is very old you can use the mysql_real_escape_string (not recommended at the moment).
even if the variable is between colons it can be injected, you just need to close the colons inside the value of the variable and then inject whatever you want afterwards.
With regard to your edit: You asked "I'd like to know if there is a specific value I can use to create an injection so I can illustrate my friend why he should change it."
According to the manual page for mysqli_real_escape_string(), the characters it escapes are as follows:
NUL (ASCII 0), \n, \r, \, ', ", and Control-Z.
The old mysql_real_escape_string() function also escapes the same characters.
This gives you a starting point as to which characters can be used to do injection attacks in MySQL. Magic quotes only escapes the quote characters and the slash character, which clearly leaves several gaping holes that can be exploited.
In an easy world, the above information would be enough for us to fix the escaping by doing a string replace on the remaining unescaped characters.
However, both the real_escape functions also require an active database connection for them to work, and this leads us to a further complication: character sets.
Further attacks are possible if the database has a different character set to PHP, particularly with variable-length character sets such as UTF-8 or UTF-16.
An attacker who knows (or can guess) the character set that PHP and the DB are using can send a crafted injection attack string that contains characters that PHP would not see as needing escaping, but which would still cause succeed in hacking MySQL. This is why the real_escape functions need to access the DB in order to know how to do the escaping.
Further resources:
Php & Sql Injection - UTF8 POC
SQL injection that gets around mysql_real_escape_string()
I hope that gives you a few pointers.
I am working on a portal and I have these few questions regarding saving data in MySQL tables :
Should I save varchar field escaped ?
i'm using now mysql_real_escape_string() for avoiding string-injection.
Why should I save them unescaped (this was proposed by a guy on this website) and how would that work for characters like single and double-quotes. Doesn't it wreck the SQL command ?
easy talking around this topic.
And one last thing....I was using addslashes and stripslashes before using mysql_real_escape_string and it worked for me (of course, with mysql-injection of malicious code chance, which I recently discovered and documented myself on it)...
thanks
The very basic thing any programmer must learn is the meaning of context.
What am I going about here? If you knew the meaning of context, you wouldn't have asked this question. Now that (I hope) you know, you won't ask how to show <test> as HTML, or how to pass a variable to javascript.
So what's it all about? It's really easy. Context is the simple fact that something in a system may mean something entirely different somewhere else.
For example, in your case, a PHP string may mean something entirely different to MySQL. You can't just pass the string and expect everything to run smoothly - it won't.
So, now that you know what context means, you need to know something else that is important. You always need to convert a value from the older context to the newer one. Always.
Again, in your case, it's mysql_real_escape_string(), but a word of warning; conversion functions are context specific, so, for example, you can't use mysql_real_escape_string() to pass a string from PHP to Javascript. Similarly, you can't just use addslashes() and expect it to work. In fact, I'd argue that addslashes() is a completely useless and misleading function. Do NOT use it unless you are very sure of what you are doing.
Should I save varchar field escaped ?
No. You should escape data so that characters (in the data) with special meaning in SQL won't cause you problems.
Once it passes through SQL and gets stored in the database, it won't be escaped any longer.
i'm using now mysql_real_escape_string() for avoiding string-injection.
Don't do that, instead use prepared statements and parameterized queries
I was using addslashes and stripslashes
addslashes is a basic form of escaping. It is pointless unless you know exactly what the target of the data is. You should use something more specific where such a thing exists (and you are – mysql_real_escape_string)
stripslashes does the opposite of addslashes. Using them together is utterly pointless.
I've heard people say (in relation to C#/SQL Server, but also in relation to PHP/MySql): Don't escape strings manually - use stored procedures instead.
Ok, I can accept this suggestion, but why? Many say (including on SO) mysql_real_escape_string() is quite enough, mysql_real_escape_string() is good, mysql_real_escape_string() is the first way of protection.
Why? Is there a case where mysql_real_escape_string() can fail? At least one... I don't need many :)
When mysql_real_escape_string FAIL:
$sql = "SELECT * FROM users WHERE id=" + mysql_real_escape_string($_GET['id']);
If $_GET['user_id'] is set to 1 OR 1=1, there are no special chars and it's not filtered.
The result: All rows are returned.
It gets worse. How about this... what if $_GET['user_id'] is set to 1 OR is_admin = 1?
The function is only designed to be used when inside single quotes.
There are two things that can go wrong with mysql_real_escape_string:
You can forget to use it
If you are using some specific multibyte connection encodings, and you have set these encodings with SET NAMES instead of mysql_set_charset as is proper, it can still leave you vulnerable
Update:
You are safe with UTF-8 (although this does not mean that you should continue using SET NAMES!)
For an explanation, see here
Just for info:
mysql_real_escape_string() does not escape % and _. These are wildcards in MySQL if combined with LIKE, GRANT, or REVOKE.
mysql_real_escape_string() can fail to clean the input.
Since mysql_real_esacpe_string() takes character set into account while cleaning strings.
There's the problem. You can change character via mysql_query function sending the query to change connection's character set. However, mysql_real_escape_string() is oblivious to the set you're using and it will escape some characters improperly.
The other thing is constantly invoking it manually. Even wrapping it in a function is a P.I.T.A. because it implies you have to create some sort of database wrapper / database abstraction layer of your own in order to be able to automate calls to mysql_real_escape_string().
That's why we have PDO in PHP which helps us alleviate all of the above and you get to use parametrized prepared statements which are preferable way of dealing with repeating queries that alter the data.
Prepare the statement, bind input variables and it will clean the input according to the database driver being used and connection's character set. It's less code and completely safe.
There is only one thing about mysql_real_escape_string() and SQL Injection -
The former does not have a slightest relation to the latter.
Although it sounds paradoxical, nothing can be more true.
Here are 2 statements proving it
You have to escape quoted strings only, as this function helps nothing else.
In fact, you have to escape every string you are adding to the query, even safiest one. just because it may contain some special character and thus break the query (just as an accident, not malicious plot).
Thus, when applicable, this function have to be used anyway, despite of whatever dangers or fears. And in any other case it will help nothing.
The only thing I have to add is that prepared statements do not provide full protection either. Here is the explanation and recommendations: https://stackoverflow.com/a/8255054/285587
So I know about MySQL injection and always escape all my user input before putting it in my database. However I was wondering, imagine a user tries to submit a query to inject, and I escape it. What if I then at a later moment take this value from the database, and use it in a query. Do I have to escape it again?
So: (sql::escape() contains my escape function)
$userinput = "'); DROP `table` --";
mysql_query("INSERT INTO `table`
(`foo`,`bar`)
VALUES
('foobar','".sql::escape($userinput)."')");
// insert php/mysql to fetch `table`.`bar` into $output here
mysql_query("INSERT INTO `table2`
(`foo`,`bar`)
VALUES
('foobar','".$output."')");
Does MySQL automatically escape their output or something like that, or should I escape in the second query as well?
This is a testcase but this occurs in some other ways within my program and I'm wondering how tight the security has to be for cases like this.
EDIT
My escape function
static function escape($string){
if(get_magic_quotes_gpc())
$string = stripslashes($string);
return mysql_real_escape_string($string);
}
Does MySQL automatically escape their output or something like that, or should I escape in the second query as well?
You need to escape in the second query as well. MySQL does not do any escaping on its output.
Long answer: MySQL string escaping does not modify the string that is being inserted, it just makes sure it doesn't do any harm in the current query. Any SQL injection attempt still remains in the data.
Yes, you have to escape the string in the second query too.
Escaping the string sounds magical to many people, something like shield against some mysterious danger, but in fact it is nothing magical. It is just the way to enable special characters being processed by the query.
The best would be just to have a look what escaping really does. Say the input string is:
'); DROP `table` --
after escaping:
\'); DROP `table` --
in fact it escaped only the single slash. That's the only thing you need to assure - that when you insert the string in the query, the syntax will be OK!
insert into table set column = '\'); DROP `table` --'
It's nothing magical like danger shield or something, it is just to ensure that the resultant query has the right syntax! (of course if it doesn't, it can be exploited)
The query parser then looks at the \' sequence and knows that it is still the variable, not ending of its value. It will remove the backslash and the following will be stored in the database:
'); DROP `table` --
which is exactly the same value as user entered. And which is exactly what you wanted to have in the database!!
So this means that the if you fetch that string from the database and want to use it in the query again, you need to escape it again to be sure that the resultant query has the right syntax.
But, in your example, very important thing to mention is the magic_quotes_gpc directive!
This feature escapes all the user input automatically (gpc - _GET, _POST and _COOKIE). This is an evil feature made for people not aware of sql injection. It is evil for two reasons. First reason is that then you have to distinguish the case of your first and second query - in the first you don't escape and in the second you do. What most people do is to either switch the "feature" off (I prefer this solution) or unescape the user input at first and then escape it again when needed. The unescape code could look like:
function stripslashes_deep($value)
{
return is_array($value) ?
array_map('stripslashes_deep', $value) :
stripslashes($value);
}
if (get_magic_quotes_gpc()) {
$_POST = stripslashes_deep($_POST);
$_GET = stripslashes_deep($_GET);
$_COOKIE = stripslashes_deep($_COOKIE);
}
The second reason why this is evil is because there is nothing like "universal quoting".
When quoting, you always quote text for some particular output, like:
string value for mysql query
like expression for mysql query
html code
json
mysql regular expression
php regular expression
For each case, you need different quoting, because each usage is present within different syntax context. This also implies that the quoting shouldn't be made at the input into PHP, but at the particular output! Which is the reason why features like magic_quotes_gpc are broken (never forget to handle it, or better, assure it is switched off!!!).
So, what methods would one use for quoting in these particular cases? (Feel free to correct me, there might be more modern methods, but these are working for me)
mysql_real_escape_string($str)
mysql_real_escape_string(addcslashes($str, "%_"))
htmlspecialchars($str)
json_encode() - only for utf8! I use my function for iso-8859-2
mysql_real_escape_string(addcslashes($str, '^.[]$()|*+?{}')) - you cannot use preg_quote in this case because backslash would be escaped two times!
preg_quote()
I'd say that whole idea of this question is wrong.
You're taking this problem absolutely wrong way.
One doesn't have to count his queries, if it's first or second or 100th.
Same goes for the the user input: it doesn't matter, where the data come from!
Data destination, not source should be your concern. Is this string going to database? Escape it! With no questions. This rule is plain and simple and require no query counting or anything.
But that's not only fault in your question.
One:
Does MySQL automatically escape their output or something like that?
That's a very bad idea. Funny part, you're fighting with a consequence of the same idea in your code, by applying get_magic_quotes_gpc(). What are these magic quotes if not such automatic escaping?
Two:
moreover, using get_magic_quotes_gpc() in your escaping function is a very bad idea again :)
imagine you have magic quotes on and using your function to protect your "second query". And there is some blob that contain \' sequence in the data. Your function will strip the slash and spoil the data. In fact, stripslashes has absolutely nothing to do with any escaping function. do it separately, on the data where it belongs - on the user input.
Three:
mysql_real_escape_string() is not some magic function that "makes everything safe". In fact, to create dynamic mysql query, one have to escape four kinds of data:
strings
numbers
identifiers
operators
while mysql_real_escape_string() escaping only one of them. And your query stand absolutely naked in all three other cases. Funny, eh?
Most disappointing part:
I know that all this ultimate knowledge is in vain and would be read scarcely by few noobs and never change either overall knowledge level of PHP community in general, nor answers quality on SO in particular. :(
Try to use PHP's PDO for database access if you can. There are two important reasons for this:
You can use PDO's prepare function to compile your query. This is efficient if you need to issue the same query with different input (as is often the case). So, compile once and execute multiple times.
Compiling the query with prepare has other nice effects. Once the query is compiled, the database engine knows the exact syntactic structure of the query, and does not allow any input that changes this syntactic structure. This is good because in SQL injection, the injected input changes the syntax of the query.
Warning: This doesn't prevent all kinds of SQL injection, but it prevents the most common kind.
References:
Are PDO prepared statements sufficient to prevent SQL injection?
http://php.net/manual/en/pdo.prepare.php
How can I secure $_REQUEST before inserting to mysql database? For example following:
$message = $message = $_REQUEST['message'];
$tags = $_REQUEST['item']['tags'];
Thanks.
Depends on what you mean by "secure", and how you intend to insert the data. $_REQUEST isn't broken or anything; it's just that the data in it can be just about anything, so you'll need to "sanitize" it before you use it.
For example, if 'some_id' should only ever be an int,
$some_id = intval($_REQUEST['some_id']);
will ensure that $some_id is always an int. (Even if it didn't exist in $_REQUEST! In which case it will be 0.)
If you use prepared statements, a lot of the issues with $_REQUEST data go away -- that is, extensions like PDO and mysqli will escape parameters for you (if you use placeholders, like all good prepared statements should!), so all you have to do is make sure the data is valid. (For example, above, it'd have been a good idea to make sure $_REQUEST['some_id'] was set first -- since we didn't, we got a 0 back, which may not be valid.)
If you don't use prepared statements, then you have a little more work ahead of you. You'll need to use mysql_real_escape_string to escape strings as you feed them into the database, like so:
$some_string_sql = mysql_real_escape_string($_REQUEST['some_string']);
$id = intval($_REQUEST['id']);
mysql_query("UPDATE stuff SET some_string = '$some_string_sql' WHERE id = $id");
Note that i did this just for the query! Too many PHP noobs think they can just apply some magic formula to everything in $_REQUEST at the beginning of their script to make everything safe. You kinda can, if you're always just feeding it directly into an SQL query -- but it trashes your data if you're using it for other stuff! For example, if you write the data to a file as well, blindly escaping the data will leave you with a bunch of ugly backslashes in your file. You should never have to *un*escape your data -- it should always be escaped as you need it, for the specific purpose you intend to use it. (htmlentities for arbitrary data being printed to the screen, mysql_real_escape_string for stuff going into an SQL query.)
Also note: If you have magic_quotes_gpc enabled on your site, disable it for the reasons mentioned in the previous paragraph. Properly escaped stuff will break in the presence of magic quotes, because it's already been "escaped" once (half-assedly, hence the quotes) by PHP! Fortunately this misfeature will be removed from PHP 6, if it ever ships. But til then, if you have magic quotes enabled, you'll need to stripslashes(anything from $_REQUEST, $_GET, $_POST, or $_COOKIE) before you can properly escape it. DO NOT rely on the magic quotes -- they're a convenience thing, and not at all designed for security.
You should just not forget escaping your data when injecting them in some SQL queries.
Either use a function to escape the data :
Depending on the API you're working with :
mysql_real_escape_string,
mysqli_real_escape_string,
or PDO::quote
Or you could use Prepared Statements :
Those might seem a bit harder to understand, at first -- but they are worth investing sometime...
With mysqli,
And with PDO.
Then, of course, when using the data from the database to generate some output, the same idea applies : escape the output.
If you are generating some HTML output, you'll typically want to use something like htmlspecialchars.
Or, to allow some specific HTML tags, see HTML Purifier.
If you are generating some other kind of output, you'll have to find how to escape your data specifically for this type of output.
Use either mysql_real_escape_string() or PDO's prepared statements. I recommend the latter, as it also helps keep your queries nice and tidy.
As secure as anything could be in them.
The user can change those values to whatever they want.
So, not secure at all. Always sanitize your inputs.
There is nothing to secure.
Your database input should be just properly formatted.
For the strings it's quoting and escaping.
As long as your input data is limited to strings and you follow formatting rules, no special security required.