I have a database class that automatically escape input strings before building the query with mysqli_real_escape_string().
But it might be that in the script, a string is escaped and then passed to database class that escape it again. Could that be wrong? What could happen?
Since you have no good way of telling on the other end (when you're pulling data out of the database) how many times it has been escaped, inconsistently double-escaping a string will wind up with you having things escaped in your end data that weren't before - because you'll add two layers of escaping but only unescape one.
Basically, you need to have a constant level of escaping - either always escape things once (and unescape them once) or always escape them twice (and unescape them twice) and so on - but never mixed.
On the first pass through mysqli_real_escape_string, the following characters are escaped by inserting a \ before each of the dangerous characters:
NUL (ASCII 0), \n, \r, \, ', ", and Control-Z:
NUL (chr(0)) becomes "\0" (chr(92).chr(48))
\n (chr(13)) becomes "\n" (chr(92).chr(110))
\r (chr(10)) becomes "\r" (chr(92).chr(114))
\ (chr(92)) becomes "\\" (chr(92).chr(92))
' (chr(39)) becomes "\'" (chr(92).chr(39))
" (chr(34)) becomes "\"" (chr(92).chr(34))
Control-Z (chr(26)) becomes "\Z" (chr(92).chr(90))
On the second pass through mysqli_real_escape_string, the \ is escaped again:
"\0" (chr(92).chr(48)) becomes "\\0" (chr(92).chr(92).chr(48))
"\n" (chr(92).chr(110)) becomes "\\n" (chr(92).chr(92).chr(110))
"\r" (chr(92).chr(114)) becomes "\\r" (chr(92).chr(92).chr(114))
"\\" (chr(92).chr(92)) becomes "\\\\" (chr(92).chr(92).chr(92).chr(92))
"\'" (chr(92).chr(39)) becomes "\\'" (chr(92).chr(92).chr(39))
"\"" (chr(92).chr(34)) becomes "\\"" (chr(92).chr(92).chr(34))
"\Z" (chr(92).chr(90)) becomes "\\Z" (chr(92).chr(92).chr(90))
Double-escaping your strings does not create any kind of vulnerability, but it will insert many extra "\" characters into the strings you are saving to in the database.
The best way to do escaping is to:
1) Turn off magic quotes
2) Use queries with named parameters only, and do not escape anything before passing it into the query. MySQL (and all other database vendors for that matter) will properly escape the strings. (However, you may run into an issue with chr(0) terminating a string).
If you absolutely have to use string queries, escape the data once, and only once, just before it is inserted into the query. Do not escape the entire query.
There is nothing wrong with double-escaping a string as long as you have a guarantee of double-unescaping it before usage.
If you forget to double-unescape it, then the user will end up reading an escaped string, which is not very nice.
A doubly-escaped string will render incorrectly unless you account for this. You've probably seen examples of text occasionally rendering in an escaped form (with extra backslashes or HTML entities, for instance).
Yes it could be wrong, because you can have things like let\'s go in your database.
You are escaping to avoid SQL Injection or query syntax error, not to "transform" the data you want to store. If you want to store let's go in the database, you should escape it only one time when building the query. You should not have to "unescape" a value that come from database. When you look into your database you should never see a value with escape characters.
Escaping is a real problem for beginners in PHP, that's perhaps due to magic_quote() function. You should take a look at http://php.net/manual/en/security.magicquotes.php You should not use that feature, it's confusing. In your script, if you can't modify the php.ini to disable this feature you can do it at runtime with :
if (get_magic_quotes_gpc()) {
$process = array(&$_GET, &$_POST, &$_COOKIE, &$_REQUEST);
while (list($key, $val) = each($process)) {
foreach ($val as $k => $v) {
unset($process[$key][$k]);
if (is_array($v)) {
$process[$key][stripslashes($k)] = $v;
$process[] = &$process[$key][stripslashes($k)];
} else {
$process[$key][stripslashes($k)] = stripslashes($v);
}
}
}
unset($process);
}
Note this code come from http://www.php.net/manual/en/security.magicquotes.disabling.php
Take your time to understand correctly how you handle escaping, you will save a lot of time in the future.
For MySQL use prepared statements and you won't need to bother with escaping the string at the database level.
And for PHP remember that you can use both " and ' to build strings. This you can use to avoid having to quote strings. If you strings starts with " then you don't have to quote your '.
Related
For example I want to store the String "That's all". MySQL automatically escapes the ' character. How do I echo that String from the database using php but remove the \ in front of escaped characters like \' ? I would also like to preserve other formatting like new lines and blank spaces.
Have you tried stripslashes(), regarding the linebreaks just use the nl2br() function.
Example:
$yourString = "That\'s all\n folks";
$yourString = stripslashes(nl2br($yourString));
echo $yourString;
Note: \\ double slashes will turn to \ single slashes
You should probably setup your own function, something like:
$yourString = "That\'s all\n folks";
function escapeString($string) {
return stripslashes(nl2br($string));
}
echo escapeString($yourString);
There are also several good examples in the nl2br() docs
Edit 2
The reason your are seeing these is because mysql is escaping line breaks, etc. I am guessing you are using mysql_* functions. You should probably look into mysqli or PDO.
Here is an example:
$yourString = "That's all
folks";
echo mysql_escape_string($yourString);
Outputs:
That\'s all\r\n folks
If you use prepared statements, those characters will not be escaped on insert.
Use stripslashes() to remove slashes if you cannot avoid adding slashes on input.
At first, magic_quotes_gpc escapes the character like ' or ". You can also disable this in your php.ini. But then you should escape the things yourself that no query can get "infected".
Lookup mysql injection for more information.
When the escaped string is been written in your database. The string doesn't contain theses escape charakters and when you output them again. You should see the result as you want it.
Me for myself prefer the method by storing everything without escapes and escape or display things when I output them. You could also easily use an str_replace("\n", "", $text) to prevent newslines are displayed.
Greetings MRu
I was reading Does $_SESSION['username'] need to be escaped before getting into an SQL query? and it said "You need to escape every string you pass to the sql query, regardless of its origin". Now I know something like this is really basic. A Google search turned up over 20, 000 results. Stackoverflow alone had 20 pages of results but no one actually explains what escaping a string is or how to do it. It is just assumed. Can you help me? I want to learn because as always I am making a web app in PHP.
I have looked at:
Inserting Escape Characters, What are all the escape characters in Java?,
Cant escape a string with addcslashes(),
Escape character,
what does mysql_real_escape_string() really do?,
How can i escape double quotes from a string in php?,
MySQL_real_escape_string not adding slashes?,
remove escape sequences from string in php I could go on but I am sure you get the point. This is not laziness.
Escaping a string means to reduce ambiguity in quotes (and other characters) used in that string. For instance, when you're defining a string, you typically surround it in either double quotes or single quotes:
"Hello World."
But what if my string had double quotes within it?
"Hello "World.""
Now I have ambiguity - the interpreter doesn't know where my string ends. If I want to keep my double quotes, I have a couple options. I could use single quotes around my string:
'Hello "World."'
Or I can escape my quotes:
"Hello \"World.\""
Any quote that is preceded by a slash is escaped, and understood to be part of the value of the string.
When it comes to queries, MySQL has certain keywords it watches for that we cannot use in our queries without causing some confusion. Suppose we had a table of values where a column was named "Select", and we wanted to select that:
SELECT select FROM myTable
We've now introduced some ambiguity into our query. Within our query, we can reduce that ambiguity by using back-ticks:
SELECT `select` FROM myTable
This removes the confusion we've introduced by using poor judgment in selecting field names.
A lot of this can be handled for you by simply passing your values through mysql_real_escape_string(). In the example below you can see that we're passing user-submitted data through this function to ensure it won't cause any problems for our query:
// Query
$query = sprintf("SELECT * FROM users WHERE user='%s' AND password='%s'",
mysql_real_escape_string($user),
mysql_real_escape_string($password));
Other methods exist for escaping strings, such as add_slashes, addcslashes, quotemeta, and more, though you'll find that when the goal is to run a safe query, by and large developers prefer mysql_real_escape_string or pg_escape_string (in the context of PostgreSQL.
Some characters have special meaning to the SQL database you are using. When these characters are being used in a query they can cause unexpected and/or unintended behavior including allowing an attacker to compromise your database. To prevent these characters from affecting a query in this way they need to be escaped, or to say it a different way, the database needs to be told to not treat them as special characters in this query.
In the case of mysql_real_escape_string() it escapes \x00, \n, \r,\, ', " and \x1a as these, when not escaped, can cause the previously mentioned problems which includes SQL injections with a MySQL database.
For simplicity, you could basically imagine the backslash "\" to be a command to the interpreter during runtime.
For e.g. while interpreting this statement:
$txt = "Hello world!";
during the lexical analysis phase ( or when splitting up the statement into individual tokens) these would be the tokens identified
$, txt, =, ", Hello world!, ", and ;
However the backslash within the string will cause an extra set of tokens and is interpreted as a command to do something with the character that immediately follows it :
for e.g.
$txt = "this \" is escaped";
results in the following tokens:
$, txt, =, ", this, \, ", is escaped, ", and ;
the interpreter already knows (or has preset routes it can take) what to do based on the character that succeeds the \ token. So in the case of " it proceeds to treat it as a character and not as the end-of-string command.
This question already has answers here:
How can I sanitize user input with PHP?
(16 answers)
How can I prevent SQL injection in PHP?
(27 answers)
Closed 7 months ago.
I am trying to put a general purpose function together that will sanitize input to a Mysql database. So far this is what I have:
function sanitize($input){
if(get_magic_quotes_qpc($input)){
$input = trim($input); // get rid of white space left and right
$input = htmlentities($input); // convert symbols to html entities
return $input;
} else {
$input = htmlentities($input); // convert symbols to html entities
$input = addslashes($input); // server doesn't add slashes, so we will add them to escape ',",\,NULL
$input = mysql_real_escape_string($input); // escapes \x00, \n, \r, \, ', " and \x1a
return $input;
}
}
If i understood the definition of get_magic_quotes_qpc(). This is set by the php server to automatically escape characters instead of needing to use addslashes().
Have I used addslashes() and mysql_real_escape_string() correctly together and is there anything else I could add to increase the sanitization.
Thanks
htmlentities() is unnecessary to make data safe for SQL. It's used when echoing data values to HTML output, to avoid XSS vulnerabilities. That's also an important security issue you need to be mindful of, but it's not related to SQL.
addslashes() is redundant with mysql_real_escape_string. You'll end up with literal backslashes in your strings in the database.
Don't use magic quotes. This feature has been deprecated for many years. Don't deploy PHP code to an environment where magic quotes is enabled. If it's enabled, turn it off. If it's a hosted environment and they won't turn off magic quotes, get a new hosting provider.
Don't use ext/mysql. It doesn't support query parameters, transactions, or OO usage.
Update: ext/mysql was deprecated in PHP 5.5.0 (2013-06-20), and removed in PHP 7.0.0 (2015-12-03). You really can't use it.
Use PDO, and make your queries safer by using prepared queries.
For more details about writing safe SQL, read my presentation SQL Injection Myths and Fallacies.
Magic quotes are deprecated. Turn them off if you can :).
The second part addslashes and mysql_real_escape_String does pretty much the same (similar) thing. Just try
addslashes( '\\')
// and
mysql_real_escape_string( '\\')
Result should be \\ so if you use
mysql_real_escape_string( addslashes( '\\'))
you should get \\ (or '\\\\' as string). Use only mysql_real_escape_string (better) OR addslashes, never both.
I recommend to use PDO instead of raw functions and manual escaping.
Why do you want to apply htmlentities before saving data to the database? What if you want to use the data for something else than just writing it out to a browser? For example for searching, partitioning data, using the data in other programming languages, etc...
The only thing you really want to apply is mysql_real_escape_string (or use PDO), nothing else.
I usually prefer to undo the effects of magic quotes entirely, always. Magic quotes is just cumbersome to work with and should never have been invented. Here's a snippet from the PHP manual to reverse the magic quotes:
if (get_magic_quotes_gpc()) {
$process = array(&$_GET, &$_POST, &$_COOKIE, &$_REQUEST);
while (list($key, $val) = each($process)) {
foreach ($val as $k => $v) {
unset($process[$key][$k]);
if (is_array($v)) {
$process[$key][stripslashes($k)] = $v;
$process[] = &$process[$key][stripslashes($k)];
} else {
$process[$key][stripslashes($k)] = stripslashes($v);
}
}
}
unset($process);
}
the worst part that adding slashes does not sanitize anything, no matter what function was used.
and it should not be used in the means of whatever "sanitization" at all.
slashes do not "sanitize" data. Slashes do escape string delimiters only. Thus, the only sanitization you can talk of, is escaping and and quoting.
Otherwise, if you won't put quotes around "sanitized" string, you will have no protection at all.
Use:
mysql_real_escape_string()
This will prevent bad data like DROP TABLE ;)
I'm doing this to all strings before inserting them:
mysql_real_escape_string($_POST['position']);
How do I remove the: \ after retriving them?
So I don't end up with: \"Piza\"
Also is this enough security or should I do something else?
Thanks
I would suggest you call $_POST['position'] directly (don't call mysql_real_escape_string on it) to get the non-escaped version.
Incidentally your comment about security suggests a bit of trouble understanding things.
One way of handling strings is to handle the escaped versions, which leads to one kind of difficulty, while another is to handle another and escape strings just before embedding, which leads to another kind of difficulty. I much prefer the latter.
use stripslashes() to get rid of the escape character.
Escaping is great. In case the value is going to be integer , I would suggest you do it like:
$value = (int) $_POST['some_int_field'];
This would make sure you always end up with an integer value.
It could be because magic quotes are enabled, so to make it versatile, use this:
if (get_magic_quotes_gpc()) { // Check if magic quotes are enabled
$position = stripslashes($_POST['position']);
} else {
$position = mysql_real_escape_string($_POST['position']);
}
mysql_real_escape_string() does add \s in your SQL strings but they should not be making it into the database as they are only there for the purpose of string parsing.
If you are seeing \s in you database then something else is escaping your stings before you call mysql_real_escape_string(). Check to make sure that magic_quotes_gpc isn't turned on.
I am trying to input data using forms into the MySQL, and also using mysql_real_escape_string for this purpose. Unfortunately I am having a problem with the output. It displays \s, or if I use stripslashes then it removes all slashes.
If I submit web's forms using backslash \ I get this output:
"web\'s forms using backslash \\"
See I got a double backslash. But if I use the stripslashes function then it removes all slashes but also removes inputed slash and the output is
"web's forms using backslash"
Here, no backslash is displayed, but there should be one backslash at the end.
The problem is that if someone uses backslash in password field and any other filed, then the backslash will be stripped or displayed twice.And also please tell me what is best for displaying output htmlentities or htmlspecialchars
You have magic quotes turned on. You need to disable them altogether as they are not good in terms of security.
Set them to off from php.ini (preferred):
; Magic quotes
;
; Magic quotes for incoming GET/POST/Cookie data.
magic_quotes_gpc = Off
; Magic quotes for runtime-generated data, e.g. data from SQL, from exec(), etc.
magic_quotes_runtime = Off
; Use Sybase-style magic quotes (escape ' with '' instead of \').
magic_quotes_sybase = Off
Or you can disable them at runtime:
if (get_magic_quotes_gpc())
{
$process = array(&$_GET, &$_POST, &$_COOKIE, &$_REQUEST);
while (list($key, $val) = each($process))
{
foreach ($val as $k => $v)
{
unset($process[$key][$k]);
if (is_array($v))
{
$process[$key][stripslashes($k)] = $v;
$process[] = &$process[$key][stripslashes($k)];
}
else
{
$process[$key][stripslashes($k)] = stripslashes($v);
}
}
}
unset($process);
}
Use the mysqli library and Prepared Statements. Then all characters go in as data and you don't need to mess with all this stuff.
What characters are NOT escaped with a mysqli prepared statement?
Why is using a mysql prepared statement more secure than using the common escape functions?
Safe way to input data to mysql
Problems in inserting data using “safe
way to input data to mysql using PHP”
You should use PDO(new improved way) prepared statement which are safer and faster then there predecessors(mysql_real_escape_string, etc). Why you Should be using PHP’s PDO for Database Access goes into deeper details.
Displaying output
The problem is that if someone uses
backslash in password field and any
other filed, then the backslash will
be stripped or displayed twice.And
also please tell me what is best for
displaying output htmlentities or
htmlspecialchars.
The new and improved way is to use filter. In particular I would advise you to read all these Performance and Security slides from PHP creator Rasmus Ledorf. http://talks.php.net/ has a lot of good slides in general which you should have a look at.