Related
I have users entering their name, as in: O'riley.
Before I enter this data into the MySQL DB, I run mysql_real_escape_string.
Problem is, when I then select this data for display and use later, it comes out as: O\'riley.
Obviously, this is the intended operation. What I'm wondering is if there's someway to be sure I can store it in the DB (still safely escaping possible malicious code), so that I don't have to use strip_slashes() on the output EVERY time I call the data throughout the entire web app? Or, am I missing something here?
Thanks.
UPDATE
Please refer to the comments in Deceze's answer.
No, it's not the intended operation to store the string as "O\'riley"; it should only be escaped in the query, but not stored this way. I'll guess that PHP puts in the backslash through Magic Quotes, and you escape it again to make it stick.
Disable Magic Quotes.
I personally always turn off magic quotes because it is doing something I haven't told it to do. If you dont have the ability to turn it off, consider including this code at the top of all of your pages.
if (get_magic_quotes_gpc()) {
function strip_array($var) {
return is_array($var)? array_map("strip_array", $var):stripslashes($var);
}
$_POST = strip_array($_POST);
$_SESSION = strip_array($_SESSION);
$_GET = strip_array($_GET);
}
If you have magic quotes disabled, and in the case that get_magic_quotes_gpc returned 1 and you did something like #Michael shows and this still occurs, magic quotes runtime may be enabled.
Magic quotes runtime will add slashes to strings when doing database queries and even when writing to files. To disable this in the currently executing script, do:
if(function_exists('get_magic_quotes_runtime') && get_magic_quotes_runtime())
{
set_magic_quotes_runtime(false);
}
You can also disable this configuration option through php.ini as well, if you can do so. However, if you can't disable it through php.ini permanently, you may need to add # before get_magic_quotes_runtime and set_magic_quotes_runtime as PHP may through E_DEPRECATED errors if error_reporting is set to log such errors (and this will get fairly annoying).
Alright, here's what you need to do. First, based on the comments, from other questions:
ISOLATE
You need to determine what is causing the problem. Is it the server? Is it some hidden code somewhere? What? Some programmers may include code like this on a configuration page:
<?php if (isset($_POST))
foreach ($_POST as $key => $value) $_POST[$key] = addslashes($value);
?>
So here are some checks to see if it is the server. This is about as solid a way of checking as possible. Create a NEW page. Leave it entirely blank and add this code:
<?php print_r($_POST); ?>
<form action="" method="POST">
<input type="text" name="test" value="O'riley" />
<input type="submit" name="submit" value="submit" />
</form>
Save it and load it in the browser. Hit submit. See if the slash is added still.
If it is still adding it, you need to troubleshoot your server / configuration. If it's a server you're paying for, you need to tell them to fix it or shove it. If it's your own server, then you're going to need to do some messing / googling to figure out how to turn off magic quotes.
If there are no quotes showing up, then there is definitely some code adding slashes to your post variables. It might not be as obvious as the code above, however. You need to run a search on your code for "addslashes", "mysql_real_escape_string", and possibly "str_replace". If you find one of these, you need to disable it. But be advised this may break other parts of your site that are assuming this action is taking place. You're other option for storing it in the database is to do a function similar to the code above, but instead, run stripslashes on it. Then you can run mysql_real_escape_string on it later. (A bit more unneeded overhead.)
Use
var_dump(get_magic_quotes_gpc());
somewhere near the actual use of the posted data to see if the Magic Quotes are really off.
Also some weird frameworks may add said quotes for you so grep your code for addslashes and other escape_string-like functions.
I would honestly suggest the use of PDO.
PDO employs the use of prepare and execute statements, which in turn adds security and removes some extra headache.
$pdo = new PDO();
$stm = $pdo->prepare('INSERT... (LastName) VALUES (:LastName)');
$stm->execute(array(':LastName' => "O'Rily"));
$stm->fetchAssoc(PDO::FETCH_ASSOC);
You no longer will need to worry about removing the escaping slashes as well as securing basic sql injection tactics.
It's possible magic quotes are turned on in your httpd.conf or in a virtual host declaration.
The exact location and file will depend on OS and distro. I'd check Apache configuration files for the php_flag setting.
Just use clean the data before inserting in to database,
function check_input($value)
{
if( get_magic_quotes_gpc() )
{
$value = stripslashes( $value );
}
//check if this function exists
if( function_exists( "mysql_real_escape_string" ) )
{
$value = mysql_real_escape_string( $value );
}
//for PHP version < 4.3.0 use addslashes
else
{
$value = addslashes( $value );
}
return $value;
}
$cleandata = check_input($value);
$sql = "INSERT INTO `table_name` SET `field_name`='".$cleandata."'";
While fetching the data and show into it use stripslashes($val)
Thank you everyone for the answers. I will award the +50 out, but I wanted to tell my real solution here, all which people did help with...
I was performing mysql_real_escape_string on all of the data AS SOON as it posted (before any processing). So, a slash was added to escape the ' character that was submitted. This, we know is normal.
However, there was no reason that the backslash \ should show up in the DB entry, right? The escape was there to be sure the ' was entered.
Turns out, AFTER escaping, I would then save the variable to be reloaded to the page in the session, in case the user had an error that PHP found while validating all of the form fields. In this case, the user's input (formerly O'riley was now printed to their screen as O\'riley. Then, the user didn't catch this - so they would often just fix their error that PHP caught during validation (unrelated to the name field), and thus the O\'riley would land in the database because mysql_real_escape_string would escape the characters.
Lesson:
When processing a form, FIRST save data for form-refill use. SECOND validate form fields. THIRD escape the data for processing into the database.
Or better yet, use PDO and avoid this =).
Comments welcome. THANKS ALL!
I have designed a comment system. This is what I am basically doing.
$story=$_POST['story'];
$story=mysql_real_escape_string($story);
$query = "INSERT INTO `comment` VALUES('$story')";
Now the problem is when i store the comment all the " are replaced by \" and all the ' are replaced by \'. So when I display the comments back these \ also show up in the comment.
Another problem is that & disappears. eg: if user comments I & you only I is stored into the database.
In fact in few cases comments don't even enter the database.
What is the correct way of processing & storing user comments so that you can display them back as written originally?
PS: I am not worried about sql injection. I just want comments to show up the way they were entered.
It looks like you have magic qoutes turned on. You should simply disable them from php.ini.
If you are worried about sql injection, consider using prepared statements.
Magic quotes may be turned on in your PHP install.
See Disabling Magic Quotes for more information
use this:
function safe_mysql( $value ) {
$magic_quotes_active = get_magic_quotes_gpc();
$new_enough_php = function_exists( "mysql_real_escape_string" );
if( $new_enough_php ) {
if( $magic_quotes_active ) { $value = stripslashes( $value ); }
$value = mysql_real_escape_string( $value );
} else {
if( !$magic_quotes_active ) { $value = addslashes( $value ); }
}
return $value;
}
The \s aren't stored in your database. If you display the escaped $story variable you'll see the backslashes, but when you retrieve the data later on with a select-query, it'll just be the original data.
Make sure you have magic quotes disabled, otherwise the already escaped string will be escaped again automatically, causing e.g. "\\" which means that a backslash will be inserted.
Are you sure & disappears from the database? I'm guessing it doesn't appear on the page because & denotes the start of a HTML entity.
Use stripslashes to first remove the backslashes in front of your quotes, then use htmlspecialchars to escape HTML entities.
The easiest way to get them into the database is to use prepared statements and let someone else down the line worry about escaping.
Then when you get them out again, you still need to make sure ampersands etc are escaped to fit into html (i.e. use htmlspecialchars() or htmlentities()). When you get them they're in UTF-8 or ASCII or something. When you output them they're inside HTML. That means "showing up the way they were entered" doesn't mean giving back what you got directly.
Personally i use the following to santize data before inserting into MySQL.
$output = filter_var($input, FILTER_SANITIZE_STRING, FILTER_SANITIZE_SPECIAL_CHARS);
Unfortunatley this is for PHP 5 >= 5.2.0 so may not work on many shared servers.
I have a simple PHP script I use to front-end an SQLite database. It's nothing fancy or complex. But I have noticed from looking at the records in the database that anything I enter in a form-field with double-quotes comes across in the form-processing as though I'd escaped the quotes with a backslash. So when I entered a record with the title:
British Light Utility Car 10HP "Tilly"
what shows up in the database is:
British Light Utility Car 10HP \"Tilly\"
I don't know where these are coming from, and what's worse, even using the following preg_replace doesn't seem to remove them:
$name = preg_replace('/\\"/', '"', $_REQUEST['kits_name']);
If I dump out $name, it still bears the unwanted \ characters.
You have most probably magic_quotes_gpc set to on in php.ini. If you want to avoid that and use your own regex, make a check like this:
if (get_magic_quotes_gpc())
{
$mytext = stripslashes($your_text);
}
// and your further code....
This means your server has magic_quotes_gpc enabled.
You can use ini_set() to disable this setting, or you can create a method to filter the $_REQUEST values()
function getRequest($key)
{
$val = $_REQUEST[$key];
if(get_magic_quotes_gpc() == 1) {
$val = stripslashes($val);
}
return $val;
}
echo getRequest('kits_name');
Is it possible magic quotes are enabled on the server?
You probably have magic quotes turned on.
You should disable these as it's bad practice and is deprecated.
View this doc to learn how to disable them.
Well because of lack of good answers.
As they said above, it is because magic quotes on.
You have to get rid of these slashes before inserting your data.
So, to get rid of it you can use either .htaccess (if any) with these settings
php_flag magic_quotes_gpc 0
php_flag magic_quotes_runtime 0
or make it manually, with code like this
if ( get_magic_quotes_gpc( ) ) {
$_GET = array_map_recursive('stripslashes', $_GET) ;
$_POST = array_map_recursive('stripslashes', $_POST) ;
$_COOKIE = array_map_recursive('stripslashes', $_COOKIE) ;
$_REQUEST = array_map_recursive('stripslashes', $_REQUEST) ;
if (isset($_SERVER['PHP_AUTH_USER'])) stripslashes($_SERVER['PHP_AUTH_USER']);
if (isset($_SERVER['PHP_AUTH_PW'])) stripslashes($_SERVER['PHP_AUTH_PW']);
}
if your php version doesn't support array_map_recursive function, you can use a recursive function like this one
function strips(&$el) {
if (is_array($el))
foreach($el as $k=>$v)
strips($el[$k]);
else $el = stripslashes($el);
}
or write your own one
You can use this code co cleanse your existing data
As for
If I dump out $name, it still bears
the unwanted \ characters.
it may be result of wrong use htmlspecialchars function
I just moved to a new hosting company and now whenever a string gets escaped using:
mysql_real_escape_string($str);
the slashes remain in the database. This is the first time I've ever seen this happen so none of my scripts use
stripslashes()
anymore.
This is on a CentOS 4.5 64bit running php 5.2.6 as fastcgi on a lighttpd 1.4 server. I've ensured that all magic_quotes options are off and the mysql client api is 5.0.51a.
I have the same issue on all 6 of my webservers.
Any help would be appreciated.
Thanks.
Edit:
Magic Quotes isn't on. Please don't recommend turning it off. THIS IS NOT THE ISSUE.
The host that you've moved probably has magic_quotes_runtime turned on. You can turn it off with set_magic_quotes_runtime(0).
Please turn off magic_quotes_runtime, and then change your code to use bind variables, rather than using the string escaping.
I can think of a number of things that could cause this. But it depends how you are invoking SQL queries. If you moved to use parameterized queries like with PDO, then escaping is unnecessary which means the call to mysql_real_escape_string is adding the extra slashes.
If you are using mysql_query etc. then there must be some code somewhere like addslashes which is doing this. This could either be before the data is going into the database, or after.
Also you say you have disabled magic quotes... if you haven't already, just do a hard check in the code with something like this:
echo htmlentities($_GET['value']); // or $_POST, whichever is appropriate
Make sure there are no slashes in that value, then also check this:
echo "Magic quotes is " . (get_magic_quotes_gpc() ? "ON" : "OFF");
I know you've said multiple times it isn't magic quotes, but for us guys trying to help we need to be sure you have checked the actual PHP output rather than just changing the config (which might not have worked).
it sounds as though you have magic quotes turned on. Turning it off isn't too hard: just create a file in your root directory called .htaccess and put this line in it:
php_flag magic_quotes off
If that's not possible for whatever reason, or you want to change your application to be able to handle magic quotes, use this technique:
Instead of accessing the request variables directly, use a function instead. That function can then check if magic quotes is on or off and strip out slashes accordingly. Simply running stripslashes() over everything won't work, because you'll get rid of slashes which you actually want.
function getVar($key) {
if (get_magic_quotes_gpc()) {
return stripslashes($_POST[$key]);
} else {
return $_POST[$key];
}
}
$x = getVar('x');
Now that you've got that, all your incoming variables are ready to be escaped again and mysql_real_escape_string() won't stuff them up.
the slashes remain in the database.
It means that your data gets double escaped.
There are 2 possible reasons:
magic quotes are on, despite of your feeling. Double-check it
There is some code in your application, that just mimic magic quotes behaviour, escaping all input.
This is very common misconception to have a general escaping function to "protect" all the incoming data. While it does no good at all, it also responsible for the cases like this.
Of so - just find that function and wipe it out.
You must probably have magic quotes turned on. Figuring out exactly how to turn it off can be quite a headache in PHP. While you can turn off magic quotes with set_magic_quotes_runtime(0), it isn't enough -- Magic quotes has already altered the input data at this point, so you must undo the change. Try with this snippet: http://talks.php.net/show/php-best-practices/26
Or better yet -- Disable magic quotes in php.ini, and any .htaccess files it may be set in.
I am not sure if I understand the issue correctly but I had a very same problem. No matter what I did the slashes were there when the string got escaped. Since I needed the inserted value to be in the exact same format as it was entered I used
htmlentities($inserted_value)
this will leave all inserted quote marks unescaped but harmless.
What might be the problem (it was with us) that you use mysql_real_escape_string() multiple times on the same var. When you use it multiple times, it will add the slashes.
Function below will correctly remove slashes before inserting into the database. I know you said magic quotes isn't on but something is adding slashes so try the following page and see the output. It'll help figure out where. Call with page.php?var=something-with'data_that;will`be|escaped
You will most likely see number three outputting more slashes than needed.
*Change the db details too.
<?php
$db = mysql_connect('host', 'user', 'pass');
$var = $_REQUEST['var'];
echo "1: $var :1<br />";
echo "2: ".stripslashes($var)." :2<br />";
echo "3: ".mysql_real_escape_string($var)." :3<br />";
echo "4: ".quote_smart($var)." :4<br />";
function quote_smart($value)
{
// Stripslashes is gpc on
if (get_magic_quotes_gpc())
{
$value = stripslashes($value);
}
// Quote if not a number or a numeric string
if ( !is_numeric($value) )
{
$value = mysql_real_escape_string($value);
}
return $value;
}
?>
mysql_real_escape_string($str); is supposed to do exactly that. it is meant to add backslashes to special characters especially when you want to pass the query to mysql. Take note that it also takes into account the character set of mysql.
For safer coding practices it would be good to edit your code and use stripslashes() to read out the data and remove the slashes.
According to the PHP manual, in order to make code more portable, they recommend using something like the following for escaping data:
if (!get_magic_quotes_gpc()) {
$lastname = addslashes($_POST['lastname']);
} else {
$lastname = $_POST['lastname'];
}
I have other validation checks that I will be performing, but how secure is the above strictly in terms of escaping data? I also saw that magic quotes will be deprecated in PHP 6. How will that affect the above code? I would prefer not to have to rely on a database-specific escaping function like mysql_real_escape_string().
Magic quotes are inherently broken. They were meant to sanitize input to the PHP script, but without knowing how that input will be used it's impossible to sanitize correctly. If anything, you're better off checking if magic quotes are enabled, then calling stripslashes() on $_GET/$_POST/$_COOKIES/$_REQUEST, and then sanitizing your variables at the point where you're using it somewhere. E.g. urlencode() if you're using it in a URL, htmlentities() if you're printing it back to a web page, or using your database driver's escaping function if you're storing it to a database. Note those input arrays could contain sub-arrays so you might need to write a function can recurse into the sub-arrays to strip those slashes too.
The PHP man page on magic quotes agrees:
"This feature has been DEPRECATED as
of PHP 5.3.0 and REMOVED as of PHP
5.4.0. Relying on this feature is highly discouraged. Magic Quotes is a
process that automagically escapes
incoming data to the PHP script. It's
preferred to code with magic quotes
off and to instead escape the data at
runtime, as needed."
Magic quotes were a design error. Their use is incompatible with retainnig your sanity.
I prefer:
if (get_magic_quotes_gpc()) {
throw new Exception("Turn magic quotes off now!");
}
Don't write code for compatibility with inherently broken setups. Instead defend aginst their use by having your code FAIL FAST.
I use the following code in the header file of my website to reverse the effects of magic_quotes:
<?php
// Strips slashes recursively only up to 3 levels to prevent attackers from
// causing a stack overflow error.
function stripslashes_array(&$array, $iterations=0) {
if ($iterations < 3) {
foreach ($array as $key => $value) {
if (is_array($value)) {
stripslashes_array($array[$key], $iterations + 1);
} else {
$array[$key] = stripslashes($array[$key]);
}
}
}
}
if (get_magic_quotes_gpc()) {
stripslashes_array($_GET);
stripslashes_array($_POST);
stripslashes_array($_COOKIE);
}
?>
Then I can write the rest of my code as if magic_quotes never existed.
"I would prefer not to have to rely on a database-specific escaping function like mysql_real_escape_string()"
Then use something like PDO. But you have to reverse the damage done by magic quotes anyway.
Put a requirement of PHP 5.2 or higher on your code and use the filter API. The filter_* functions access the raw input data directly (they don't ever touch $_POST etc.) so they're completely unaffected by magic_quotes_gpc.
Then this example:
if (!get_magic_quotes_gpc()) {
$lastname = addslashes($_POST['lastname']);
} else {
$lastname = $_POST['lastname'];
}
Can become this:
$lastname = filter_input(INPUT_POST, 'lastname');
Right, it's not the best way to do it and not the most secure. Escaping is best done in relation to what you are escaping for. If it is to store in a mysql database, use mysql_real_escape_string which takes into account other locales, character sets. For HTML, htmlentities. For use in code, escapeshellcmd, escapeshellarg. Yes, you probably need to stirpslashes first if magic quotes is on. But best not to count on it or use it.
Regarding using a database specific escaping function, you pretty much need to. I have found just using addslashes() to fail in rare cases with MySQL. You can write a function to escape which determines which DB you are using and then use the approriate escape function.
You may try this:
if (get_magic_quotes_gpc()) {
$_REQUEST = array_map('stripslashes', $_REQUEST);
$_GET = array_map('stripslashes', $_GET);
$_POST = array_map('stripslashes', $_POST);
$_GET = array_map('stripslashes', $_COOKIES);
}
"I would prefer not to have to rely on a database-specific escaping function like mysql_real_escape_string()"
Also addslashes can be tricked as well check out this post:
http://shiflett.org/blog/2006/jan/addslashes-versus-mysql-real-escape-string
Your sample code is backwards, you should be doing the following:
if (get_magic_quotes_gpc()) {
$lastname = stripslashes($_POST['lastname']);
} else {
$lastname = $_POST['lastname'];
}
Note that this leaves your input data in a 'raw' state exactly as the user typed it - no extra backslashes and potentially loaded with SQL Injection and XSRF attacks - and that's exactly what you want. Then, you make sure you always use one of the following:
When echoing the variable into HTML, wrap it in htmlentities()
When putting it into mysql, use prepared statements or else mysql_real_escape_string() as a minimum.
When echoing the variable into Javascritpt code, use json_encode()
Joel Spolsky has some good starting advice in Making Wrong Code Look Wrong
Just found this over on the PHP manual pages, looks like a pretty clever way to strip em (deals with keys and values...):
if (get_magic_quotes_gpc())
{
$_GET = json_decode(stripslashes(json_encode($_GET, JSON_HEX_APOS)), true);
$_POST = json_decode(stripslashes(json_encode($_POST, JSON_HEX_APOS)), true);
$_COOKIE = json_decode(stripslashes(json_encode($_COOKIE, JSON_HEX_APOS)), true);
$_REQUEST = json_decode(stripslashes(json_encode($_REQUEST, JSON_HEX_APOS)), true);
ini_set('magic_quotes_gpc', 0);
}
Prepared statements of PDO and Mysqli are the better way to prevent SQL injection.
But if you are migrating a legacy code which is base on Magic Quotes for every SQL queries, you can refer yidas/php-magic-quotes for implementing Magic Quotes on the environment with PHP 5.4 above version.
https://github.com/yidas/php-magic-quotes