I just moved to a new hosting company and now whenever a string gets escaped using:
mysql_real_escape_string($str);
the slashes remain in the database. This is the first time I've ever seen this happen so none of my scripts use
stripslashes()
anymore.
This is on a CentOS 4.5 64bit running php 5.2.6 as fastcgi on a lighttpd 1.4 server. I've ensured that all magic_quotes options are off and the mysql client api is 5.0.51a.
I have the same issue on all 6 of my webservers.
Any help would be appreciated.
Thanks.
Edit:
Magic Quotes isn't on. Please don't recommend turning it off. THIS IS NOT THE ISSUE.
The host that you've moved probably has magic_quotes_runtime turned on. You can turn it off with set_magic_quotes_runtime(0).
Please turn off magic_quotes_runtime, and then change your code to use bind variables, rather than using the string escaping.
I can think of a number of things that could cause this. But it depends how you are invoking SQL queries. If you moved to use parameterized queries like with PDO, then escaping is unnecessary which means the call to mysql_real_escape_string is adding the extra slashes.
If you are using mysql_query etc. then there must be some code somewhere like addslashes which is doing this. This could either be before the data is going into the database, or after.
Also you say you have disabled magic quotes... if you haven't already, just do a hard check in the code with something like this:
echo htmlentities($_GET['value']); // or $_POST, whichever is appropriate
Make sure there are no slashes in that value, then also check this:
echo "Magic quotes is " . (get_magic_quotes_gpc() ? "ON" : "OFF");
I know you've said multiple times it isn't magic quotes, but for us guys trying to help we need to be sure you have checked the actual PHP output rather than just changing the config (which might not have worked).
it sounds as though you have magic quotes turned on. Turning it off isn't too hard: just create a file in your root directory called .htaccess and put this line in it:
php_flag magic_quotes off
If that's not possible for whatever reason, or you want to change your application to be able to handle magic quotes, use this technique:
Instead of accessing the request variables directly, use a function instead. That function can then check if magic quotes is on or off and strip out slashes accordingly. Simply running stripslashes() over everything won't work, because you'll get rid of slashes which you actually want.
function getVar($key) {
if (get_magic_quotes_gpc()) {
return stripslashes($_POST[$key]);
} else {
return $_POST[$key];
}
}
$x = getVar('x');
Now that you've got that, all your incoming variables are ready to be escaped again and mysql_real_escape_string() won't stuff them up.
the slashes remain in the database.
It means that your data gets double escaped.
There are 2 possible reasons:
magic quotes are on, despite of your feeling. Double-check it
There is some code in your application, that just mimic magic quotes behaviour, escaping all input.
This is very common misconception to have a general escaping function to "protect" all the incoming data. While it does no good at all, it also responsible for the cases like this.
Of so - just find that function and wipe it out.
You must probably have magic quotes turned on. Figuring out exactly how to turn it off can be quite a headache in PHP. While you can turn off magic quotes with set_magic_quotes_runtime(0), it isn't enough -- Magic quotes has already altered the input data at this point, so you must undo the change. Try with this snippet: http://talks.php.net/show/php-best-practices/26
Or better yet -- Disable magic quotes in php.ini, and any .htaccess files it may be set in.
I am not sure if I understand the issue correctly but I had a very same problem. No matter what I did the slashes were there when the string got escaped. Since I needed the inserted value to be in the exact same format as it was entered I used
htmlentities($inserted_value)
this will leave all inserted quote marks unescaped but harmless.
What might be the problem (it was with us) that you use mysql_real_escape_string() multiple times on the same var. When you use it multiple times, it will add the slashes.
Function below will correctly remove slashes before inserting into the database. I know you said magic quotes isn't on but something is adding slashes so try the following page and see the output. It'll help figure out where. Call with page.php?var=something-with'data_that;will`be|escaped
You will most likely see number three outputting more slashes than needed.
*Change the db details too.
<?php
$db = mysql_connect('host', 'user', 'pass');
$var = $_REQUEST['var'];
echo "1: $var :1<br />";
echo "2: ".stripslashes($var)." :2<br />";
echo "3: ".mysql_real_escape_string($var)." :3<br />";
echo "4: ".quote_smart($var)." :4<br />";
function quote_smart($value)
{
// Stripslashes is gpc on
if (get_magic_quotes_gpc())
{
$value = stripslashes($value);
}
// Quote if not a number or a numeric string
if ( !is_numeric($value) )
{
$value = mysql_real_escape_string($value);
}
return $value;
}
?>
mysql_real_escape_string($str); is supposed to do exactly that. it is meant to add backslashes to special characters especially when you want to pass the query to mysql. Take note that it also takes into account the character set of mysql.
For safer coding practices it would be good to edit your code and use stripslashes() to read out the data and remove the slashes.
Related
I have users entering their name, as in: O'riley.
Before I enter this data into the MySQL DB, I run mysql_real_escape_string.
Problem is, when I then select this data for display and use later, it comes out as: O\'riley.
Obviously, this is the intended operation. What I'm wondering is if there's someway to be sure I can store it in the DB (still safely escaping possible malicious code), so that I don't have to use strip_slashes() on the output EVERY time I call the data throughout the entire web app? Or, am I missing something here?
Thanks.
UPDATE
Please refer to the comments in Deceze's answer.
No, it's not the intended operation to store the string as "O\'riley"; it should only be escaped in the query, but not stored this way. I'll guess that PHP puts in the backslash through Magic Quotes, and you escape it again to make it stick.
Disable Magic Quotes.
I personally always turn off magic quotes because it is doing something I haven't told it to do. If you dont have the ability to turn it off, consider including this code at the top of all of your pages.
if (get_magic_quotes_gpc()) {
function strip_array($var) {
return is_array($var)? array_map("strip_array", $var):stripslashes($var);
}
$_POST = strip_array($_POST);
$_SESSION = strip_array($_SESSION);
$_GET = strip_array($_GET);
}
If you have magic quotes disabled, and in the case that get_magic_quotes_gpc returned 1 and you did something like #Michael shows and this still occurs, magic quotes runtime may be enabled.
Magic quotes runtime will add slashes to strings when doing database queries and even when writing to files. To disable this in the currently executing script, do:
if(function_exists('get_magic_quotes_runtime') && get_magic_quotes_runtime())
{
set_magic_quotes_runtime(false);
}
You can also disable this configuration option through php.ini as well, if you can do so. However, if you can't disable it through php.ini permanently, you may need to add # before get_magic_quotes_runtime and set_magic_quotes_runtime as PHP may through E_DEPRECATED errors if error_reporting is set to log such errors (and this will get fairly annoying).
Alright, here's what you need to do. First, based on the comments, from other questions:
ISOLATE
You need to determine what is causing the problem. Is it the server? Is it some hidden code somewhere? What? Some programmers may include code like this on a configuration page:
<?php if (isset($_POST))
foreach ($_POST as $key => $value) $_POST[$key] = addslashes($value);
?>
So here are some checks to see if it is the server. This is about as solid a way of checking as possible. Create a NEW page. Leave it entirely blank and add this code:
<?php print_r($_POST); ?>
<form action="" method="POST">
<input type="text" name="test" value="O'riley" />
<input type="submit" name="submit" value="submit" />
</form>
Save it and load it in the browser. Hit submit. See if the slash is added still.
If it is still adding it, you need to troubleshoot your server / configuration. If it's a server you're paying for, you need to tell them to fix it or shove it. If it's your own server, then you're going to need to do some messing / googling to figure out how to turn off magic quotes.
If there are no quotes showing up, then there is definitely some code adding slashes to your post variables. It might not be as obvious as the code above, however. You need to run a search on your code for "addslashes", "mysql_real_escape_string", and possibly "str_replace". If you find one of these, you need to disable it. But be advised this may break other parts of your site that are assuming this action is taking place. You're other option for storing it in the database is to do a function similar to the code above, but instead, run stripslashes on it. Then you can run mysql_real_escape_string on it later. (A bit more unneeded overhead.)
Use
var_dump(get_magic_quotes_gpc());
somewhere near the actual use of the posted data to see if the Magic Quotes are really off.
Also some weird frameworks may add said quotes for you so grep your code for addslashes and other escape_string-like functions.
I would honestly suggest the use of PDO.
PDO employs the use of prepare and execute statements, which in turn adds security and removes some extra headache.
$pdo = new PDO();
$stm = $pdo->prepare('INSERT... (LastName) VALUES (:LastName)');
$stm->execute(array(':LastName' => "O'Rily"));
$stm->fetchAssoc(PDO::FETCH_ASSOC);
You no longer will need to worry about removing the escaping slashes as well as securing basic sql injection tactics.
It's possible magic quotes are turned on in your httpd.conf or in a virtual host declaration.
The exact location and file will depend on OS and distro. I'd check Apache configuration files for the php_flag setting.
Just use clean the data before inserting in to database,
function check_input($value)
{
if( get_magic_quotes_gpc() )
{
$value = stripslashes( $value );
}
//check if this function exists
if( function_exists( "mysql_real_escape_string" ) )
{
$value = mysql_real_escape_string( $value );
}
//for PHP version < 4.3.0 use addslashes
else
{
$value = addslashes( $value );
}
return $value;
}
$cleandata = check_input($value);
$sql = "INSERT INTO `table_name` SET `field_name`='".$cleandata."'";
While fetching the data and show into it use stripslashes($val)
Thank you everyone for the answers. I will award the +50 out, but I wanted to tell my real solution here, all which people did help with...
I was performing mysql_real_escape_string on all of the data AS SOON as it posted (before any processing). So, a slash was added to escape the ' character that was submitted. This, we know is normal.
However, there was no reason that the backslash \ should show up in the DB entry, right? The escape was there to be sure the ' was entered.
Turns out, AFTER escaping, I would then save the variable to be reloaded to the page in the session, in case the user had an error that PHP found while validating all of the form fields. In this case, the user's input (formerly O'riley was now printed to their screen as O\'riley. Then, the user didn't catch this - so they would often just fix their error that PHP caught during validation (unrelated to the name field), and thus the O\'riley would land in the database because mysql_real_escape_string would escape the characters.
Lesson:
When processing a form, FIRST save data for form-refill use. SECOND validate form fields. THIRD escape the data for processing into the database.
Or better yet, use PDO and avoid this =).
Comments welcome. THANKS ALL!
I have found stripslashes function but I would rather find where I am adding more slashes than I should. My functions use mysql_real_escape_string once for each variable and I am querying database using "insert into foo(bar,bar) values($baz,$baz)"
maybe this is the problem.
phpinfo gives
magic_quotes_gpc On On
magic_quotes_runtime Off Off
magic_quotes_sybase Off Off
static function insert($replyto,$memberid,$postid,$comment)
{
$message=array();
$lenmax=1000;
$lenmin=5;
$toolong="comment is too long.";
$tooshort="comment is too short.";
$notarget="replied comment is deleted";
$nomember="you are not a member";
$notpost="commented post is deleted";
switch(true)
{
case strlen($comment)<$lenmin: $message[]= $tooshort; break;
case strlen($comment)>$lenmax: $message[]=$toolong; break;
case $replyto!=NULL && !commentexists($replyto): $message[]=$notarget; break;
case !memberexists($memberid): $message[]=$nomember; break;
case !postexists($postid): $message[]=$nopost; break;
case count($message)>0:return $message; break;
}
$replyto=mysql_real_escape_string($replyto);
$memberid=mysql_real_escape_string($memberid);
$postid=mysql_real_escape_string($postid);
$comment=mysql_real_escape_string($comment);
if($replyto==NULL)
mysql_query("insert into fe_comment(memberid,postid,comment) values($memberid,$postid,'$comment')");
else
mysql_query("insert into fe_comment(replyto,memberid,postid,comment) values($replyto,$memberid,$postid,'$comment')");
}
my hosting firm has magic_quotes_gpc on and I don't have access to php.ini file I am using plesk panel to configure things.
php documentation says
An example use of stripslashes() is when the PHP directive magic_quotes_gpc is on (it's on by default), and you aren't inserting this data into a place (such as a database) that requires escaping. For example, if you're simply outputting data straight from an HTML form.
My insert queries are inserted with slashes in the database and My php version is 5.2.3
documentation also says
If magic_quotes_gpc is enabled, first apply stripslashes() to the data. Using this function on data which has already been escaped will escape the data twice.
So I am checking if I escaped values twice I am not able to find anywhere I escaped the values twice.
now I am using
$comment=mysql_real_escape_string(stripslashes($comment));
but I think it shouldn't become a standard in my codes because it doesn't look like "the right way" even though it saves the day.
magic_quotes_gpc automaticly escapes all and also is not reliable because it is deprecated.
so I have created a .htaccess file and copied it into all directories I have an index.php file, .htaccess files have this text only
php_flag magic_quotes_gpc Off
I ran phpinfo and it still gives
magic_quotes_gpc On On
magic_quotes_runtime Off Off
magic_quotes_sybase Off Off
now I need a way to disable the magic quotes gpc and I have no access to the php.ini file. I am looking for the ways to edit .htaccess files now.
I think it shouldn't become a standard in my codes because it doesn't look like "the right way"
You are right.
magic quotes stuff has nothing to do with sql stuff and shouldn't be connected to it.
Because magic quotes is a site-wide problem and sql escaping is sql only related problem.
So, they need different treatment an should be never used in conjunction.
You have to get rid of magic quotes unconditionally, because it spoiling not only SQL stuff but every data manipulation of your site.
So, it would be wise to put some stripslashes code in whatever bootstrap file to be run on every call of the script. The code you can find in numerous implementations of such a code, just google for the 'stripslashes_deep' function.
It would be wise to have this code always run (of course under the condition checking get_magic_quotes_gpc()) despite of the actual state of magic quotes, just for sake of compatibility.
But there is another possibility to turn them off: try to create a php.ini file in the root of your application.
However, there is a grave mistake in your code. In fact, it doesn't protect anything.
You are escaping $memberid and $postid but don't quote them!. Thus, there is no protection at all. Just because escaping works only when used with quoting.
Please, remember:
Escaping is not a synonym for security!
Escaping alone can help nothing. There is a whole set of rules to be followed.
I wrote a decent explanation recently, so, I wouldn't repeat myself: Replacing mysql_* functions with PDO and prepared statements
Various ways of disabling magic quotes are provided in the php documentation. Failing that it provides a way of removing the slashes recursively from all of your request variables.
This question already has answers here:
Why are $_POST variables getting escaped in PHP?
(6 answers)
Closed 7 years ago.
I have a php page which contains a form.
Sometimes this page is submitted to itself (like when pics are uploaded).
I wouldn't want users to have to fill in every field again and again, so I use this as a value of a text-input inside the form:
value="<?php echo htmlentities(#$_POST['annonsera_headline'],ENT_COMPAT,'UTF-8');?>">
This works, except it adds a "\" sign before every double-quote...
For instance writing 19" wheels gives after page is submitted to itself:
19\" wheels
And if I don't even use htmlentities then everything after the quotes dissappears.
What is the problem here?
UPDATE:
Okay, so the prob is magic_quotes... This is enabled on my server...
Should I disable it? I have root access and it is my server :)
Whats the harm in disabling it?
Looks like you have magic quotes turned on. Use below condition using stripslashes with whatever text you want to process:
if(get_magic_quotes_gpc())
{
$your_text = stripslashes($your_text);
}
Now you can process $your_text variable normally.
Update:
Magic quotes are exaplained here. For well written code there is normally no harm in disabling it.
You likely have magic quotes turned on. You need to stripslashes() it as well.
Nicest way would be to wrap this in a function:
function get_string($array, $index, $default = null) {
if (isset($array[$index]) && strlen($value = trim($array[$index])) > 0) {
return get_magic_quotes_gpc() ? stripslashes($value) : $value;
} else {
return $default;
}
}
Which you can use as
$annonsera_headline = get_string($_POST, 'annonsera_headline');
By the way:
And if I don't even use htmlentities then everything after the quotes dissappears.
It's actually still there in the HTML source, you only don't see it. Do a View Source ;)
Update as per your update: the magic quotes is there to prevent SQL injection attacks in code of beginners. You see this often in 3rd party hosts. If you really know what you're doing in the code, then you can safely turn it off. But if you want to make your code distributable, then you'll really take this into account when gathering request parameters. For this the above function example is perfectly suitable (you only need to write simliar get_boolean(), get_number(), get_array() functions yourself).
Yes, you should disable magic quotes if you can. The feature is deprecated, and will likely go away completely in the future.
If you've relied on magic quotes for escaping data (for instance when inserting it into a database) you will may be opening yourself up to sql injection vulnerabilities if you disable it. You should check all your queries and make sure you're using mysql_real_escape_string().
I include the following file to undo magic quotes in apps that are deployed to servers not under my control.
<?php
set_magic_quotes_runtime(0);
function _remove_magic_quotes(&$input) {
if(is_array($input)) {
foreach(array_keys($input) as $key) _remove_magic_quotes($input[$key]);
}
else $input = stripslashes($input);
}
if(get_magic_quotes_gpc()) {
_remove_magic_quotes($_REQUEST);
_remove_magic_quotes($_GET);
_remove_magic_quotes($_POST);
_remove_magic_quotes($_COOKIE);
}
return true;
?>
This is actually a function of PHP trying to be security conscious, luckily there is an easy fix for it that looks something like this:
if (get_magic_quotes_gpc()) {$var = stripslashes($var);}
There isn't a huge problem in having it enabled, it comes down to personal preference. If you code will be moving servers much and you can't disable it through your php.ini file, it's best to use something as described above.
If you have access to your php.ini file and you want to change it, because you don't want to have to validate it each time you can add the following line to php.ini
magic_quotes_gpc = Off
Or the following to your .htaccess:
php_flag magic_quotes_gpc Off
Hope this helps clear things up.
Looks like your server is setup to use Magic Quotes.
You can fix it by stripping them with stripslashes, or better, by turning off Magic Quotes.
you shouldn't use htmlentities() when writing something to an input field value.
is magic_quotes enabled on your server? try out stripslashes before the output.
if(get_magic_quotes_gpc())
{
$location_name = trim(mysql_real_escape_string(trim(stripslashes($_GET['location_name']))));
}
else
{
$location_name = trim(mysql_real_escape_string(trim($_GET['location_name'])));
}
That's the code I have so far. seems to me this code is fundamentally ... OK. Do you think I can safely remove the inner trim(). Please try not a spam me with endless version of this, I want to try to learn how to do this better.
UPDATE
So after reading some of the responses, I think I have come to understand a good method for safely getting data from a user, storing it and then displaying it back.
When you first load the page
$foo = trim($_GET['foo']);
if(get_magic_quotes_gpc())
{
$foo = stripslashes($foo);
}
Then when you come to use this variable as part of a SQL string, even if not storing the data in the database, you should escape it.
mysql_real_escape_string($foo);
And finally, if reading data from the database and wanting to display it as HTML, such a post on a blog or forum, you should pass the variable using htmlspecialchars
echo(htmlspecialchars($bar));
Would any one like to suggest a better set of functions to use? other then obviously wrapping these functions to make them simpler to call.
Here:
$location_name = trim($_GET['location_name']);
if(get_magic_quotes_gpc()) $location_name=stripslashes($location_name);
Then there is also the SQL injection protection, but don't do this until the very last moment before sticking this var in an SQL query. And even then don't apply the changes to the var itself, but rather a copy. You might want to show $location_name to the user afterwards (for example if the form fails). So
$sql="UPDATE whatever(location) VALUES('" . mysql_real_escape_string($location_name) . "')"
I'm assuming of course that $location_name will end up in the database; otherwise you don't need mysql_real_escape_string.
Finally you want to use htmlspecialchars if you're going to display $location_name on your page somewhere.
Edit: You want to use htmlspecialchars() just before displaying the data (definately don't save data that has already been transformed via htmlspecialchars in your database). In general you want to use escaping functions at the last moment and then on a copy of your var. That way you know that at any point during the script the var is the original one and is not carrying some random escape characters from a transformation that happened somewhere before.
You also know where your escape functions are/should be. sql escaping is near/at your sql query. XSS escaping (htmlspecialchars) is near the part where you display data in a web page.
Finally once you get the grip of things, you could always forego SQL escaping by using PHP's PDO functions. Also in the future you might want to take at look at this: Do htmlspecialchars and mysql_real_escape_string keep my PHP code safe from injection?
I am sorry to say but everything in your question is wrong.
First, it has nothing to do with performance, by any means. these functions never become a bottleneck and never cause any performance issue.
Next, You've choose wrong place to get rid of magic quotes. Magic quotes is input data related, not database related. It it is better to make a distinct function and place it in your configuration file, being included into every script. You can use one from here
So, the code become like this:
$location_name = mysql_real_escape_string(trim($_GET['location_name']));
But i strongly advise you not to mix database escaping with anything else, as anything else is optional while database escaping is strict and unconditional.
I'm trying to figure out why this function does not work correctly.
It's adding an extra \ every time I edit my entries.
Online server has these settings:
magic_quotes_gpc On
magic_quotes_runtime Off
magic_quotes_sybase Off
Here is the code:
function esc($s)
{
if (get_magic_quotes_gpc()) {
if (ini_get('magic_quotes_sybase'))
$s = str_replace("''", "'", $s);
else
$s = stripslashes($s);
} //if
return mysql_real_escape_string($s);
}
Edit note:
I have tried completely removing this function to see what it does... and it does the same thing, so I have realized that addslashes is also use in the code for the same thing.
The extra \ were there because magic_quote was ON
Your function makes little sense. If magic quotes is on (eg. input is escaped), you unescape it. If it's not on, you escape it. So you'll get different results, depending on if you have magic quote on or not.
In any case, relying on magic quotes is a really bad practice. You should:
Disable magic quotes or reverse its effect globally.
Either escape strings when you construct SQL queries or (better) use prepared statements.
Not unescape/strip/whatever anything when you get it back from the database.
You probably want to stripslashes even if magic_quotes_sybase is on:
function esc($s)
{
if (get_magic_quotes_gpc()) {
if (ini_get('magic_quotes_sybase'))
$s = str_replace("''", "'", $s);
$s = stripslashes($s);
} //if
return mysql_real_escape_string($s);
}
You might also want to take a look at PHP's get_magic_quotes_gpc function page, there are several user comments on the page with fairly elegant solutions for ensuring slashes are stripped.
Ok I have fixed the problem. A quick solution for now, I have removed function esc($s).
I changed Magic_Quote to OFF in php.ini.
I'm keeping addslashes solution.