As I was trying to add a quote (via a PHP/HTML form) to my site that was particularly long, I noticed that the longer quotes did not get added via the MySQL query. The shorter ones got added just fine.
This is not a problem with the length of the columns in my table, the quotes are most definitely shorter than the limit (which is VARCHAR(600)).
When I try to manually enter the quotes into the database from the MySQL command line, hitting enter after entering the query properly does not result in a new prompt (it goes to the next line of the query instead). The query never executes.
The only way I can get to a new prompt is to force exit out of the MySQL command line and re-opening it. Anyone know why this is happening?
EDIT: Here is my query. The first 3 values are irrelevant, I won't confuse anyone by explaining them. The last 2 are the source and the quote.
EDIT2: MySQL version is 5.5.8
THE QUERY THAT DOES NOT WORK
INSERT INTO quotes VALUES(NULL, 2, 20, 'http://www.mlb.com/news/article.jsp?ymd=20091104&content_id=7620238&vkey=news_nyy&fext=.jsp&c_id=nyy', 'He's the reason we're here. First of all, we wouldn't be in this stadium if it wasn't for him. We wouldn't have this group together if it wasn't for him. It's a special moment. We all wanted to win it for him.');
EXAMPLE OF A QUERY THAT WORKS
INSERT INTO quotes VALUES(NULL, 2, 20, 'http://www.mlb.com/news/article.jsp?ymd=20091104&content_id=7620238&vkey=news_nyy&fext=.jsp&c_id=nyy', 'I guess you could say that this is the best moment of my life right now," Matsui said. "If I were to look back, yes, this would be the best.');
To those downvoting, mind telling me why?
FINAL EDIT Got it... it's because of the single quotes in my quote. Thank you.
The following things could cause the MySQL console to think that you have not yet finished your query, and there is more to enter:
An unclosed quote, or backtick.
Usually due to the fact that you didn't escape a quote by doing \' (for single quotes) or \" (for double quotes).
No semi-colon at the end of the query.
Make sure you don't have either of those problems and the query should run.
Edit following poster's edit:
Yeah, looking at your query, you need to escape quotes by replacing ' with \' whenever it is inside an item of data. PHP has a function to escape for you which is mysql_real_escape_string(). You could also use prepared statements, which prevents this problem. See this question for more information.
It's absolutely obvious, my friend ... you have single quotes in your last string, while using those as your string boundary ...
When in the MySQL console, make sure you add a semicolon (;) to the end of the query. Otherwise, it will expect more to the query.
You've got single quotes inside your string, which is encapsulated in single quotes.
You should be escaping your strings with mysql_real_escape_string (or something similar, like PDO): http://php.net/manual/en/function.mysql-real-escape-string.php
it sounds like you did not escape the ' or " characters in the quote properly. look for any ' or " or \ characters and put a \ before them.
You post no PHP code but your SQL suggests that you are omitting two basic steps:
Use prepared statements to inject data into your queries (or proper escaping functions)
Test the result of your DB calls and print/log error messages.
Related
I'm trying to update an old program that uses regular MySql queries, runs all the inputs through addslashes() prior to inserting them, and runs the retrieved data through stripslashes() before returning it. I'm trying to update the program to run with Mysqli and use prepared statements, but I'm not sure how to make the transition with the existing data in the database.
My first thought was to write a function to test if slashes have been added, but the general consensus in the answers to similar questions I found was that this isn't doable.
I'm trying to avoid doing a mass update to the database, because it's an open source program and this seems likely to potentially cause problems for existing users when they try to upgrade.
If I continue to use addslashes() prior to inserting using MySqli and prepared statements would this work? What would be the reasons for not doing it this way and instead going the full database upgrade route?
Edit:
Based on a comment that appears to be deleted now, I went and looked at the database directly. All I could find was an ' that had been converted to ' and no slashes. When I pulled out the data it came out fine without running stripslashes(). So do the added slashes just tell mysql to escape the data when it's inserted? Will all the data come out find if I remove the stripslashes()? If so what's the point of stripslashes()?
You have several questions:
If I continue to use addslashes() prior to inserting using mysqli_ and prepared statements would this work?
No, it would not: the added backslashes would then be stored in your database, which is not what you want. In prepared statements the provided arguments are taken literally as you pass them, as in that mechanism there is no more need to escape quotes.
What would be the reasons for not doing it this way and instead going the full database upgrade route?
See above.
So do the added slashes just tell MySql to escape the data when it's inserted?
The added slashes are interpreted when you embed a string literal in your SQL, as that necessarily is wrapped in quotes. For example:
$text = "My friend's wedding";
$sql = "INSERT INTO mytable VALUES ('$text')";
$result = mysqli_query($con, $sql);
The mysqli_query will fail, as the SQL statement that is passed to it looks like this:
INSERT INTO mytable VALUES ('My friend's wedding')
This is not valid SQL, as the middle quote ends the string literal, and so the s that follows is not considered part of the string any more. As the SQL engine does not understand the s (nor wedding and the dangling quote after it), it produces a syntax error. You can even see something is wrong in the way the syntax is highlighted in the above line.
So that is where addslashes() is (somewhat) useful:
$text = "My friend's wedding";
$text = addslashes($text);
$sql = "INSERT INTO mytable VALUES ('$text')";
$result = mysqli_query($con, $sql);
This will work because now the SQL statement sent to MySql looks like this:
INSERT INTO mytable VALUES ('My friend\'s wedding')
The backslash tells MySql that the quote that follows it has to be taken as a literal one, and not as the end of the string. MySql will not store the backslash itself, because it is only there to escape the quote (see list of escape sequences in the documentation).
So in your table you will get this content inserted:
My friend's wedding
Then:
Will all the data come out fine if I remove the stripslashes()?
Yes. In fact, it should never have been in your code in the first place, as it will negatively effect what you get when there truly are (intended!) backslashes in your data. You are maybe lucky that a backslash is a rare character in normal text. Apparently you have not found any backslash in your database data, so it probably did not badly affect you until now. It is an error to think that the addslashes() call during the saving of data had to be countered with a stripslashes() call while reading the data, because, again, MySql had already removed them while writing the data into the database.
If so what's the point of stripslashes()?
There is no point in the context of what you are doing. It only is useful if you have received somehow a string in which there are characters that have a backslash before them, and have the meaning to escape the next character when the string is used in some context. But since you only briefly have such strings (after calling addslashes()) and MySql interprets these escapes by removing the backslashes in the written data, you never again see that string with the additional backslashes.
Situations where you would need stripslashes() are very rare and very specific. For your scenario you don't need it.
Confused about escape string and how it is stored in the database
In my MySQL call, I escaping a string with a backslash:
UPDATE `TABLE` SET `PERSONAL_BELONGINGS` = 'Tom\'s things'
But when I look in the phpadmin - the value was saved like this:
|Tom's things|
Why is the backslash not saved into the database?
This causes problems when I read this value into javascript and then try passing it around - my javascript strings will terminate. This is why I escaped the character to begin with.
Why is MySQL removing the '\' backslash before it is saved into the database?
If not saving it in the database with the '\' - What then is best way to deal with this as you are passing it back to the javascript as a string? To escape it again when passed as a string to javascript?
Let me begin by saying that you should really not really store data in any particular escaped format in the database, you'll regret it later if you need to extract it in another format or search the data for some reason later. The format you're saving now looks good, and adding backslashes for Javascript is better done in code when passing the data to the actual Javascript.
Now this is why it currently behaves like it does;
In the string 'Tom\'s things', \' is a character escape sequence and is really only used to let MySQL understand how to parse the SQL string, it's never saved as is to the database.
The reason you escape the character ' in the SQL statement you're showing to begin with is that otherwise MySQL has no way of knowing that the string does not end at the single quote after 'Tom.
If you use MySQLi or PDO prepared statements instead of building your SQL statements yourself, MySQL will let you save values entirely unchanged without having to ever escape anything. This is definitely the preferred option, since the MySQL API that does not support prepared statements is deprecated anyway.
The backslash is treated as an "escape character". If there was no backslash your string would end at Tom but the remaining s things would cause a syntax error.
The \ tells MySQL to not treat the escaped ' as a string delimiter but carry on until the next unescaped ' is found.
This escape character is only used for the query purposes and is not treated as part of the string you want to update.
Like Alvin suggested in comments, if you want to keep the backslash in your database you have to add it by adding another escaped backslash, i.e. \\. This would make your query look like:
UPDATE `TABLE` SET `PERSONAL_BELONGINGS` = 'Tom\\\'s things'
And the data in database would look like:
|Tom\'s things|
You can read more about string literals and escaping special characters in MySQL Manual
It's worth noting though, that storing an already escaped string in database is a bad practice. You should take care of escaping special characters in your code.
Because, in MySQL, the backslash is an escape character (just like it is in PHP). You have to escape the backslash itself to store it -- so, \\ would store a single backslash. \\\' would store a backslash followed by a quote, since the first backslash escapes the second, and the third escapes the quote.
This causes problems when I read this value into javascript
Surely this doesn't.
Then you lock your money in a safe, you can blame a safe maker if it gets robbed.
But as soon as you get your money back and it got stolen - you cannot still blame a safe for this.
It is yours responsibility now.
So, if you need your data escaped for javascript - just escape it.
But of course not by using mysql_escape_string()
try using this:
Function EscapeBack(ByVal msData As Object) As String
Return (Replace(msData, "\", "\\"))
End Function
to use this:
EscapeQuote(filename.Text)
When I use mysqli->real_escape_string and add a string value to the db such as "who's", it adds the string without modifying the quote in any way. When I check the db, the value within it is "who's".
When I do not use mysqli->real_escape_string and add a string with a single quote like "who's", it just doesn't get added to the database. It is nowhere to be found.
This isn't normal, is it?
Yes, it is. It's a little difficult to answer this question in any more detailed a way than that.
real_escape_string() only escapes the data so it can be safely used in a query, it doesn't modify it in any way. When you don't escape it, the query has a syntax error in it so it fails - and the data is not inserted.
However the safest way to escape your data is to use prepared statements.
It is normal, and correct. the real_escape_string allows your string to be added to the database (including the quote). Without it, the apostrophe is interfering with the SQL, thus likely throwing an error, and that's why it's "nowhere to be found".
What real_escape_string does is, it escapes necessary characters only while adding these in the database. So it's like escaping the same type of quote in a string in PHP, for example:
$str = 'We escape single quotes like in O\'Connor';
And the single quote (apostrophe) there is to prevent the code from breaking, but the string value does not actually contain it. Likewise…
INSERT INTO table (`name`) VALUES ('O\'Connor')
in this example the backslash will not be inserted in the row. This is for security reasons, of course. Check Hackipedia's page to see some possible cases.
Hoping this was what you were asking.
I feel silly for asking such a rookie question, and I tried searching but all I could find are sources that show how to do the escaping, which is not what I'm asking.
I am using PDO to add stuff to a database. As a result stuff gets escaped. The problem is, when I display that stuff back to the user it gets displayed exactly as it is in the db. In other words an escaped single quote displays as a left slash plus a single quote.
I could just run some sort of find/replace on all escaped character sequences but I am sure there most be a 'proper' way to do this. A way that remains secure (or improves security)
Then you're doing it wrong. SQL escaping is only done to properly insert into the database and not break SQL syntax, the values are stored in their unescaped form. Values only need to be escaped for their transition into the database, the values are never supposed to be altered permanently. Especially with PDO you shouldn't need any escaping at all.
Most likely you have magic quotes turned on, which causes double escapism. Please read this carefully: http://php.net/magic_quotes
use stripslashes
echo stripslashes($var);
http://php.net/manual/en/function.stripslashes.php
Firstly magic quotes & runtime are disabled correctly in php.ini, and confirmed by phpinfo().
PHP version: 5.3.4
MySQL version: 5.1.52
I'm only using mysql_real_escape_string on the data, after htmlspecialchars and a trim, that's all the data cleaning on the variable.
Yet, when I submit a single quote, the slash remains in the database.
When running mysql_query I'm using "' . $var . '", although in the past this hasn't changed anything (could be due to the double quotes?).
Any ideas? and please don't tell me about PDO/prepared statements, I'm aware of them and I have my reasons for doing it this way.
Thanks!
Code example (this is the only thing done to the data):
mysql_real_escape_string( htmlspecialchars( trim( $data ) ) );
I'm only use mysql_real_escape_string on the data, after htmlspecialchars and a trim, that's all the data cleaning on the variable.
No. Only use mysql_real_escape_string for storing data in the database. Don't mangle your data when you store it.
The function htmlspecialchars is used to encode a string to HTML (< becomes < etc.) and it should only be used for this purpose.
Perhaps the massively misguided, unhelpful and damaging option
magic_quotes_gpc
Has been enabled?
You can check that in the output of phpinfo(), but there's not a lot you can do if the server admin has enabled it globally without the ability to overrride.
I recommend checking if it's on (on every page of the app of course), and if so, causing the application to die quickly and painfully to ensure that you avoid data corruption (which chiefly manifests itself as the proliferation of backslashes you described).
Then go around to the server admin's house with a blunt weapon of your choice.
Hopefully you can do all this before your database becomes overrun with hoards of evil self-multiplying backslashes.
your storing procedure is correct. (altough htmlspecialchars and/or trim is probably not needed - but i dunno about your application)
from the information you are providing there is no reason to be seen for your problem.
the next debugging approach would then be remembering whatever else you may changed or has been changed on your system (maby you are using some 3rd party installation image?).
if that fails ie is left to wild guessing possible causes, for which i will offer a first one:
mysql could be running in NO_BACKSLASH_ESCAPES -mode, which would cause backlashes to be treated as regular characters.
furthermore it looks like you are wrapping your strings in double quotes, which would then insert a single quote - which usually gets escaped - straight into your database, preceded by a backslash.
it may very likely be also possible that - as you are wrapping your strings with double quotes inside your sql statements, which is not how it should be like and i am baffled you dont get a syntax violation error, you end up with some query like "john\'s house" which is caused by the single quote escaping from mysql_real_escape, which would be correct if you had your query correctly wrapped by single quotes.
which leads me to the question. do you get a syntax error (or an injected query) when trying to insert double quotes?
as for your comment. you could very well prepare statements with pdo and, then get the query string form it, and execute them using mysql functions. however i realise that this is no solution to your problem.
please also try putting your whole query in only one variable and print that out directly before executing it. then have a look at it and follow any wrong manipulation back operation by operation that is done to produce the string.
If you use double quotes within the SQL commands after escaping the data:
SELECT "1\'2"
then it will store and return the value as 1\'2 with the backslash still intact.
The proper syntax for SQL strings is using single quotes. That's what mysql_real_escape_string is escaping for. Else it would have to escape double quotes, whose usage however it is completely unaware of.
Use double quotes in PHP. Use single quotes for SQL. Rewrite your code like that:
$x = escapy($x);
$y = escapy($y);
sql_query("INSERT INTO tbl (x,y) VALUES ('$x', '$y')");