I have a notification system on my website where users will be notified of a certain event. The message of the notification will be stored in a MySQL database.
When I insert links into the message, it returns a MySQL error saying it cannot insert links.
How can I achieve that? I am using php.
Almost all links will include characters that will be interpreted by a sql database in an odd manner, leading to unpredictable results based on the varied input. You should ensure these characters are properly escaped:
$some_input = getInputFromWebPage();
$safe_input = mysql_real_escape_string($someInput);
insertIntoDB($safe_input);
That is just an example, and obviously not working code, but hopefully it heads you in the right direction. There are several functions that will add escape characters to strings for you.
Related
We're running into a weird edge case where we are trying to store a json blob in a table in our database, and that blob needs to be able to contain the \ character. So a user were to enter in \test it needs to come back as exactly that, but instead its coming back as a tab followed by "est"
As far as I can tell, whats happening is that when a user enters and submits "\test" it gets evaluated into "\ \test" (remove the space, cant put two backslashes in here and have it display right?) by the client and then entered into the table. I can verify that in the SQL that gets called against the table there are two backslashes. When I look at it in the table after this step its back to "\test". When the client loads it up again it gets evaluated into a tab followed by an "est".
We are under the impression that the second backslash is necessary so that the first backslash will get escaped and not evaluated but maybe that is what is causing issues? I sort of assume when the query runs one of the backslashes gets escaped anyway but I'm not really sure what to do about that. Is there something with out our database is handling backslashes that we need to be looking out for? Is there a way to handle this that we haven't considered?
It's a Postgres database if that's helpful. I'd say I'm beginner to intermediate on this sort of thing, I'm looking through documentation but if anyone can even point me in the right direction that would be very helpful.
Postgres version as far as i can tell through Amazon Aws is 9.3
EDIT
I think ive tracked this issue down to a line in our php backend that I don't really understand. I'm looking at the documentation for that now and will mark this as answered since I've verified that its not an issue with SQL.
Blockquote A backslash as - by default - no special meaning in SQL. This might be caused by whatever code is processing those values (and sending them to the database). See here for an online example: rextester.com/QLLYG57275 – a_horse_with_no_name
I'm accepting this as the answer as I've verified that the issue is with out backend code constructing the SQL, and not how the SQL is being handled on the database end.
Storing some values in a mysql database, the input is being sanitized with mysql_real_escape_string($value) and it displays fine. However while performing a direct query on the database, I'm seeing characters like †on each text field that has been edited using this form. How does that happen? It doesn't show when I display values on webpages but how can I prevent these characters from appearing at all?
I looked at this question: Strange characters in mysql dbase which seemed to have some advice on setting character names on the input, but how can I fix this for once and for all? I believe the person who's updating these values is copying and pasting directly from Microsoft Word, so I'm sure it has something to do with the "smart quotes" and other such fancy formatting that MS Word likes to use.
As the answer you linked shows, it comes from PHP which connects to mysql with latin1 encoding by default. So the datas are not correctly inserted in database.
Another problem is that if you query back the data in php, you get correct data as they are "decoded" the same way they are encoded. But if you perform direct query in database (say, with mysql client on console), data seem broken.
That's why the answer is to query "SET NAMES UTF8" before anything else.
You may parameter the mysql server to force utf8 on any connection. I do not see any other solution.
I've got a weird scenario going on here.
On my localhost running WAMP server (Apache, MySQL, PHP), I've created a webpage that displays a list of messages from a table in my database.
Let's say the DIV container was called: #message-list
This list gets displayed correctly (when the page is launched, PHP renders the whole page).
The HTML markup that PHP echoes-out works just fine.
The MySQL database lookup therefore also works! Great.
Now...
With a bit of AJAX and jQuery magic, I've created a form to add more messages on-the-fly, by sending a POST request to a PHP scripts that uses the SAME underlying code that generated the initial #message-list DIV.
The AJAX'ed PHP script does two things:
add a record for the new message from the user;
echoes the list (which should be updated with the new message now);
When the AJAX response comes back to the browser, the JavaScript side replaces the old list with the new #message-list content.
Now this... partially works.
What goes wrong - On one given page, it seems some of my previous posts are somehow "corrupting" something inside the AJAX request on the PHP side, resulting a null response (basically no HTML code gets generated to replace the #message-list DIV tag).
On some other pages though, the AJAX response works fine.
So my question is:
Is it possible that some String data in the Database breaks the execution of my PHP script because of some invalid character, badly encoded, or a quote / double-quote?
I've tried using PHP's htmlentities() and mysql_real_escape_string() functions to solve this, but one of my pages still doesn't properly refreshes the list after the AJAX response is received.
Could it just be that I just need to cleanup / sanitize the existing content in my table?
If so, is there any easy script / query I can use to do this?
Thanks!
EDIT #1:
MySQL version = 5.5.24-log
By using mysql_client_encoding, this shows "latin1" (ah HA! That may be the issue then!)
In PHP, using the mysql_... methods (such as mysql_connect, mysql_select_db, mysql_query, etc.);
Sample of database table with possible issue:
http://pastebin.com/PjLVmXEF
By the looks of it, many developers say PDO is recommended. I'll give that a shot and see if all errors vanish. Thanks a lot for your help so far everyone!
EDIT #2:
My current solution has been this:
I've used these queries to modify my database and the table with the encoding problem:
// SQL queries:
ALTER DATABASE timegrasp charset=utf8;
ALTER TABLE tg_messages CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
Second, I noticed some characters in a specific record was not displaying correctly (an  around some double-quoted sentences). So I manually backspaced and reinserted the double-quote in MySQL Query Browser to be sure it was completely gone.
On the PHP side, I only encode the messages on the way "in" to the database, with this:
$htmlConverted = htmlentities( $pMessage, ENT_COMPAT | ENT_HTML401, "UTF-8" );
return mysql_real_escape_string( $htmlConverted );
And make sure I begin my MySQL connection with this:
mysql_set_charset("utf8", $DB_LINK);
Then, I can just read the String directly from the table without any decoding / conversion.
Finally, to test this - I copied the same message from the source (a Skype chat with my client) which had the special characters, pasted it in my web form, and now it works fine! :)
I'm not certain all the steps and parameters above were necessary, but this is what helped fix my issue.
It would be good to know for future reference though if any of this is bad practice or common "dont's" mistake when handling special characters in MySQL tables.
The PHP json_encode function refuses to process strings that are invalid in UTF-8: it returns null. If you don't set a character encoding for the database connection it's possible that some other encoding is being used -- not UTF-8 -- and the data you pass to this function is in fact not valid UTF-8.
If you mentioned details such as which database API and which connection parameters you are using I could give further advice...
If a full name is submitted to the name column of database and it's pulled onto a web page it adds a + sign instead on the space.
Also if theres a " within the message text that in the message column and its pulled onto a web page it displays a \ before every "
Is there any way of fixing these issues
From the code that you added in your comments, expanding my comment in to an answer. The '+' is because you are urlencodeing some of your rows. urlencode is meant for data that will be part of a URL, what I think you are wanting to do is display it in HTML, in which you would want to use htmlentities. But right after pulling from your DB, you'll want to use stripslashes before using the htmlentities.
It appears that when you get your POST data, your server is already adding slashes. Depending on your server version, you'll want to check the Magic Quotes, and if enabled, stripslashes before pushing it through the mysql_real_escape_string. However, since your DB is already set up, it might be easier to skip this paragraph completely and deal with what you already have.
Side note, using 'prepared statements' is a better practice, and eliminates the need to use mysql_real_escape_string. ^^
So I have this website that has a search feature which searches a table in my mysql database. The database at the moment has 1108 rows. It contains music info such as Artist and Album. Since its possible for every character to be in an artist name or album name, I've urlencoded each of those variables before being added to the database. See below:
$artist = urlencode($_POST['artist']);
$album = urlencode($_POST['album']);
So now lets pretend that I have added a new entry to the database and it contain characters that needed to be urlencoded. The database shows it fine.
Now I want to go search.
Foreign characters worked. You can see here: http://albumarrrt.net/details.php?artist=Ai%20Otsuka clicking the album link for each one works.
But now a few problems occur.
1 - If you search for '&' the search reads the %26 as nothing. It shows %26 in the address bar, but it reads it as nothing.
Here is how it is being read:
$search = $_GET['search'];
if($search == '') {
echo "Please enter a search term :(";
}
That is the only thing done with $search before it starts getting read by the database.
2 - If you search for a single or double quotes, it does some weird stuff example:
Search for " and get No matches found
for "%5C%5C%26quot%3B" Search for '
and get No matches found for
"%5C%5C%26%23039%3B"
I don't understand why it does this, because the database only contains the code for the quote and nothing else.
Those are the only two things I have found wrong with my search. Maybe I have just been looking at it too long and can't figure it out, but it perplexes my why it doesn't read '&' as anything.
Onto my last question.
My current searching method separates each word and adds %'s around it and then uses the LIKE statement to find matches. example:
Search: A bunch of Stuff (word)
the mysql query would be like:
SELECT * FROM TABLE WHERE (album LIKE '%A%' AND album LIKE '%bunch%' AND album LIKE '%of%' AND album LIKE '%Stuff%' AND album LIKE '%%28word%29%') OR (artist LIKE '%A%' AND artist LIKE '%bunch%' AND artist LIKE '%of%' AND artist LIKE '%Stuff%' AND artist LIKE '%%28word%29%')
Obviously this is putting a lot of strain on the server, and I know using LIKE statements for such large database searching is a bad idea, so what would be an alternate way of searching FULL TEXT or some other method?
Sorry for the overwhelming amount of questions, But they all sorta go hand-in-hand with each other.
edit:
Ok I've fixed my database up, but still have a few questions.
Someone suggested to convert my text from utf8 to plain utf, how would I do this?
and I am still getting the problem with the & sign.
for example:
if you search for & on google it works, however on my site, my POST result for the search query reveals nothing when searching for &.
First: don't urlencode data in the database. Urlencode data after you fetch it, as you output to HTML.
Second: do use query parameters when you use user-supplied values in SQL queries. Then you don't have to worry about quotes in the form data causing syntax errors or SQL injection risks.
Third: don't use the LIKE '%pattern%' hack; instead use a real fulltext search solution instead (either FULLTEXT or Lucene/Solr or Sphinx Search). It'll have performance hundreds or thousands of times better than using ad-hoc text searching (depending on your volume of data).
See the presentation I did for MySQL University: Practical Full Text Search in MySQL.
I don't see why you need to urlencode, I would simply use mysql_real_escape_string.
'&' is a separator in a url so it won't be passed to your script unless you urlencode it first
Another problem with urlencode is the large number of extra characters. mySQL may silently truncate the artist or title if you haven't allowed for enough characters.
DC
are you sure you don't want to be decoding the things coming from your URL's (and POSTS) before placing them in the database? If I were storing various strings, I would want to decode them to plain UTF or something and store them that way. Then I would re-encode them to display them. This might solve your search problem in and of itself.
Second, to speed up strings search access, you could create a strings table with all of your strings tokenized, and linked back to the strings that contain them. Then instead of doing a "like %$1%" you can say where $1 = stringTable.String and join against that ID. By no means count this as the optimal solution as I haven't done those performance tunes myself, it's just a suggestion.