PHP Mysql CodeIgniter Converting characters to symbols in very bizarre circumstances
Application Built on CodeIgniter.
Has been running for over a year. No problems.
Client fills in a form about a customer.
A simple trim($_POST['notes']) captures textarea form field text and saves to MySQL
no error reported in PHP or JavaScript
The other day I notice some text the client has entered, has had the brackets used in the text "()" replaced with the equivalent "()
I think... "That's strange... I don't recall any reason why those characters would have been replaced like that.!"
I take a look ... and a day later... here is my madness revealed:
The text in question is verbatim "
Always run credit card on file (we do not charge this customer for pick-up or return)
"
No matter what I did or changed on the code side.. I could not prevent the PHP... OR Javascript... Or MySQL... OR alien beings... - or whoever the heck is doing it - from converting the "()" in the text to "(). And I tried many things like cleaning the string in all ways known to man or god. Capturing the string previous to sending just before saving to the database. And the conversion would always take place just before the save to MySQL. I tried posting in different forms and fields... Same thing every time... could not stop the magic conversion to "().
What in the name of batman is in this magical text that is causing this to happen?? is it magic pixie dust sprinkled on to godaddy server it is running on??? 0_o
.......
Being the genius that I am 0_0 I decide to remove one word from the paragraph at a time.
Magically... as all the creatures of the forest gathered around - as I finally got to the word "file" in the paragraph, and removed it !!! Like magic - the "()" stay as "()" and are NOT converted to "()?!?!???!?!? :\ How come??I simply removed the word "file" from the text... How could this change anything?? What is the word "file" causing to change with how the string is saved or converted??
OK -So I tested this out on any and every form field in the app. Every single time, in any field, if you type the word "file" followed by a "(" it will convert the first "(" to "(; and the very next ")" to ")
So.. if the string is:
"file ( any number of characters or text ) any other text or characters"
On post, it will be converted mysteriously to:
"file ( any number of characters or text ) any other text or characters"
Remove the word "file" from the string, and you get:
"( any number of characters or text ) any other text or characters"
The alien beings return the abducted "()"
Anyone have a clue what the heck could be going on here?
What is causing this?
Is the word "file" a keyword that is tripping some sort of security measures? interpereting it as "file()"???
I dunno :\
It's the strangest thing I ever saw... Except for that time I walked in on Mom and Dad 0_o
Any help would be greatly appreciated, and I will buy you a beer for sure :)
The very large headed, - (way to much power for such tender egos) -, Noo-Noos here at stack have paused this question as "Off topic" LOL... honest to God these guys are so silly.
So - in an effort to placate the stack-gestapo - I will attempt to edit this question so that it is... "on topic"??? 0_o ... anything for you oh so "King" Stack Guys O_O - too bad you would never have the whit to ever notice such a bug... maybe some day. ;)
Sample code:
<textarea name="notes">Always run credit card on file (we do not charge this customer for pick-up or return) blah blah</textarea>
<?php
if(isset($_POST['notes']){
$this->db->where("ID = ".$_POST['ID']);
$this->db->update('OWNER', $_POST['notes']);
}
?>
Resulting MySQL storage:
"Always run credit card on file (we do not charge this customer for pick-up or return) blah blah"
InnoDB - Type text utf8_general_ci
I am not looking for a way to prevent it, or clean it... I am clearly asking "What causes it"
/*
* Sanitize naughty scripting elements
*
* Similar to above, only instead of looking for
* tags it looks for PHP and JavaScript commands
* that are disallowed. Rather than removing the
* code, it simply converts the parenthesis to entities
* rendering the code un-executable.
*
* For example: eval('some code')
* Becomes: eval('some code')
*/
$str = preg_replace('#(alert|cmd|passthru|eval|exec|expression|system|fopen|fsockopen|file|file_get_contents|readfile|unlink)(\s*)\((.*?)\)#si', "\\1\\2(\\3)", $str);
This is the part of XSS Clean. (system/core/Security.php)
If you want the filter to run automatically every time it encounters POST or COOKIE data you can enable it by opening your application/config/config.php file and setting this:
$config['global_xss_filtering'] = TRUE;
https://www.codeigniter.com/user_guide/libraries/security.html
try something like this
$this->db->set('OWNER', $_POST['notes'],FALSE);
$this->db->where('ID ', $_POST['ID']);
$this->db->update('table_name');
Men I think Is in your server. If Ur using Wamp try to check if you have miss Install some arguments in xhtml. This is my Idea. it's related on my experience in CodeIgniter. hope U will response if you want some advice.
Use utf8 encoding to store these values.
To avoid injections use mysql_real_escape_string() (or prepared statements).
To protect from XSS use htmlspecialchars.
How ever not sure what is the issue in ur case..
Probably try using some other sql keywords in the string and verify the solution.
Try replacing the ( and the ) with ( and ) using str_replace
If you are storing ( and ) in your database then you should try replacing it on output if not try and replace it before input.
I'm not sure if this would work, but you could try inserting a slash in or before the word 'file':
fi\le ( any number of characters or text ) any other text or characters
Related
OK, so I shave my head, but if I had hair I wouldn't need a razor because I'd have torn it all out tonight. It's gone 3am and what looked like a simple solution at 00:30 has become far from it.
Please see the code extract below..
$psusername = substr($list[$count],16);
if ($psusername == $psu_value){
$answer = "YES";
}
else {
$answer = "NO";
}
$psusername holds the value "normann" which is taken from a URL in a text based file (url.db)
$psu_value also holds the value "normann" which is retrieved from a cookie set on the user's computer (or a parameter in the browser address bar - URL).
However, and I'm sure you can guess my problem, the variable $answer contains "NO" from the test above.
All the PHP I know I've picked up from Google searches and you guys here, so I'm no expert, which is perhaps evident.
Maybe this is a schoolboy error, but I cannot figure out what I'm doing wrong. My assumption is that the data types differ. Ultimately, I want to compare the two variables and have a TRUE result when they contain the same information (i.e normann = normann).
So if you very clever fellows can point out why two variables echo what appears to be the same information but are in fact different, it'd be a very useful lesson for me and make my users very happy.
Do they echo the same thing when you do:
echo gettype($psusername) . '\n' . gettype($psu_value);
Since i can't see what data is stored in the array $list (and the index $count), I cannot suggest a full solution to yuor problem.
But i can suggest you to insert this code right before the if statement:
var_dump($psusername);
var_dump($psu_value);
and see why the two variables are not identical.
The var_dump function will output the content stored in the variable and the type (string, integer, array ec..), so you will figure out why the if statement is returning false
Since it looks like you have non-printable characters in your string, you can strip them out before the comparison. This will remove whatever is not printable in your character set:
$psusername = preg_replace("/[[:^print:]]/", "", $psusername);
0D 0A is a new line. The first is the carriage return (CR) character and the second is the new line (NL) character. They are also known as \r and \n.
You can just trim it off using trim().
$psusername = trim($psusername);
Or if it only occurs at the end of the string then rtrim() would do the job:
$psusername = rtrim($psusername);
If you are getting the values from the file using file() then you can pass FILE_IGNORE_NEW_LINES as the second argument, and that will remove the new line:
$contents = file('url.db', FILE_IGNORE_NEW_LINES);
I just want to thank all who responded. I realised after viewing my logfile the outputs in HEX format that it was the carriage return values causing the variables to mismatch and a I mentioned was able to resolve (trim) with the following code..
$psusername = preg_replace("/[^[:alnum:]]/u", '', $psusername);
I also know that the system within which the profiles and usernames are created allow both upper and lower case values to match, so I took the precaution of building that functionality into my code as an added measure of completeness.
And I'm happy to say, the code functions perfectly now.
Once again, thanks for your responses and suggestions.
I have a MySQL table that contains names and e-mail addresses. The data was originally imported from a .csv file, and it did not originally contain complete e-mail addresses. We had to append the #place.domain to the user's alias.
When the data is sitting in the MySQL table, it looks normal: person#place.domain; however, when I output the content in PHP, I get this: person #place.domain. There's always a space between the person and the #. It doesn't look like that in the MySQL column, nor does it look like that when I copy/paste the data into Notepad, Word, Excel, etc. Furthermore, if I erase the data in the column and manually replace it with person#place.domain, it displays normally in my PHP app. So I'm guessing there's some hidden character that PHP is picking up that I can't detect. Is there a way for me to clean this up? I've tried TRIM(), REPLACE,(), etc., all to no avail.
UPDATE: I've discovered that, when I click in the MySQL field and move among the characters using my arrow keys, I have to hit the arrow key TWICE to move past the # symbol, yet there is no visible space.
I made this sample code for you:
<?php
$test = "user #mail.com";
$aux = explode("#",$test);
$mailok = trim($aux[0])."#".trim($aux[1]);
echo $test." vs ".$mailok;
?>
This is likely something like non-breaking space (ascii 160). To get rid of it:
UPDATE my_table SET e_mail = REPLACE(e_mail, CHAR(160), '');
Try with a foreach cycle, and do the chr($char) function to every character in the string, it will display you the ascii code of each character in the string, and you will find the wrong character. It's the only solution I found. Hope to be useful
In a project I am building I would like to use markdown as follows
*text* = <em>text</em>
**text** = <strong>text</strong>
***text*** = <strong><em>text</em><strong>
As those are the only three markdown formats I require, I would like to remain lightweight and avoid importing the entire PHP markdown library as that would introduce features I do not require and create issues.
So I have been trying to build some simple regex replaces. Using preg_replace I run:
'/(\*\*\*)(.*?)\1/' to '<strong><em>\2</em></strong>'
'/(\*\*)(.*?)\1/' to '<strong>\2</strong>'
'/(\*)(.*?)\1/' to '<em>\2</em>',
And this works great! em, bold, and the combo all work fine...
But if the user makes a mistake or enters to many stars, everything breaks.
i.e.
****hello**** = <strong><em><em>hello</em></strong></em>
*****hello***** = <strong><em><strong>hello</em></strong></strong>
******hello****** = <strong><em></em></strong>hello<strong><em></em></strong>
etc
When ideally it would create
****hello**** = *<strong><em>hello</em></strong>*
*****hello***** = **<strong><em>hello</em></strong>**
******hello****** = ***<strong><em>hello</em></strong>***
etc
Ignoring the un-required stars (so it would become clear to the user they made a mistake, and more importantly, the rendered HTML remains valid).
I presume there must be some way to modify my regex to do this but I cannot for the life of my work it out, even after a whole day trying!
I would also be happy with the result of
******hello****** = <strong><em>hello</em></strong>
So please, can anybody help me?
Also please consider uneven stars. In this case the below scenario would be ideal.
***hello* = **<em>hello</em>
And the time when a star should be part of the body and not detected, such as if a user inputs:
'terms and conditions may apply*'
or
'I give the film 5* out of 10'
Many many thanks
Try different capturing pattern (match anything except * one or more times),
'/(\*\*\*)([^*]+)\1/'
I am having hard time getting a form submitted textarea where people put in line breaks when they hit enter on their address..For example.
<textarea name="address"></textarea>
I am using this to get the post
$address = mysql_real_escape_string(trim($_POST['address']));
They are entering 123 test <return>city,state.
Then after they submit, in the mysql database the address is showing 123 testnrcity state
So how can I handle this?
Thanks!
Use nl2br?
$address = mysql_real_escape_string(trim(nl2br($_POST['address'])));
HTH :)
Have you checked what's in your database field? Just show the database contents with phpMyAdmin or a similar tool. The line break can either be a line break character or two characters: \n. If it's the latter, then your input probably was escaped twice. Or is it just "n". Then there is probably another escaping somewhere along the way. Do you use a database abstraction layer? Maybe it escapes the values too.
I was not able to resolve this the way I wanted to and just settled with using str_replace() to replace all "\r\n" stuff with a space...
I have a form that, among other things, accepts an image for upload and sticks it in the database. Previously I had a function filtering the POSTed data that was basically:
function processInput($stuff) {
$formdata = $stuff;
$formdata = htmlentities($formdata, ENT_QUOTES);
return "'" . mysql_real_escape_string(stripslashes($formdata)) . "'";
}
When, in an effort to fix some weird entities that weren't getting converted properly I changed the function to (all that has changed is I added that 'UTF-8' bit in htmlentities):
function processInput($stuff) {
$formdata = $stuff;
$formdata = htmlentities($formdata, ENT_QUOTES, 'UTF-8'); //added UTF-8
return "'" . mysql_real_escape_string(stripslashes($formdata)) . "'";
}
And now images will not upload.
What would be causing this? Simply removing the 'UTF-8' bit allows images to upload properly but then some of the MS Word entities that users put into the system show up as gibberish. What is going on?
**EDIT: Since I cannot do much to change the code on this beast I was able to slap a bandaid on by using htmlspecialchars() rather than htmlentities() and that seems to at least leave the image data untouched while converting things like quotes, angle brackets, etc.
bobince's advice is excellent but in this case I cannot now spend the time needed to fix the messy legacy code in this project. Most stuff I deal with is object oriented and framework based but now I see first hand what people mean when they talk about "spaghetti code" in PHP.
function processInput($stuff) {
$formdata = $stuff;
$formdata = htmlentities($formdata, ENT_QUOTES);
return "'" . mysql_real_escape_string(stripslashes($formdata)) . "'";
}
This function represents a basic misunderstanding of string processing, one common to PHP programmers.
SQL-escaping, HTML-escaping and input validation are three separate functions, to be used at different stages of your script. It makes no sense to try to do them all in one go; it will only result in characters that are ‘special’ to any one of the processes getting mangled when used in the other parts of the script. You can try to tinker with this function to try to fix mangling in one part of the app, but you'll break something else.
Why are images being mangled? Well, it's not immediately clear via what path image data is going from a $_FILES temporary upload file to the database. If this function is involved at any point though, it's going to completely ruin the binary content of an image file. Backslashes removed and HTML-escaped... no image could survive that.
mysql_real_escape_string is for escaping some text for inclusion in a MySQL string literal. It should be used always-and-only when making an SQL string literal with inserted text, and not globally applied to input. Because some things that come in in the input aren't going immediately or solely to the database. For example, if you echo one of the input values to the HTML page, you'll find you get a bunch of unwanted backslashes in it when it contains characters like '. This is how you end up with pages full of runaway backslashes.
(Even then, parameterised queries are generally preferable to manual string hacking and mysql_real_escape_string. They hide the details of string escaping from you so you don't get confused by them.)
htmlentities is for escaping text for inclusion in an HTML page. It should be used always-and-only in the output templating bit of your PHP. It is inappropriate to run it globally over all your input because not everything is going to end up in an HTML page or solely in an HTML page, and most probably it's going to go to the database first where you absolutely don't want a load of < and & rubbish making your text fail to search or substring reliably.
(Even then, htmlspecialchars is generally preferable to htmlentities as it only encodes the characters that really need it. htmlentities will add needless escaping, and unless you tell it the right encoding it'll also totally mess up all your non-ASCII characters. htmlentities should almost never be used.)
As for stripslashes... well, you sometimes need to apply that to input, but only when the idiotic magic_quotes_gpc option is turned on. You certainly shouldn't apply it all the time, only when you detect magic_quotes_gpc is on. It is long deprecated and thankfully dying out, so it's probably just as good to bomb out with an error message if you detect it being turned on. Then you could chuck the whole processInput thing away.
To summarise:
At start time, do no global input processing. You can do application-specific validation here if you want, like checking a phone number is just numbers, or removing control characters from text or something, but there should be no escaping happening here.
When making an SQL query with a string literal in it, use SQL-escaping on the value as it goes into the string: $query= "SELECT * FROM t WHERE name='".mysql_real_escape_string($name)."'";. You can define a function with a shorter name to do the escaping to save some typing. Or, more readably, parameterisation.
When making HTML output with strings from the input or the database or elsewhere, use HTML-escaping, eg.: <p>Hello, <?php echo htmlspecialchars($name); ?>!</p>. Again, you can define a function with a short name to do echo htmlspecialchars to save on typing.