I'm not really sure why I'm having this issue but at the moment I'm blocked. The problem that I'm having is that when using Doctrine to insert into MySQL, some values are automatically escaped. For example, on a form where a user inputs something such as he/she, the data stored into the MySQL table is he\/she.
Now, the issue is that sometimes the user will use a search function, but using he/she won't give them any results. How can I solve this? I'm thinking on escaping the characters on the string that I pass to the LIKE function, is this a correct approach? I'm thinking that the same issue will happen if the user has values such as this is 50% (the percent would need to be converted as well before given to the LIKE function?)
You need to solve this at the core - the data should never enter into your database. A probable cause of this is a misconfigured web server that still has magic quotes enabled with MSSQL mode, or using a library that tries to do input sanitation in a (very) wrong way. You should never ever ever want to fix this on the search end of things - once you start patching there for corrupted data you're in for an endless cesspool of problems.
Debug why the data is going in wrong, and fix it there.
Related
I have discovered that some old (2009) code that was written for a website, did, under certain circumstances on a search query save the SQL as a a $_GET variable!
When the search was carried out, the details are POSTED and then sanitized, and the results are paginated with the LIMIT clause in MySQL. If there is more than one page (ie +30 results) the pages are anchor links in the HTML with a GET var containing the SQL statement.
I know, this is absolutely not the way to do this. It's old code I've just seen it by chance. This needs to be fixed.
So I've fixed it, sanitized it and used an alternative method to reload the SQL, BUT:
My question is thus:
The page outputs the data relating to thumbnail images, all data is output as named array var (the original clause is a SELECT * clause), so if someone does abuse the GET variable, the page itself will only output the columns named,
I have managed to DELETE rows from the DB using the GET abuse, I would like to think the abuse is only effective if the result is not involving any returned output (such as DELETE) but I don't know; so given that the user can input anything into the GET clause but only get the displayed output of what's coded (ie named columns in a 30 row array) -- what other abuses can this gaping hole be open to?
Further details: The code is MySQLi
A tool like SQLMAP can probably take over the entire server and do with it whatever the user wants.
Having an unsanitized database input isn´t even hacking anymore, it´s waiting for someone to run a script on your machine and basically own it from that point on.
What the attacker can do depends on your database configuration and database user access. If you create a new user with a permission to only SELECT that one specified table, and use that user for that particular script, the harm it can do is reading data from that table.
Still this is bad practice. Never use it.
Ok, I am trying to store data in a MySQL database from an Android app. I use PHP as the go between.
What I submit to the database is this (for example):
Sakè
What is stored in MySQL is:
Sakè
When I print the line back out in Android it knows to convert back to:
Sakè
The problem is, when I do certain select statements, yeah it basically doesn't work out like it should. Is there any way I can keep this consistent?
Just a note in MySQL: I use utf8_general_ci
You need to enforce correct character for your database connection for your PHP scripts. Simplest approach would be to execute this query:
SET NAMES 'utf8';
just once, best after you connect(). If you use Zend Framework, just add this:
resources.db.params.charset = UTF8
to your application.ini. For mysqli see set_charset(). If you use anything else, check the docs for something similar. If there's nothing, the SET NAMES trick would always work.
One important note though - depending on how you put your data to the DB, your database content most likely require fixing as it is corrupted by broken charset during INSERTs. So if even you do SET NAMES now, you most likely still see old data shown wrong. But that is, ironically, OK - you are now showing all fine, but data itself is broken. You just now need to fix your data (just re-insert it).
I'm not sure if this is a duplicate of another question, but I have a small PHP file that calls some SQL INSERT and DELETE for an image tagging system. Most of the time both insertions and deletes work, but on some occasions the insertions don't work.
Is there a way to view why the SQL statements failed to execute, something similar to when you use SQL functions in Python or Java, and if it fails, it tells you why (example: duplicate key insertion, unterminated quote etc...)?
There are two things I can think of off the top of my head, and one thing that I stole from amitchhajer:
pg_last_error will tell you the last error in your session. This is awesome for obvious reasons, and you're going to want to log the error to a text file on disk in case the issue is something like the DB going down. If you try to store the error in the DB, you might have some HILARIOUS* hi-jinks in the process of figuring out why.
Log every query to this text file, even the successful ones. Find out if the issue affects identical operations (an issue with your DB or connection, again) or certain queries every time (issue with your app.)
If you have access to the guts of your server (or your shared hosting is good,) enable and examine the database's query log. This won't help if there's a network issue between the app and server, though.
But if I had to guess, I would imagine that when the app fails it's getting weird input. Nine times out of ten the input isn't getting escaped properly or - since you're using PHP, which murders variables as a matter of routine during type conversions - it's being set to FALSE or NULL or something and the system is generating a broken query like INSERT INTO wizards (hats, cloaks, spell_count) VALUES ('Wizard Hat', 'Robes', );
*not actually hilarious
Start monitoring your SQL queries by starting the log. There you can look what all queries are fired and errors if any.
This tutorial to start the logger will help.
Depending on which API your PHP file uses (let's hope it's PDO ;) you could check for errors in your current transaction with s.th. like
$naughtyPdoStatement->execute();
if ($naughtyPdoStatement->errorCode() != '00000')
DebuggerOfChoice::log( implode (' ', $naughtyPdoStatement->errorInfo() );
When using the legacy-APIs there's equivalents like mysql_errno, mysql_error, pg_last_error, etc... which should enable to do the same. DebuggerOfChoice::Log of course can be whatever log function you'd like to utilise
I've been coding with PHP and MySQL for about a year now and have gotten the hang of it pretty well; constructing really complicated queries with joins and calculated fields and all the other joys of MySQL hasn't been a problem for me in months.
BUT there's something syntactically screwy with the following chunk of code that I can't figure out - even though it's impossibly simple, and, even more infuriating, is closely related to other parts of the project that I'm working on (and which works flawlessly).
Here's the problem code I'm trying to run, followed by the bugchecking I've already done to try to isolate the problem.
If anyone has any suggestions, I'd be totally grateful because I'm beginning to lose my mind.
Problem:
I'm really losing my mind over this, so please don't laugh when you see the code:
$query="SELECT count(somefield) FROM db_name WHERE otherfield='".$myvariable."'";
My query finds no results when using a certain variable as part of a field search - even though I know that there are over 900 records in the database that should match.
Bugchecking:
Because I know the value of the variable I'm passing to the query, I've tried hardcoding it into the query and it works fine.
I've run the query in the MySQL console (again, of course, hardcoded instead of with the variable) and it works fine.
To my mind, these two facts eliminate the possibility that there's something syntactically incorrect with the PHP version of the query.
In order to eliminate all possible database connection issues and to make sure the problem isn't related to iterating through the results returned, instead of trying to get the actual results, I've altered my original query to return only the count of the results and have incorporated the standard or die(mysql_error()) statements during the connection sequence. The query is executing, but is finding 0 results, so that eliminates the possibility that it's a connection issue.
I've verified that the field I'm checking is the correct field for the information I'm looking for (like I said, it runs fine if I hardcode the variable into the query... which, of course, will not be an option in the finished code).
I've checked the type of the variable before trying to pass it into the query (figuring that maybe, because it's pulled from a returned xml earlier in the script, that maybe it was showing up as an array or something). It typed as string.
I've verified that the variable is formatted in the way that I expect it to be found in the database; strtoupper, etc.
I've tried using LIKE '%".$myvariable."'"; still no dice.
Anyone have any suggestions for what I can do to figure out what the hell is going wrong? Thanks so much!
It's not a PHP issue so quotes have nothing to do with it.
The query has no error, so you'll need to debug it step by step.
I. SELECT count(*) FROM table_name
II. SELECT count(*) FROM table_name WHERE field='$myvariable'
Where you're dealing with PHP variables in query, echo the query and run it directly in database to omit PHP's side of error.
III. SELECT count(somefield) FROM table_name WHERE field='$myvariable'
Is the $myVariable escaped? If not, escape it using
$escapedVariable=mysql_real_escape_string($myVariable);
and then run
$query="SELECT count(somefield) FROM db_name WHERE otherfield='$escapedVariable'";
Thanks so much to everyone who tried to help, but I figured it out several hours after posting: The first problem was that I forgot to use mysql_real_escape_string($myvariable)... the reasons for how and why I forgot are manifold, but there it is.
So, after plugging that guy back in (which I had ASSUMED had been in this particular module of my code in the first place, but that's where "assuming" gets you, lol), I thought I had the whole thing licked. Three hours later, and still nothing. THEN I realized that it had to be related to XML that was being parsed into the $myvariable... so around and around we went with that one for a few more hours.
FINALLY, I realized that the real culprit was my eyes (which aren't so great).. what looked like a perfectly legit quoted string while reading a debug echo of the query before running turned out to have leading and trailing white space (which, of course, I instantly removed with $myvariable=trim($myvariable, " "), and, that, of course, solved the entire problem...:<
... Yes, I am an idiot, and I'm sorry, but, after working over this UTTERLY INFURIATINGLY stupid line of code for over 48 hours (I'm used to writing things like:
$query="UPDATE db_one.table_one SET item1='".(string)$result_array[$i][1]."', item2='".(string)$result_array[$i][2]."' WHERE thing3=".(string)$result_array[$i][19];
... and other assorted fun nonsense), I had to resort to asking (because I - no pun intended - couldn't "see" the problem... ugh)... SO... I am an idiot, and I'm sorry (but encouraged by the efforts of all of you nice people who tried to help) and am sorry for wasting everyone's time. I need to learn how to handle XML much *more*.
Sorry and thanks again!
I always wrote it like this
$query="SELECT count(somefield) FROM db_name WHERE otherfield='$myvariable' ";
Try removing the double quotes and dot at the variable name
You shouldn't need the parenthesis around the var. Plus look at other changes.
$query="SELECT count(*) FROM table_name WHERE field='$myvariable'";
I just saw the first comment to this question Inserting into a serialized array in PHP and it made me wonder why? Especially seeing that when you use database managed sessions (database based session handling) that is exactly what happens, the session handler inserts a serialized array into a database field.
There's nothing wrong with this in certain contexts. Session management is definitely one of those instances where this would be deemed acceptable. The thing to remember is that if you ever find yourself trying to relate data between the serialized data and any fields in your database you've made a huge design flaw and unfortunately this is something that I have seen people try to do.
Take any "never do x" with a grain of salt as almost any technique can be the correct one in certain circumstances. The advice is usually directed towards noobies who are very apt to misunderstand proper usage and code themselves into a very nasty corner.
How certain are you that you'll never want to get at that data from any platform other than PHP?
I don't know about PHP's form of serialization, but the default binary serialization format from every platform I do know about is inoperable with other platforms... typically it's not a good idea to data encoded for just a single frontend into a database.
Even if you don't end up using any other languages, it means the database itself isn't going to know anything about the information - so you won't be able to query on it etc. Maybe that's not a problem in your case - but it's definitely something to bear in mind.
The main argument against serialized data is that serialized data are hard to search through and impossible to do so efficiently i.e., without retrieving the records in the first place.
Depends on the data. By storing a language-specific data structure in a field you're tied to that language and you're also giving up anything the DB can give you. You won't have indexes on specific fields, can't run simple updates, can't extract partial data, can't have data check, referential integrity and so on.