Does using hex encoding to prevent SQL injections have any weaknesses? - php

Reading some of the SQL injection questions and answers on SO, I saw this answer that suggests you can convert untrusted user input to hex, which will by its very nature not require escaping of any kind, and thus completely and utterly avoid the possibility of SQL injection.
What database abstraction layer you use (PDO, mysqli, mysql, Pear DB, etc.) does not matter.
Example of a normal working query:
$DBH = new PDO('mysql:host=127.0.0.1;dbname=test', 'test', 'testpassword');
// could have been:
//$bookTitle = bin2hex($_GET['title']);
$bookTitle = bin2hex('Catch-22');
$query = "SELECT * from `books` WHERE `title` = UNHEX('$bookTitle')";
foreach ($DBH->query($query) as $row) {
echo "\n<br />\n";
print_r($row);
echo "\n<br />\n";
}
I've included enough code here for you to quickly run a test if you have a database with a table such as:
CREATE TABLE `books` (`id` INT, `title` VARCHAR(100), `author` VARCHAR(100)) ENGINE=InnoDB CHARACTER SET='utf8';
INSERT INTO `books` VALUES(1, 'Catch-22', 'Joseph Heller');
Of course this is oversimplified - you'd normally have input validation, output sanitizing and lots of other abstractions but we want to focus on the question - no more fluff than necessary to help provide easily executed examples.
What I'd like to know is if there are any technical weaknesses to this technique. I am specifically not asking if this technique has human weaknesses (easier to futz as a sloppy programmer, since it is clearly not as clean as using parameterized queries).
Yes, we can all agree that parameterized queries are less susceptible to bad programming or unfortunate oversights. So please stick to the question - does this technique help avoid SQL injections of all kinds, unconditionally?
Can someone show an example of user input that would break this technique? Even a corner case, some particular MySQL server settings or old PHP version that breaks it?
It works just as well for integer replacements:
// could have been:
//$bookID = bin2hex($_GET['id']);
$bookID = bin2hex(1);
$query = "SELECT * from `books` WHERE `id` = UNHEX('$bookID')";
Other thoughts:
With this technique you avoid two round trips to the database as happens if using (non-emulated) prepared statements (although prepared statements != parameterized queries).
The caveats to some other suggested techniques start to make one go cross-eyed what with the corner case exceptions such as this and this. Does the hex encoding technique avoid all possibility of attackers wreaking havoc using character set encoding tricks and whatever else is out there?
It appears that decoding on the database side may be limited to MySQL/MariaDB, although there may be third-party solutions for adding UNHEX() to PostgreSQL (or - not sure - in some databases, you might be able to use the other method of placing the hex literals in the query without using any UNHEX function)

We use this hex-encoding technique on an ecommerce system that has been live for a few years.
I've yet to come across a disadvantage and, once you have an object written for every table in the database (with the corresponding retrieve & store functions), it's all so hidden from the UI developers, they really don't even notice.
This one technique works for
blocking SQL-Injection
supports any client's character encoding
storing binary data
Its also very fast (we had considered base64 encoding, which would produce shorter SQL strings, but require more CPU). In tests we saw no measurable performance difference between hex-encoding all string input and not.
does this technique help avoid SQL injections of all kinds, unconditionally?
I'm yet to see how anybody could SQLi if this technique was always used (when binding any external data into an SQL statement) -
As is common to all techniques (DCoder) - it has to be universally used to provide universal protection - mysql_real_escape_string also suffers this.
user "N.B" rants, but provides no technical argument (may be he was talking about PokemonGo vs CSGO, who knows?), and the only answer clearly misunderstands the technique.
However, I do find the lack of people using / documenting this technique highly surprising - this is the only reason I have any doubt about it.
It's so simply, easy, comprehensive and provides a range of other useful benefits, it's hard to get my head round why everybody isn't doing it and SQLi is a thing of the past???

Seems logical since all the data, even the user input that could cause injections, would be converted. I've been trying to find ways of bypassing it but I can't. So far the biggest disadvantages I see are: 1) From a database view you won't be able to make sense of the user's input. You'll have pull everything from the DB and convert it just to see it or manipulate it. 2) Whatever input you provide, the string length will get doubled after conversion.

Related

Not using php prepare statement when I am not getting an input from a user

P.S: I know it is a best practice to user prepared statements all the time, and it should be a habit to use prepared statements. So putting this aside, please just let me know technically what can go wrong if I don't write below scenario with prepare.
1. I am not getting any input from users or any other class file, and I won't in the future.
2. I am getting an input from a variable in the same PHP file (an array for example).
Example: ($myID will be a variable hardcoded in the same PHP file)
$myID=12;
$wpdb->query("UPDATE `$table_name` SET `your_column_1` = 1 WHERE `myTable`.`your_column_id` = $myID");
Tl;DR This is a very bad idea. You are introducing long-term risk to save a few seconds of coding effort. It is overwhelmingly likely that you will, sooner or later, introduce a SQL injection risk as your code and data evolve.
If you:
are extremely careful about validating your data,
are extremely careful about constructing your queries,
are absolutely certain that your data is safe and free of any user input (from any source at all, including forms, sensors, APIs, scraping websites, etc.), and
are absolutely certain that no one will ever modify your data or your code (or reuse your code inappropriately), including you,
Then we can say:
it would be safe to execute queries without prepared statements, and - and this is the important part -
you would be living in a fantasy world.
You can never assume safely that you can avoid preparing statements. It’s highly, highly likely that your code will break if you do. You could have bad data, overlook a problem, change the code yourself somehow, repurpose the code for something insecure, or make any number of other errors. You may end up doing harmless SQL injection yourself (with, say, a name or spelling like O’Brien) or you may get massively hacked...
...all to save a few seconds of coding time.
Short version: either (1) use prepared statements on every query that has any kind of variable information or (2) learn the hard way why that’s the rule.
This is a wordpress plugin and will only be used by me in admin panel. So the problem caused because of using "in" statement because it is hard to write the query like update column where color in ('black','white').
If you're developing for WordPress, have you considered using the wpdb API? It makes it pretty easy to add parameters to your SQL queries.
Example of using parameters for an IN( ) predicate:
$colors_array = ["black", "white"];
$placeholders = array_fill(0, count($colors_array), "%s");
$placeholder_list = implode(",", $placeholders);
$wpdb->query( $wpdb->prepare(
"
UPDATE $wpdb->stock
SET quantity = 327
WHERE color IN ($placeholder_list)
",
$colors_array
));
See https://codex.wordpress.org/Class_Reference/wpdb#Protect_Queries_Against_SQL_Injection_Attacks
I agree with Ed Cottrell's advice that you should not compromise on secure programming methods. Use the most secure method and use it consistently.
You don't have to waste time thinking about whether any given case is "safe enough" to skip using the secure method.
You don't have to worry if it's still safe after your PHP variables are no longer hard-coded.
You don't have to worry that someone will copy & paste your code as an example, and they use it in an unsafe way.

Security with PDO select querys?

Could anyone help me with my query?
I am currently converting all my old PHP code to use PDO.
I was wondering when is it necessary to use the prepare function for my querys?
I currently have this query.
$sql = "SELECT deckName FROM decks WHERE deleted = '0' ORDER by deckName";
None of it is dynamic apart from maybe the deleted column.
I am using the below to reiterate my data on screen.
foreach($DBH->query($sql) as $row){ echo $row['deckName']?> }
Should I still be using this as good practice or is the above good enough?
$sth = $DBH->prepare("SELECT deckName FROM decks WHERE deleted = '0' ORDER by deckName");
I am not quite sure how to use the fetch statement correctly when reiterating data from the row?
Thanks in advance :)
Hayley
If you send a constant string to the database and do never mix in "untrusted input" the security concern is in my experience irrelevant, and only a potential performance aspect remains, which depends on the underlying database. IF the preparation step of an expensive to parse query may be spared for the execute phase (prepare, bind=NOOP here, execute) of all invocations after the first.
So guessing from the tags, that you use mysql and do not use memcached or the like as an intermediate layer, you might just want to measure the performance, otherwise you could migrate to direct queries.
When the "Project" later needs variable parametrization of the query (based on untrusted input e.g. from the network), you should use again prepare, to shield against injection attacks. Maybe then it will be good to find out "how to use the fetch statement correctly when reiterating data from the row" ...
Amending to the answer to accomodate the "uppercased nevers" for using variables in queries from the comments:
There is a practical reason for me writing "receiving untrusted input" instead of "never this and never that". In my experience there is a slippery slope, when it comes to variables: Without auditing the wider context of the $sql variable in above question - even the whole query is potentially vulernable to injection attacks.
Also internal refactorings of building query strings from constants may lead to mixing further variables to compose a (more complex) string that acts as a query.
For a more balanced / gap minimizing hardening I would suggest:
Documenting a specific map of say "trusted boundaries" in a code realizing an app, and based on that, enforcing strict validation of any input crossing these boundaries - i.e. variables transporting values accross - is in most cases effective and efficient.
Adding the common sense concept of layered security / separation of concerns will then naturally lead to using prepared statements when "such input" (that crossed a boundary) is to be "injected" as parameter.
I am reluctant to hand out simple recipes giving an optimistic sense of security, when one deals with such a scyscraper like application stack as db based web apps usually do.

Preventing SQL Injection in PHP without parameterized queries?

I know this topic has been covered to death but I would like some feedback from the community regarding security within our web application.
We have standard LAMP stack web app which contains a large number of database queries which are executed using mysqli_query. These queries are not parameterized and at the moment but there is some naive escaping of the inputs using addslashes.
I have been tasked with making this system safer as we will be penetration tested very shortly. The powers above know that parameterized queries are the way to go to make the system safer however they don't want to invest the time and effort into re-writing all the queries in the application and also changing the framework we have to make them all work correctly.
So basically I'm asking what my options are here?
I've run mysqli_real_escape_string over the inputs. I've setup a filter which doesn't allow words like SELECT, WHERE, UNION to be passed in which I guess makes it safer. I know mysqli_query only allows one query to be run at once so there's some security there (from concatenating updates onto the end of of selects).
Do I have any other options here?
Edit: I should probably add that if anyone is able to provide an example of an attack which is completely unavoidable without parameterized queries that would also be helpful. We have a query which looks like this:
SELECT
pl.created
p.LoginName,
pl.username_entered,
pl.ip_address
FROM loginattempts pl
LEFT JOIN people p ON p.PersonnelId = pl.personnel_id
WHERE p.personnelid = $id
AND pl.created > $date1
AND pl.created < $date2
I've substituted a UNION query into the $id UNION SELECT * FROM p WHERE 1 = 1 sort of thing and I can prevent that by not allowing SELECT/UNION but then I'm sure there are countless other types of attack which I can't think of. Can anyone suggest a few more?
Update
I've convinced the powers that be above me that we need to rewrite the queries to parameterized statements. They estimate it will take a few months maybe but it has to be done. Win. I think?
Update2
Unfortunately I've not been able to convince the powers that be that we need to re-write all of our queries to parameterized ones.
The strategy we have come up with is to test every input as follows:
If the user supplied input is_int that cast it as so.
Same for real numbers.
Run mysqli_real_escape_string over the character data.
Change all the parameters in the queries to quoted strings i.e.
WHERE staffName = ' . $blah . '
In accordance with this answer we are 100% safe as we are not changing the character set at any time and we are using PHP5.5 with latin1 character set at all times.
Update 3
This question has been marked as a duplicate however in my mind the question is still not followed answered. As per update no.2 we have found some strong opinion that the mysqli_real_escape string function can prevent attacks and is apparently "100% safe". No good counter argument has since been provided (i.e. a demonstration of an attack which can defeat it when used correctly).
check every single user input for datatype and where applicabile with regular expressions (golden rule is: never EVER trust user input)
use prepared statements
seriously: prepared statements :)
it's a lot of work especially if your application is in bad shape (like it seems to be in your case) but it's the best way to have a decent security level
the other way (which i'm advising against) could be virtual patching using mod_security or a WAF to filter out injection attempts but first and foremost: try to write robust applications
(virtual patching might seem to be a lazy way to fix things but takes actually a lot of work and testing too and should really only be used on top of an already strong application code)
Do I have any other options here?
No. No external measure, like ones you tried to implement, has been proven to be of any help. Your site is still vulnerable.
I've run mysqli_real_escape_string over the inputs
Congratulations, you just reinvented the notorious magic_quotes feature, that proven to be useless and now expelled from the language.
JFYI, mysqli_real_escape_string has nothing to do with SQL injections at all.
Also, combining it with existing addslashes() call, you are spoiling your data, by doubling number of slashes in it.
I've setup a filter which I guess makes it safer.
It is not. SQL injection is not about adding some words.
Also, this approach is called "Black-listing" it is proven to be essentially unreliable. A black list is essentially incomplete, no matter how many "suggestions" you can get.
I know mysqli_query only allows one query to be run at once so there's some security there
There is not. SQL injection is not about adding another query.
Why did I close this question as a duplicate for "How can I prevent SQL-injection in PHP?"?
Because these questions are mutually exclusive, and cannot coexist on the same site.
If we agree, that the only proper answer is using prepared statements, then a question asks "How can I protect using no prepared statements" makes very little sense.
At the same time, if the OP manages to force us to give the positive answer they desperately wants, it will make the other question obsoleted. Why use prepared statements if everything is all right without them?
Additionally, this particular question is too localized as well. It seeks not insight but excuse. An excuse for nobody but the OP personally only. An excuse that let them to use an approach that proven to be insecure. Although it's up to them, but this renders this question essentially useless for the community.

Shortcomings of mysql_real_escape_string?

I have seen a few people on here state that concatenating queries using mysql_real_escape_string will not protect you (entirely) from SQL injection attacks.
However, I am yet to see an example of input that illustrates an attack that mysql_real_escape_string would not protect you from. The majority of examples forget that mysql_query is limited to one query and use mysql_real_escape_string incorrectly.
The only example I can think of is the following:
mysql_query('DELETE FROM users WHERE user_id = '.mysql_real_escape_string($input));
This would not protect you from the following input:
5 OR 1=1
I would see this as incorrect usage of mysql_real_escape_string rather than a shortcoming, it is designed for strings not numeric values. You should either cast to a numeric type or if you are going to treat the input as a string when sanitising you should do the same in your query and wrap quotation marks around it.
Can anyone provide an example of input that can get around mysql_real_escape_string that does not rely on incorrect handling of numeric values or forget that mysql_query can only execute one query?
Edit: I am interested in the limitations of mysql_real_escape_string and not comparing it to alternatives, I realise there are better options for new projects and am not disputing that.
The main shortcoming of mysql_real_escape_string, or of the mysql_ extension in general, is that it is harder to apply correctly than other, more modern APIs, especially prepared statements. mysql_real_escape_string is supposed to be used in exactly one case: escaping text content that is used as a value in an SQL statement between quotes. E.g.:
$value = mysql_real_escape_string($value, $link);
$sql = "... `foo` = '$value' ...";
^^^^^^
mysql_real_escape_string makes sure that the $value in the above context does not mess up the SQL syntax. It does not work as you may think here:
$sql = "... `foo` = $value ...";
or here:
$sql = "... `$value` ...";
or here:
$sql = mysql_real_escape_string("... `foo` = '$value' ...");
If applied to values which are used in any context other than a quoted string in an SQL statement, it is misapplied and may or may not mess up the resulting syntax and/or allow somebody to submit values which may enable SQL injection attacks. The use case of mysql_real_escape_string is very narrow, but is seldom correctly understood.
Another way to get yourself into hot water using mysql_real_escape_string is when you set the database connection encoding using the wrong method. You should do this:
mysql_set_charset('utf8', $link);
You can also do this though:
mysql_query("SET NAMES 'utf8'", $link);
The problem is that the latter bypasses the mysql_ API, which still thinks you're talking to the database using latin1 (or something else). When using mysql_real_escape_string now, it will assume the wrong character encoding and escape strings differently than the database will interpret them later. By running the SET NAMES query, you have created a rift between how the mysql_ client API is treating strings and how the database will interpret these strings. This can be used for injection attacks in certain multibyte string situations.
There are no fundamental injection vulnerabilities in mysql_real_escape_string that I am aware of if it is applied correctly. Again though, the main problem is that it is terrifyingly easy to apply it incorrectly, which opens up vulnerabilities.
Ok, so apart from mysql_* being deprecated, I understand your wanting to know about any possible workaround that might exist. perhaps this blog post and the slides might reveal some of them.But as this older question here shows, casting and quoting isn't full proof. There's just So many things that can wrong, and Murphy's law, twined with that ever valid mantra "Never trust the network", will go horribly wrong.
Perhaps this article, but most importantly, the follow-up to that article can reveal even more security issues. To be honest, I know mysql_real_escape_string isn't fullproof, even in combination with type casting and string formats:
printf('WHERE id = \'%d\'',(int)mysql_real_escape_string($_REQUEST['id']));
doesn't cover every possible attack. I'm no expert on this matter, but what I can tell you is sanitizing every input, will, if anything, give you a FALSE sense of security. Most of the time, you'll know (initially) what and why and how you protect against the attacks, but your colleagues might not. They might forget something, and your entire system is compromized.
In summary: Yes, you might be able to prevent any form of malicious input from getting to your DB, but every additional action it requires is an added risk. In that scenario, the greatest liability (as always) is the developer that hasn't had is fourth cup of coffee on a monday morning. No code, no matter how defensive and well thought out, can protect itself from the monster that is a tired developer with a bad temper, going cold turkey on caffeine and nicotine.

How to secure project against hacks and attacks?

I have started in web development not long time ago. I know some stuff now, but I'm really concerned about security issues that may arise. I know simple security solutions like preg_replace , but I'm not confident with that.
So I would like to ask you for any sort of speaking "universal" security standards that can be applied in the following cases. As I mentioned, I'm not pro so it would be great if you can start with something simple, yet useful. If possible could you provide examples please?
I did have a look at php manual, although I would like to know additional info from person.
Here are some typical MySQL / PHP things I use in my projects. Could you suggest any improvements to make them more secure?
$sql = mysql_query("SELECT * FROM stories WHERE showing = 1 ORDER BY cr_date DESC LIMIT 5") or die (mysql_error("There was an error in connection"));
while($row = mysql_fetch_assoc($sql)){
$story_id = $row["id"];
// etc...
}
$username = $_POST['username'];
$sql = mysql_query("INSERT INTO myMembers (username, //etc... )
VALUES('$username' //etc.. ")or die (mysql_error());
$username = $_GET['username'];
//gets username from url like http://myweb.com/profile.php?username=blabla
First of all, thank you for caring about web security. Many PHP developers don't know anything about it, and don't care to learn. They are the ones who are exposing our passwords and bank accounts to hackers. Be part of the solution! :-)
1. Treat the mysql extension as if it is deprecated.
Use the PDO or mysqli extensions instead. The plain mysql extension does not support prepared statements, and some other features, such as transaction control. No one should be using mysql if they have PDO_mysql or mysqli available to them.
2. Do not interpolate external data into SQL.
Anytime you get a value from $_GET or $_POST, you should consider it to be unsafe to use in any SQL statement, or shell_exec(), or other instance where you execute the string as some kind of code.
3. Use prepared query parameters instead of interpolation.
It's really easy. In fact, it's easier to use query parameters than it is to interpolate variables into SQL strings. You don't need to worry about escaping, or long complex string-concatenation.
See example code here: http://us.php.net/manual/en/pdo.prepare.php
4. For corner cases, use careful filtering.
A query parameter takes the place for one literal value in an SQL expression. Not table names, not column names, not SQL keywords, not lists of values or full expressions. For those, you do need to use string interpolation, but see my presentation SQL Injection Myths and Fallacies for examples of how you can "whitelist" values to interpolate.
Also check out the PHP filter extension, which offers a flexible way of validating inputs or stripping off invalid characters to make sure only the valid part of the input is used.
Looking at your examples, the SELECT query has no dynamic values interpolated from external sources like $_GET. So that one is safe.
The INSERT query takes a value from the request, which could contain malicious content that changes the way your query runs. This one is a good candidate for using query parameters.
Also consider that SQL injection is one of the two most prevalent security issues with PHP. The other one is Cross-Site Scripting (XSS). This is not directly related to SQL, but you should learn about it too.
Here's a good resource for learning more about web programming security: OWASP.org cheat sheets.
Many frameworks have a good set of security measures already in place that will do a great deal in preventing things like SQL injections. Yii, CakePhP, CodeIgnitre all may be of some use.
Although it's almost impossible to beat Bill, I feel I must clarify answers stating that "you have to trust no user input".
In fact, quite contrary - SQL injection protection will do any good only if it would be totally ignorant of the data source. And treat ALL the data as potentially malicious. And process it accordingly.
Thus, to summarize all the advises:
Prepared statements is a good approach but not a complete one.
It has a brilliant idea of using some placeholder, a proxy to represent the actual value in the query. Thus this value can be properly sanitized.
But these placeholders, as Bill said, are limited to the strings and numbers only. So, it would be a good idea to add another placeholder of your own - for identifiers. But you still have to watch out SQL syntax keywords manually.
So, instead of "Do not interpolate external data into SQL." statement one have to use
"2. Do not interpolate values into query directly but only by some proxy, performing necessary precautions"
The most important thing to remember is never trust anything from an external source (eg user input, responses from other web services etc). Always sanitise all input and where possible use code from your database provider to do so.
In the case of MySQL parameterising all queries is essential, so use a prepared statement, eg
$statement = $db->prepare('SELECT * FROM stories WHERE title = :title');
$statement->execute(array(':title' => $title));
$rows = $statement->fetchAll();
Your current insert statement is vulnerable to an SQL injection attack, modify it to be closer to:
$username = $_POST['username'];
$statement = $db.prepare("INSERT INTO myMembers (username) VALUES(':username');
$statement->execute(array(':username' => $username));
You should also ensure that you never store any passwords in plain text, always store a hashed version of a password (along with a salt) and check that the hash matches rather than the actual string. This means that should your database become compromised, figuring out your user's credentials becomes a non-trivial task.
These are only a couple of ways of making your app more secure, I'd highly recommend reading OWASPs top 10 site vulnerabilities and researching these individually as each one in itself is quite a big topic!

Categories