Could anyone help me with my query?
I am currently converting all my old PHP code to use PDO.
I was wondering when is it necessary to use the prepare function for my querys?
I currently have this query.
$sql = "SELECT deckName FROM decks WHERE deleted = '0' ORDER by deckName";
None of it is dynamic apart from maybe the deleted column.
I am using the below to reiterate my data on screen.
foreach($DBH->query($sql) as $row){ echo $row['deckName']?> }
Should I still be using this as good practice or is the above good enough?
$sth = $DBH->prepare("SELECT deckName FROM decks WHERE deleted = '0' ORDER by deckName");
I am not quite sure how to use the fetch statement correctly when reiterating data from the row?
Thanks in advance :)
Hayley
If you send a constant string to the database and do never mix in "untrusted input" the security concern is in my experience irrelevant, and only a potential performance aspect remains, which depends on the underlying database. IF the preparation step of an expensive to parse query may be spared for the execute phase (prepare, bind=NOOP here, execute) of all invocations after the first.
So guessing from the tags, that you use mysql and do not use memcached or the like as an intermediate layer, you might just want to measure the performance, otherwise you could migrate to direct queries.
When the "Project" later needs variable parametrization of the query (based on untrusted input e.g. from the network), you should use again prepare, to shield against injection attacks. Maybe then it will be good to find out "how to use the fetch statement correctly when reiterating data from the row" ...
Amending to the answer to accomodate the "uppercased nevers" for using variables in queries from the comments:
There is a practical reason for me writing "receiving untrusted input" instead of "never this and never that". In my experience there is a slippery slope, when it comes to variables: Without auditing the wider context of the $sql variable in above question - even the whole query is potentially vulernable to injection attacks.
Also internal refactorings of building query strings from constants may lead to mixing further variables to compose a (more complex) string that acts as a query.
For a more balanced / gap minimizing hardening I would suggest:
Documenting a specific map of say "trusted boundaries" in a code realizing an app, and based on that, enforcing strict validation of any input crossing these boundaries - i.e. variables transporting values accross - is in most cases effective and efficient.
Adding the common sense concept of layered security / separation of concerns will then naturally lead to using prepared statements when "such input" (that crossed a boundary) is to be "injected" as parameter.
I am reluctant to hand out simple recipes giving an optimistic sense of security, when one deals with such a scyscraper like application stack as db based web apps usually do.
Related
P.S: I know it is a best practice to user prepared statements all the time, and it should be a habit to use prepared statements. So putting this aside, please just let me know technically what can go wrong if I don't write below scenario with prepare.
1. I am not getting any input from users or any other class file, and I won't in the future.
2. I am getting an input from a variable in the same PHP file (an array for example).
Example: ($myID will be a variable hardcoded in the same PHP file)
$myID=12;
$wpdb->query("UPDATE `$table_name` SET `your_column_1` = 1 WHERE `myTable`.`your_column_id` = $myID");
Tl;DR This is a very bad idea. You are introducing long-term risk to save a few seconds of coding effort. It is overwhelmingly likely that you will, sooner or later, introduce a SQL injection risk as your code and data evolve.
If you:
are extremely careful about validating your data,
are extremely careful about constructing your queries,
are absolutely certain that your data is safe and free of any user input (from any source at all, including forms, sensors, APIs, scraping websites, etc.), and
are absolutely certain that no one will ever modify your data or your code (or reuse your code inappropriately), including you,
Then we can say:
it would be safe to execute queries without prepared statements, and - and this is the important part -
you would be living in a fantasy world.
You can never assume safely that you can avoid preparing statements. It’s highly, highly likely that your code will break if you do. You could have bad data, overlook a problem, change the code yourself somehow, repurpose the code for something insecure, or make any number of other errors. You may end up doing harmless SQL injection yourself (with, say, a name or spelling like O’Brien) or you may get massively hacked...
...all to save a few seconds of coding time.
Short version: either (1) use prepared statements on every query that has any kind of variable information or (2) learn the hard way why that’s the rule.
This is a wordpress plugin and will only be used by me in admin panel. So the problem caused because of using "in" statement because it is hard to write the query like update column where color in ('black','white').
If you're developing for WordPress, have you considered using the wpdb API? It makes it pretty easy to add parameters to your SQL queries.
Example of using parameters for an IN( ) predicate:
$colors_array = ["black", "white"];
$placeholders = array_fill(0, count($colors_array), "%s");
$placeholder_list = implode(",", $placeholders);
$wpdb->query( $wpdb->prepare(
"
UPDATE $wpdb->stock
SET quantity = 327
WHERE color IN ($placeholder_list)
",
$colors_array
));
See https://codex.wordpress.org/Class_Reference/wpdb#Protect_Queries_Against_SQL_Injection_Attacks
I agree with Ed Cottrell's advice that you should not compromise on secure programming methods. Use the most secure method and use it consistently.
You don't have to waste time thinking about whether any given case is "safe enough" to skip using the secure method.
You don't have to worry if it's still safe after your PHP variables are no longer hard-coded.
You don't have to worry that someone will copy & paste your code as an example, and they use it in an unsafe way.
Everyone knows or should know parameterized queries help to protect against SQL injection. All of the tutorials and documentation I have seen have revolved around using prepared SQL queries to process form input. But what about when there isn't any form input? I.e. a subsequent query after a user has been logged in such as $stmt = "SELECT theme_preference FROM users WHERE user_id = '1234'";
$query = mysqli_query($conn, $stmt);
Is there any possible way an attacker could exploit this? (Let's say I'm using PHP).
The question is not whether the source of the data written in a SQL query is a http form. It's not even if it's from the current request.
The question is whether you trust the source of the data. And that may be a complex question.
You obviously do not trust something that comes from the current request. You also don't trust something that may have come from an earlier request, like for examples fields in a database that are modified by request data. But you also may or may not trust other fields in your database. For example you have IT ops staff, or DB admins. Do you trust them to not inject some kind of an XSS or secondary SQLi attack into a database field to steal user credit card data, which is stored in an audited table, so they cannot just go in and dump it without being noticed? If they injected javascript or a clever SQLi in the right place in a table that is not audited, they may steal credit card info by exploiting the vulnerable application, then change it back and remove all traces.
Also an application may have different sources for data, other systems may for example upload files (say XML) on APIs, data from those will be processed, some of it will eventually make it to the UI or used in SQL queries. If you trust those sources, you may choose to not implement protection against SQLi or XSS. But why would you, when it is easy? Multiple layers of defenses is always better than walking on thin ice.
So in short, the question is trust. If you absolutely trust the source of the data (like for example because it's static, hard-coded, or for some other reason), that's fine to use it directly in queries. But in case of SQL injection, using it properly (ie. in parameters) is so easy that you should just do that.
Also consider future changes. You are writing it in a SQL string without parameters because you know that it's safe now. Months pass, somebody adds a new feature, modifies your query, adds a few more parameters, one is from the request. But the pattern was already there, he will probably just copy-paste and go with the pattern - and your application is vulnerable.
My final point is static security scanners, those that look at your source code. Pretty much all of those will flag your code for SQLi if a variable is included in the query string itself without using parameters. That may of course be a false positive, but I doubt you want to bother with those findings, when you can avoid them in the first place.
So sometimes it's not just about the technical exploitability, there are other aspects too.
I know this topic has been covered to death but I would like some feedback from the community regarding security within our web application.
We have standard LAMP stack web app which contains a large number of database queries which are executed using mysqli_query. These queries are not parameterized and at the moment but there is some naive escaping of the inputs using addslashes.
I have been tasked with making this system safer as we will be penetration tested very shortly. The powers above know that parameterized queries are the way to go to make the system safer however they don't want to invest the time and effort into re-writing all the queries in the application and also changing the framework we have to make them all work correctly.
So basically I'm asking what my options are here?
I've run mysqli_real_escape_string over the inputs. I've setup a filter which doesn't allow words like SELECT, WHERE, UNION to be passed in which I guess makes it safer. I know mysqli_query only allows one query to be run at once so there's some security there (from concatenating updates onto the end of of selects).
Do I have any other options here?
Edit: I should probably add that if anyone is able to provide an example of an attack which is completely unavoidable without parameterized queries that would also be helpful. We have a query which looks like this:
SELECT
pl.created
p.LoginName,
pl.username_entered,
pl.ip_address
FROM loginattempts pl
LEFT JOIN people p ON p.PersonnelId = pl.personnel_id
WHERE p.personnelid = $id
AND pl.created > $date1
AND pl.created < $date2
I've substituted a UNION query into the $id UNION SELECT * FROM p WHERE 1 = 1 sort of thing and I can prevent that by not allowing SELECT/UNION but then I'm sure there are countless other types of attack which I can't think of. Can anyone suggest a few more?
Update
I've convinced the powers that be above me that we need to rewrite the queries to parameterized statements. They estimate it will take a few months maybe but it has to be done. Win. I think?
Update2
Unfortunately I've not been able to convince the powers that be that we need to re-write all of our queries to parameterized ones.
The strategy we have come up with is to test every input as follows:
If the user supplied input is_int that cast it as so.
Same for real numbers.
Run mysqli_real_escape_string over the character data.
Change all the parameters in the queries to quoted strings i.e.
WHERE staffName = ' . $blah . '
In accordance with this answer we are 100% safe as we are not changing the character set at any time and we are using PHP5.5 with latin1 character set at all times.
Update 3
This question has been marked as a duplicate however in my mind the question is still not followed answered. As per update no.2 we have found some strong opinion that the mysqli_real_escape string function can prevent attacks and is apparently "100% safe". No good counter argument has since been provided (i.e. a demonstration of an attack which can defeat it when used correctly).
check every single user input for datatype and where applicabile with regular expressions (golden rule is: never EVER trust user input)
use prepared statements
seriously: prepared statements :)
it's a lot of work especially if your application is in bad shape (like it seems to be in your case) but it's the best way to have a decent security level
the other way (which i'm advising against) could be virtual patching using mod_security or a WAF to filter out injection attempts but first and foremost: try to write robust applications
(virtual patching might seem to be a lazy way to fix things but takes actually a lot of work and testing too and should really only be used on top of an already strong application code)
Do I have any other options here?
No. No external measure, like ones you tried to implement, has been proven to be of any help. Your site is still vulnerable.
I've run mysqli_real_escape_string over the inputs
Congratulations, you just reinvented the notorious magic_quotes feature, that proven to be useless and now expelled from the language.
JFYI, mysqli_real_escape_string has nothing to do with SQL injections at all.
Also, combining it with existing addslashes() call, you are spoiling your data, by doubling number of slashes in it.
I've setup a filter which I guess makes it safer.
It is not. SQL injection is not about adding some words.
Also, this approach is called "Black-listing" it is proven to be essentially unreliable. A black list is essentially incomplete, no matter how many "suggestions" you can get.
I know mysqli_query only allows one query to be run at once so there's some security there
There is not. SQL injection is not about adding another query.
Why did I close this question as a duplicate for "How can I prevent SQL-injection in PHP?"?
Because these questions are mutually exclusive, and cannot coexist on the same site.
If we agree, that the only proper answer is using prepared statements, then a question asks "How can I protect using no prepared statements" makes very little sense.
At the same time, if the OP manages to force us to give the positive answer they desperately wants, it will make the other question obsoleted. Why use prepared statements if everything is all right without them?
Additionally, this particular question is too localized as well. It seeks not insight but excuse. An excuse for nobody but the OP personally only. An excuse that let them to use an approach that proven to be insecure. Although it's up to them, but this renders this question essentially useless for the community.
Reading some of the SQL injection questions and answers on SO, I saw this answer that suggests you can convert untrusted user input to hex, which will by its very nature not require escaping of any kind, and thus completely and utterly avoid the possibility of SQL injection.
What database abstraction layer you use (PDO, mysqli, mysql, Pear DB, etc.) does not matter.
Example of a normal working query:
$DBH = new PDO('mysql:host=127.0.0.1;dbname=test', 'test', 'testpassword');
// could have been:
//$bookTitle = bin2hex($_GET['title']);
$bookTitle = bin2hex('Catch-22');
$query = "SELECT * from `books` WHERE `title` = UNHEX('$bookTitle')";
foreach ($DBH->query($query) as $row) {
echo "\n<br />\n";
print_r($row);
echo "\n<br />\n";
}
I've included enough code here for you to quickly run a test if you have a database with a table such as:
CREATE TABLE `books` (`id` INT, `title` VARCHAR(100), `author` VARCHAR(100)) ENGINE=InnoDB CHARACTER SET='utf8';
INSERT INTO `books` VALUES(1, 'Catch-22', 'Joseph Heller');
Of course this is oversimplified - you'd normally have input validation, output sanitizing and lots of other abstractions but we want to focus on the question - no more fluff than necessary to help provide easily executed examples.
What I'd like to know is if there are any technical weaknesses to this technique. I am specifically not asking if this technique has human weaknesses (easier to futz as a sloppy programmer, since it is clearly not as clean as using parameterized queries).
Yes, we can all agree that parameterized queries are less susceptible to bad programming or unfortunate oversights. So please stick to the question - does this technique help avoid SQL injections of all kinds, unconditionally?
Can someone show an example of user input that would break this technique? Even a corner case, some particular MySQL server settings or old PHP version that breaks it?
It works just as well for integer replacements:
// could have been:
//$bookID = bin2hex($_GET['id']);
$bookID = bin2hex(1);
$query = "SELECT * from `books` WHERE `id` = UNHEX('$bookID')";
Other thoughts:
With this technique you avoid two round trips to the database as happens if using (non-emulated) prepared statements (although prepared statements != parameterized queries).
The caveats to some other suggested techniques start to make one go cross-eyed what with the corner case exceptions such as this and this. Does the hex encoding technique avoid all possibility of attackers wreaking havoc using character set encoding tricks and whatever else is out there?
It appears that decoding on the database side may be limited to MySQL/MariaDB, although there may be third-party solutions for adding UNHEX() to PostgreSQL (or - not sure - in some databases, you might be able to use the other method of placing the hex literals in the query without using any UNHEX function)
We use this hex-encoding technique on an ecommerce system that has been live for a few years.
I've yet to come across a disadvantage and, once you have an object written for every table in the database (with the corresponding retrieve & store functions), it's all so hidden from the UI developers, they really don't even notice.
This one technique works for
blocking SQL-Injection
supports any client's character encoding
storing binary data
Its also very fast (we had considered base64 encoding, which would produce shorter SQL strings, but require more CPU). In tests we saw no measurable performance difference between hex-encoding all string input and not.
does this technique help avoid SQL injections of all kinds, unconditionally?
I'm yet to see how anybody could SQLi if this technique was always used (when binding any external data into an SQL statement) -
As is common to all techniques (DCoder) - it has to be universally used to provide universal protection - mysql_real_escape_string also suffers this.
user "N.B" rants, but provides no technical argument (may be he was talking about PokemonGo vs CSGO, who knows?), and the only answer clearly misunderstands the technique.
However, I do find the lack of people using / documenting this technique highly surprising - this is the only reason I have any doubt about it.
It's so simply, easy, comprehensive and provides a range of other useful benefits, it's hard to get my head round why everybody isn't doing it and SQLi is a thing of the past???
Seems logical since all the data, even the user input that could cause injections, would be converted. I've been trying to find ways of bypassing it but I can't. So far the biggest disadvantages I see are: 1) From a database view you won't be able to make sense of the user's input. You'll have pull everything from the DB and convert it just to see it or manipulate it. 2) Whatever input you provide, the string length will get doubled after conversion.
I am thinking of rewriting some open-source application for my purposes to PDO and transactions using InnoDB (mysql_query and MyISAM now).
My question is: Which cases are reasonable for using prepared statements?
Because everywhere I am reading is stated (even in many posts here) that I should use prepared statements every time and everywhere because of the 1. security and 2. performance. Even PHP manual recommends using prepared statements and not mentioning the escape-thing.
You can't deny the security mechanism. But thinking it over and over it comes to mind that having to prepare the statement every time and then use it once.. It doesn't make sense. While having to insert 1000 times some variables in single statement, that makes sense but it is obvious. But this is not what common eshop or board is built upon.
So how to overcome this? May I prepare my statements application-wide and to name them specifically? Can I prepare several different statements and to use them by name? Because this is the only reasonable solution I am thinking of (except the 1000x thing).
I found there is this mysql_real_escape called $pdo->quote as well for the purpose of single query. Why not to use this? Why to bother with preparing?
And what do you think of this excellent article?
http://blog.ulf-wendel.de/2008/pdo_mysqlnd-prepared-statements-again/
Do you agree with the overhead caused by preparing the statements?
Thanks
I think this falls in the "premature optimization" category.
How significant is the overhead? Have you measured it? Does it affect your server performance at all?
Odds are it doesn't.
On the plus side, you have an undeniable gain in terms of security (which should be a major concern for any internet-based shop).
On the downside, you have the risk that it might affect performance. In the link you provided, it shows that poorly implemented PDO preparation results in slightly lower performance than non prepared statement in some circumstances. Performance difference on 5000 runs is 0.298 seconds.
Insignificant. Even more so when you realize that the "non prepared" queries are run without the input sanitizing routines that would be required to make them safe in a live environment. If you don't use the prepared queries, you need some form of input sanitizing to prevent SQL attacks, and depending on how it is done, you may need to massage back the result sets.
Bottom line, there is no significant performance issue, but there is a significant security benefit. Thus the official recommendation of using prepared statements.
In your question, you speak of "the common eshop". The "common eshop" will never have enough traffic to worry about the performance issue, if there is one. The security issue on the other end...
My question is: Which cases are reasonable for using prepared statements?
All of them. The community is openly-opposed to the usage of mysql_* functions.
Note: Suggested alternatives
Use of this extension is discouraged. Instead, the MySQLi or PDO_MySQL extension should be used. See also MySQL: choosing an API for more information.
Alternatives to this function include:
mysqli_connect()
PDO::__construct()
source
But thinking it over and over it comes to mind that having to prepare the statement every time and then use it once.. It doesn't make sense
You're trading in a Geo for a Jaguar and you're complaining that you don't like the Jaguar because you don't always use the seat-heaters. You don't have to be consistently using every function of a library to mean it's good.
I found there is this mysql_real_escape called $pdo->quote as well for the purpose of single query. Why not to use this? Why to bother with preparing?
If you are using this function to build SQL statements, you are strongly recommended to use PDO::prepare() to prepare SQL statements with bound parameters instead of using PDO::quote() to interpolate user input into an SQL statement. Prepared statements with bound parameters are not only more portable, more convenient, immune to SQL injection, but are often much faster to execute than interpolated queries, as both the server and client side can cache a compiled form of the query. source
My question is: Which cases are reasonable for using prepared statements?
Well actually, that's hard to say. Especially as you didn't even tell which open source application you speak about here.
To give you an example: For a ultra-lame guestbook app PDO with prepared statements will be the perfect choice, as well for 99% of all other open source apps out there. But for some this actually can make a difference. The important part here is: You have not told anything about the application.
As the database is not unimportant to an application, it's the other way round as well: the application is not unimportant to the database.
So you either need to share more about that "mysterious" open-source application you ask about or you need to tell us, what exactly you would like to know. Because generally, it's simple: Take PDO. But in specific, there are differences, so you need to tell us what the application in specific is, otherwise your question is already answered.
And btw., if the application is mysql_* style, it's much easier to just replace with mysqli_* interface. If you had done some actually rewriting, even just for fun, you would have seen that.
So better add more meat here or live with some not-so-precise answers.
While this question is rather old, some topics were not really discussed that should be outlined here for others researching the same as the OP.
To summarize everything below:
Yes always use prepare statements
Yes use PDO over mysqli over mysql. This way if you switch database systems all you need to do is update the queries instead of queries, function calls, and arguments given it supports prepared statements.
Always sanitize user supplied data despite using prepared statements with parameters
Look into a DBAL (Database Abstraction Layer) to ease working with all of these factors and manipulating queries to suit your needs.
There is the topic of PDO::ATTR_EMULATE_PREPARES which will increase the performance of calling cached queries in MySQL >= 5.1.21 when emulation is turned OFF, which is ENABLED by default. Meaning PHP will emulate the prepare before execute sends it to the actual database. The time between emulated and non-emulated is normally negligible unless working with an external database (not localhost), such as on a cloud, that may have an abnormally high ping rate.
The caching depends on your MySQL settings in my.cnf as well, but MySQL optimization outside the scope of this post.
<?php
$pdo = new \PDO($connection_string);
$pdo->setAttribute( \PDO::ATTR_EMULATE_PREPARES, false );
?>
So keep this in mind since mysqli_ does not provide an API for client side emulation and is always going to use MySQL for preparing statements.
http://www.php.net/manual/en/mysqli.quickstart.prepared-statements.php
Despite having similar features there are differences and you may need features that one API provides while the other does not. See PHP's reference on choosing one API over the other: http://www.php.net/manual/en/mysqlinfo.api.choosing.php
So this pretty much goes along with what you asked with defining your statements application-wide, as cacheable queries would be cached on the MySQL server, and wouldn't need to be prepared application-wide.
The other benefit is that exceptions in your Query would be thrown at prepare() instead of execute() which aids in development to ensure your Queries are correct.
Regardless there is no real world performance benefits of using prepare or not.
Another benefit of prepared statements is working with Transactions if you use InnoDB for MySQL. You can start a transaction, insert a record, get the last insert id, update another table, delete from another, and if anything fails along the way you can rollBack() to before the transaction took place. Otherwise commit the changes if you choose to. For example working with a new order and setting the user's last order column to the new order id, and delete a pending order, but the supplied payment type did not meet the criteria for placing orders from the order_flags table, so you can rollBack() and show the user a friendly error message.
As for security, I am rather baffled no one touched on this. When sending any user supplied data to ANY system including PHP and MySQL, sanitize and standardize it.
Yes prepared statements do provide some security when it comes to escaping the data but it is NOT 100% bullet proof.
So always using prepared statements is far more beneficial than not with no real performance loss, and some benefits with caching, but you should still sanitize your user supplied data.
One step is to typecast the variables to the desired data type you are working with. Using objects would further ease this since you work within a single Model for the data types as opposed to having to remember it each time you work with the same data.
To add on to the above you should look into a database abstraction layer that uses PDO.
For example Doctrine DBAL: http://docs.doctrine-project.org/projects/doctrine-dbal/en/latest/reference/query-builder.html
The added benefits of working with a DBAL+PDO are that
You can standardize and shorten the amount of work you have to do.
Aid in sanitization of user supplied data
Easily manipulate complex queries
Use nested transactions
Easily switch between databases
Your code becomes more portable and usable in other projects
For example I extended PDO and overrode the query(), fetchAll(), and fetch() methods so that they would always use prepared statements and so that I could write SQL statements inside fetch() or fetchAll() instead of having to write everything out again.
EG:
<?php
$pdo = new PDOEnhanced( $connection );
$pdo->fetchAll( "SELECT * FROM foo WHERE bar = 'hi'", PDO::FETCH_OBJ );
//would automatically provide
$stmt = $pdo->prepare( "SELECT * FROM foo WHERE bar=?" );
$stmt->execute( array( 'hi' ) );
$resultSet = $stmt->fetchAll( PDO::FETCH_OBJ )
?>
As for people suggesting that mysql_* style, is much easier to just replace with mysqli_* API. It is not the case. A large portion of mysql_* functions were left out or had arguments changes with mysqli_*
See: http://php.net/manual/en/mysqli.summary.php
You can however get a converter released by Oracle to ease the process: https://wikis.oracle.com/display/mysql/Converting+to+MySQLi
Keep in mind that it is a file source text parser and is not 100% accurate so validate the changes before merging them. It will also add a significant amount of overhead for the globals it creates.