I'm re-engineering a PHP-driven web site which uses a minimal database. The original version used "pseudo-prepared-statements" (PHP functions which did quoting and parameter replacement) to prevent injection attacks and to separate database logic from page logic.
It seemed natural to replace these ad-hoc functions with an object which uses PDO and real prepared statements, but after doing my reading on them, I'm not so sure. PDO still seems like a great idea, but one of the primary selling points of prepared statements is being able to reuse them… which I never will. Here's my setup:
The statements are all trivially simple. Most are in the form SELECT foo,bar FROM baz WHERE quux = ? ORDER BY bar LIMIT 1. The most complex statement in the lot is simply three such selects joined together with UNION ALLs.
Each page hit executes at most one statement and executes it only once.
I'm in a hosted environment and therefore leery of slamming their servers by doing any "stress tests" personally.
Given that using prepared statements will, at minimum, double the number of database round-trips I'm making, am I better off avoiding them? Can I use PDO::MYSQL_ATTR_DIRECT_QUERY to avoid the overhead of multiple database trips while retaining the benefit of parametrization and injection defense? Or do the binary calls used by the prepared statement API perform well enough compared to executing non-prepared queries that I shouldn't worry about it?
EDIT:
Thanks for all the good advice, folks. This is one where I wish I could mark more than one answer as "accepted" — lots of different perspectives. Ultimately, though, I have to give rick his due… without his answer I would have blissfully gone off and done the completely Wrong Thing even after following everyone's advice. :-)
Emulated prepared statements it is!
Today's rule of software engineering: if it isn't going to do anything for you, don't use it.
I think you want PDO::ATTR_EMULATE_PREPARES. That turns off native database prepared statements, but still allows query bindings to prevent sql injection and keep your sql tidy. From what I understand, PDO::MYSQL_ATTR_DIRECT_QUERY turns off query bindings completely.
When not to use prepared statements? When you're only going to be running the statement once before the db connection goes away.
When not to use bound query parameters (which is really what most people use prepared statements to get)? I'm inclined to say "never" and I'd really like to say "never", but the reality is that most databases and some db abstraction layers have certain circumstances under which they won't allow you to bind parameters, so you're forced to not use them in those cases. Any other time, though, it will make your life simpler and your code more secure to use them.
I'm not familiar with PDO, but I'd bet it provides a mechanism for running parametrized queries with the values given in the same function call if you don't want to prepare, then run as a separate step. (e.g., Something like run_query("SELECT * FROM users WHERE id = ?", 1) or similar.)
Also, if you look under the hood, most db abstraction layers will prepare the query, then run it, even if you just tell it to execute a static SQL statement. So you're probably not saving a trip to the db by avoiding explicit prepares anyhow.
Prepared statements are being used by thousands of people and are therefore well-tested (and thus one can infer they are reasonably secure). Your custom solution is only used by you.
The chance that your custom solution is insecure is pretty high. Use prepared statements. You have to maintain less code that way.
The benefits of prepared statements are as follows:
each query is only compiled once
mysql will use a more efficient transport format to send data to the server
However, prepared statements only persist per connection. Unless you're using connection pooling, there would be no benefit if you're only doing one statement per page. Trivially simple queries would not benefit from the more efficient transport format, either.
Personally I wouldn't bother. The pseudo-prepared statements are likely to be useful for the safe variable quoting they presumably provide.
Honestly, I don't think you should worry about it. However, I remember that a number of PHP data access frameworks supported prepare statement modes and non-prepare statement modes. If I remember correctly, PEAR:DB did back in the day.
I have ran into the same issue as you and I had my own reservations, so instead of using PDO I ended up writing my own light-weight database layer that supported prepares and standard statements and performed correct escaping (sql-injection prevention) in both cases. One of my other gripes with prepares is that sometimes it is more efficient to append some non-escapable input to a statement like ... WHERE id IN (1, 2, 3...).
I don't know enough about PDO to tell you what other options you have using it. However, I do know that PHP has escaping functions available for all database vendors it supports and you could roll your own little layer on top of any data access layer you are stuck with.
Related
I'm re-engineering a PHP-driven web site which uses a minimal database. The original version used "pseudo-prepared-statements" (PHP functions which did quoting and parameter replacement) to prevent injection attacks and to separate database logic from page logic.
It seemed natural to replace these ad-hoc functions with an object which uses PDO and real prepared statements, but after doing my reading on them, I'm not so sure. PDO still seems like a great idea, but one of the primary selling points of prepared statements is being able to reuse them… which I never will. Here's my setup:
The statements are all trivially simple. Most are in the form SELECT foo,bar FROM baz WHERE quux = ? ORDER BY bar LIMIT 1. The most complex statement in the lot is simply three such selects joined together with UNION ALLs.
Each page hit executes at most one statement and executes it only once.
I'm in a hosted environment and therefore leery of slamming their servers by doing any "stress tests" personally.
Given that using prepared statements will, at minimum, double the number of database round-trips I'm making, am I better off avoiding them? Can I use PDO::MYSQL_ATTR_DIRECT_QUERY to avoid the overhead of multiple database trips while retaining the benefit of parametrization and injection defense? Or do the binary calls used by the prepared statement API perform well enough compared to executing non-prepared queries that I shouldn't worry about it?
EDIT:
Thanks for all the good advice, folks. This is one where I wish I could mark more than one answer as "accepted" — lots of different perspectives. Ultimately, though, I have to give rick his due… without his answer I would have blissfully gone off and done the completely Wrong Thing even after following everyone's advice. :-)
Emulated prepared statements it is!
Today's rule of software engineering: if it isn't going to do anything for you, don't use it.
I think you want PDO::ATTR_EMULATE_PREPARES. That turns off native database prepared statements, but still allows query bindings to prevent sql injection and keep your sql tidy. From what I understand, PDO::MYSQL_ATTR_DIRECT_QUERY turns off query bindings completely.
When not to use prepared statements? When you're only going to be running the statement once before the db connection goes away.
When not to use bound query parameters (which is really what most people use prepared statements to get)? I'm inclined to say "never" and I'd really like to say "never", but the reality is that most databases and some db abstraction layers have certain circumstances under which they won't allow you to bind parameters, so you're forced to not use them in those cases. Any other time, though, it will make your life simpler and your code more secure to use them.
I'm not familiar with PDO, but I'd bet it provides a mechanism for running parametrized queries with the values given in the same function call if you don't want to prepare, then run as a separate step. (e.g., Something like run_query("SELECT * FROM users WHERE id = ?", 1) or similar.)
Also, if you look under the hood, most db abstraction layers will prepare the query, then run it, even if you just tell it to execute a static SQL statement. So you're probably not saving a trip to the db by avoiding explicit prepares anyhow.
Prepared statements are being used by thousands of people and are therefore well-tested (and thus one can infer they are reasonably secure). Your custom solution is only used by you.
The chance that your custom solution is insecure is pretty high. Use prepared statements. You have to maintain less code that way.
The benefits of prepared statements are as follows:
each query is only compiled once
mysql will use a more efficient transport format to send data to the server
However, prepared statements only persist per connection. Unless you're using connection pooling, there would be no benefit if you're only doing one statement per page. Trivially simple queries would not benefit from the more efficient transport format, either.
Personally I wouldn't bother. The pseudo-prepared statements are likely to be useful for the safe variable quoting they presumably provide.
Honestly, I don't think you should worry about it. However, I remember that a number of PHP data access frameworks supported prepare statement modes and non-prepare statement modes. If I remember correctly, PEAR:DB did back in the day.
I have ran into the same issue as you and I had my own reservations, so instead of using PDO I ended up writing my own light-weight database layer that supported prepares and standard statements and performed correct escaping (sql-injection prevention) in both cases. One of my other gripes with prepares is that sometimes it is more efficient to append some non-escapable input to a statement like ... WHERE id IN (1, 2, 3...).
I don't know enough about PDO to tell you what other options you have using it. However, I do know that PHP has escaping functions available for all database vendors it supports and you could roll your own little layer on top of any data access layer you are stuck with.
I am thinking of rewriting some open-source application for my purposes to PDO and transactions using InnoDB (mysql_query and MyISAM now).
My question is: Which cases are reasonable for using prepared statements?
Because everywhere I am reading is stated (even in many posts here) that I should use prepared statements every time and everywhere because of the 1. security and 2. performance. Even PHP manual recommends using prepared statements and not mentioning the escape-thing.
You can't deny the security mechanism. But thinking it over and over it comes to mind that having to prepare the statement every time and then use it once.. It doesn't make sense. While having to insert 1000 times some variables in single statement, that makes sense but it is obvious. But this is not what common eshop or board is built upon.
So how to overcome this? May I prepare my statements application-wide and to name them specifically? Can I prepare several different statements and to use them by name? Because this is the only reasonable solution I am thinking of (except the 1000x thing).
I found there is this mysql_real_escape called $pdo->quote as well for the purpose of single query. Why not to use this? Why to bother with preparing?
And what do you think of this excellent article?
http://blog.ulf-wendel.de/2008/pdo_mysqlnd-prepared-statements-again/
Do you agree with the overhead caused by preparing the statements?
Thanks
I think this falls in the "premature optimization" category.
How significant is the overhead? Have you measured it? Does it affect your server performance at all?
Odds are it doesn't.
On the plus side, you have an undeniable gain in terms of security (which should be a major concern for any internet-based shop).
On the downside, you have the risk that it might affect performance. In the link you provided, it shows that poorly implemented PDO preparation results in slightly lower performance than non prepared statement in some circumstances. Performance difference on 5000 runs is 0.298 seconds.
Insignificant. Even more so when you realize that the "non prepared" queries are run without the input sanitizing routines that would be required to make them safe in a live environment. If you don't use the prepared queries, you need some form of input sanitizing to prevent SQL attacks, and depending on how it is done, you may need to massage back the result sets.
Bottom line, there is no significant performance issue, but there is a significant security benefit. Thus the official recommendation of using prepared statements.
In your question, you speak of "the common eshop". The "common eshop" will never have enough traffic to worry about the performance issue, if there is one. The security issue on the other end...
My question is: Which cases are reasonable for using prepared statements?
All of them. The community is openly-opposed to the usage of mysql_* functions.
Note: Suggested alternatives
Use of this extension is discouraged. Instead, the MySQLi or PDO_MySQL extension should be used. See also MySQL: choosing an API for more information.
Alternatives to this function include:
mysqli_connect()
PDO::__construct()
source
But thinking it over and over it comes to mind that having to prepare the statement every time and then use it once.. It doesn't make sense
You're trading in a Geo for a Jaguar and you're complaining that you don't like the Jaguar because you don't always use the seat-heaters. You don't have to be consistently using every function of a library to mean it's good.
I found there is this mysql_real_escape called $pdo->quote as well for the purpose of single query. Why not to use this? Why to bother with preparing?
If you are using this function to build SQL statements, you are strongly recommended to use PDO::prepare() to prepare SQL statements with bound parameters instead of using PDO::quote() to interpolate user input into an SQL statement. Prepared statements with bound parameters are not only more portable, more convenient, immune to SQL injection, but are often much faster to execute than interpolated queries, as both the server and client side can cache a compiled form of the query. source
My question is: Which cases are reasonable for using prepared statements?
Well actually, that's hard to say. Especially as you didn't even tell which open source application you speak about here.
To give you an example: For a ultra-lame guestbook app PDO with prepared statements will be the perfect choice, as well for 99% of all other open source apps out there. But for some this actually can make a difference. The important part here is: You have not told anything about the application.
As the database is not unimportant to an application, it's the other way round as well: the application is not unimportant to the database.
So you either need to share more about that "mysterious" open-source application you ask about or you need to tell us, what exactly you would like to know. Because generally, it's simple: Take PDO. But in specific, there are differences, so you need to tell us what the application in specific is, otherwise your question is already answered.
And btw., if the application is mysql_* style, it's much easier to just replace with mysqli_* interface. If you had done some actually rewriting, even just for fun, you would have seen that.
So better add more meat here or live with some not-so-precise answers.
While this question is rather old, some topics were not really discussed that should be outlined here for others researching the same as the OP.
To summarize everything below:
Yes always use prepare statements
Yes use PDO over mysqli over mysql. This way if you switch database systems all you need to do is update the queries instead of queries, function calls, and arguments given it supports prepared statements.
Always sanitize user supplied data despite using prepared statements with parameters
Look into a DBAL (Database Abstraction Layer) to ease working with all of these factors and manipulating queries to suit your needs.
There is the topic of PDO::ATTR_EMULATE_PREPARES which will increase the performance of calling cached queries in MySQL >= 5.1.21 when emulation is turned OFF, which is ENABLED by default. Meaning PHP will emulate the prepare before execute sends it to the actual database. The time between emulated and non-emulated is normally negligible unless working with an external database (not localhost), such as on a cloud, that may have an abnormally high ping rate.
The caching depends on your MySQL settings in my.cnf as well, but MySQL optimization outside the scope of this post.
<?php
$pdo = new \PDO($connection_string);
$pdo->setAttribute( \PDO::ATTR_EMULATE_PREPARES, false );
?>
So keep this in mind since mysqli_ does not provide an API for client side emulation and is always going to use MySQL for preparing statements.
http://www.php.net/manual/en/mysqli.quickstart.prepared-statements.php
Despite having similar features there are differences and you may need features that one API provides while the other does not. See PHP's reference on choosing one API over the other: http://www.php.net/manual/en/mysqlinfo.api.choosing.php
So this pretty much goes along with what you asked with defining your statements application-wide, as cacheable queries would be cached on the MySQL server, and wouldn't need to be prepared application-wide.
The other benefit is that exceptions in your Query would be thrown at prepare() instead of execute() which aids in development to ensure your Queries are correct.
Regardless there is no real world performance benefits of using prepare or not.
Another benefit of prepared statements is working with Transactions if you use InnoDB for MySQL. You can start a transaction, insert a record, get the last insert id, update another table, delete from another, and if anything fails along the way you can rollBack() to before the transaction took place. Otherwise commit the changes if you choose to. For example working with a new order and setting the user's last order column to the new order id, and delete a pending order, but the supplied payment type did not meet the criteria for placing orders from the order_flags table, so you can rollBack() and show the user a friendly error message.
As for security, I am rather baffled no one touched on this. When sending any user supplied data to ANY system including PHP and MySQL, sanitize and standardize it.
Yes prepared statements do provide some security when it comes to escaping the data but it is NOT 100% bullet proof.
So always using prepared statements is far more beneficial than not with no real performance loss, and some benefits with caching, but you should still sanitize your user supplied data.
One step is to typecast the variables to the desired data type you are working with. Using objects would further ease this since you work within a single Model for the data types as opposed to having to remember it each time you work with the same data.
To add on to the above you should look into a database abstraction layer that uses PDO.
For example Doctrine DBAL: http://docs.doctrine-project.org/projects/doctrine-dbal/en/latest/reference/query-builder.html
The added benefits of working with a DBAL+PDO are that
You can standardize and shorten the amount of work you have to do.
Aid in sanitization of user supplied data
Easily manipulate complex queries
Use nested transactions
Easily switch between databases
Your code becomes more portable and usable in other projects
For example I extended PDO and overrode the query(), fetchAll(), and fetch() methods so that they would always use prepared statements and so that I could write SQL statements inside fetch() or fetchAll() instead of having to write everything out again.
EG:
<?php
$pdo = new PDOEnhanced( $connection );
$pdo->fetchAll( "SELECT * FROM foo WHERE bar = 'hi'", PDO::FETCH_OBJ );
//would automatically provide
$stmt = $pdo->prepare( "SELECT * FROM foo WHERE bar=?" );
$stmt->execute( array( 'hi' ) );
$resultSet = $stmt->fetchAll( PDO::FETCH_OBJ )
?>
As for people suggesting that mysql_* style, is much easier to just replace with mysqli_* API. It is not the case. A large portion of mysql_* functions were left out or had arguments changes with mysqli_*
See: http://php.net/manual/en/mysqli.summary.php
You can however get a converter released by Oracle to ease the process: https://wikis.oracle.com/display/mysql/Converting+to+MySQLi
Keep in mind that it is a file source text parser and is not 100% accurate so validate the changes before merging them. It will also add a significant amount of overhead for the globals it creates.
Normally you'd prepare a statement, and then you'd only send different values when you execute it inside of loops.
But I've seen MVC code where the statements are being prepared for every call of the update method, for example. So my question is - does this hurt performance? And if so, by how much? Is it worse or still better than not preparing statements in the first place? Or does PHP and/or MySQL realize it and only prepare it once?
Some databases and application servers do cache prepared statements on the client side and realize that you're trying to prepare a statement that has already been prepared, and serve up the reference to the pstmt from the cache. This is very common in Java app servers, but may not be so common in PHP.
Preparing a statement is certainly more expensive than not. It requires some extra round trips to the database, so it is only worth it, from a performance perspective, if you're going to be calling the same SQL (with different parms) many, many times.
However, there is another aspect beyond performance - security. Prepared statements are the single best line of defense against SQL Injection attacks - which are incredibly prevalent. Because of this, I would always use prepared statements, even if it may be technically more performant to just skip the prepare step in some cases.
I doubt PHP and/or MySQL are that smart.
So, AFAIK, it hurts performance, as it makes additional roundtrip to the database, but I doubt you will ever notice that.
I am currently writing a CRUD class in PHP using PDO.
I like the security that prepared statements provide, but I have heard that they also prevent databases like mysql from using the queryCache.
Is it better to use a prepared Select statement when you are only doing one select at a time? or would just $pdo->quote() suffice the security standpoint (or have any other advantages like caching?).
All my update, delete and inserts are done using prepared statements already. I am just curious about the selects.
MySQLPerformanceBlog.com did some benchmarks in an article about "Prepared Statements." Peter Zaitsev wrote:
I’ve done a simple benchmark (using
SysBench) to see performance of simple
query (single row point select) using
standard statement, prepared statement
and have it served from query cache.
Prepared statements give 2290
queries/sec which is significantly
better than 2000 with standard
statements but it is still well below
4470 queries/sec when results are
served from query cache.
This seems to say that the "overhead" of using prepared statements is that they are 14.5% faster than using a straight query execution, at least in this simple test. The relative difference probably diminishes with a more complex query or a larger result set.
It seems counter-intuitive that prepared queries would be faster, given the double round-trip to the server and other factors. Peter's benchmark lacks details. Anyway, you should run your own tests, because the type of query you run, and your environment and hardware, are definitely important factors.
As for Query Cache, it was true in the past that prepared statements were incompatible with caching query results, but this was changed. See "How the Query Cache Operates" in the MySQL documentation:
Before MySQL 5.1.17, prepared
statements do not use the query cache.
Beginning with 5.1.17, prepared
statements use the query cache under
certain conditions, which differ
depending on the preparation method: ...
The documentation goes on to describe these conditions. Go read it.
I do recommend using prepared statements for SELECT queries. Quoting variables as you interpolate them into SQL statements can be effective if you do it consistently. But even quoting may have some subtle security vulnerabilities, e.g. with multi-byte character sets (see MySQL bug #8378). It's easier to use prepared queries in a secure way in these cases.
Yes, use prepared statements. I seriously doubt you will run into performance problems with prepared statements running much slower than just a regular literal query. However, on mysql, you appear to be correct. I would opt for prepared statements nevertheless.
Here is one reference:
http://www.mysqlperformanceblog.com/2006/08/02/mysql-prepared-statements/
Although, if you are worried about caching, you might want to look at things like memcached.
This is my understanding, as confirmed by discussion from: here
A normal query is taken as a single
string, parsed, executed, and
returned. End of story. A prepared
statement is taken as a template
string, parsed, and cached. It then
has variables passed into it, almost
like a function call.
Caching the query once tends to cost a
little bit more than just executing it
straight. The savings come in later
calls, when you skip the compilation
step. You save per repeated query the
amount of the compilation.
So, in short, on MySQL, if you're executing a query once, preparing it just adds a needless extra amount of processing.
Prepared statements are generally considered to be better practice.
I would suggest reading the MySql article on prepared statements and their practicalities and advantages over conventional plain-vanilla interpolated stringy queries.
Are you only doing a select "once" in the application lifetime, or "once" per call to the function?
Because if the latter, you should still benefit from the caching in the prepared statement anyway.
Just a reminder that MySQL > 5.1.17 does use the query cache for prepared statements.
From the code POV, i believe prepared statements are, for the most part, the way to go in terms of readability, maintainability, etc...
The one reason not to use them would be expensive queries that get called with some frequency. (queries that take a lot of time to run and have a real benefit on being on the query cache).
Here are a few possibilities to get the conversation started:
Escape all input upon initialization.
Escape each value, preferably when generating the SQL.
The first solution is suboptimal, because you then need to unescape each value if you want to use it in anything other than SQL, like outputting it on a web page.
The second solution makes much more sense, but manually escaping each value is a pain.
I'm aware of prepared statements, however I find MySQLi cumbersome. Also, separating the query from the inputs concerns me, because although it's crucial to get the order correct it's easy to make a mistake, and thus write the wrong data to the wrong fields.
Prepared statements are the best answer. You have testing because you can make mistakes!
See this question.
as #Rob Walker states, parameterized queries are your best bet. If you're using the latest and greatest PHP, I'd highly recommend taking a look at PDO (PHP Data Objects). This is a native database abstraction library that has support for a wide range of databases (including MySQL of course) as well as prepared statements with named parameters.
I would go with using prepared statements. If you want to use prepared statements, you probably want to check out the PDO functions for PHP. Not only does this let you easily run prepared statements, it also lets you be a little more database agnostic by not calling functions that begin with mysql_, mysqli_, or pgsql_.
PDO may be worth it some day, but it's not just there yet. It's a DBAL and it's strengh is (supposedly) to make switching between vendors more easier. It's not really build to catch SQL injections.
Anyhow, you want to escape and sanatize your inputs, using prepared statements could be a good measure (I second that). Although I believe it's much easier, e.g. by utilizing filter.
I've always used the first solution because 99% of the time, variables in $_GET, $_POST, and $_COOKIE are never outputted to the browser. You also won't ever mistakenly write code with an SQL injection (unless you don't use quotes in the query), whereas with the second solution you could easily forget to escape one of your strings eventually.
Actually, the reason I've always done it that way was because all my sites had the magic_quotes setting on by default, and once you've written a lot of code using one of those two solutions, it takes a lot of work to change to the other one.