I learnt about prepared statements when making a JDBC-enabled Java application, and my app uses a connection pooling layer that assures me that prepared statements are cached server-side and this gives a performance benefit.
However, with PHP everything I've read says that they are only cached for the life of the page load. Generally I don't repeat the same query many times, but run several different queries, on a given page load, but will repeat them across multiple page loads.
As my PHP processes are persistent (i.e. they will serve hundreds of pages in their lifetime instead of just one, using PHP-FPM), I was wondering if they will re-use database connections, rather than spawning and killing them off for each hit.
Will using PHP-FPM with mysqli or PDO keep connections longer than a single page load?
If it doesn't, can I make it?
If it does, or I do #2, will this persist the caching of prepared statements longer than just one page load?
Edit:
Just to clarify, I'm not talking about the query cache, which is another beast entirely, or caching the output of queries. I want to cache the compiled prepared statement and its execution plan server-side.
When a request is served php "cleans" the instance and frees resources and other variables. This is done in several steps. Since fastcgi keeps the process alive after a request not all steps are executed and not all memory is freed. There is e.g. EG(persistent_list) which is used by mysql_pconnect(), pg_pconnect(), ... This list isn't emptied between requests as long as the process keeps alive (could be, depending on the actual implementation, but that would defy the purpose of EG(persistent_list)). If you use persistent connections your script might get a "re-used" connection established during a previous request.
To (re-)use a prepared statement directly you need the identifier for that statement (and that connection). When using (php-)postgresql this is simply a (connection-wise) unique string you pass to pg_execute(), so your script has no problem to gain access to the statement previously prepared by another instance (using the same connection).
Using mysqli or PDO-mysql you need a resource/object as statement identifier. That's kind of a problem since neither the mysqli nor the pdo extension seem to offer a way of storing the resource in EG(persist_list) between requests and you can't recreate it either. Unless php-fpm offers such a "service" it's seems impossible to re-use a mysql prepared statement directly.
All you can hope for is MySQL's server-side query cache. In recent versions (see link) it may recognize the statement when using prepared statements. But even then it doesn't re-use the actual prepared statement:
For a prepared statement executed via the binary protocol, comparison with statements in the query cache is based on the text of the statement after expansion of ? parameter markers. The statement is compared only with other cached statements that were executed via the binary protocol. That is, for query cache purposes, statements issued via the binary protocol are distinct from statements issued via the text protocol.
So, if I'm not mistaken, currently you can't re-use a mysql statement prepared during a previous request in php.
You're confusing what is happening at the PHP/Java tier with what's happenning in the database.
Yes, using prepared statements (usually) means that the execution plan is cached by the database itself (NOT the PHP/Java tier). However it does not follow that this always results in better performance - and an explanation of this would take several hundred pages. However I infer from what you've said elsewhere you are using MySQL as the DBMS which makes the discussion somewhat simpler (IIRC none of the storage engines implement histograms). Typically MySQL will be able to cache enough information about a schema to be able to generate a plan without any disk I/O. OTOH, using prepared statements mean a minimum of three round trips to the DBMS for each query (present statement, present params, retrieve results) while using inlined values eliminates on of these round trips. In the absence of histogram indexes, the value of the variables is irrelevant to the optimal plan detectable by the optimizer.
The fact that you are using PHP, or PHP-FPM or Java with single or persistent or pooled connections is irrelevant to whether or not prepared-statements are cached/re-used by the DBMS.
HTH
C.
If your PHP application uses connection pooling to the database, and the database caches prepared statements, then yes, the caching will persist between pages. If the prepared statement caching is done by the client library, then this is more nebulous.
You need to look at the docs for PHP-FPM and/or PDO to see how to tell them to use connection pooling. There should be an option in both to do it.
You should be aware that MySQL connection setup and teardown is actually very fast and many PHP installations do not use connection pooling because of this. Either way, you should also invest time in your server settings, particularly the wait_timeout parameter. PHP is also designed around the idea that you create everything you need when your pages starts and it all goes away when the page finishes. Most PHP code and libraries assume this is the case. It is quite a different paradigm than under Java.
The only true answer is it depends.
Prepared statements are finicky beasts when it comes to MySQL. There are a great number of factors that determine whether or not a prepared statement is cached.
The general idea is if your version is < 5.1.17, the prepared statement is never cached in the query cache, and if using >= 5.1.17, it depends.
Please see the following page in the MySQL 5.1 manual:
http://dev.mysql.com/doc/refman/5.1/en/query-cache-operation.html
you can force mysqli to create a persistent connection by prepending p: to the hostname, as per the php doc: http://www.php.net/manual/en/mysqli.persistconns.php
However, prepared statements are always closed between page loads, as discussed here: http://dev.mysql.com/doc/refman/5.0/en/apis-php-mysqli.persistconns.html
Sorry, it can't be done as far as I know. Prepared statements are for a single page load.
Prepared statements have nothing to do with result caching.
Result caching can be controlled via db server configuration or forced via memcached and the like.
I suggest you look into memcached, especially for PHP http://www.php.net/manual/en/book.memcached.php
PHP does not cache queries nor query results in most cases. MySQL will perform this kind of caching regardless or what thread or connection is issuing the query.
If you want server-side caching across multiple page loads or multiple servers, then use MySQL query caching and server-side caching (APC, file-based caching, memcached, etc).
Related
I'm not exactly sure why mysqli_stmt::close is needed. Is it enough to just call mysqli::close?
Is it needed only when you want to execut multiple prepared statements? Or is it also needed if you execute just one?
It all depends on the nature of your script.
If it's a regular web-page, that runs for a fraction of second, neither stmt::close nor mysqli::close is ever needed - PHP will close all the resources automatically.
While in a long-running PHP script it's good to close particular statements when you don't need them anymore, as every statement allocates some resources on the server that otherwise can be used by other connections.
Normally one doesn't need to close any database connections, or most other connections in PHP. As the garbage handler and the associated cleanup code does this automatically.
However, as noted in the comment by #Phil, sometimes you might want to close something earlier. Most common situations are either by not wanting the entire resultset, should a specific condition occur and you need to run another prepared statement; Or when you need to conserve resources.
Of these two the former is by far the most common, and even that isn't really common. So most of the time you do not need to worry about this.
Also, I recommend using the PDO library. It has a few more features than MySQLi, and a few other advantages over the older MySQLi-library.
How to take advantage of prepared statements for performance? I understand that something like this might benefit if I put it in a loop:
SELECT `Name` FROM `Hobbits` WHERE `ID` = :ID;
I've read that looping with prepared statements is faster than looping without, but otherwise prepared statements would slightly decrease performance. So - how big may that loop be?
If I run a complex SQL query at the beginning of my code and repeat it with one different parameter at the end - will the second query run faster? (We are using a single connection for each page load). Is there a limit on cached queries, so I better repeat my queries right away?
What about executing the entire script twice with the exact same parameters (reload the page or 2 users)?
A prepared query is given to the SQL server, which parses it and possibly already prepares an execution plan. You're then basically given an id for these allocated resources and can execute this prepared statement by just filling in the blanks in the statement. You can run this statement as often as you like and the database will not have to repeat the parsing and execution planning, which may bring a speed improvement.
As long as you do not throw away the statement, there's no hard timeout for how long the statement will "stay prepared". It's not a cache, it's an allocated resource on the SQL server. At least as long as your database driver uses native prepared statements in the SQL API. PDO for example does not do so by default, unless you set PDO::ATTR_EMULATE_PREPARES to false.
At the end of the script execution though, all those resources will always be deallocated, they do not persist across different page loads. Beyond that, the SQL server may or may not cache the query and its results for some time regardless of the client script.
How long are prepared mysql queries cached?
This is not actually "a cache". Prepared statement lasts as little as during script execution.
If I run a complex SQL query at the beginning of my code and repeat it with one different parameter at the end - will the second query run faster?
The more complex a query, the less effect you will see. Frankly, prepared statement saves you only parsing, while if execution involves temporary or filesort, or table scan - prepared statement would speed up none of them.
On the other hand, for the simple primary-key lookups, which involve no complex query parsing nor building sophisticated query plans, the benefit would be negligible to none.
So - how big may that loop be?
The more iterations it gets - the more benefit. However, in a sane web-application one have to avoid looping queries at all.
What about executing the entire script twice with the exact same parameters (reload the page or 2 users)?
As I said above, there will be no benefit from a prepared statement at all. A classical query cache, however, most likely would fire.
How to take advantage of prepared statements for performance?
Noway. Not in web-serving PHP, at least. In a some long-running cli-based script - may be.
However, prepared statements ought to be used anyway, for the purpose of producing syntactically correct queries.
FYI, from
http://dev.mysql.com/doc/refman/5.7/en/statement-caching.html
The max_prepared_stmt_count system variable controls the total number of statements the server caches. (The sum of the number of prepared statements across all sessions.)
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Are prepared statements cached server-side across multiple page loads with PHP?
I'm working on a new project and using parameterized queries for the first time (PHP with a MySQL DB). I read that they parameterized queries are cached, but I'm wondering how long they are cached for. For example, let's say I have a function 'getAllUsers()' that gets a list of all active user ID's from the user table and for each ID, a User object is created and a call to function 'getUser($user)' is made to set the other properties of the object. The 'getUser()' function has it's own prepared query with a stmt->close() at the end of the function.
If I do it this way, does my parameterized query in 'getUser()' take advantage of caching at all or is the query destroyed from cache after each stmt->close()?
Note: I also use the getUser() function if a page only requires data for a single user object so I wanted to do it this way to ensure that if the user table changes I only ever need to update one query.
Is this the right way of doing something like this or is there a better way?
Update: Interesting, just saw this on php.net's manual for prepared statements (http://php.net/manual/en/mysqli.quickstart.prepared-statements.php)
Using a prepared statement is not always the most efficient way of executing a statement. A prepared statement executed only once causes more client-server round-trips than a non-prepared statement.
So I guess the main benefit for parameterized queries is to protect against SQL injection and not necessarily to speed things up unless it's a query that will repeated at one time.
Calling mysqli_stmt::close will:
Closes a prepared statement. mysqli_stmt_close() also deallocates the
statement handle.
therefore not being able to use the cached version of the statement for further executions.
I wouldn't mind of freeing resources or closing statements since PHP will do it for you at the end of the script anyway.
Also if you are working with loops (as you described) take a look at mysqli_stmt::reset which will reset the prepared statement to its original state (after the prepare call).
That's good question, from some point of view.
First, about "caching".
There is some special thing about prepared queries - you can send it to server once and then execute it multiple times. It can give some small theoretical benefit for using already parsed and prepared query.
As it seems, you're not using such mechanism, every time preparing every your query. So, there is no caching at all.
Next, about premature optimization.
You've heard of some caching, and it occupied your imagination.
While there is no real need or cause for you to concern about caching or whatever performance issue.
So, there is a rule: do not occupy yourself with performance issues until they are real.
Otherwise you'll waste your time.
I was just wondering how prepared queries work. I am using PHP and MySQL.
I know that prepared queries compile the query and then repeated uses only have to change the parameters, so it saves time. But how long does that compiled query have an effect? At which point does it have to be reevaluated? Is it just as long as the PHP script is running? Or is it as long as the connection to the database is present? In that case, would a persistent connection have any effect on it?
ryeguy, they last for the length of the connection. Per the MySQL manual:
A prepared statement is specific to the session in which it was created. If you terminate a session without deallocating a previously prepared statement, the server deallocates it automatically.
If you are not using persistent connections, then this will be deallocated when your script finishes executing (or you explicitly deallocate it, or close the connection). If using persistent connections, then it will persist across multiple PHP sessions using the same persistent connection.
As far as I know, a prepared query will only "last" for as long as the variable storing it is within scope. I suppose there could be some ways to cache prepared queries for later use, but I don't know if MySQL does this.
hobodave is right, they just last as long as the session they were created in.
There's one other thing to consider, too:
7.5.5. The MySQL Query Cache
Note
The query cache is not used for
prepared statements. If you are using
prepared statements, consider that
these statements will not be satisfied
by the query cache.
According to the documentation, a prepared query provides a significant performance benefit if you're running a query multiple times because the overhead of the MySQL server parsing the query only happens once. I'm wondering what exactly they mean by "multiple times" there.
I.e., say you have a web page that runs a query one time. Now say that page gets called 50 times per second. Does it make more sense from a performance standpoint to prepare() the query (thereby requiring two roundtrips to the DB server; one to prepare the query, one to run it) or to just send the query normally (which only requires one roundtrip)? Is MySQL and/or the PHP mysqli driver smart enough to realize when a query was prepare()'d in a previous invocation?
No. PHP is a "shared nothing" architecture, so every resource associated with one request (one page view) is discarded at the end of that request. Prepared queries are not available to a subsequent database connections.
The scenario in which you would get benefit from a prepared query is when you prepare it and execute it many times during the same PHP request.