I was just wondering how prepared queries work. I am using PHP and MySQL.
I know that prepared queries compile the query and then repeated uses only have to change the parameters, so it saves time. But how long does that compiled query have an effect? At which point does it have to be reevaluated? Is it just as long as the PHP script is running? Or is it as long as the connection to the database is present? In that case, would a persistent connection have any effect on it?
ryeguy, they last for the length of the connection. Per the MySQL manual:
A prepared statement is specific to the session in which it was created. If you terminate a session without deallocating a previously prepared statement, the server deallocates it automatically.
If you are not using persistent connections, then this will be deallocated when your script finishes executing (or you explicitly deallocate it, or close the connection). If using persistent connections, then it will persist across multiple PHP sessions using the same persistent connection.
As far as I know, a prepared query will only "last" for as long as the variable storing it is within scope. I suppose there could be some ways to cache prepared queries for later use, but I don't know if MySQL does this.
hobodave is right, they just last as long as the session they were created in.
There's one other thing to consider, too:
7.5.5. The MySQL Query Cache
Note
The query cache is not used for
prepared statements. If you are using
prepared statements, consider that
these statements will not be satisfied
by the query cache.
Related
How to take advantage of prepared statements for performance? I understand that something like this might benefit if I put it in a loop:
SELECT `Name` FROM `Hobbits` WHERE `ID` = :ID;
I've read that looping with prepared statements is faster than looping without, but otherwise prepared statements would slightly decrease performance. So - how big may that loop be?
If I run a complex SQL query at the beginning of my code and repeat it with one different parameter at the end - will the second query run faster? (We are using a single connection for each page load). Is there a limit on cached queries, so I better repeat my queries right away?
What about executing the entire script twice with the exact same parameters (reload the page or 2 users)?
A prepared query is given to the SQL server, which parses it and possibly already prepares an execution plan. You're then basically given an id for these allocated resources and can execute this prepared statement by just filling in the blanks in the statement. You can run this statement as often as you like and the database will not have to repeat the parsing and execution planning, which may bring a speed improvement.
As long as you do not throw away the statement, there's no hard timeout for how long the statement will "stay prepared". It's not a cache, it's an allocated resource on the SQL server. At least as long as your database driver uses native prepared statements in the SQL API. PDO for example does not do so by default, unless you set PDO::ATTR_EMULATE_PREPARES to false.
At the end of the script execution though, all those resources will always be deallocated, they do not persist across different page loads. Beyond that, the SQL server may or may not cache the query and its results for some time regardless of the client script.
How long are prepared mysql queries cached?
This is not actually "a cache". Prepared statement lasts as little as during script execution.
If I run a complex SQL query at the beginning of my code and repeat it with one different parameter at the end - will the second query run faster?
The more complex a query, the less effect you will see. Frankly, prepared statement saves you only parsing, while if execution involves temporary or filesort, or table scan - prepared statement would speed up none of them.
On the other hand, for the simple primary-key lookups, which involve no complex query parsing nor building sophisticated query plans, the benefit would be negligible to none.
So - how big may that loop be?
The more iterations it gets - the more benefit. However, in a sane web-application one have to avoid looping queries at all.
What about executing the entire script twice with the exact same parameters (reload the page or 2 users)?
As I said above, there will be no benefit from a prepared statement at all. A classical query cache, however, most likely would fire.
How to take advantage of prepared statements for performance?
Noway. Not in web-serving PHP, at least. In a some long-running cli-based script - may be.
However, prepared statements ought to be used anyway, for the purpose of producing syntactically correct queries.
FYI, from
http://dev.mysql.com/doc/refman/5.7/en/statement-caching.html
The max_prepared_stmt_count system variable controls the total number of statements the server caches. (The sum of the number of prepared statements across all sessions.)
If I want to execute the same query on two different requests, and I use prepared statements with Doctrine2... Will the prepared statement be sent only the first time and be stored by the database for some time? Or will it be removed after each script finishes?
On PostgreSQL a prepared statement is valid only till the end of the session and is not saved in the memory and shared between many sessions, see doc:
http://www.postgresql.org/docs/9.2/static/sql-prepare.html
Prepared statements only last for the duration of the current database
session. When the session ends, the prepared statement is forgotten,
so it must be recreated before being used again. This also means that
a single prepared statement cannot be used by multiple simultaneous
database clients; however, each client can create their own prepared
statement to use.
However, they also say, that PostgreSQL may (but not need to) save a plan for this query in memory for future reusing:
If a prepared statement is executed enough times, the server may
eventually decide to save and re-use a generic plan rather than
re-planning each time. This will occur immediately if the prepared
statement has no parameters; otherwise it occurs only if the generic
plan appears to be not much more expensive than a plan that depends on
specific parameter values. Typically, a generic plan will be selected
only if the query's performance is estimated to be fairly insensitive
to the specific parameter values supplied.
To examine the query plan PostgreSQL is using for a prepared statement, use EXPLAIN. If a generic plan is in use, it will contain
parameter symbols $n, while a custom plan will have the current actual
parameter values substituted into it.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Are prepared statements cached server-side across multiple page loads with PHP?
I'm working on a new project and using parameterized queries for the first time (PHP with a MySQL DB). I read that they parameterized queries are cached, but I'm wondering how long they are cached for. For example, let's say I have a function 'getAllUsers()' that gets a list of all active user ID's from the user table and for each ID, a User object is created and a call to function 'getUser($user)' is made to set the other properties of the object. The 'getUser()' function has it's own prepared query with a stmt->close() at the end of the function.
If I do it this way, does my parameterized query in 'getUser()' take advantage of caching at all or is the query destroyed from cache after each stmt->close()?
Note: I also use the getUser() function if a page only requires data for a single user object so I wanted to do it this way to ensure that if the user table changes I only ever need to update one query.
Is this the right way of doing something like this or is there a better way?
Update: Interesting, just saw this on php.net's manual for prepared statements (http://php.net/manual/en/mysqli.quickstart.prepared-statements.php)
Using a prepared statement is not always the most efficient way of executing a statement. A prepared statement executed only once causes more client-server round-trips than a non-prepared statement.
So I guess the main benefit for parameterized queries is to protect against SQL injection and not necessarily to speed things up unless it's a query that will repeated at one time.
Calling mysqli_stmt::close will:
Closes a prepared statement. mysqli_stmt_close() also deallocates the
statement handle.
therefore not being able to use the cached version of the statement for further executions.
I wouldn't mind of freeing resources or closing statements since PHP will do it for you at the end of the script anyway.
Also if you are working with loops (as you described) take a look at mysqli_stmt::reset which will reset the prepared statement to its original state (after the prepare call).
That's good question, from some point of view.
First, about "caching".
There is some special thing about prepared queries - you can send it to server once and then execute it multiple times. It can give some small theoretical benefit for using already parsed and prepared query.
As it seems, you're not using such mechanism, every time preparing every your query. So, there is no caching at all.
Next, about premature optimization.
You've heard of some caching, and it occupied your imagination.
While there is no real need or cause for you to concern about caching or whatever performance issue.
So, there is a rule: do not occupy yourself with performance issues until they are real.
Otherwise you'll waste your time.
Should PDO prepared statements be freed up after use? And if so, how? Specifically I'm asking about MySQL - how can you, and should you, call DEALLOCATE PREPARE though PDO. (Edit: To clarify, this question is not referring to emulated prepares, but real prepares. )
Also - will this free the results set (when large)?
Explanation:
I have seen code along the lines of
$stmnt = $db->prepare($sql);
$stmnt->execute($aParams);
$stmnt = null;
which led me to wondering what this does, when, and if f unset($stmnt); would be different?
The manual indicates that
When the query is prepared, the database will analyze, compile and
optimize its plan for executing the query. [...] By using a prepared
statement the application avoids repeating the
analyze/compile/optimize cycle.
which tends to suggest you should unallocate the statement, and MySQL has the capability. So,
Can you call DEALLOCATE PREPARE, and how
Should you do it?
And can anyone confirm that setting statement to null (or unsetting the statement) will do the same as "free_result" for mysql_ and mysqli_?
Does it happen immediately, or does it wait for garbage collector to kick in?
For completeness, another SO question referring to "free_result" and "close" functions for mysqli_() suggests that freeing the statement actually adds time (unless you have large memory usage and need the space). But "free_result" is different from freeing the SQL server from having the prepared statment cached.
Should PDO prepared statements be freed up after use? And if so, how?
In the context of MySQL? No. Why?
PDO emulates prepared statements by default. This means that PDO itself does the parameter replacement, escaping, etc, and sends chunks of SQL down the line instead of using native prepared statements.
While you can turn it on, you still don't need to expressly close the handle unless you are also using unbuffered queries. Merely letting the statement handle go out of scope or setting it to null will not close the cursor. Again, this only matters if you're using unbuffered queries. If you are not, then letting it go out of scope or setting it to null is enough to close the handle cleanly.
You also linked to DEALLOCATE PREPARE. That syntax is only needed when manually calling PREPARE with an SQL string. This is a completely and totally separate action than MySQL C-level API-based prepared statements, which is what PDO_MYSQL is using. (Okay, maybe you're using mysqlnd, but it's effectively the same thing.)
Yes. When you are done with the prepare statement you can set it to NULL or use unset().
For a script with multiple queries and large databases, this makes a difference. You can test with:
$before = memory_get_usage();
$stmt = NULL;
die(memory_get_usage() - $before);
For me, this saved 20MB of memory, which was crashing the script later.
I learnt about prepared statements when making a JDBC-enabled Java application, and my app uses a connection pooling layer that assures me that prepared statements are cached server-side and this gives a performance benefit.
However, with PHP everything I've read says that they are only cached for the life of the page load. Generally I don't repeat the same query many times, but run several different queries, on a given page load, but will repeat them across multiple page loads.
As my PHP processes are persistent (i.e. they will serve hundreds of pages in their lifetime instead of just one, using PHP-FPM), I was wondering if they will re-use database connections, rather than spawning and killing them off for each hit.
Will using PHP-FPM with mysqli or PDO keep connections longer than a single page load?
If it doesn't, can I make it?
If it does, or I do #2, will this persist the caching of prepared statements longer than just one page load?
Edit:
Just to clarify, I'm not talking about the query cache, which is another beast entirely, or caching the output of queries. I want to cache the compiled prepared statement and its execution plan server-side.
When a request is served php "cleans" the instance and frees resources and other variables. This is done in several steps. Since fastcgi keeps the process alive after a request not all steps are executed and not all memory is freed. There is e.g. EG(persistent_list) which is used by mysql_pconnect(), pg_pconnect(), ... This list isn't emptied between requests as long as the process keeps alive (could be, depending on the actual implementation, but that would defy the purpose of EG(persistent_list)). If you use persistent connections your script might get a "re-used" connection established during a previous request.
To (re-)use a prepared statement directly you need the identifier for that statement (and that connection). When using (php-)postgresql this is simply a (connection-wise) unique string you pass to pg_execute(), so your script has no problem to gain access to the statement previously prepared by another instance (using the same connection).
Using mysqli or PDO-mysql you need a resource/object as statement identifier. That's kind of a problem since neither the mysqli nor the pdo extension seem to offer a way of storing the resource in EG(persist_list) between requests and you can't recreate it either. Unless php-fpm offers such a "service" it's seems impossible to re-use a mysql prepared statement directly.
All you can hope for is MySQL's server-side query cache. In recent versions (see link) it may recognize the statement when using prepared statements. But even then it doesn't re-use the actual prepared statement:
For a prepared statement executed via the binary protocol, comparison with statements in the query cache is based on the text of the statement after expansion of ? parameter markers. The statement is compared only with other cached statements that were executed via the binary protocol. That is, for query cache purposes, statements issued via the binary protocol are distinct from statements issued via the text protocol.
So, if I'm not mistaken, currently you can't re-use a mysql statement prepared during a previous request in php.
You're confusing what is happening at the PHP/Java tier with what's happenning in the database.
Yes, using prepared statements (usually) means that the execution plan is cached by the database itself (NOT the PHP/Java tier). However it does not follow that this always results in better performance - and an explanation of this would take several hundred pages. However I infer from what you've said elsewhere you are using MySQL as the DBMS which makes the discussion somewhat simpler (IIRC none of the storage engines implement histograms). Typically MySQL will be able to cache enough information about a schema to be able to generate a plan without any disk I/O. OTOH, using prepared statements mean a minimum of three round trips to the DBMS for each query (present statement, present params, retrieve results) while using inlined values eliminates on of these round trips. In the absence of histogram indexes, the value of the variables is irrelevant to the optimal plan detectable by the optimizer.
The fact that you are using PHP, or PHP-FPM or Java with single or persistent or pooled connections is irrelevant to whether or not prepared-statements are cached/re-used by the DBMS.
HTH
C.
If your PHP application uses connection pooling to the database, and the database caches prepared statements, then yes, the caching will persist between pages. If the prepared statement caching is done by the client library, then this is more nebulous.
You need to look at the docs for PHP-FPM and/or PDO to see how to tell them to use connection pooling. There should be an option in both to do it.
You should be aware that MySQL connection setup and teardown is actually very fast and many PHP installations do not use connection pooling because of this. Either way, you should also invest time in your server settings, particularly the wait_timeout parameter. PHP is also designed around the idea that you create everything you need when your pages starts and it all goes away when the page finishes. Most PHP code and libraries assume this is the case. It is quite a different paradigm than under Java.
The only true answer is it depends.
Prepared statements are finicky beasts when it comes to MySQL. There are a great number of factors that determine whether or not a prepared statement is cached.
The general idea is if your version is < 5.1.17, the prepared statement is never cached in the query cache, and if using >= 5.1.17, it depends.
Please see the following page in the MySQL 5.1 manual:
http://dev.mysql.com/doc/refman/5.1/en/query-cache-operation.html
you can force mysqli to create a persistent connection by prepending p: to the hostname, as per the php doc: http://www.php.net/manual/en/mysqli.persistconns.php
However, prepared statements are always closed between page loads, as discussed here: http://dev.mysql.com/doc/refman/5.0/en/apis-php-mysqli.persistconns.html
Sorry, it can't be done as far as I know. Prepared statements are for a single page load.
Prepared statements have nothing to do with result caching.
Result caching can be controlled via db server configuration or forced via memcached and the like.
I suggest you look into memcached, especially for PHP http://www.php.net/manual/en/book.memcached.php
PHP does not cache queries nor query results in most cases. MySQL will perform this kind of caching regardless or what thread or connection is issuing the query.
If you want server-side caching across multiple page loads or multiple servers, then use MySQL query caching and server-side caching (APC, file-based caching, memcached, etc).