There are quite a few blog/links which discourage usage of persistent connections, mainly because the cleanup needs to be done on client side, and cases where transactions/locks have to be correctly rolled back. However, those links are old, and not enirely in context of mysqli PHP interface.
I read the link : The mysqli Extension and Persistent Connections
It clearly suggests that it does most of the desired cleanup when a client terminates unexpectedly:
Rollback active transactions
Close and drop temporary tables
Unlock tables
Reset session variables
Close prepared statements (always happens with PHP)
Close handler
Release locks acquired with GET_LOCK()
Now that pretty much does most of the cleanup, including READ/WRITE locks on tables if any acquired. So I believe it should be safe. Can I be wrong?
Also, it says there are some performance penalty in form of extra time needed to do cleanup. I would like to know how much that may be in terms of millisecs? Can it ever be as large as say 100 ms?
The automatic cleanup feature has advantages and disadvantages though.
The advantage is that the programmer no longer needs to worry about
adding cleanup code, as it is called automatically. However, the
disadvantage is that the code could potentially be a little slower, as
the code to perform the cleanup needs to run each time a connection is
returned from the connection pool.
I wonder if you really think you can trust an answer from anonymous passer-by more than official documentation page, that clearly answers your question.
But if you do - yes, you can believe it should be safe.
As of performance penalty, from the way it is asked, I believe you don't really need persistent connections at all.
Related
I am searching for a efficient way to use PHP MySQL innoDB connection but not able to found conclusive information on the web.
As I know, persistent connection is much faster than non-persistent one,
we can set up the connection in following way:
$instance_mysqli = new mysqli('p:127.0.0.1', 'username', 'password', 'db');
However, from the official website, it said the default behavior is "reset" on reuse, which is slower. http://php.net/manual/en/mysqli.persistconns.php
The mysqli extension does this cleanup by automatically calling the
C-API function mysql_change_user(). The automatic cleanup feature has
advantages and disadvantages though. The advantage is that the
programmer no longer needs to worry about adding cleanup code, as it
is called automatically. However, the disadvantage is that the code
could potentially be a little slower, as the code to perform the
cleanup needs to run each time a connection is returned from the
connection pool.
So, there is no way to pass parameter to the above constructor to avoid "reset"? The only way is to recompile extension from source code as the document suggested?
And my anther question is... if mysqli is so smart that it can automatically reset connection by default, what is the point many people still use non-persistent connection, which is even slower.
The cost of a connection is quite small, whether it is persistent or not, whether there is cleanup or not.
Normally, one should acquire one connection at the beginning of the program, and keep it until the end. (There are some exceptions.)
The only time a connection is really noticeable is if you acquire a connection before each and every SQL query.
Bottom line: Worry about your indexes, system design, etc, not about acquiring the connection.
I was always in assumption that it is always a good practice to close database connection, regardless of database/ORM, like mysql_close(), Propel::close() etc.
With reference to one of my other question and some other research on Internet, I came to know a surprising face that most people recommends it doesn't really matter if you close connection as connection always gets closed after the request.
However I'm finding those answers little difficult to digest. Reason is, why all DB lib, ORM provide close method? If it is there, in every ORM/lib, there must be some good use of it.
Can someone please shed some light on under what circumstances, we should use close method to close DB connection? & if these methods are not useful at all, why they are present there in all db libs/ORM?
EDIT
My Conclusion
It was a good discussion between Bondye and Fluffeh and it cleared my doubts about use of connection closing. Thanks to both of them.
If your script is expected to last of less than 100 ms, dont bother closing connection.
BUT: if script is expected to last longer and there is some time between last DB operation and close of script, free connection for others by calling *close().
It is really very difficult for me to accept one answer as both answer are correct on its place. Just accepting answer with all comments so that it remain on top. But +1 to both correct answers.
Using mysql_close() isn't usually necessary, as non-persistent open links are automatically closed at the end of the script's execution.
Freeing resources
Thanks to the reference-counting system introduced with PHP 4's Zend Engine, a resource with no more references to it is detected automatically, and it is freed by the garbage collector. For this reason, it is rarely necessary to free the memory manually.
Hope this helps you more.
(source)
edit:
The purpose of mysql_close() is also to save computer resources, but another key reason for using it is because there is a limited number of connections that a MySQL server can accept, and if you have several clients holding connections open for no reason then the server may well need to turn away other, waiting clients. Naturally this is a bad thing, so, as with mysql_free_result(), it is good to call mysql_close() if you think there will be some time between your last database use and your script ending.
It is always good practice to close a database connection when you no longer need it. Even if it gets closed automatically after the script ends - that might be another second or a number of split seconds later. If you no longer need it, one user hitting a page and wasting the database connection for half a second won't make a difference - but twenty doing it at once is suddenly 10 seconds of open connection - and that does make a difference.
At the same time, re-using a connection can be a good practise - making and opening the connection normally takes at least a few milliseconds - and if you are for example inserting a few hundred thousand rows, that few milliseconds each time adds up really fast.
In a way, it is no different to setting a variable to NULL or unsetting it. You don't have to do it, but clean elegant code and resource management is always a good thing.
Database connections are not unlimited. Commercial database software, especially, often have licenses that limit the number of simultaneous connections to a relatively small number. In such a situation, you definitely want to close the connection when your script is no longer actively using. While PHP does automatically close database connection when a script terminates, it doesn't do so until the visitor has finished downloading the page. If his connection is slow (dial-up or mobile), that could take ten, twenty seconds for all you know.
Well developed ORM's like Doctrine and Propel are good at closing MySQL connections. But if you are using straight php, I've seen a lot of database problems tracked back to unclosed connections. It's wise to close all db connections at the end of each script.
Just started using mysqli. If I'm working with small data sets on small websites (traffic-wise), do I really need to use these all the time?
$result->close();
$mysqli->close();
Also, for someone doing custom PHP and MySQL work without a framework, is mysqli the general preferred way of interacting with MySQL?
PHP will close all open files and DB connections at the end of the script. It's good practice to do it manually when you are done with the connections, but it's no disaster if you don't. If you have a DB connection that will be used throughout the whole script you can as well leave it open.
+1 on PDO
According to current documentation, you should always use $mysql->kill() in addition to $mysql->close().
$thread = $mysqli->thread_id;
$mysqli->kill($thread);
$mysqli->close();
You should get in the habit of doing cleanup right (calling close as soon as you're done), or the resource leaks can gradually accumulate until they impact performance.
As far as what DB layer, learning PDO should be worthwhile because it is well-designed and compatible with all the major databases.
It is a good practice to release resource early when it is no more needed, this may avoid resource peek out when there are more number of concurrent user accessing the same page
I learnt about prepared statements when making a JDBC-enabled Java application, and my app uses a connection pooling layer that assures me that prepared statements are cached server-side and this gives a performance benefit.
However, with PHP everything I've read says that they are only cached for the life of the page load. Generally I don't repeat the same query many times, but run several different queries, on a given page load, but will repeat them across multiple page loads.
As my PHP processes are persistent (i.e. they will serve hundreds of pages in their lifetime instead of just one, using PHP-FPM), I was wondering if they will re-use database connections, rather than spawning and killing them off for each hit.
Will using PHP-FPM with mysqli or PDO keep connections longer than a single page load?
If it doesn't, can I make it?
If it does, or I do #2, will this persist the caching of prepared statements longer than just one page load?
Edit:
Just to clarify, I'm not talking about the query cache, which is another beast entirely, or caching the output of queries. I want to cache the compiled prepared statement and its execution plan server-side.
When a request is served php "cleans" the instance and frees resources and other variables. This is done in several steps. Since fastcgi keeps the process alive after a request not all steps are executed and not all memory is freed. There is e.g. EG(persistent_list) which is used by mysql_pconnect(), pg_pconnect(), ... This list isn't emptied between requests as long as the process keeps alive (could be, depending on the actual implementation, but that would defy the purpose of EG(persistent_list)). If you use persistent connections your script might get a "re-used" connection established during a previous request.
To (re-)use a prepared statement directly you need the identifier for that statement (and that connection). When using (php-)postgresql this is simply a (connection-wise) unique string you pass to pg_execute(), so your script has no problem to gain access to the statement previously prepared by another instance (using the same connection).
Using mysqli or PDO-mysql you need a resource/object as statement identifier. That's kind of a problem since neither the mysqli nor the pdo extension seem to offer a way of storing the resource in EG(persist_list) between requests and you can't recreate it either. Unless php-fpm offers such a "service" it's seems impossible to re-use a mysql prepared statement directly.
All you can hope for is MySQL's server-side query cache. In recent versions (see link) it may recognize the statement when using prepared statements. But even then it doesn't re-use the actual prepared statement:
For a prepared statement executed via the binary protocol, comparison with statements in the query cache is based on the text of the statement after expansion of ? parameter markers. The statement is compared only with other cached statements that were executed via the binary protocol. That is, for query cache purposes, statements issued via the binary protocol are distinct from statements issued via the text protocol.
So, if I'm not mistaken, currently you can't re-use a mysql statement prepared during a previous request in php.
You're confusing what is happening at the PHP/Java tier with what's happenning in the database.
Yes, using prepared statements (usually) means that the execution plan is cached by the database itself (NOT the PHP/Java tier). However it does not follow that this always results in better performance - and an explanation of this would take several hundred pages. However I infer from what you've said elsewhere you are using MySQL as the DBMS which makes the discussion somewhat simpler (IIRC none of the storage engines implement histograms). Typically MySQL will be able to cache enough information about a schema to be able to generate a plan without any disk I/O. OTOH, using prepared statements mean a minimum of three round trips to the DBMS for each query (present statement, present params, retrieve results) while using inlined values eliminates on of these round trips. In the absence of histogram indexes, the value of the variables is irrelevant to the optimal plan detectable by the optimizer.
The fact that you are using PHP, or PHP-FPM or Java with single or persistent or pooled connections is irrelevant to whether or not prepared-statements are cached/re-used by the DBMS.
HTH
C.
If your PHP application uses connection pooling to the database, and the database caches prepared statements, then yes, the caching will persist between pages. If the prepared statement caching is done by the client library, then this is more nebulous.
You need to look at the docs for PHP-FPM and/or PDO to see how to tell them to use connection pooling. There should be an option in both to do it.
You should be aware that MySQL connection setup and teardown is actually very fast and many PHP installations do not use connection pooling because of this. Either way, you should also invest time in your server settings, particularly the wait_timeout parameter. PHP is also designed around the idea that you create everything you need when your pages starts and it all goes away when the page finishes. Most PHP code and libraries assume this is the case. It is quite a different paradigm than under Java.
The only true answer is it depends.
Prepared statements are finicky beasts when it comes to MySQL. There are a great number of factors that determine whether or not a prepared statement is cached.
The general idea is if your version is < 5.1.17, the prepared statement is never cached in the query cache, and if using >= 5.1.17, it depends.
Please see the following page in the MySQL 5.1 manual:
http://dev.mysql.com/doc/refman/5.1/en/query-cache-operation.html
you can force mysqli to create a persistent connection by prepending p: to the hostname, as per the php doc: http://www.php.net/manual/en/mysqli.persistconns.php
However, prepared statements are always closed between page loads, as discussed here: http://dev.mysql.com/doc/refman/5.0/en/apis-php-mysqli.persistconns.html
Sorry, it can't be done as far as I know. Prepared statements are for a single page load.
Prepared statements have nothing to do with result caching.
Result caching can be controlled via db server configuration or forced via memcached and the like.
I suggest you look into memcached, especially for PHP http://www.php.net/manual/en/book.memcached.php
PHP does not cache queries nor query results in most cases. MySQL will perform this kind of caching regardless or what thread or connection is issuing the query.
If you want server-side caching across multiple page loads or multiple servers, then use MySQL query caching and server-side caching (APC, file-based caching, memcached, etc).
I use lazy connection to connect to my DB within my DB object. This basically means that it doesn't call mysql_connect() until the first query is handed to it, and it subsequently skips reconnecting from then on after.
Now I have a method in my DB class called disconnectFromDB() which pretty much calls mysql_close() and sets $_connected = FALSE (so the query() method will know to connect to the DB again). Should this be called after every query (as a private function) or externally via the object... because I was thinking something like (code is an example only)
$students = $db->query('SELECT id FROM students');
$teachers = $db->query('SELECT id FROM teachers');
Now if it was closing after every query, would this slow it down a lot as opposed to me just adding this line to the end
$db->disconnectFromDB();
Or should I just include that line above at the very end of the page?
What advantages/disadvantages do either have? What has worked best in your situation? Is there anything really wrong with forgetting to close the mySQL connection, besides a small loss of performance?
Appreciate taking your time to answer.
Thank you!
As far as I know, unless you are using persistent connections, your MySQL connection will be closed at the end of the page execution.
Therefore, you calling disconnect will add nothing and because you do the lazy connection, may cause a second connection to be created if you or another developer makes a mistake and disconnects at the wrong time.
Given that, I would just allow my connection to close automatically for me. Your pages should be executing quickly, therefore holding the connection for that small amount of time shouldn't cause any problems.
I just read this comment on PHP website regarding persistent connection and it might be interesting to know:
Here's a recap of important reasons
NOT to use persistent connections:
When you lock a table, normally it is unlocked when the connection
closes, but since persistent
connections do not close, any tables
you accidentally leave locked will
remain locked, and the only way to
unlock them is to wait for the
connection to timeout or kill the
process. The same locking problem
occurs with transactions. (See
comments below on 23-Apr-2002 &
12-Jul-2003)
Normally temporary tables are dropped when the connection closes,
but since persistent connections do
not close, temporary tables aren't so
temporary. If you do not explicitly
drop temporary tables when you are
done, that table will already exist
for a new client reusing the same
connection. The same problem occurs
with setting session variables. (See
comments below on 19-Nov-2004 &
07-Aug-2006)
If PHP and MySQL are on the same server or local network, the
connection time may be negligible, in
which case there is no advantage to
persistent connections.
Apache does not work well with persistent connections. When it
receives a request from a new client,
instead of using one of the available
children which already has a
persistent connection open, it tends
to spawn a new child, which must then
open a new database connection. This
causes excess processes which are just
sleeping, wasting resources, and
causing errors when you reach your
maximum connections, plus it defeats
any benefit of persistent connections.
(See comments below on 03-Feb-2004,
and the footnote at
http://devzone.zend.com/node/view/id/686#fn1)
(I was not the one that wrote the text above)
Don't bother disconnecting. The cost of checking $_connected before each query combined with the cost of actually calling $db->disconnectFromDB(); to do the closing will end up being more expensive than just letting PHP close the connection when it is finished with each page.
Reasoning:
1: If you leave the connection open till the end of the script:
PHP engine loops through internal array of mysql connections
PHP engine calls mysql_close() internally for each connection
2: If you close the connection yourself:
You have to check the value of $_connected for every single query. This means PHP has to check that the variable $_connected A) exists B) is a boolean and C) is true/false.
You have to call your 'disconnect' function, and function calls are one of the more expensive operations in PHP. PHP has to check that your function A) exists, B) is not private/protected and C) that you provided enough arguments to your function. It also has to create a copy of the $connection variable in the new local scope.
Then your 'disconnect' function will call mysql_close() which means PHP A) checks that mysql_close() exists and B) that you have provided all needed arguments to mysql_close() and C) that they are the correct type (mysql resource).
I might not be 100% correct here but I believe the odds are in my favour.
You may want to look at a using persistent connections. Here are two links to help you out
http://us2.php.net/manual/en/features.persistent-connections.php
http://us2.php.net/manual/en/function.mysql-pconnect.php
The basic unit of execution presumably is an entire script. What you first of all are wanting to apply resources (i.e. the database) to, efficiently and effectively, is the entirety of a single script.
However, PHP, Apache/IIS/whatever, have lives of their own; and they are capable of using the connections you open beyond the life of your script. That's the signficance of persistent (or pooled) connections.
Back to your script. It turns out you have a great deal of opportunity to be creative about using that connection during its execution.
The typical naive script will tend to hit the connection again and again, picking up locally appropriate scraps of data associated with given objects/modules/selected options. This is where procedural methodology can inflict a penalty on that connection by opening, requesting, receiving, and closing. (Note that any single query will remain alive until it is explicitly closed, or the script ends. Be careful to note that a connection and a query are not the same thing at all. Queries tie up tables; connections tie up ... connections (in most cases mapped to sockets). So you should be conscious of proper economy in the use of both.
The most economical strategy with regard to queries is to have as few as possible. I'll often try to construct a more or less complex joined query that brings back a full set of data rather than parceling out the requests in small pieces.
Using a lazy connection is probably a good idea, since you may not need the database connection at all for some script executions.
On the other hand, once it's open, leave it open, and either close it explicitly as the script ends, or allow PHP to clean up the connection - having an open connection isn't going to harm anything, and you don't want to incur the unnecessary overhead of checking and re-establishing a connection if you are querying the database a second time.