What is the best way to ensure I'm writing a scheme to minimize connection count to my MySQL database?
Currently, I have a ubuntu / PHP / apache setup on my server, and I'm using the following code with PDO to ensure a persistent connection:
$dbh = new PDO('mysql:host=localhost;dbname=test', $user, $pass, array(
PDO::ATTR_PERSISTENT => true
));
However, I seem to have 10 or more connections on at all times. I am not sure why this is. Quote from php.net:
Many web applications will benefit from making persistent connections
to database servers. Persistent connections are not closed at the end
of the script, but are cached and re-used when another script requests
a connection using the same credentials. The persistent connection
cache allows you to avoid the overhead of establishing a new
connection every time a script needs to talk to a database, resulting
in a faster web application.
Does this mean I have to write $dbh = null to "close" and cache the connection?
A persistent connection is, by definition, one that is not closed (i.e. it persists); the benefit is that PHP remains connected to the database for other scripts to use (i.e. there is no need for it to teardown and setup new database connections each time a script runs).
It is not necessary to write $dbh = null, as that is effectively implied on termination of the script's execution; you can however explicitly write it if you want to signal that you are finished with the connection earlier than at the end of the script (PHP's garbage collector will then, at some point, free up the resource and return it to the connection pool for other scripts' use).
If you want connections to get closed and destroyed at the end of each script, then don't use persistent connections!
The 'I seem to have 10 more connections' does not seem a legitimate question or concern, because that attribute will just hold the database resource object, as long as PHP is running. Therefore, as #eggyal has said, if you want connection to get closed or destroyed, then don't use persistent connection.
Please read also:
What are the disadvantages of using persistent connection in PDO
Why You Should Use Persistent Connections with MySQL
Related
Question:
What are the rules/logic behind persistent connection management when using PDO?
Environment:
Web Server
Windows 7 x64
Dual-core with 16GB RAM
Apache 2.2.17
PHP 5.3.5
Connecting through DSN string with IP address, port, service name, etc...
No ODBC for DB conn (been trying to create one for 2 hours now, thanks Oracle!)
DB Server
Oracle 10g on Linux
Multi-core with 4GB RAM
Username specifically created for my web app (yes, it's fake)
user: webuser
My understanding/observations:
Non-persistent connections
<?php
// Open a new connection
// Session created in Oracle
$dbh = new PDO('DSN', 'webuser', 'password');
// webuser is active in v$session with a SID=1
$dbh = NULL;
// webuser removed from v$session
// Manually calling $dbh = NULL; will remove the session from v$session
// OR
// Wait for script EOL so a kill-session command is sent to Oracle?
?>
Script reliably takes about ~.09 seconds to execute with framework overhead, etc...
Persistent connections
<?php
// Open a new connection and make it persistent
// Session created in Oracle
// Is Apache maintaining some sort of keep-alive with Oracle here?
// because I thought php.exe is only alive for the duration of the script
$dbh = new PDO('DSN', 'webuser', 'password', array(PDO::ATTR_PERSISTENT => TRUE));
// webuser is active in v$session with a SID=1
$dbh = NULL;
// webuser is still active in v$session with a SID=1
$dbh = new PDO('DSN', 'webuser', 'password', array(PDO::ATTR_PERSISTENT => TRUE));
// webuser is still active in v$session with a SID=1
// Manually calling $dbh = NULL; does not kill session
// OR
// Script EOL does not kill session
// ^^ this is good, just as expected
?>
Script takes ~.12 seconds to execute upon initial visit with framework overhead, etc...
Sub-sequent executes take ~.04
The issue:
I visit the page and webuser gets a SID=1
My colleague visits the page and webuser gets an additional SID=2 <- rinse, repeat, and increment SID for new computers visiting this page
Shouldn't a new visitor be re-using SID=1?
All answers, suggestions, requests for alternate testing, links to reading material are welcomed.
I have RTFM'ed for a while and Googling has only produced meager Advantages of Persistent vs. Non-persistent blogs.
Apaches point of view
Apache has one parent process. This process creates child processes that will handle any requests coming to the web server.
The initial amount of child processes being started when the web server starts is configured by the StartServers directive in the apache configuration. The number goes up as needed with a raising amount of requests hitting the web server until ServerLimit is reached.
PHP and persistent connections
If PHP (ran as mod_php, as CGI all resources are freed at the end of script execution) is now being told to establish a persistent connection with a database for a request, this connection is hold even after the script finishes.
The connection being now hold is a connection between the apache child process which the request was handled by and the database server and can be re-used by any request that is being handled by this exact child process.
If, for some reason (do not ask me exactly why), the child process is being occupied longer than the actual request and another request comes in, the parent apache process redirects this request to a (new) child process which may has not established a connection to the database up to this time. If it has to during the execution of the script, it raises the SID as you have observed. Now there are two connections be hold by two different child processes of apache.
Keep in mind that...
It is important to know, that this can also cause a lot of trouble.
If there is an endless loop or an aborted transaction or some other may be even unpredictable error during the script execution, the connection is blocked and can not be re-used.
Also it could happen that all of the available connections of the database are used, but there is another child process of the apache server trying to access the database.
This process is blocked for the time being until a connection is freed by the database or apache (timeout or voluntarily by termination).
Any further information about this topic on this page: http://www.php.net/manual/en/features.persistent-connections.php
I hope I got all that we have discussed in our comment conversation summarized correctly and did not forget anything.
If so, please, leave me a hint and I will add it. :)
Edit:
I just finished reading the article #MonkeyZeus mentioned in this comment.
It describes the process I summarized above and provides useful information on how to optimize your apache server to work better together with persistent connections.
It can be used with or without oracle database backends, though.
You should give a look: http://www.oracle.com/technetwork/articles/coggeshall-persist-084844.html
Advantages
From the manual page of php for persistent connections on this link:
Persistent connections are links that do not close when the execution of your script ends. When a persistent connection is requested, PHP checks if there's already an identical persistent connection (that remained open from earlier) - and if it exists, it uses it. If it does not exist, it creates the link.
The reason behind using persistent connections is, of course, reducing the number of connections which are rather expensive; Even though they are much faster with MySQL than with most other databases.
Issues
There are some issues with table locking while using persistent connections.
if the script for whatever reason cannot release the lock, then subsequent scripts using the same connection will block indefinitely and may require that you either restart the httpd server or the database server.
Another is that when using transactions by mysql commit.
A transaction block will also carry over to the next script which uses that connection if script execution ends before the transaction block does. In either case, you can use register_shutdown_function() to register a simple cleanup function to unlock your tables or roll back your transactions.
I suggest you read this question about disatvantages of persistent connections.
PDO is kinda funny that way. Even the same user/visitor can cause a second or even third instance to be created. The same thing happened to me on my local machine, while testing the performance of my db queries.
That is nothing to worry about, because these instances will timeout sooner or later, the exact timeout depends on your server configuration.
Why that happens? If the current instance is busy, then a new instance will be created and the older one will timeout sooner or later. At least that seems logical to me.
Here are my observations as I recently experienced an issue similar to yours.
MySQL server kept opening new connections and eventually maxed out the number of concurrent connections to MySQL server even though there were a lot of Idle connections that can be used.
Setting PDO::ATTR_PERSISTENT => true does re-use available idle connections. It may not seem like this at first glance when you try to monitor MySQL process because in the time that the report is sent back to you, that idle connection may have become active by another process.
Overall, you should notice a drop in the number of idle connections as opposed to not using a persistent connection. As regards the table locking issue, I decided to use InnoDB Storage Engine for my tables as it uses row-level locking as opposed to table locking with MyISAM Storage Engine.
I have not had an issue yet with concurrency when using this combination of InnoDB Storage Engine and PDO persistent Connection.
Also, as a safeguard on badly executed queries locking tables, keep queries within a try-catch Block.
I've read a ton about persistent database connections between PHP and MySQL (mysql_connect vs. mysql_pconnect). Same with PDO and MySQLi. It's definitely just my lack of understanding on this one, but how can a database connection be persistent between webpages? In this code:
$conn = mysql_pconnect( $server , $user, $pass );
mysql_select_db( $dbname );
If two users load this page at the same time, with two different $dbname variables, will PHP only make one connection to the database or two? I am fairly certain that
$conn = mysql_connect( $server , $user, $pass );
would make two connections.
If pconnect reuses the connection opened by the first user, will the mysql_select_db call work for the second user?
Ideally, what I am looking for is a way to have fewer database connections but still be able to set the default database in each PHP script. I have clients who all use the same PHP scripts, but the data is stored in their own client database (hence, $dbname is always different, but the MySQL connection parameters are the same - same mysql ip address, user and password).
Hope that makes sense. We can use MySQL, MySQLi or PDO, just need to know how to accomplish this the best way without having the possibility for clients to accidently write data to someone else's database! Thanks in advance.
The persistence is done by the copy of the PHP that's embedded in the webserver. Ordinarily you'd be right- if PHP was running in CGI mode, it would be impossible to have a persistent connection, because there'd be nothing left to persist when the request is done and PHP shuts down.
However, since there's a copy of PHP embedded in the webserver, and the webserver itself keeps running between requests, it is possible to maintain a pool of persistent connections within that "permanent" PHP.
However, note that on Apache multi-worker type server models, the connection pools are maintained PER-CHILD. If you set your pool limit to 10, you'll have 10 connections per Apache child. 20 children = 200 connections.
Persistent connections will also lead to long-term problems with deadlocks and other hard-to-debug problems. Remember - there's no guarantee that a user's HTTP requests will be serviced by the SAME apache child/mysql connection. If a script dies part-way through a database transaction, that transaction will NOT be rolled back, because MySQL does not see the HTTP side of things - all it sees is that the mysql<->apache connection is still open and assumes all's well.
The next user to hit that particular apache/mysql child/connection combination will now magically end up in the middle of that transaction, with no clue that the transaction is open. Basically, it's the Web equivalent of an unflushed toilet - all the "garbage" from the previous user is still there.
With non-persistent connections, you're guaranteed to have a 'clean' environment each time you connect.
From my reading of documentation and comments, I see:
Docs on mysql_pconnect (deprecated method)
Second, the connection to the SQL server will not be closed when the execution of the script ends. Instead, the link will remain open for future use ( mysql_close() will not close links established by mysql_pconnect()).
and a comment on that page
Persistent connections work well for CGI PHP managed by fastCGI, contrary to the suggestion above that they only work for the module version. That's because fastCGI keeps PHP processes running between requests. Persistent connections in this mode are easily made immune to connection limits too, because you can set PHP_FCGI_CHILDREN << mysql's max_connections <<< Apache's MaxClients. This also saves resources.
Docs on mysqli_connect (new method)
Prepending host by p: opens a persistent connection. mysqli_change_user() is automatically called on connections opened from the connection pool.
Docs for mysqli_change_user:
Changes the user of the specified database connection and sets the current database.
So my understanding is as follows: pconnect keeps the connection open after a script ends but while a process (or maybe group of processes) is still alive (like in a server with FCGI set up). Only one script at a time uses a connection, and when a new script grabs that connection the user and database are updated.
Thus if you use FCGI and persistent connections you can reduce the number of db connections open, but scripts running simultaneously will not be sharing the same connection. There is no problem with the connection being confused as to which database is selected.
so suppose I connect to mysql with my php script as usual, but then suppose I never actually call any code that would close that connection and just let it be until the page is served to the user
Is there any reason why I shouldn't be doing this? What negative consequences may arise (if any) if I do this?
Depending on the mysql extension you are using not closing the connection is actually beneficial as this will allow the use of persistent connections. That means the connection is only opened the first time the script runs, and every subsequent run the already-open mysql connection is used, reconnecting as needed. This is a bit faster and less resource intensive then opening and closing the connection on every single request. If you are using mysqli then persistent connections should be possible and are preferable.
Even if your mysql extension doesn't support persistent connections, or isn't configured to use them, that just means the connection will close when the script exits, which is no different than closing it yourself, which means closing it manually gains you nothing.
Conclusion: closing the connection yourself probably gains you nothing and might actually hurt performance.
edit: See mysql.allow_persistent and mysqli.allow_persistent to configure whether or not persistent connections are used for mysql, depending on which mysql extension you are using.
Checkout out the php option
; http://php.net/mysql.allow-persistent
mysql.allow_persistent = On
This should help, if you can set this.
Like any unoptimized code, failing to close() or kill() the socket will usually not become evident until the site/application is under heavy demand, i.e., lots of page requests are being sent to the server. By not killing+closing and freeing up the socket after the request has been serviced, the socket sits in a a 'wait' state for a short period of time, adding to the load of any new socket connections occurring when new page loads and connection attempts are made. Given very high traffic conditions, these sockets will continue to accumulate in process memory, possibly (based on your max connections and max_user_connections params) leading to the ubiquitous "Max connections reached...." message.
Garbage collection that occurs after the script has completed will not necessarily recycle the socket endpoint. As far as I know, only calling mysqli_kill() after mysqli_close() will actually kill the socket.
As for persistent connections, you don't mention which mysql api you're using or the server and how php is loaded into the server and network topology. Persistent connections allow you to avoid the overhead of opening a connection to mysql. If mysql runs on the same box as the web server, persistent connections probably won't buy much scalability. If php runs in a cgi process, persistent connections will not work because the php process only exists during the lifetime of the request. Don't confuse persistent connections with JDBC connection pooling, they're not the same.
Since PHP has garbage collection, when the script finishes on the server, the connection is closed. Unless you specified it as a persistent connection.
From the php reference:
Using mysql_close() isn't usually necessary, as non-persistent open links are automatically closed at the end of the script's execution.
You can read more at the mysql_close reference here.
PHP supports the use of persistent SQLite connections. However, a problem arises in trying to run maintenance scripts (like a file backup), with such a connection opened. These scripts are likely to be run without a server restart every now and then, in periods of low traffic.
How can I check if there is currently an SQLite persistent connection, without opening / calling one (hence creating the connection)?
See: php SQLite persistent connection
If you have access to shell_exec() or exec, you can run a shell command to check to see if a SQLite process is running using something like top or maybe a command like lsof -i -n -P | grep sqlite assuming sqlite is the name of the process.
If you use PDO you can just check if the handler is null or not before you run the maintenance scripts. That way you wont be creating a connection like with sqlite_popen()
Create a persistent connection with PDO if you want: $handler = new PDO('sqlite:test.db', NULL, NULL, array(PDO::ATTR_PERSISTENT => TRUE));
...Then you can just close the connection before the maintenance script is called, assuming it is on some sort of schedule:
if(!is_null($handler)){
$handler = null;
//run maintenance script, recreate connection once finished
}
To quote the manual at https://www.php.net/manual/en/features.persistent-connections.php:
'When a persistent connection is requested, PHP checks if there's already an identical persistent connection (that remained open from earlier) - and if it exists, it uses it. If it does not exist, it creates the link. An 'identical' connection is a connection that was opened to the same host, with the same username and the same password (where applicable).'
That seems to imply that there no need to check prior to trying to connect.
Note that the article goes into a lot of considerations about how persistent connections work out with web servers, particularly about how subsequent processes attempting to use the connection may be blocked by what a fault in an earlier process created, implying that persistent connections may not be reliable in a web server environment given how web sessions can terminate at any time.
In particular, it recommends that persistent connections are not used for scripts that use table locks or transactions.
Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 7 years ago.
Improve this question
I have this doubt, I've searched the web and the answers seem to be diversified. Is it better to use mysql_pconnect over mysql_connect when connecting to a database via PHP? I read that pconnect scales much better, but on the other hand, being a persistent connection... having 10 000 connections at the same time, all persistent, doesn't seem scalable to me.
Thanks in advance.
Persistent connections should be unnecessary for MySQL. In other databases (such as Oracle), making a connection is expensive and time-consuming, so if you can re-use a connection it's a big win. But those brands of database offer connection pooling, which solves the problem in a better way.
Making a connection to a MySQL database is quick compared to those other brands, so using persistent connections gives proportionally less benefit for MySQL than it would for another brand of database.
Persistent connections have a downside too. The database server allocates resources to each connection, whether the connections are needed or not. So you see a lot of wasted resources for no purpose if the connections are idle. I don't know if you'll reach 10,000 idle connections, but even a couple of hundred is costly.
Connections have state, and it would be inappropriate for a PHP request to "inherit" information from a session previously used by another PHP request. For example, temporary tables and user variables are normally cleaned up as a connection closes, but not if you use persistent connections. Likewise session-based settings like character set and collation. Also, LAST_INSERT_ID() would report the id last generated during the session -- even if that was during a prior PHP request.
For MySQL at least, the downside of persistent connections probably outweighs their benefits. And there are other, better techniques to achieve high scalability.
Update March 2014:
MySQL connection speed was always low compared to other brands of RDBMS, but it's getting even better.
See http://mysqlserverteam.com/improving-connectdisconnect-performance/
In MySQL 5.6 we started working on optimizing the code handling connects and disconnects. And this work has accelerated in MySQL 5.7. In this blog post I will first show the results we have achieved and then describe what we have done to get them.
Read the blog for more details and speed comparisons.
Basically you have to balance the cost of creating connections versus keeping connections. Even though MySQL is very fast at setting up a new connection, it still costs -- in thread setup time, and in TCP/IP setup time from your web server. This is noticeable on a high-enough traffic site. Unfortunately, PHP does not have any controls on the persistence of connections. So the answer is to lower the idle timeout in MySQL a long way (like down to 20 seconds), and to up the thread cache size. Together, this generally works remarkably well.
On the flip side, your application needs to respect the state of the connection. It is best if it makes no assumptions about what state the session is in. If you use temporary tables, then using CREATE IF NOT EXISTS and TRUNCATE TABLE helps a lot, as does naming them uniquely (such as including as userid). Transactions are bit more problematic; but your code can always do ROLLBACK at the top, just in case.
mysql_connect() and mysql_pconnect() both are working for database connection but with little difference. In mysql_pconnect(), p stands for persistance connection.
When we are using mysql_connect() function, every time it is opening and closing the database connection, depending on the request.
But in case of mysql_pconnect() function:
First, when connecting, the function would try to find a (persistent) connection that's already open with the same host, username and password. If one is found, an identifier for it will be returned instead of opening a new connection.
Second, the connection to the SQL server will not be closed when the execution of the script ends. Instead, the connection will remain open for future use (mysql_close() will not close connection established by mysql_pconnect()).
mysql_pconncet() is useful when you have a lot of traffice on your site. At that time for every request it will not open a connection but will take it from the pool. This will increase the efficiency of your site. But for general use mysql_connect() is best.
It's very unlikely that you'll reach 10000 connections. Anyhow, go to the official source. (Emphasis mine).
If persistent connections don't have
any added functionality, what are
they good for?
The answer here is extremely simple --
efficiency. Persistent connections are
good if the overhead to create a link
to your SQL server is high. Whether or
not this overhead is really high
depends on many factors. Like, what
kind of database it is, whether or
not it sits on the same computer on
which your web server sits, how
loaded the machine the SQL server sits
on is and so forth. The bottom line
is that if that connection overhead is
high, persistent connections help you
considerably. They cause the child
process to simply connect only once
for its entire lifespan, instead of
every time it processes a page that
requires connecting to the SQL server.
This means that for every child that
opened a persistent connection will
have its own open persistent
connection to the server. For example,
if you had 20 different child
processes that ran a script that made
a persistent connection to your SQL
server, you'd have 20 different
connections to the SQL server, one
from each child.
Note, however, that this can have some
drawbacks if you are using a database
with connection limits that are
exceeded by persistent child
connections. If your database has a
limit of 16 simultaneous connections,
and in the course of a busy server
session, 17 child threads attempt to
connect, one will not be able to. If
there are bugs in your scripts which
do not allow the connections to shut
down (such as infinite loops), the
database with only 16 connections may
be rapidly swamped. Check your
database documentation for information
on handling abandoned or idle
connections.
MYSQL_CONNECT()
1.mysql_connect can be used to close the connection.Every time it is opening and closing the database connection, depending on the request .
2.Here database is opened everytime when the page is loaded in MYSQL connect
3.When the page is loaded, the database is loaded everytime
4.It is used to close the connection
Example:
<?php $conn = mysql_connect(‘host’, ‘mysql_user’, ‘mysql_password’); if(!$conn){ die(‘Could not connect: ‘ . mysql_error()); } echo ‘Connected successfully’; mysql_close($conn); ?>
Description:
host: Specifies a host name or an IP address like localhost.
mysql_user: Specifies the MySQL username
mysql_password: Specifies the MySQL password
MYSQL_PCONNECT()
1.We use the mysql_pconncet(), it initially tries to find an open persistent connection.
2.The mysql_pconncet() opens persistant connection
3.The mysql_pconnect() does not support the close connection
4.mysql_pconnect() cannot close the connection. Here open a persistant connection to the database
5.Here database need not be connected everytime.
6.The database need not be connected every time in mysql_pconncet().
more details:http://prittytimes.com/difference-between-mysql_connect-and-mysql_pconnect/