A framework or an application automatically connect the database and we have to just use the database object for DB related operation. In CMS or framework, a term "connection pooling" is very popular. You can opt CMS or framework of PHP.
What is connection pooling?
Can someone describe this with an example?
What is the advantage of connection pooling?
Without connection pooling:
Every time you want to talk to the database, you have to open a connection, use it, then close it again.
With connection pooling:
The connections are kept open all the time (in a pool). When you want to talk to the database, you take an already connection, that isn't already in a use, use it, then put it back.
This is more efficient then opening and closing them all the time.
Connection pooling generally refers to, well, having a pool of connections which is being reused. To contrast this with non-pooled connections: typically every program instance connects to the database by itself every time it is run. In a PHP program, you just have the line $db = new PDO(...), which connects to the database. If you have 100 simultaneous visitors, 100 separate instances of that script will be run simultaneously, and 100 separate connections will be established to the database simultaneously. This may be very inefficient and/or temporarily overwhelm the database server.
A connection pool works by establishing, say, 50 permanent connections to the database which stay open the whole time. A PHP script would then simply pick one of these open connections to talk to the database and drop it back into the pool when it's done. If suddenly more than 50 PHP scripts try to use connections from this pool at once, the first 50 will succeed, and the rest will have to wait in line until an unused connection becomes available. This is more efficient, because connections aren't opened and torn down all the time, and it doesn't overwhelm the database server when sudden spikes occur.
Related
I have an issue concerning PDO persistent connection. Now this may not be an actual problem, but I can't seem to find any post addressing this behavior.
I'm using the good old PDO in a persistent connection mode for my web app. Now I'm creating a new connection via new PDO(...).
When I run this script a new connection (C#1) is getting established and a MySql process (P#1) to accommodate the persistent connection.
So, I run the script again creating a new conction (C#2) and expecting C#2 to use the P#1 from the last connection. Every time I run this script a new process appears while the last one is still alive (in sleep mode).
On my production server there are about 350 prossers (in sleep) at any given time from 3 defrent users (all users connect from the same apache server).
The question: is this situation valid?
found my answer
They cause the child process to simply connect only once for its entire lifespan, instead of every time it processes a page that requires connecting to the SQL server. This means that for every child that opened a persistent connection will have its own open persistent connection to the server. For example, if you had 20 different child processes that ran a script that made a persistent connection to your SQL server, you'd have 20 different connections to the SQL server, one from each child.
http://php.net/manual/en/features.persistent-connections.php
I've read a ton about persistent database connections between PHP and MySQL (mysql_connect vs. mysql_pconnect). Same with PDO and MySQLi. It's definitely just my lack of understanding on this one, but how can a database connection be persistent between webpages? In this code:
$conn = mysql_pconnect( $server , $user, $pass );
mysql_select_db( $dbname );
If two users load this page at the same time, with two different $dbname variables, will PHP only make one connection to the database or two? I am fairly certain that
$conn = mysql_connect( $server , $user, $pass );
would make two connections.
If pconnect reuses the connection opened by the first user, will the mysql_select_db call work for the second user?
Ideally, what I am looking for is a way to have fewer database connections but still be able to set the default database in each PHP script. I have clients who all use the same PHP scripts, but the data is stored in their own client database (hence, $dbname is always different, but the MySQL connection parameters are the same - same mysql ip address, user and password).
Hope that makes sense. We can use MySQL, MySQLi or PDO, just need to know how to accomplish this the best way without having the possibility for clients to accidently write data to someone else's database! Thanks in advance.
The persistence is done by the copy of the PHP that's embedded in the webserver. Ordinarily you'd be right- if PHP was running in CGI mode, it would be impossible to have a persistent connection, because there'd be nothing left to persist when the request is done and PHP shuts down.
However, since there's a copy of PHP embedded in the webserver, and the webserver itself keeps running between requests, it is possible to maintain a pool of persistent connections within that "permanent" PHP.
However, note that on Apache multi-worker type server models, the connection pools are maintained PER-CHILD. If you set your pool limit to 10, you'll have 10 connections per Apache child. 20 children = 200 connections.
Persistent connections will also lead to long-term problems with deadlocks and other hard-to-debug problems. Remember - there's no guarantee that a user's HTTP requests will be serviced by the SAME apache child/mysql connection. If a script dies part-way through a database transaction, that transaction will NOT be rolled back, because MySQL does not see the HTTP side of things - all it sees is that the mysql<->apache connection is still open and assumes all's well.
The next user to hit that particular apache/mysql child/connection combination will now magically end up in the middle of that transaction, with no clue that the transaction is open. Basically, it's the Web equivalent of an unflushed toilet - all the "garbage" from the previous user is still there.
With non-persistent connections, you're guaranteed to have a 'clean' environment each time you connect.
From my reading of documentation and comments, I see:
Docs on mysql_pconnect (deprecated method)
Second, the connection to the SQL server will not be closed when the execution of the script ends. Instead, the link will remain open for future use ( mysql_close() will not close links established by mysql_pconnect()).
and a comment on that page
Persistent connections work well for CGI PHP managed by fastCGI, contrary to the suggestion above that they only work for the module version. That's because fastCGI keeps PHP processes running between requests. Persistent connections in this mode are easily made immune to connection limits too, because you can set PHP_FCGI_CHILDREN << mysql's max_connections <<< Apache's MaxClients. This also saves resources.
Docs on mysqli_connect (new method)
Prepending host by p: opens a persistent connection. mysqli_change_user() is automatically called on connections opened from the connection pool.
Docs for mysqli_change_user:
Changes the user of the specified database connection and sets the current database.
So my understanding is as follows: pconnect keeps the connection open after a script ends but while a process (or maybe group of processes) is still alive (like in a server with FCGI set up). Only one script at a time uses a connection, and when a new script grabs that connection the user and database are updated.
Thus if you use FCGI and persistent connections you can reduce the number of db connections open, but scripts running simultaneously will not be sharing the same connection. There is no problem with the connection being confused as to which database is selected.
In PDO, a connection can be made persistent using the PDO::ATTR_PERSISTENT attribute. According to the php manual -
Persistent connections are not closed at the end of the script, but
are cached and re-used when another script requests a connection using
the same credentials. The persistent connection cache allows you to
avoid the overhead of establishing a new connection every time a
script needs to talk to a database, resulting in a faster web
application.
The manual also recommends not to use persistent connection while using PDO ODBC driver, because it may hamper the ODBC Connection Pooling process.
So apparently there seems to be no drawbacks of using persistent connection in PDO, except in the last case. However., I would like to know if there is any other disadvantages of using this mechanism, i.e., a situation where this mechanism results in performance degradation or something like that.
Please be sure to read this answer below, which details ways to mitigate the problems outlined here.
The same drawbacks exist using PDO as with any other PHP database interface that does persistent connections: if your script terminates unexpectedly in the middle of database operations, the next request that gets the left over connection will pick up where the dead script left off. The connection is held open at the process manager level (Apache for mod_php, the current FastCGI process if you're using FastCGI, etc), not at the PHP level, and PHP doesn't tell the parent process to let the connection die when the script terminates abnormally.
If the dead script locked tables, those tables will remain locked until the connection dies or the next script that gets the connection unlocks the tables itself.
If the dead script was in the middle of a transaction, that can block a multitude of tables until the deadlock timer kicks in, and even then, the deadlock timer can kill the newer request instead of the older request that's causing the problem.
If the dead script was in the middle of a transaction, the next script that gets that connection also gets the transaction state. It's very possible (depending on your application design) that the next script might not actually ever try to commit the existing transaction, or will commit when it should not have, or roll back when it should not have.
This is only the tip of the iceberg. It can all be mitigated to an extent by always trying to clean up after a dirty connection on every single script request, but that can be a pain depending on the database. Unless you have identified creating database connections as the one thing that is a bottleneck in your script (this means you've done code profiling using xdebug and/or xhprof), you should not consider persistent connections as a solution to anything.
Further, most modern databases (including PostgreSQL) have their own preferred ways of performing connection pooling that don't have the immediate drawbacks that plain vanilla PHP-based persistent connections do.
To clarify a point, we use persistent connections at my workplace, but not by choice. We were encountering weird connection behavior, where the initial connection from our app server to our database server was taking exactly three seconds, when it should have taken a fraction of a fraction of a second. We think it's a kernel bug. We gave up trying to troubleshoot it because it happened randomly and could not be reproduced on demand, and our outsourced IT didn't have the concrete ability to track it down.
Regardless, when the folks in the warehouse are processing a few hundred incoming parts, and each part is taking three and a half seconds instead of a half second, we had to take action before they kidnapped us all and made us help them. So, we flipped a few bits on in our home-grown ERP/CRM/CMS monstrosity and experienced all of the horrors of persistent connections first-hand. It took us weeks to track down all the subtle little problems and bizarre behavior that happened seemingly at random. It turned out that those once-a-week fatal errors that our users diligently squeezed out of our app were leaving locked tables, abandoned transactions and other unfortunate wonky states.
This sob-story has a point: It broke things that we never expected to break, all in the name of performance. The tradeoff wasn't worth it, and we're eagerly awaiting the day we can switch back to normal connections without a riot from our users.
In response to Charles' problem above,
From : http://www.php.net/manual/en/mysqli.quickstart.connections.php -
A common complain about persistent connections is that their state is
not reset before reuse. For example, open and unfinished transactions
are not automatically rolled back. But also, authorization changes
which happened in the time between putting the connection into the
pool and reusing it are not reflected. This may be seen as an unwanted
side-effect. On the contrary, the name persistent may be understood as
a promise that the state is persisted.
The mysqli extension supports both interpretations of a persistent
connection: state persisted, and state reset before reuse. The default
is reset. Before a persistent connection is reused, the mysqli
extension implicitly calls mysqli_change_user() to reset the state.
The persistent connection appears to the user as if it was just
opened. No artifacts from previous usages are visible.
The mysqli_change_user() function is an expensive operation. For
best performance, users may want to recompile the extension with the
compile flag MYSQLI_NO_CHANGE_USER_ON_PCONNECT being set.
It is left to the user to choose between safe behavior and best
performance. Both are valid optimization goals. For ease of use, the
safe behavior has been made the default at the expense of maximum
performance.
Persistent connections are a good idea only when it takes a (relatively) long time to connect to your database. Nowadays that's almost never the case. The biggest drawback to persistent connections is that it limits the number of users you can have browsing your site: if MySQL is configured to only allow 10 concurrent connections at once then when an 11th person tries to browse your site it won't work for them.
PDO does not manage the persistence. The MySQL driver does. It reuses connections when a) they are available and the host/user/password/database match. If any change then it will not reuse a connection. The best case net effect is that these connections you have will be started and stopped so often because you have different users on the site and making them persistent doesn't do any good.
The key thing to understand about persistent connections is that you should NOT use them in most web applications. They sound enticing but they are dangerous and pretty much useless.
I'm sure there are other threads on this but a persistent connection is dangerous because it persists between requests. If, for example, you lock a table during a request and then fail to unlock then that table is going to stay locked indefinitely. Persistent connections are also pretty much useless for 99% of your apps because you have no way of knowing if the same connection will be used between different requests. Each web thread will have it's own set of persistent connections and you have no way of controlling which thread will handle which requests.
The procedural mysql library of PHP, has a feature whereby subsequent calls to mysql_connect will return the same link, rather than open a different connection (As one might expect). This has nothing to do with persistent connections and is specific to the mysql library. PDO does not exhibit such behaviour
Resource Link : link
In General you could use this as a rough "ruleset"::
YES, use persistent connections, if:
There are only few applications/users accessing the database, i.e.
you will not result in 200 open (but probably idle) connections,
because there are 200 different users shared on the same host.
The database is running on another server that you are accessing over
the network
An (one) application accesses the database very often
NO, don't use persistent connections, if:
Your application only needs to access the database 100 times an hour.
You have many, many webservers accessing one database server
Using persistent connections is considerable faster, especially if you are accessing the database over a network. It doesn't make so much difference if the database is running on the same machine, but it is still a little bit faster. However - as the name says - the connection is persistent, i.e. it stays open, even if it is not used.
The problem with that is, that in "default configuration", MySQL only allows 1000 parallel "open channels". After that, new connections are refused (You can tweak this setting). So if you have - say - 20 Webservers with each 100 Clients on them, and every one of them has just one page access per hour, simple math will show you that you'll need 2000 parallel connections to the database. That won't work.
Ergo: Only use it for applications with lots of requests.
On my tests I had a connection time of over a second to my localhost, thus assuming I should use a persistent connection. Further tests showed it was a problem with 'localhost':
Test results in seconds (measured by php microtime):
hosted web: connectDB: 0.0038912296295166
localhost: connectDB: 1.0214691162109 (over one second: do not use localhost!)
127.0.0.1: connectDB: 0.00097203254699707
Interestingly: The following code is just as fast as using 127.0.0.1:
$host = gethostbyname('localhost');
// echo "<p>$host</p>";
$db = new PDO("mysql:host=$host;dbname=" . DATABASE . ';charset=utf8', $username, $password,
array(PDO::ATTR_EMULATE_PREPARES => false,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION));
Persistent connections should give a sizable performance boost. I disagree with the assement that you should "Avoid" persistence..
It sounds like the complaints above are driven by someone using MyIASM tables and hacking in their own versions of transactions by grabbing table locks.. Well of course you're going to deadlock! Use PDO's beginTransaction() and move your tables over to InnoDB..
seems to me having a persistent connection would eat up more system resources. Maybe a trivial amount, but still...
The explanation for using persistent connections is obviously reducing quantity of connects that are rather costly, despite the fact that they're considerably faster with MySQL compared to other databases.
The very first trouble with persistent connections...
If you are creating 1000's of connections per second you normally don't ensure that it stays open for very long time, but Operation System does. Based on TCP/IP protocol Ports can’t be recycled instantly and also have to invest a while in “FIN” stage waiting before they may be recycled.
The 2nd problem... using a lot of MySQL server connections.
Many people simply don't realize you are able to increase *max_connections* variable and obtain over 100 concurrent connections with MySQL others were beaten by older Linux problems of the inability to convey more than 1024 connections with MySQL.
Allows talk now about why Persistent connections were disabled in mysqli extension. Despite the fact that you can misuse persistent connections and obtain poor performance which was not the main reason. The actual reason is – you can get a lot more issues with it.
Persistent connections were put into PHP throughout occasions of MySQL 3.22/3.23 when MySQL was not so difficult which means you could recycle connections easily with no problems. In later versions quantity of problems however came about – Should you recycle connection that has uncommitted transactions you take into trouble. If you recycle connections with custom character set configurations you’re in danger again, as well as about possibly transformed per session variables.
One trouble with using persistent connections is it does not really scale that well. For those who have 5000 people connected, you'll need 5000 persistent connections. For away the requirement for persistence, you may have the ability to serve 10000 people with similar quantity of connections because they are in a position to share individuals connections when they are not with them.
I was just wondering whether a partial solution would be to have a pool of use-once connections. You could spend time creating a connection pool when the system is at low usage, up to a limit, hand them out and kill them when either they've completed or timed out. In the background you're creating new connections as they're being taken. At worst case this should only be as slow as creating the connection without the pool, assuming that establishing the link is the limiting factor?
Trying to separate out my LAMP application into two servers, one for php and one for mysql. So far the application connects locally through a file socket and works fine.
I'm worried about the number connections I can establish if it is over the network. I have been testing tcp connections on unix for benchmark purposes and I know that you cannot exceed a certain amount of connections per second otherwise it halts due to the lack of resources (be it sockets, or file handles or whatever). I also understand that php does not implement connection pooling so for each page load a new connection over the network must be made. I also looked into pconnect for php and it seems to bring more problems.
I know this is a very very common setup (php+mysql), can anyone provide some typical usage and statistics they get out of their servers? Thanks!
The problem is not related to running out of connections allowed my MySQL. The main problem is that unix cannot very quickly create and tear down tcp connections. Sockets end up in TIME_WAIT and you have to wait for a period before you free up more sockets to connect again. These two screenshots clearly shows this pattern. MySQL does work up to a certain point and then pauses because the web server ran out of sockets. After certain amount of time passed, the web server was able to make new connections.
alt text http://img35.imageshack.us/img35/3809/picture4k.png
alt text http://img35.imageshack.us/img35/4580/picture2uyw.png
I think the limit is at 65535. So you'd have to have 65535 connections at the same time to hit that limit since a regular mysql connection closes automatically.
mysql_connect()
Note: The link to the server will be closed as soon as the execution of the script ends, unless it's closed earlier by explicitly calling mysql_close().
But if you're using a persistent mysql connection, then you can run into trouble.
Using persistent connections can require a bit of tuning of your Apache and MySQL configurations to ensure that you do not exceed the number of connections allowed by MySQL.
Each MySQL connection actually uses several meg of ram for various buffers, and takes a while to set up, which is why MySQL is limited to 100 concurrent open connections by default. You can up that limit, but it's better to spend your time trying to limit concurrent connections, via various methods.
Beware of raising the connection limit too high, as you can run out of memory (which, I believe, crashes mysql), or you may push important things out of memory. e.g. MySQL's performance is highly dependent on the OS automatically caching the data it reads from disk in memory; if you set your connection limit too high, you'll be contending for memory with the cache.
If you don't up your connection limit, you'll run out of connections long before your run out of sockets/file handles/etc. If you do increase your connection limit, you'll run out of RAM long before you run out of sockets/file handles/etc.
Regarding limiting concurrent connections:
Use a connection pooling solution. You're right, there isn't one built in to PHP, but there are plenty of standalone ones out there to choose from. This saves expensive connection setup/tear down time.
Only open database connections when you absolutely need them. In my current project, we automatically open a database connection when the first query is issued, and not a moment before; we also release the connection after we've done all our database work, but before the page's HTML is actually generated. The shorter the period of time you hold connections open, the fewer connections will be open simultaneously.
Cache what you can in a lighter-weight solution like memcached. My current project temporarily caches pages displayed to anonymous users (since every anonymous user gets the same HTML, in the end -- why bother running the same database queries all over again a few scant milliseconds later?), meaning no database connection is necessary at all. This is especially useful for bursts of anonymous traffic, like a front-page digg.
Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 7 years ago.
Improve this question
I have this doubt, I've searched the web and the answers seem to be diversified. Is it better to use mysql_pconnect over mysql_connect when connecting to a database via PHP? I read that pconnect scales much better, but on the other hand, being a persistent connection... having 10 000 connections at the same time, all persistent, doesn't seem scalable to me.
Thanks in advance.
Persistent connections should be unnecessary for MySQL. In other databases (such as Oracle), making a connection is expensive and time-consuming, so if you can re-use a connection it's a big win. But those brands of database offer connection pooling, which solves the problem in a better way.
Making a connection to a MySQL database is quick compared to those other brands, so using persistent connections gives proportionally less benefit for MySQL than it would for another brand of database.
Persistent connections have a downside too. The database server allocates resources to each connection, whether the connections are needed or not. So you see a lot of wasted resources for no purpose if the connections are idle. I don't know if you'll reach 10,000 idle connections, but even a couple of hundred is costly.
Connections have state, and it would be inappropriate for a PHP request to "inherit" information from a session previously used by another PHP request. For example, temporary tables and user variables are normally cleaned up as a connection closes, but not if you use persistent connections. Likewise session-based settings like character set and collation. Also, LAST_INSERT_ID() would report the id last generated during the session -- even if that was during a prior PHP request.
For MySQL at least, the downside of persistent connections probably outweighs their benefits. And there are other, better techniques to achieve high scalability.
Update March 2014:
MySQL connection speed was always low compared to other brands of RDBMS, but it's getting even better.
See http://mysqlserverteam.com/improving-connectdisconnect-performance/
In MySQL 5.6 we started working on optimizing the code handling connects and disconnects. And this work has accelerated in MySQL 5.7. In this blog post I will first show the results we have achieved and then describe what we have done to get them.
Read the blog for more details and speed comparisons.
Basically you have to balance the cost of creating connections versus keeping connections. Even though MySQL is very fast at setting up a new connection, it still costs -- in thread setup time, and in TCP/IP setup time from your web server. This is noticeable on a high-enough traffic site. Unfortunately, PHP does not have any controls on the persistence of connections. So the answer is to lower the idle timeout in MySQL a long way (like down to 20 seconds), and to up the thread cache size. Together, this generally works remarkably well.
On the flip side, your application needs to respect the state of the connection. It is best if it makes no assumptions about what state the session is in. If you use temporary tables, then using CREATE IF NOT EXISTS and TRUNCATE TABLE helps a lot, as does naming them uniquely (such as including as userid). Transactions are bit more problematic; but your code can always do ROLLBACK at the top, just in case.
mysql_connect() and mysql_pconnect() both are working for database connection but with little difference. In mysql_pconnect(), p stands for persistance connection.
When we are using mysql_connect() function, every time it is opening and closing the database connection, depending on the request.
But in case of mysql_pconnect() function:
First, when connecting, the function would try to find a (persistent) connection that's already open with the same host, username and password. If one is found, an identifier for it will be returned instead of opening a new connection.
Second, the connection to the SQL server will not be closed when the execution of the script ends. Instead, the connection will remain open for future use (mysql_close() will not close connection established by mysql_pconnect()).
mysql_pconncet() is useful when you have a lot of traffice on your site. At that time for every request it will not open a connection but will take it from the pool. This will increase the efficiency of your site. But for general use mysql_connect() is best.
It's very unlikely that you'll reach 10000 connections. Anyhow, go to the official source. (Emphasis mine).
If persistent connections don't have
any added functionality, what are
they good for?
The answer here is extremely simple --
efficiency. Persistent connections are
good if the overhead to create a link
to your SQL server is high. Whether or
not this overhead is really high
depends on many factors. Like, what
kind of database it is, whether or
not it sits on the same computer on
which your web server sits, how
loaded the machine the SQL server sits
on is and so forth. The bottom line
is that if that connection overhead is
high, persistent connections help you
considerably. They cause the child
process to simply connect only once
for its entire lifespan, instead of
every time it processes a page that
requires connecting to the SQL server.
This means that for every child that
opened a persistent connection will
have its own open persistent
connection to the server. For example,
if you had 20 different child
processes that ran a script that made
a persistent connection to your SQL
server, you'd have 20 different
connections to the SQL server, one
from each child.
Note, however, that this can have some
drawbacks if you are using a database
with connection limits that are
exceeded by persistent child
connections. If your database has a
limit of 16 simultaneous connections,
and in the course of a busy server
session, 17 child threads attempt to
connect, one will not be able to. If
there are bugs in your scripts which
do not allow the connections to shut
down (such as infinite loops), the
database with only 16 connections may
be rapidly swamped. Check your
database documentation for information
on handling abandoned or idle
connections.
MYSQL_CONNECT()
1.mysql_connect can be used to close the connection.Every time it is opening and closing the database connection, depending on the request .
2.Here database is opened everytime when the page is loaded in MYSQL connect
3.When the page is loaded, the database is loaded everytime
4.It is used to close the connection
Example:
<?php $conn = mysql_connect(‘host’, ‘mysql_user’, ‘mysql_password’); if(!$conn){ die(‘Could not connect: ‘ . mysql_error()); } echo ‘Connected successfully’; mysql_close($conn); ?>
Description:
host: Specifies a host name or an IP address like localhost.
mysql_user: Specifies the MySQL username
mysql_password: Specifies the MySQL password
MYSQL_PCONNECT()
1.We use the mysql_pconncet(), it initially tries to find an open persistent connection.
2.The mysql_pconncet() opens persistant connection
3.The mysql_pconnect() does not support the close connection
4.mysql_pconnect() cannot close the connection. Here open a persistant connection to the database
5.Here database need not be connected everytime.
6.The database need not be connected every time in mysql_pconncet().
more details:http://prittytimes.com/difference-between-mysql_connect-and-mysql_pconnect/