Im trying to optimize some of my PHP code. I found out that most time of my PHP script is spent during the connection to my mysql database at the beginning of the script.
I only connect to the database once at the beginning and close the database connection at the end of the script.
But for each user requesting this page a new connection has to be established.
Is there a way to hold a reference to the database and share it for all requests?
Yes, you can achieve that.
If you're using the MySQLi extension (the old one without the i at the end is outdated!), you can create a persistent connection by passing p: as a prefix to the hostname when creating the instance:
See mysqli::__construct.
If you're really using the old MySQL extension, there's mysql_pconnect for persistent connections.
Alternatively, if you're using PDO, then you can use setAttribute() to use PDO::ATTR_PERSISTENT.
Documentation on using persistent connection in PDO.
You can make your connection persistent (for PDO use PDO::ATTR_PERSISTENT => true), but my recomandation is to find out why your script is spending a lot of time connecting to mysql and make some improvments there.
Consider this:
Use MySQL server IP instead of hostname, to eliminate time needed to resolve dns.
Disable mysql autocompletition: (comment skip-auto-rehash from my.cnf)
If you make a lot of queries which will return the same thing over and over again, consider using a caching system to cache for query results.
Post your code on codereview.stackexchange.com to benefit from others expertise on further improving your code.
To further debug the problem, connect from console to MySQL server and check the time needed to open connection, change database, select, etc.
Related
I'm using Yii with postgres database in support library of PDPPDO.
I am facing issue in PDO library to conenct Postgres Database.
I'm using some background process to work so ultimately it will be 24X7 continues task. Now, As per use case it's possible that my Postgres will go OFF/Crash somehow. That time, pg_prepare() function returning me FALSE.. That's tottaly Corrent !!!
Now, after few minutes, if Postgres get start. So ultimately pg_prepare() should give me resource Id. But it's still giving me FALSE !!
I believe that due to restarting postgres server it's not allowing me to use same resource again. In that case I need infinite loop that willy continuously ping over Postgres for new connection.
If need to open connection again then I think following like would help,
// THOUGH IT'S CHECKING FOR $this->_pdo
// I need to make it NULL before process...
Yii:app()->db->open();
Now, Is that right flow? or do you guys have any other idea for this?
Thanks ...
After doing plenty of research, found that there isn't any solution for automatically inform Yii that connection is started again. So to maintain this, I am keeping one flag internally to check connection.
In Yii, to reconnect any database you need to delete previous connection using,
Yii::app()->db->setActive(false);
It will remove all your previous connection and sources and allow you to make connection again using,
Yii::app()->db->setActive(true);
If still not able to find connection then we can delay our background job for few minutes. Benefit, you don't need to restart your background workers.
Regards
I have no idea how PDO's connection pooling works, and I'm getting nowhere with searches.
First, for clarity, is PDO's connection pooling automatic?
If not, how can I utilize it on an AJAX page?
I've clocked the time cost on my local wampserver dev of making a database connection via mysqli at 10ms per, and I'm trying to drive wait times as close to 0 as possible. This is one of my last issues.
If you are talking about persistent connections look here: http://php.net/manual/en/pdo.connections.php
Do persistent connections remain open across different AJAX requests?
Yes and the risk that they are never reused. Thanks to #carefulnow. I'll quote his comments here:
Persistent connections and connection pooling are not the same thing
Connection pooling creates a lot of connections to start with, and then the application just grabs one of those already established connections. Persistent connections do establish new connections, but they do not close unless explicitly told to (a normal connection will close at the end of the script). Persistent connections are dangerous if not used carefully, as most databases implement a maximum connections count, and will refuse further connections when hit. For most cases, a normal connection (just calling the PDO constructor with a DSN and login details) will be fine.
I'll explain it by showing the PHP code, and the resulting connection count in the MariaDB console monitor. Look at the following screenshots, and pay attention the the "run count" in the bottom right. You'll see when using persistent connections, connections do not close, but instead go into sleep mode. Ignore connection #231, that's the console monitor's. http://i.imgur.com/IL42tjF.png, http://i.imgur.com/aDvl7F7.png, http://i.imgur.com/IuFEEvO.png. As you can see, the time column indicates the length of inactivity, not the length of time connected, hence the console monitor's time is always zero, because I've just executed SHOW PROCESSLIST;. I hope this clears everything up a bit, database connection management is quite tricky sometimes.
So looking at the screenshots the questions is open, how to pickup these persistent connections again. Some answers on StackOverflow say, that they are cached on thread level, so if you have more than one thread (which is highly expected) than you get many connections. This opens the risk to run into the connection limit of mysql. Before using persistant connections make sure you understand how they work (which I don't, but it seems #carefulnow :-))
Is it possible to set a persistent connection to expire?
PHP doc says they are cached, therefore I assume they are cleanup after a time.
More on the php doc
also on the same link given by #fabian you can see the following:
<?php
$dbh = new PDO('mysql:host=localhost;dbname=test', $user, $pass, array(
PDO::ATTR_PERSISTENT => true
));
?>
Note: If you wish to use persistent connections, you must set PDO::ATTR_PERSISTENT in the array of driver options passed to the PDO constructor. If setting this attribute with PDO::setAttribute() after instantiation of the object, the driver will not use persistent connections.
Pdo persistent uses the any URL with multiple mysql request as one connection. If the page reloaded or access any other php page with pdo connection makes only the new pdo connection. We need the static SQL connection already created in the server and reuse it for the pages. Like global declaration before the page load. It only possible with node mysql package with express. In express js we can configure the mysql pool connection as global declaration.
You can use the connection pooling in nodejs with mysql and express npm modules . And setup a http server in node, then make the request to express server via API like http://127.0.0.1:1000/getdata in a post request. Then use the php as the getter using the file() method with stream context for creating the post request. Don't use curl because it takes more time for the execution. Use while statement for check the data return from the express js server
This post I created via mobile, so I can't give the explained code for the use. If anyone need the example code. I'll post to the GitHub repo. Share your comments
Is it possible to cache database connections when using PHP like you would in a J2EE container? If so, how?
There is no connection pooling in php.
mysql_pconnect and connection pooling are two different things.
There are many problems connected with mysql_pconnect and first you should read the manual and carefully use it, but this is not connection pooling.
Connection pooling is a technique where the application server manages the connections. When the application needs a connection it asks the application server for it and the application server returns one of the pooled connections if there is one free.
We can do connection scaling in php for that please go through following link: http://www.oracle.com/technetwork/articles/dsl/white-php-part1-355135.html
So no connection pooling in php.
As Julio said apache releases all resources when the request ends for the current reques. You can use mysql_pconnect but you are limited with that function and you must be very careful. Other choice is to use singleton pattern, but none of this is pooling.
This is a good article: https://blogs.oracle.com/opal/highly-scalable-connection-pooling-in-php
Also read this one http://www.apache2.es/2.2.2/mod/mod_dbd.html
Persistent connections are nothing like connection pooling. A persistent connection in php will only be reused if you make multiple db connects within the same request/script execution context. In most typical web dev scenarios you'll max out your connections way faster if you use mysql_pconnect because your script will have no way to get a reference to any open connections on your next request. The best way to use db connections in php is to make a singleton instance of a db object so that the connection is reused within the context of your script execution. This still incurs at least 1 db connect per request, but it's better than making multiple db connects per reqeust.
There is no real db connection pooling in php due to the nature of php. Php is not an application server that can sit there in between requests and manage references to a pool of open connections, at least not without some kind of major hack. I think in theory you could write an app server in php and run it as a commandline script that would just sit there in the background and keep a bunch of db connections open and pass references to them to your other scripts, but I don't know if that would be possible in practice, how you'd pass the references from your commandline script to other scripts, and I sort of doubt it would perform well even if you could pull it off. Anyway that's mostly speculation. I did just notice the link someone else posted to an apache module to allow connection pooling for prefork servers such as php. Looks interesting:
https://github.com/junamai2000/mod_namy_pool#readme
I suppose you're using mod_php, right?
When a PHP file finishes executing all it's state is killed so there's no way (in PHP code) to do connection pooling. Instead you have to rely on extensions.
You can mysql_pconnect so that your connections won't get closed after the page finishes, that way they get reused in the next request.
This might be all that you need but this isn't the same as connection pooling as there's no way to specify the number of connections to maintain opened.
You can use MySQLi.
For more info, scroll down to Connection pooling section # http://www.php.net/manual/en/mysqli.quickstart.connections.php#example-1622
Note that Connection pooling is also dependent on your server (i.e. Apache httpd) and its configuration.
If an unused persistent connection for a given combination of "host, username, password, socket, port and default database can not be found" in the open connection pool, then only mysqli opens a new connection otherwise it would reuse already open available persistent connections, which is in a way similar to the concept of connection pooling. The use of persistent connections can be enabled and disabled using the PHP directive mysqli.allow_persistent. The total number of connections opened by a script can be limited with mysqli.max_links (this may be interesting to you to address max_user_connections issue hitting hosting server's limit). The maximum number of persistent connections per PHP process can be restricted with mysqli.max_persistent.
In wider programming context, it's a task of web/app server however in this context, it's being handled by mysqli directive of PHP itself in a way supporting connection re-usability. You may also implement a singleton class to get a static instance of connection to reuse just like in Java. Just want to remind that java also doesn't support connection pooling as part of its standard JDBC, they're being different module/layers on top of JDBC drivers.
Coming to PHP, the good thing is that for the common databases in the PHP echosystem it does support Persistent Database Connections which persists the connection for 500 requests (config of max_requests in php.ini) and this avoids creating a new connection in each request. So check it out in docs in detail, it solves most of your challenges. Please note that PHP is not so much sophisticated in terms of extensive multi-threading mechanism and concurrent processing together with powerful asynchronous event handling, when compared to strictly object oriented Java. So in a way it is very less effective for PHP to have such in-built mechanism like pooling.
You cannot instantiate connection pools manually.
But you can use the "built in" connection pooling with the mysql_pconnect function.
I would like to suggest PDO::ATTR_PERSISTENT
Persistent connections are links that do not close when the execution of your script ends. When a persistent connection is requested, PHP checks if there's already an identical persistent connection (that remained open from earlier) - and if it exists, it uses it. If it does not exist, it creates the link.
Connection pooling works at MySQL server side like this.
If persistence connection is enabled into MySQL server config then MySQL keep a connection open and in sleep state after requested client (php script) finises its work and die.
When a 2nd request comes with same credential data (Same User Name, Same Password, Same Connection Parameter, Same Database name, Maybe from same IP, I am not sure about the IP) Then MySQL pool the previous connection from sleep state to active state and let the client use the connection. This helps MySQL to save time for initial resource for connection and reduce the total number of connection.
So the connection pooling option is actually available at MySQL server side. At PHP code end there is no option. mysql_pconnect() is just a wrapper that inform PHP to not send connection close request signal at the end of script run.
For features such as connection pooling - you need to install swoole extension first: https://openswoole.com/
It adds async features to php.
After that its trivial to add mysql and redis connection pooling:
https://github.com/open-smf/connection-pool
Some PHP frameworks come with pooling built-in: https://hyperf.wiki/2.2/#/en/pool
I read specifications of mysql_pconnect of the function of PHP..
"mysql_pconnect is that connection with the SQL server is not closed even if the practice of the script is finished".
There is it unless connection is closed, but will it be that connection stays until a timing of the reboot of mysql?
I think, and I may be wrong, but...
I think the pconnect is a persistant connection held not for the running of the script, but for the duration of the PHP session / MySQL session, that there is a socket connection maintained by PHP regardless of the which script is being run. A bit like having multiple documents open in word instead of multiple instances of notepad running. by using a common database link, the processing power for creating individual links is not needed.
However, after reading, I think that it only seems to be of benefit if PHP is being run as an Apache module, not in CGI mode.
Call me out on my misinformation.
mysql_pconnect allows Apache's mod_php module to do connection pooling. You can still call mysql_close, but mod_php will not actually close it; it will just invalidate your resource handle. Unfortunately, the module doesn't have any configuration for this, so pooled connections are reaped by the MySQL server via its wait_timeout parameter. The default value of this is quite high so if you want to take advantage of it, you will probably want to lower that variable.
Connection pooling saves two things: connection setup and MySQL thread creation. Connection setup is very fast with MySQL compared to other databases, but a highly in-demand web site could still see a benefit from reducing this step. The cost of thread creation in MySQL is more dependant on the underlying OS but it can still be a win for a busy site.
Both aspects need to be looked at in the bigger picture of website speed and the load it presents on your database. It is possible to run out of connection threads on the database with a busy enough site using connection pooling. There is also the aspect that your application needs to do its best to leave the connection in a consistent state, as you can no longer rely on closing the connection to do things like unlock tables and rollback transactions.
There is more information in the PHP documentation.
yes, but you can not close it using mysql___close.
It will close when you reboot mysql server, or when the connection stays idle (not used) for a specific amount of time, defined in the configuration variable wait_timeout.
This question already has answers here:
mysqli persistent connection
(3 answers)
Closed 18 days ago.
I am planning to use MYSQL. Is there a connection pooling extension available? Or what is the normal practice for connection?
Is this the one used in every where...
mysqli_connect("localhost", "xxx", "xxx", "test");
Do people use just normal mysql_connect or pconnect..? How better is pconnect and what setting should I do for PConnect?
have you ever used mysql_pconnect() ?
mysql_pconnect() acts very much like mysql_connect() with two major differences.
First, when connecting, the function would first try to find a (persistent) link that's already open with the same host, username and password. If one is found, an identifier for it will be returned instead of opening a new connection.
Second, the connection to the SQL server will not be closed when the execution of the script ends. Instead, the link will remain open for future use (mysql_close() will not close links established by mysql_pconnect()).
This type of link is therefore called 'persistent'
Check it here
Persistent connection support was
introduced in PHP 5.3 for the mysqli
extension. Support was already present
in PDO MYSQL and ext/mysql. The idea
behind persistent connections is that
a connection between a client process
and a database can be reused by a
client process, rather than being
created and destroyed multiple times.
This reduces the overhead of creating
fresh connections every time one is
required, as unused connections are
cached and ready to be reused.
Unlike the mysql extension, mysqli
does not provide a separate function
for opening persistent connections. To
open a persistent connection you must
prepend p: to the hostname when
connecting.
source: http://www.php.net/manual/en/mysqli.persistconns.php
sample code:
$GLOBALS["mysqli"] = new mysqli('p:localhost', 'username', 'password', 'db_name');
edit: Sorry for the dupe, didn't see the other answers.
use the mysqli or the PDO extension instead of the old mysql extension.
you can tell the mysqli_connect or mysqli::__construct to use persistent connection, if you prefix your hostname with 'p:'
http://php.net/manual/en/mysqli.construct.php
As of PHP 5.3, mysqli supports persistent connections, you just need to prepend p: to the front of the host name.
If you are running Apache, have you tried looking into connection pooling with mysql through the Apache mod_dbd module? It supports connection pooling for MySQL. http://httpd.apache.org/docs/2.2/mod/mod_dbd.html
There are 3 connection functions:
mysql_connect: normal connection, no pooling, you cannot execute stored procedures (just sql)
mysql_pconnect: pooled connection, you cannot execute stored procedures (just sql)
mysqli_connect: normal connection, no pooling, you can execute stored procedures (needs mysql 5 or higher)
mysqli_pconnect: DOES NOT EXIST. There is no built in connection function both handling stored procedures and pooling
My advice (via experience and surfing):
If you need stored procedures, omit pooling and use mysqli_connect
If you do not need stored procedures, use mysql_pconnect
Connection pooling is absolutely possible in PHP - however you should use swoole or openswoole extension that brings along a lot of async/parallelization features:
https://openswoole.com/docs/get-started/installation
After that you can use normal mysqli or pdo php extensions to set up connection pools.
See these examples:
https://github.com/open-smf/connection-pool
https://openswoole.com/docs/modules/connection-pool
Some frameworks already handle that for you:
https://hyperf.wiki/2.2/#/en/pool
edit: i use smf connection pool with mezzio micro-framework and openswoole and its rock solid for mysql and redis.