I'm using php::memcache module to connect a local memcached server (#127.0.0.1), but I don't know that which one I should use, memcache::connect() or memcache::pconnect ? Does memcache::pconnect will consume many resource of the server?
Thank you very much for your answer!
Memcached uses a TCP connection (handshake is 3 extra packets, closing is usually 4 packets) and doesn't require any authentication. Therefore, the only upside to using a persistent connection is that you don't need to send those extra 7 packets and don't have to worry about having a leftover TIME-WAIT port for a few seconds.
Sadly, the downside of sacrificing those resources is far greater than the minor upsides. So I recommend not using persistent connections in memcached.
pconnect stands for persistant connection. This means that the client (in your case the script) will constantly have a connection open to your server which might not be a resouces problem - more a lack of connections available.
You should probably be wanting the standard connect unless you know you need to use persistant connections.
As far as I know, the same rules that govern persistent vs. regular connections when connecting to MySQL apply to memcached as well. The upshot is, you probably shouldn't use persistent connections in either case.
"Consumes" TCP port.
In application I'm developing I use pconnect as it uses connection pool and from the view of hardware - one server keeps one connection to memcache. I don't know exactly how it works but I think memcached is smart enough to track IP of memcached client machine.
I've played with memcached for a long time and found that using memcache::getStatus shows that connections count doesn't increased when using pconnect.
You can use debug page which show memcached stats and try to tweak pconnect or connect and see what's going on.
One downside is that PHP gets no blatant error or warning if one or all of the persistently-connected memcached daemons vanish(es). That's a pretty darn big downside.
Related
I've noticed that while reasonably fast, the connection to the database (google's cloud SQL mysql-compatible one) is a large part of the request. I'm using PDO for the abstraction.
So, since that's the case the obvious solution is to enable PHP's PDO persistent connections.
To my understanding, and I've verified this in PHP's source code (links bellow), the way these work is as follows:
when you connect with the persistent flag on, PHP caches the connection using a hash of the connection string, username and password for the key
when you try to re-connect in another request it checks if a persistent connection exists then checks the liveness of the connection (which is driver specific; mysql version is what is executed in my case) and kills the cached version it if it fails the test
if the cached version was killed a new connection is created and returned; otherwise you get to skip the overhead of creating the connection (around 30x faster creation process based on xdebug profiles executed directly in devel versions on the cloud)
Everything sounds good so far? Not sure how all of this works when say you have a cached connections and two requests hit it (stress testing it doesn't appear to cause issues), but otherwise sounds okey and in testing works fine.
Well, here's what happens in the real world after some time passes...
Once a connection "dies" PDO will stall the entire request for 60s or more. This happens I believe after maybe 1h or more; so for a short while everything will work just fine and PDO will connect super fast to Cloud SQL. I've tried several ways to at least mitigate the stalling being more then 1s but to no result (ini_set socket timeout wont affect it, expires flag on PDO is ignored I believe, exception and status checks for the "has gone away" are useless since it stalls on making the connection, etc). I assume most likely the connection "expires" but reasons are unknown to me. I assume Cloud SQL drops it since its not in "show processlist;", but it's possible I'm not looking at it correctly.
Is there any secret sauce that makes PDO persistent connections work with Cloud SQL for more then a brief time?
Are persistent connections to Cloud SQL not supported?
You haven't described where your application is running (e.g. Compute Engine, App Engine, etc), so I will make an educated guess based on the described symptoms.
It's likely your TCP keepalive time is set too high on the application host. You can change the settings via these instructions.
Assuming a Linux host, the following command will show your current setting, in seconds:
cat /proc/sys/net/ipv4/tcp_keepalive_time
TCP connections without any activity for a period of time may be dropped anywhere along the path between the application host and Cloud SQL, depending on how everything in the communication path is configured.
TCP keepalive sends a periodic "ping" on idle connections to work around this problem/feature.
Cloud SQL supports long-running connections.
The target is simple： clients post http requests to query data and update record by some keys。 Highest request: 500/sec (the higher the better, but the best is to fulfil this requirement while making the system easy to achieve and using less mashines)
what I've done: nginx + php-cgi(using php) to serve http request, the php use thrift RPC to retrieve data from a DB proxy which is only used to query and update DB(mysql). The DB proxy uses mysql connection pool and thrift's TNonblockingServer. (In my country, there are 2 ISP, DB Proxy will be deployed in multi-isp machine and so is the db, web servers can be deployed on single-isp mashine according to the experience)
what trouble me: when I do stress test(when >500/sec), I found " TSocket: Could not connect to 172.19.122.32:9090 (Connection refused [111]" from php log. I think it may be caused by the port's running out(may be incorrect conclusion). So I design to use thrift connection bool to reduce thrift connection. But there is no connection pool in php (there seem to be some DB connection pool tech) and php does not support the feature.
So I think maybe the project is designed in the wrong way from the beginning(like use php ,thrift). Is there a good way to solve this based on what i've done? And I think most people will doubt my awkward scheme. Well, your new scheme will help a lot
thanks.
"TSocket: Could not connect to 172.19.122.32:9090 (Connection refused [111])" from php log shows the ports running out because of too many short connections in a short time. So I config the tcp TIME_WAIT status to recycle port in time using:
sysctl -w net.ipv4.tcp_timestamps=1
sysctl -w net.ipv4.tcp_tw_recycle=1
it works!
what droubles me is sloved, but to change the kernal parameter will affect the NAT. It's not a perfect solution. I think a new good design of this system can be continue to discuss.
The situation is: I have one Debian Server running LAMP with one Virtual Host with one Website. My MySQL has only one user from that website.
In this case would I benefit from using a persistent connection?
The PHP documentation seems to advise against persistent connections in any case.
Thanks
Edit: Yes, the MySQL server is on the same machine.
There's a discussion here http://groups.google.com/group/comp.databases.mysql/browse_thread/thread/4ae68befe1b488e7/e843f0b9e59ad710?#e843f0b9e59ad710 :
"No, it is not (better). Contrary, using mysql_pconnect() is considered harmful, as it tends to hog the MySQL server with idle connections."
If you connect via 'localhost', the connection will automatically be established via the MySQL socket, which is really cheap anyways.
(Groups link taken from MySQL Persistent Connections)
While you can get some performance benefits from using a persistent connection, but if the mysql server is on the same machine and you're not experiencing problems then it is probably not worth it. It is too easy to accidentally leave connections open, and the actual performance benefit is only going to be noticeable at high volumes.
I read specifications of mysql_pconnect of the function of PHP..
"mysql_pconnect is that connection with the SQL server is not closed even if the practice of the script is finished".
There is it unless connection is closed, but will it be that connection stays until a timing of the reboot of mysql?
I think, and I may be wrong, but...
I think the pconnect is a persistant connection held not for the running of the script, but for the duration of the PHP session / MySQL session, that there is a socket connection maintained by PHP regardless of the which script is being run. A bit like having multiple documents open in word instead of multiple instances of notepad running. by using a common database link, the processing power for creating individual links is not needed.
However, after reading, I think that it only seems to be of benefit if PHP is being run as an Apache module, not in CGI mode.
Call me out on my misinformation.
mysql_pconnect allows Apache's mod_php module to do connection pooling. You can still call mysql_close, but mod_php will not actually close it; it will just invalidate your resource handle. Unfortunately, the module doesn't have any configuration for this, so pooled connections are reaped by the MySQL server via its wait_timeout parameter. The default value of this is quite high so if you want to take advantage of it, you will probably want to lower that variable.
Connection pooling saves two things: connection setup and MySQL thread creation. Connection setup is very fast with MySQL compared to other databases, but a highly in-demand web site could still see a benefit from reducing this step. The cost of thread creation in MySQL is more dependant on the underlying OS but it can still be a win for a busy site.
Both aspects need to be looked at in the bigger picture of website speed and the load it presents on your database. It is possible to run out of connection threads on the database with a busy enough site using connection pooling. There is also the aspect that your application needs to do its best to leave the connection in a consistent state, as you can no longer rely on closing the connection to do things like unlock tables and rollback transactions.
There is more information in the PHP documentation.
yes, but you can not close it using mysql___close.
It will close when you reboot mysql server, or when the connection stays idle (not used) for a specific amount of time, defined in the configuration variable wait_timeout.
Is it possible to cache database connections when using PHP like you would in a J2EE container? If so, how?
There is no connection pooling in php.
mysql_pconnect and connection pooling are two different things.
There are many problems connected with mysql_pconnect and first you should read the manual and carefully use it, but this is not connection pooling.
Connection pooling is a technique where the application server manages the connections. When the application needs a connection it asks the application server for it and the application server returns one of the pooled connections if there is one free.
We can do connection scaling in php for that please go through following link: http://www.oracle.com/technetwork/articles/dsl/white-php-part1-355135.html
So no connection pooling in php.
As Julio said apache releases all resources when the request ends for the current reques. You can use mysql_pconnect but you are limited with that function and you must be very careful. Other choice is to use singleton pattern, but none of this is pooling.
This is a good article: https://blogs.oracle.com/opal/highly-scalable-connection-pooling-in-php
Also read this one http://www.apache2.es/2.2.2/mod/mod_dbd.html
Persistent connections are nothing like connection pooling. A persistent connection in php will only be reused if you make multiple db connects within the same request/script execution context. In most typical web dev scenarios you'll max out your connections way faster if you use mysql_pconnect because your script will have no way to get a reference to any open connections on your next request. The best way to use db connections in php is to make a singleton instance of a db object so that the connection is reused within the context of your script execution. This still incurs at least 1 db connect per request, but it's better than making multiple db connects per reqeust.
There is no real db connection pooling in php due to the nature of php. Php is not an application server that can sit there in between requests and manage references to a pool of open connections, at least not without some kind of major hack. I think in theory you could write an app server in php and run it as a commandline script that would just sit there in the background and keep a bunch of db connections open and pass references to them to your other scripts, but I don't know if that would be possible in practice, how you'd pass the references from your commandline script to other scripts, and I sort of doubt it would perform well even if you could pull it off. Anyway that's mostly speculation. I did just notice the link someone else posted to an apache module to allow connection pooling for prefork servers such as php. Looks interesting:
https://github.com/junamai2000/mod_namy_pool#readme
I suppose you're using mod_php, right?
When a PHP file finishes executing all it's state is killed so there's no way (in PHP code) to do connection pooling. Instead you have to rely on extensions.
You can mysql_pconnect so that your connections won't get closed after the page finishes, that way they get reused in the next request.
This might be all that you need but this isn't the same as connection pooling as there's no way to specify the number of connections to maintain opened.
You can use MySQLi.
For more info, scroll down to Connection pooling section # http://www.php.net/manual/en/mysqli.quickstart.connections.php#example-1622
Note that Connection pooling is also dependent on your server (i.e. Apache httpd) and its configuration.
If an unused persistent connection for a given combination of "host, username, password, socket, port and default database can not be found" in the open connection pool, then only mysqli opens a new connection otherwise it would reuse already open available persistent connections, which is in a way similar to the concept of connection pooling. The use of persistent connections can be enabled and disabled using the PHP directive mysqli.allow_persistent. The total number of connections opened by a script can be limited with mysqli.max_links (this may be interesting to you to address max_user_connections issue hitting hosting server's limit). The maximum number of persistent connections per PHP process can be restricted with mysqli.max_persistent.
In wider programming context, it's a task of web/app server however in this context, it's being handled by mysqli directive of PHP itself in a way supporting connection re-usability. You may also implement a singleton class to get a static instance of connection to reuse just like in Java. Just want to remind that java also doesn't support connection pooling as part of its standard JDBC, they're being different module/layers on top of JDBC drivers.
Coming to PHP, the good thing is that for the common databases in the PHP echosystem it does support Persistent Database Connections which persists the connection for 500 requests (config of max_requests in php.ini) and this avoids creating a new connection in each request. So check it out in docs in detail, it solves most of your challenges. Please note that PHP is not so much sophisticated in terms of extensive multi-threading mechanism and concurrent processing together with powerful asynchronous event handling, when compared to strictly object oriented Java. So in a way it is very less effective for PHP to have such in-built mechanism like pooling.
You cannot instantiate connection pools manually.
But you can use the "built in" connection pooling with the mysql_pconnect function.
I would like to suggest PDO::ATTR_PERSISTENT
Persistent connections are links that do not close when the execution of your script ends. When a persistent connection is requested, PHP checks if there's already an identical persistent connection (that remained open from earlier) - and if it exists, it uses it. If it does not exist, it creates the link.
Connection pooling works at MySQL server side like this.
If persistence connection is enabled into MySQL server config then MySQL keep a connection open and in sleep state after requested client (php script) finises its work and die.
When a 2nd request comes with same credential data (Same User Name, Same Password, Same Connection Parameter, Same Database name, Maybe from same IP, I am not sure about the IP) Then MySQL pool the previous connection from sleep state to active state and let the client use the connection. This helps MySQL to save time for initial resource for connection and reduce the total number of connection.
So the connection pooling option is actually available at MySQL server side. At PHP code end there is no option. mysql_pconnect() is just a wrapper that inform PHP to not send connection close request signal at the end of script run.
For features such as connection pooling - you need to install swoole extension first: https://openswoole.com/
It adds async features to php.
After that its trivial to add mysql and redis connection pooling:
https://github.com/open-smf/connection-pool
Some PHP frameworks come with pooling built-in: https://hyperf.wiki/2.2/#/en/pool