Does the phpdriver for mongoDB provide a feature for starting the connection on demand - only.
Maybe this can save some load, although persistent connection is already really fast.
However, if there is a chance to optimize the application, i'd try this, too.
The alternative, calling the function to connect only on demand seems to be a bit tricky and makes my source very redundant.
The link H Hatfield posted addresses this question. Mongo::__construct() has a connect option that defaults to true, but you can specify false to delay the connection. The documentation isn't clear how lazy this is, as it only guarantees that the constructor will return without waiting for a connection to be established.
That said, you would be better off relying on persistent connections, which are being actively improved. I wouldn't be surprised if the lazy connection option was removed in a future major version of the driver.
On the documentation page when initiating the Manager it says:
this constructor performs no I/O. Connections will be initialized on demand, when the first operation is executed. So this is actually what you need already.
Related
I have a question I can't seem to find the answer to.
We are building a gRPC microservice in Go, to serve our main application written in PHP. I am running some tests on one of the functions now, to see the performance.
My results indicate that setting up the connection takes about 2 seconds, but after that, each call takes less than a microsecond.
How does it work in a real-life application? Does it open one shared connection that is kept alive for a while, or does each request to our application have to open its own connection to the service?
If each request has to open its own connection, is it possible to get around this to get rid of the overhead that comes with establishing a new connection?
What you need is called connection pool.
I found this PR https://github.com/grpc/grpc/issues/15426 after a short search on Google, but I am not sure if it is actually for pooling connections.
If you get able to create connections polls, you may just pick up one available connection and use for your request.
[EDITED]
I just found something that may work for your needs:
https://github.com/swoole/swoole-src#the-simplest-example-of-a-connection-pool
If you follow the example of the Redis connection pool you will be able to make a gRPC connection pool.
The latest php-grpc extension will keep your connections around and re-use them between requests.
Those are called "persistent" connections. A previous connection can be re-used if the set of parameters given to the connection constructor are exactly the same.
Source: php-grpc documentation: https://grpc.github.io/grpc/php/class_grpc_1_1_channel.html
By default, the underlying grpc_channel is "persistent". That is, given the same set of parameters passed to the constructor, the same underlying grpc_channel will be returned.
I just ran a test creating 1000 non-persistent connections to mongodb via nginx/php fastcgi which took about 2.1 seconds on my dev machine. I then tried the same test using persistent connections, same result. I think I read somewhere that persistence in the php driver is now always enabled anyway. Next, I tried storing the connections to APC which resulted in a 7-9ms response time after the first request. Now I'm wondering a few things here:
There's almost never a time I can think of where I'd want to create more than one connection in my app at once and with a persistent connection from what I understand, new connections are created as needed by the mongo driver.
Creating a single connection seems to take about the same time as pulling the stored connection object from APC. Will caching the connection object ever really provide a benefit?
Caching the connection I know of course would still require some sort of check to see if it's even still a valid connection.. in performing this check each time, I wonder if it would negate the performance gain (if any) from pulling it from cache.
I can't seem to really find any material really covering any of this so I'm assuming it's because I'm confused in my understanding. Have any of you experimented with this?
Thanks!
First, as far as i know, APC is serializing data while storing it. so it would not make any sense to store any connection in APC.
Then, persistend connections will be reused by the php process for various requests. So a non persistend connection will be reestablished for each request the php process will receive.
I'm implementing an application that will have a lot of clients querying lots of small data packages from my webserver. Now I'm unsure whether to use persistent data connections to the database or not. The database is currently on the same system as the webserver and could connect via the socket, but this may change in the near future.
As I know a few releases of PHP ago mysqli_pconnect was removed because it behaved suboptimally. In the meantime it seems to be back again.
Based on my scenario I suppose I won't have an other chance to handle thousands of queries per minute but with loads of persistent connections and a MySQL configuration that reserves only little resources per connection, right?
Thanks for your input!
What happened when you tested it?
With the nest will in the world, there's no practical way you can convey all the informaton required for people to provide a definitive answer in a SO response. However usually there is very little overhead in establishing a mysql connection, particularly if it resides on the same system as the database client (in this case the webserver). There's even less overhead if you use a filesystem rather than a network socket.
So I'd suggest abstracting all your database calls so you can easily switch between connection types, but write your system to use on-demand connections, and ensure you code explicitly releases the connection as soon as practical - then see how it behaves.
C.
Are PHP persistant connections evil?
The problem is there can be only so
many connections active between Host
“Apache” and Host “MySQL”
Persistant connections usually give problems in that you hit the maximum number of connections. Also, in your case it does not give a great benefit since your database server is on the same host. Keep it to normal connections for now.
As they say, your mileage may vary, but I've never had good experiences using persistent connections from PHP, including MySQL and Oracle (both ODBC and OCI8). Every time I've tested it, the system fails to reuse connections. With high load, I end up hitting the top limit while I have hundreds of idle connections.
So my advice is that you actually try it and find out whether your set-up is reusing connections properly. If it isn't working as expected, it won't be a big lose anyway: opening a MySQL connection is not particularly costly compared to other DBMS.
Also, don't forget to reset all relevant settings when appropriate (whatever session value you change, it'll be waiting for you next time to stablish a connection and happen to reuse that one).
When querying the db is it plausible to feel extremely paranoid? I go as far as opening and closing mysql connection every time a new query has to be done. I am afraid that (especially with the ajax enabled pages) this would cause great performance downgrading.
Should I continue in this method or at least open and close connections once in everypage (instead of per-query)? (i'm writing in php btw)
thank you.
Yes, the overhead of connecting every time will be considerable. I suggest you just close it when you're done, it's very unlikely that simply having an open connection without running queries on it will open you to vulnerabilities.
I'd recommend a connection pool if it's possible with PHP. It's a way to simultaneously maximize performance and minimize connection time.
You should not close MySQL connections immediately. It's better to use a single connection for the entire PHP script. PHP will automatically close the connection if you don't explicitly.
Opening a new connection incurs some small time penalty, particularly if MySQL lives on another server on the network. New TCP connections require a three-way handshake, and each TCP connection consumes kernel resources for at least two minutes.
Although PHP doesn't support full-fledged connection pooling, the procedural MySQL API does support persistent connections. See mysql_pconnect() for more details. At my office we use pconnect to avoid crashing the TCP stacks on our high-traffic PHP site.
Consider PHP Data Objects, which can keep a persistent connection for you. You really only connect with credentials once and then cache the connection. Create the connection with something along these lines:
$dbh = new PDO('mysql:host=HOSTNAME;dbname=DBNAME',
'USER', 'PASSWORD',
array(PDO::ATTR_PERSISTENT => true));
For more information, see this page in the php manual.
Can't speak for PHP, but when I write this kind of thing in Apache/Perl I generally do two things to boost performance:
a) allow MySQL handles to stay open as long as the Apache daemon does.
b) keep cached statement handles around (using an LRU cache).
On production servers, some of those MySQL handles have been around for days, serving many thousands of quite complicated SQL queries. It isn't a problem.
In our current use of memcached, we are running into problems in a high volume server because so much time is used setting up and tearing down connections to our memcache server. Would using persistent connections to memcached help alleviate this problem?
Also, what is the preferred way to connect and use persistent memcahced connections? I was thinking of setting a "pool_size" variable then randomly choosing from 1-$POOL_SIZE and using that connection
$mem = new Memcached(rand(1, $pool_size));
Either I am looking in the wrong place or there is not a lot of information on this out there.
Both pecl/memcache and pecl/memcached support persistent connections per process. However, the bug does exist in pecl/memcached at this time.
The php client doesn't handle persistent connections. you either need to use your pooling idea, or use a 3rd party memcached client for php that supports persistent connections.
like this one:
http://github.com/andreiz/php-memcached/tree/master
I have read that persistent connections feature is broken in the "memcached" PHP extension.
First: the "persistent" connection is not destroyed. (This is ok.)
Second: when you try to reuse it, it creates a new one! (This is bad!)
Result: memory leaks, increasingly consuming all available RAM.
Check here: http://brian.moonspot.net/php-memcached-issues
As I said, I haven't experienced this myself - I just read this information in the linked article.