Colleagues!
I'm running php 5.3 (5.3.8) with memcache (2.2.6) client library (http://pecl.php.net/package/memcache) to deal with memcached server.
My goal is to have failover solution for sessions engine, namely:
Only native php sessions support (no custom handlers)
Few memcached servers in the pool
What I expect is that in case if one of memcached servers is down, php will attempt to utilize the second server in the pool [will successfully connect it and become happy], however when first memcached server in the pool is down I'm receiving the following error:
Session start failed. Original message: session_start(): Server 10.0.10.111 (tcp 11211) failed with: Connection refused (111)
while relevant php settings are:
session.save_handler memcache
session.save_path tcp://10.0.10.111:11211?persistent=1&weight=1&timeout=1&retry_interval=10, tcp://10.0.10.110:11211?persistent=1&weight=1&timeout=1&retry_interval=10
and memcache settings (while I think that it's near to standard) are:
Directive Local Value
memcache.allow_failover 1
memcache.chunk_size 8192
memcache.default_port 11211
memcache.default_timeout_ms 1000
memcache.hash_function crc32
memcache.hash_strategy standard
memcache.max_failover_attempts 20
Memcached still running on the second server and perfectly accessible from the WEB server:
telnet 10.0.10.110 11211
Trying 10.0.10.110...
Connected to 10.0.10.110 (10.0.10.110).
Escape character is '^]'.
get aaa
END
quit
Connection closed by foreign host.
So in other words, instead of querying all of the listed servers sequentially it crashes after unsuccessful attempt to connect the first server in the queue. Finally I do realize that there are releases of 3.0.x client library available, however it does not look too reliable for me as it still in beta version.
Please advice how can I get desired behavior with standard PHP, client lib and server.
Thanks a lot!
Best,
Eugene
Use the Memcached extension. Note that there are two memcache plugins for PHP. One is called Memcache, the other is called Memcached. Yes, that's confusing, but true anyway.
The Memcache plugin supports those complex URL's you're using, with the protocol identifier (tcp) and the parameters (persistency and so on), while the Memcached plugin supports connection pools.
The documentation you're mentioning in the comments above (http://www.php.net/manual/en/memcached.sessions.php) is about the Memcached extension, not about Memcache.
Update: Some interesting read: https://serverfault.com/questions/164350/can-a-pool-of-memcache-daemons-be-used-to-share-sessions-more-efficiently
I would like to thank everybody who participated this question, the answer is the following: in reality memcache (not memcached) as session handler supports comma separated servers as the session.save_path, moreover it supports failover. The error mentioned above Session start failed. Original message: session_start(): Server 10.0.10.111 (tcp 11211) failed with: Connection refused (111) had only 8th (Notice) level. In fact engine just informs you about the fact that one of the servers is unavailable (which is logical, as otherwise how will you know?) and then successfully connects to the second server and using it.
So all of the misunderstanding has been caused by weak documentation, memcache/memcached confusions and paranoid (E_ALL) settings of my custom error handler. In the meantime the issue has been resolved by ignoring notices referring to error Connection refused (111) in the session establishing context
You must change the hash strategy
Change your config to
memcache.hash_strategy consistent
When you make the hash strategy to consistent memcache copies the data across multiple servers. If one of the servers is down, it retries to copy it on the next request.
Related
We have experienced a problem with AWS Aurora failover and looking for pointers as to how to resolve.
Scenario
AWS Aurora set up with two end points:
Writer:
host: stackName-dbcluster-ID.cluster-ID.us-west-2.rds.amazonaws.com
resolves to IP: 10.1.0.X
Reader:
host: stackName-dbcluster-ID.cluster-ro-ID.us-west-2.rds.amazonaws.com
resolves to IP: 10.1.0.Y
So therefore our PDO MySQL Connection string is stackName-dbcluster-ID.cluster-ID.us-west-2.rds.amazonaws.com (for writing)
After failover
On failover, the DNS entries are flipped to point as follows:
Reader:
host: stackName-dbcluster-ID.cluster-ro-ID.us-west-2.rds.amazonaws.com
resolves to IP: 10.1.0.X
Writer:
host: stackName-dbcluster-ID.cluster-ID.us-west-2.rds.amazonaws.com
resolves to IP: 10.1.0.Y
Critically, the PDO Connection string (for writing) remains the same "stackName-dbcluster-ID.cluster-ID.us-west-2.rds.amazonaws.com" b ut points to a different IP address.
What Happened
We had error 1290 "SQLSTATE[HY000]: General error: 1290 The MySQL server is running with the --read-only option so it cannot execute this statement".
As the DB engines are stopped started, our initial persistent connections will have "gone away" and been invalidated (something we immediately handle in a reconnect/retry code).
However the error above means new connections will have been made to the old node, but then not further invalidated with propagation of the DNS change. They lasted 10/15 minutes (well beyond TTL of the DNS).
My Questions
Does anyone know if a persistent connection on PDO is retrieved based on the connection string, or is more reliable using the IP or other signature? Evidence suggests it's hostname, but would like confirmation.
Does anyone know a way to mark a persistent connection as "invalid" in PDO, so that is it not used again?
Or, is there something I missed?
Side notes
We already have code in place to handle the retry, and they retry is told to get a new non-persistent connection (which works). It's at this point we could "invalidate" the PDO connection so the next run of a script does not repeat this cycle over and over.
The failover can happen at any time, so we're not in a position to do manual actions such as restart php (as we had to do this time).
Without persistent connections, performance is notably slower.
FastCGI, Centos 16, PHP 7.2, MySQLD 5.0.12-dev (which is normal on Centos - see https://superuser.com/questions/1433346/php-shows-outdated-mysqlnd-version)
Persistent connections must be terminated and restarted.
Reminds me of a 2-minute TTL that took 20 minutes to be recognized. I don't know whether Amazon does a better job, or even if they have any say in DNS.
5.0.12?? That was released in 2005! Maybe a typo. Anyway, I don't think the version matters in this Question.
DNS may not be the optimal way to failover; there are several Proxy servers out there. I would expect them to flip within seconds. However, they need to know who's who rather than depending on DNS.
Can you modify the code to disconnect+reconnect when that error occurs? (It may not help.)
Unfortunately, this error is documented:
https://github.com/jeremydaly/serverless-mysql/issues/7
everything said revolves around migrating to: mysqlnd driver for mysqlnd_ms
I will continue looking for a more efficient solution.
I have classical server configuration schema with Nginx + PHP-FPM. Most of pages on my site contain data that saved into redis. Thus it is too many indirect (through php-fpm) lite requests to redis from many independent users. I use phpredis PHP extension to communicate with redis from PHP code. Can I use phpredis pconnect() method to decrease number of TCP connections between my backend servers and redis server? Should I expect no mush up of different users data within shared connections?
PHP version is 5.3.x
phpredis version is 2.2.4
Using pconnect will definitely enable using existing connection and help decreasing the number of tcp connections thus enabling a better performance
We have a PHP application (hosted on Linux) which uses Zend Framework components to query a Microsoft SQL Server 2008 database. The PHP application is hosted in a datacenter with reliable internet connection, but the SQL Server database is at the far end of a VPN connection that drops out regularly.
The issue we have is that VPN drop outs occasionally occur while queries against the SQL server are in progress. When this occurs our application can wait up to 2 hours before finally raising the following exception:
SQLSTATE[HY000]: General error: 20004 Read from the server failed [20004] (severity 9) [(null)]
What I'd like to do is set an overall query timeout and/or read timeout of around 2-3 minutes so that the application gets an exception much earlier and con recover from it without blocking for 2 hours.
We're using the pdo_dblib extension to connect to SQL Server, and I've been through the php.net docs and I can't find any timeout options either for the connection of php.ini.
Try using the PDO::ATTR_TIMEOUT attribute. I'm not sure what the default value is for pdo_dblib, it may differ between the drivers.
PHP works with the Unix library FreeTDS which interacts with MSSQL/Sybase servers.
FreeTDS is an open-source implementation of the TDS (Tabular DataStream) database client access protocol and related libraries used by Sybase and Microsoft. FreeTDS supports all versions of the protocol from both vendors, and includes DB-Lib, CT-Lib, and ODBC libraries.
Settings for a timeout must be set therefore in the freetds.conf file.
Please see http://freetds.schemamania.org/userguide/freetdsconf.htm
My company php web site is connected to an Advantage Database Server where are stored all necessary data such as users, passwords and customer registry.
Lately we started to get an error requesting web pages:
Warning: SQL error: [Extended Systems][Advantage SQL][ASA] Error 6303: Maximum Advantage Database Server connections exceeded. axServerConnect, SQL state HY000 in SQLDriverConnect in C:\...\www\... on line...
It's becoming critical day by day and it can happen once a week or twice a day without an apparent reason.
When website crashes, database service still working great with other applications connected and the only way to restore web service is to restart apache web server.
On database server we've got ads.ini configuration file in C:\Windows folder where we raised max connections setting with "MAX_CONNECTIONS=1000" which is really big compared to our needs.
Can it be useful if we set also "RETRY_ADS_CONNECTS = 1" ??
I found this post where R&D confirms a bug in may 2009:
Is this a bug with Advantage Database?
Has this been fixed? In wich release?
Where can i see the real number of connections open by apache on db?
Each php page closes ads connection on footer, what can cause connections to exceed??
Thanks in advance for help.
-
ENVIRONMENT INFO
Database:
Advantage Database Server 10.10.0.6 on Windows 2003 server
Web server:
Apache/2.0.59 (Win32) mod_ssl/2.0.59 OpenSSL/0.9.8d PHP/4.4.7 on Windows XP pro
On phpinfo() page we get "Advantage Version" "8.00.0.0".
Why this? Do we need to upgrade php-advantage extension?
Lots of questions, but I will try and address each.
1) 6303 Error. Using MAX_CONNECTIONS is the correct way to resolve this.
Make sure MAX_CONNECTIONS is in the [SETTINGS] section
Check if Apache / PHP / ADS driver is using the correct ads.ini file. You can use Process Monitor from Sysinternals to see what ads.ini file was opened successfully. If you upgrade your PHP driver you can set an environment variable adsini_path to point to the directory where the ads.ini lives.
2) Setting RETRY_ADS_CONNECTS=1 will be helpful. This also goes under the [SETTINGS] section of the ads.ini. When an ADS client receives a networking error (generally a 6000 class error) then the error is cached by the client driver and subsequent attempts to connect will use the cached error vs. retrying. Setting RETRY_ADS_CONNECTS will tell the ADS client to ignore the cached error and retry the connection
3) Bug: Looks like this was fixed in 9.10.0.9 version of the driver based on the release notes on http://devzone.advantagedatabase.com.
Fixed an issue where the garbage collection reference count on a
connection would be incorrect if multiple SQL statements were opened
on it.
Since you are running a 10.1 server you may look at updating to a 10.1 client which will also contain the fix. 10.1 Advantage PHP Driver
4)See the real number of connection
I would recommend using the stored procedure sp_mgGetConnectedUsers, you can use ARC (Advantage Data Architect) but it can be difficult to group, order, etc.
Since you are using 10.1 you can include the results of the stored procedure in a query such as
SELECT COUNT(*) FROM (EXECUTE PROCEDURE sp_mgGetConnectedUsers()) u WHERE ADDRESS='xxx.xxx.xxx.xxx'
You could also use other fields to identify the PHP application such as UserName (the server name), DictionaryUser (assuming the php application uses a unique user), ApplicationID
5) PHPINFO shows Advantage Version of the Advantage PHP client driver. You may want to upgrade the client driver for reasons mentioned above. Should be as simple as swapping DLL files (ace32.dll, axces32.dll, adsodbc.dll and php_advantage.dll), but I would recommend testing first to ensure you get everything.
I'm using php::memcache module to connect a local memcached server (#127.0.0.1), but I don't know that which one I should use, memcache::connect() or memcache::pconnect ? Does memcache::pconnect will consume many resource of the server?
Thank you very much for your answer!
Memcached uses a TCP connection (handshake is 3 extra packets, closing is usually 4 packets) and doesn't require any authentication. Therefore, the only upside to using a persistent connection is that you don't need to send those extra 7 packets and don't have to worry about having a leftover TIME-WAIT port for a few seconds.
Sadly, the downside of sacrificing those resources is far greater than the minor upsides. So I recommend not using persistent connections in memcached.
pconnect stands for persistant connection. This means that the client (in your case the script) will constantly have a connection open to your server which might not be a resouces problem - more a lack of connections available.
You should probably be wanting the standard connect unless you know you need to use persistant connections.
As far as I know, the same rules that govern persistent vs. regular connections when connecting to MySQL apply to memcached as well. The upshot is, you probably shouldn't use persistent connections in either case.
"Consumes" TCP port.
In application I'm developing I use pconnect as it uses connection pool and from the view of hardware - one server keeps one connection to memcache. I don't know exactly how it works but I think memcached is smart enough to track IP of memcached client machine.
I've played with memcached for a long time and found that using memcache::getStatus shows that connections count doesn't increased when using pconnect.
You can use debug page which show memcached stats and try to tweak pconnect or connect and see what's going on.
One downside is that PHP gets no blatant error or warning if one or all of the persistently-connected memcached daemons vanish(es). That's a pretty darn big downside.