Unable to establish SSL connection on first attempt only - php

Question

I have an ecommerce website that has been running for several months with no code changes (and for several years with only minimal changes to the card processing path). I now have a problem where when first opening a connection to the credit card processor secure server, the connection fails. On a second (or third, or fourth, etc.) attempt the connection succeeds. After some length of time--perhaps 5 minutes--the initial connection will fail again and subsequent connections will succeed.
Sample code that comes from the credit card processor's PHP API file:
$url = 'https://esplus.moneris.com:443/gateway_us/servlet/MpgRequestArray';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt ($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$dataToSend);
curl_setopt($ch,CURLOPT_TIMEOUT,$gArray[CLIENT_TIMEOUT]);
curl_setopt($ch,CURLOPT_USERAGENT,$gArray[API_VERSION]);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, TRUE);
$response=curl_exec ($ch);
if(!$response) {
print curl_error($ch);
print "\n";
print curl_errno($ch);
print "\n";
} else {
print "Success\n";
}
Output:
% php tester_curl.php
error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
35
% php tester_curl.php
Success
% php tester_curl.php
Success
% php tester_curl.php
Success
There are some similar questions, but I haven't been able to resolve the problem and I do not see any with the same error message and symptom of subsequent connection attempts succeeding after an initial fail, e.g.:
Unable to establish SSL connection, how do I fix my SSL cert?
curl errno 35 (Unknown SSL protocol error in connection to [secure site]:443) (same error message)
How to fix cURL SSL connection timeout that only happens the first time the script is called (different error msg, but SSL connections fails first attempt, subsequently succeeds)

Answer 1

The server is kind of broken. It does support TLS1.2 and TLS1.0, but not TLS1.1 (replies with TLS1.0 which is ok). This is usually not a problem unless you have client code which tries to enforce specific protocols by excluding others.
The behavior you describe looks like a client which downgrades the connection on failed connection, keeps this downgrade cached for a while but retries with the originally failed version again after some time. To trace the problem down:
check if the problem is also with other servers
check if other clients have the problem with the same server
check the underlying implementation. Curl can use GnuTLS, NSS, OpenSSL and maybe more. From the error message it looks like OpenSSL, but which version?
check for any middlebox (firewall, load balancer...) in the path to the server which might cause problems
do a packet capture and post it here in a form usable with wireshark (e.g. cloudshark)
For more information on how to debug this kind of problems and which additional information would be useful check http://noxxi.de/howto/ssl-debugging.html#aid_external_debugging

Answer 2

I had the exact thing happen with a client using an old VirtualMerchant gateway. It started failing at 5:00 PM on a Monday and magically started working again at 10 AM the next day.
Whether on the command line via openssl, or curl, or through curl in PHP the connection would fail the first time and then if you ran the same command a second latter it would work.
I tried forcing IPv4 (instead of IPv6), setting timeouts, forcing different protocols, downgrading openssl, etc, and none of it work.
The assumption is that this was something DNS and/or server related on the gateway side because nothing we did fixed it and it fixed itself.
We were running an older openssl that only supported up to TLS 1.1, but it was working and then it started working again, so it wasn't only our client. Though, the age of our client must have been part of the issue because other newer clients didn't experience the "first attempt failure" during the same window of time.
Long story short, if this happens it's probably not YOU (besides you have an older OpenSSL) and the other gateway/server you're calling likely will possibly need to fix/tweak something for it to start working again.
Keep in mind, openssl is part of the Linux core packages, so you can't simply upgrade openssl without serious risk of messing up your server. You'll have to upgrade to a newer version of the operating system to get a more modern openssl.