SoapClient Error in PHP on EC2 AWS server - php

I am tearing my hair out with this problem.
So I have a PHP app which I am moving from a "personal" server to an AWS-EC2.
I have everything installed on the AWS-EC2 server. Apache2, PHP 7.3, MySql etc.
I am testing the app before making the jump, and when testing SoapCliet call I get the following PHP error:
WSDL SOAP-ERROR: Parsing WSDL: Couldn t load from "https://celcer.sri.gob.ec/comprobantes-electronicos-ws/RecepcionComprobantesOffline?wsdl"
I look into the error trying to debug the problem and try a wget call and I get this:
$ sudo wget https://celcer.sri.gob.ec/comprobantes-electronicos-ws/RecepcionComprobantesOffline?wsdl
--2019-04-24 14:29:58-- https://celcer.sri.gob.ec/comprobantes-electronicos-ws/RecepcionComprobantesOffline?wsdl
Resolving celcer.sri.gob.ec (celcer.sri.gob.ec)... 186.42.213.26
Connecting to celcer.sri.gob.ec (celcer.sri.gob.ec)|186.42.213.26|:443... connected.
Unable to establish SSL connection.
Then I try a cURL call:
$ sudo curl -v https://celcer.sri.gob.ec/comprobantes-electronicos-ws/RecepcionComprobantesOffline?wsdl
Trying 186.42.213.26...
TCP_NODELAY set
Connected to celcer.sri.gob.ec (186.42.213.26) port 443 (#0)
ALPN, offering h2
ALPN, offering http/1.1
successfully set certificate verify locations:
CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
(304) (OUT), TLS handshake, Client hello (1):
OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to celcer.sri.gob.ec:443
stopped the pause stream!
Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to celcer.sri.gob.ec:443
Then I try openSSL:
$ sudo openssl s_client -connect celcer.sri.gob.ec:443
CONNECTED(00000003)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 319 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
Versions of stuff:
Ubuntu Linux 18.04.2
PHP 7.3
OpenSSL 1.1.1b 26 Feb 2019
I have been searching for an answer for days with no solution.
Obviously, all these commands work perfectly on my old server (not EC2) although Ubuntu's version is 16.04.1 and OpenSSL is 1.0.2g. I also installed OpenSSL 1.1.1b (to check if it was a version issue) for windows on my pc with a successful connection to the mentioned site.
I have tried with no success or change whatsoever:
-Disabling IPv6
-Using additional options like -servername or a specific protocol
-Specifying the server IP on hosts file
-Restarting
-Updating
-Updating CA certificates
-Fiddling with the firewall (both servers firewall are equally configured)
I must say that this error does not happen in all sites, in fact, I have only found this problem on this particular site celcer.sri.gob.ec which is a local government site and it's useless and futile to even think that they change anything (even if it's badly configured) also this IS the one an only site I need to soap with.
After several hours of reading and probing, I am now desperate. The only difference I can find is that this new server is an AWS EC2 server and some middle layer amazon firewall is causing this problem, but I have no idea how to even debug such a claim. My security group of EC2 just has open port 80 and port 443 for the web server and SSH for a specific IP.
Please Help
Edit 1:
I did try this before, same results:
$ openssl s_client -connect celcer.sri.gob.ec:443 -servername celcer.sri.gob.ec
CONNECTED(00000003)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 319 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
And yes it does work with several other https sites, for example:
s$ openssl s_client -connect stackoverflow.com:443
CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
verify return:1
depth=0 C = US, ST = NY, L = New York, O = "Stack Exchange, Inc.", CN = *.stackexchange.com
verify return:1
---
Certificate chain
0 s:C = US, ST = NY, L = New York, O = "Stack Exchange, Inc.", CN = *.stackexchange.com
i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
1 s:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA
---
Server certificate, etc, etc....
Edit 2:
So as I continue to fiddle around this and since with every passing minute I am convinced this has AWS Issue written all over the place I went to my AWS account created a security group allowing ALL inbound and outbound traffic and launched a fresh new Ubuntu instance, as soon as I logged in I went and try openssl, first with any secure site and then with the mentioned site and SAME RESPONSE on both cases, one successful one errno=104.
Then launched another fresh new instance this time with Amazon Linux on it, first thing after login and boom SAME RESPONSE on both cases, one successful one errno=104.
This is interesting cause I now can almost be certain it is not a configuration of my server, but a block on some firewall between amazon and celcer.sri.gob.ec. The question is how could I find this out, networking is the lesser of my IT skills.
Please help.

Although we may never find out what was causing the problem, I followed the suggestion of the AWS forum member, who found that the solution was to move to another AWS region. Somehow, somewhere the government server I needed to communicate with was breaking the communication to the specific original region I chose to launch my instance.
I chose another region, and problem solved.

Related

cURL randomly throws "curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL" after some failed requests

I have a PHP script that quickly sends a bunch of requests to the Apple API (APNs). Sometimes 10k requests are sent totally fine (just for the record, it takes ~30sec). However, when the API returns some non-200 codes, establishing new connections to this API throws the following error:
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to api.push.apple.com:443
With the debug mode enabled I get strange results:
When everything is fine:
* Found bundle for host api.push.apple.com: 0x55eba8b11660 [serially]
* Found bundle for host api.push.apple.com: 0x55eba8b11660 [serially]
* Trying 17.188.140.151...
* TCP_NODELAY set
* Hostname 'api.push.apple.com' was found in DNS cache
* Trying 17.188.140.151...
...
...
(full output is here).
After the issue happened:
* Found bundle for host api.push.apple.com: 0x561d83bb1f00 [serially]
* Server doesn't support multiplex (yet)
* Connection #0 is still name resolving, can't reuse
* Found bundle for host api.push.apple.com: 0x561d83bb1f00 [serially]
* Server doesn't support multiplex (yet)
* Connection #0 is still name resolving, can't reuse
* Connection #1 is still name resolving, can't reuse
* Trying 17.188.156.30:443...
* TCP_NODELAY set
* Hostname 'api.push.apple.com' was found in DNS cache
* Trying 17.188.156.30:443...
...
...
(full output is here).
So the IP is changed and something about multiplexing.
After it happens, sending requests to this API using the cli curl also stops working:
$ curl -v -I https://api.push.apple.com
* Trying 17.188.156.30:443...
* TCP_NODELAY set
* Connected to api.push.apple.com (17.188.156.30) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to api.push.apple.com:443
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to api.push.apple.com:443
(full output is here).
The error may disappear after a few minutes, may not. The code runs in a docker container (php:7.3-cli-alpine), restarting the container usually resets the issue until some future requests get non-200 codes. Performing restarts isn't an option.
Presumably, curl somehow stores opened connections and tries to re-use them, but for some reason something is broken inside the curl and it doesn't allow curl to correctly re-use the connections.
While curl stops working, openssl works fine:
$ openssl s_client -connect api.push.apple.com:443
CONNECTED(00000003)
write:errno=0
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 320 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
Software versions:
$ curl --version
curl 7.67.0 (x86_64-alpine-linux-musl) libcurl/7.67.0 OpenSSL/1.1.1d zlib/1.2.11 nghttp2/1.40.0
Release-Date: 2019-11-06
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS HTTP2 HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL TLS-SRP UnixSockets
$ openssl version
OpenSSL 1.1.1d 10 Sep 2019
Looks like a curl issue, but how to fix it?
Any help will be appreciated
Just in case it's helpful to you or someone else, the "one error breaks things for a long time" nature of this (and the re-use of Curl connections) reminded me a bit of the Curl bug:
https://github.com/curl/curl/issues/3966
Not sure whether the fix for that is in your Curl version though - from the date it might well be.

curl server certificate verification failed

I have a bunch of PHP scripts that use curl to communicate with various services. Currently, one of those services' SSL certificate got updated and my curl started crying about it when I try to get it from my server's CLI:
~$ curl https://example.com
curl: (60) server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle" of Certificate Authority (CA) public keys (CA certs). If the default bundle file isn't adequate, you can specify an alternate file using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in the bundle, the certificate verification probably failed due to a problem with the certificate (it might be expired, or the name might not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use the -k (or --insecure) option.
Currently, I hardcoded verify => false to all of my requests in order to keep my scripts operating but that's not something I would like to have laying around.
I got the latest cacert file from mozilla, put it in /etc/ssl/certs/ca-certificates.crt and then ran sudo update-ca-certificates which ran successfully (I suppose..)
~$ sudo update-ca-certificates
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
But then again curl is not too happy about it, still can't get my resource without passing the -k flag.
You can use the openssl s_client command to further debug the issue, in order to find out what exactly seems to be the problem with the certificate.
openssl s_client -showcerts -servername myservice.com -connect myservice.com:443

Handshake failure although the root certificate is installed (PayPal upgrades - g5 certificate - openssl)

I have (and cannot upgrade for several reasons) several virtual machines running Centos 5.11
As a check to verify the machines are compatible with new PayPal updates described here:
https://www.paypal-knowledge.com/infocenter/index?page=content&widgetview=true&id=FAQ1766&viewlocale=en_US
I ran this in a shell:
grep -C 5 --color=always "VeriSign Class 3 Public Primary Certification Authority - G5" /etc/pki/tls/certs/ca-bundle.crt
Everything looked fine, this is the output:
Data:
Version: 3 (0x2)
Serial Number:
18:da:d1:9e:26:7d:e8:bb:4a:21:58:cd:cc:6b:3b:4a
Signature Algorithm: sha1WithRSAEncryption
Issuer: C=US, O=VeriSign, Inc., OU=VeriSign Trust Network, OU=(c) 2006 VeriSign, Inc. - For authorized use only, CN=VeriSign Class 3 Public Primary Certification Authority - G5
Validity
Not Before: Nov 8 00:00:00 2006 GMT
Not After : Jul 16 23:59:59 2036 GMT
Subject: C=US, O=VeriSign, Inc., OU=VeriSign Trust Network, OU=(c) 2006 VeriSign, Inc. - For authorized use only, CN=VeriSign Class 3 Public Primary Certification Authority - G5
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:af:24:08:08:29:7a:35:9e:60:0c:aa:e7:4b:3b:
To me, this means that the new required G5 root certificate is actually in there.
However, when testing against the sandbox which should already be using the new specs with this command:
openssl s_client -connect api-3t.sandbox.paypal.com:443 -showcerts
The response is:
CONNECTED(00000003)
2052:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:586:
Of course, no issues when doing the same on the non-sandbox link.
I am 101% stuck here. The cert is there, but I still get a failure?
Installed Openssl (and latest avail through official repos) is openssl-0.9.8e-40.el5_11.
I saw several other questions about the handshake issue, but none of them seem to address an issue like this (certificate in place but still issues connecting).
Any idea about why this is happening?
EDIT:
by also trying to folrce ssl3 in the call this way:
openssl s_client -ssl3 -connect api-3t.sandbox.paypal.com:443 -showcerts
I get this instead:
CONNECTED(00000003)
6064:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:s3_pkt.c:1092:SSL alert number 40
6064:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:536:
openssl-0.9.8e-40.el5_11.
Paypal requires you to support TLS 1.2. TLS 1.2 is only supported since OpenSSL version 1.0.1 which was released 4 years ago. And since your old version of OpenSSL does not support TLS 1.2 but at most TLS 1.0 you get the handshake error. This is unrelated to the certificates.

Unable to connect to test.salesforce.com with SSL

I'm having some problems connecting to a SOAP Service at https://test.salesforce.com. I use the Toolkit-for-PHP v20.0 (https://github.com/developerforce/Force.com-Toolkit-for-PHP) which is based on PHP's native SoapClient.
Software:
MacOS 10.8
Macports 2.1.2
PHP 5.3.15
OpenSSL 1.0.1_c
The only error message I receive after 30 seconds (timeout?) is:
[SoapFault]
Could not connect to host
Strangely, connecting to http://test.salesforce.com (without SSL) or connecting to https://login.salesforce.com (with SSL) works as expected.
I even managed to log into https://test.salesforce.com using soapUI.
So my guess is there has to be some certification/handshake problem but i can't figure out how to get a more detailed error message or how to change anything about the toolkit setup.
I searched google, stackoverflow and the SalesForce discussion boards but nobody seems to have this specific sandbox+SSL problem.
Does anyone have a clue how to debug this problem?
OK, i think it's an issue with macports' openssl binary. Apparently the handshake fails because my client is attempting a SSLv2/SSLv3 handshake which the server does not understand.
openssl s_client -connect test.salesforce.com:443 -state
CONNECTED(00000003)
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
[...end...]
Same command with forced SSLv3:
openssl s_client -ssl3 -connect test.salesforce.com:443 -state
CONNECTED(00000003)
SSL_connect:before/connect initialization
SSL_connect:SSLv3 write client hello A
SSL_connect:SSLv3 read server hello A
depth=1 O = VeriSign Trust Network, OU = "VeriSign, Inc.", OU = VeriSign International Server CA - Class 3, OU = www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
verify error:num=20:unable to get local issuer certificate
verify return:0
SSL_connect:SSLv3 read server certificate A
SSL_connect:SSLv3 read server done A
SSL_connect:SSLv3 write client key exchange A
SSL_connect:SSLv3 write change cipher spec A
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
SSL_connect:SSLv3 read finished A
[...and so forth...]
Not really sure what to make out of this... the SalesForce toolkit-for-php uses PHP's native SoapClient and I don't know how to force it to use SSLv3.
This is a known bug of the latest versions of macports' port of openssl 1.0.1:
http://trac.macports.org/ticket/33715
Possible solution: install an older openssl version, in this case openssl 1.0.0h:
cd /opt/local/src
sudo svn checkout -r 90715 http://svn.macports.org/repository/macports/trunk/dports/devel/openssl
cd openssl
sudo port install
Taken from:
https://trac.macports.org/wiki/howto/InstallingOlderPort
http://trac.macports.org/ticket/33715#comment:30

How do I solve ldap_start_tls() "Unable to start TLS: Connect error" in PHP?

I'm getting:
Warning: ldap_start_tls()
[function.ldap-start-tls]: Unable to
start TLS: Connect error in
/var/www/X.php on line Y
/etc/ldap/ldap.conf:
TLS_CACERT /etc/ssl/certs/ca.crt
ca.crt is the CA which signed the LDAP server certificate. The certificate on the LDAP server is expired and I can't change it.
You can ignore the validity in windows by issuing
putenv('LDAPTLS_REQCERT=never');
in your php code. In *nix you need to edit your /etc/ldap.conf to contain
TLS_REQCERT never
Another thing to be aware of is that it requires version 3 (version 2 is php default):
//$hostnameSSL example would be "ldaps://just.example.com:636" , just make sure it has ldaps://
$con = ldap_connect($hostnameSSL);
ldap_set_option($con, LDAP_OPT_PROTOCOL_VERSION, 3);
To get a better idea of what's going on, you can enable debug logging by:
ldap_set_option(NULL, LDAP_OPT_DEBUG_LEVEL, 7);
This can be done before the ldap_connect takes place.
The specific scenario presented in the question--with an expired certificate that can't be changed--does appear to require disabling certificate validation on the LDAP client.
However, I suspect a lot of people, like me, reach this page for other root causes of receiving opaque LDAP TLS errors, where disabling validation of TLS certificates is not an appropriate answer.
In my case--using the LDAP Authentication extension for Mediawiki on an Ubuntu 18.04 LTS server, and authenticating against Active Directory on a Windows Server 2012 server--authentication stopped working in January/February 2020. The server certificate and the CA certificate were still both valid, and openssl s_client -verify 2 -connect <AD server>:636 from the Mediawiki server passed just fine.
Eventually I noticed that the signature algorithm in the SSL certificate served by AD/LDAP was SHA1, which I remembered recently suffered from the first known chosen-prefix collision exploit. This led me to investigate the changelog for packages that had recently been updated on the system, which turned up "Mark SHA1 as insecure for certificate signing" in the gnutls28 changelog circa January 8th, 2020. (The chain of dependencies from the php-ldap package in Ubuntu 18.04 goes to php7.2-ldap -> libldap-2.4-2 -> libgnutls30, whose source package is gnutls28.)
I followed some instructions to update the Windows CA to use SHA256 and then selectively followed instructions to renew the AD/LDAP cert, installed the new CA cert on my Mediawiki server, and the problem was solved! Briefly, these steps included:
In an Admin PowerShell on the AD server, run certutil -setreg ca\csp\CNGHashAlgorithm SHA256
In the Certification Authority MMC, right click on the CA -> All Tasks -> Renew CA Certificate
In a blank MMC, add snap-in for Certificates; select Local Computer
Under Personal -> Certificates, find the current entry used by LDAPS (Kerberos Authentication template type) -> All Tasks -> Advanced Options -> Renew This Certificate with the Same Key
In the same window, open the new CA certificate -> Details -> Copy to file -> no private key -> base64-encoded X.509
Copy the resulting file to /usr/share/ca-certificates/ on the Mediawiki server, then run sudo dpkg-reconfigure ca-certificates and select the new CA cert for inclusion.
P.S. For SEO purposes, depending on the mode I was using, error messages included:
ldap_start_tls(): Unable to start TLS: Connect error in /var/www/mediawiki/extensions/LdapAuthentication/LdapAuthenticationPlugin.php in the HTTP error log
ldap_start_tls(): Unable to start TLS: Can't contact LDAP server in [...]
Failed to start TLS. in the Mediawiki debug log (when using wgLDAPEncryptionType = ssl, i.e. encrypted LDAP port, 636)
Failed to bind as CN=foobar,CN=Users,DC=myOrgName,DC=local in the Mediwiki debug log (when using wgLDAPEncryptionType = tls, i.e. STARTTLS on the unencrypted LDAP port, 389)
My solution/workaround is to use
/etc/ldap/ldap.conf:
#TLS_CACERT /etc/ssl/certs/ca.crt
TLS_REQCERT never
If you have any better idea, please post another answer.
The path for ldap.conf in Windows is fixed:
c:\openldap\sysconf\ldap.conf
A restart of the web server may be required to apply changes.
In debian based systems:
Install the package: ldap-utils and in the file
/etc/ldap/ldap.conf, edit the line:
TLS_CACERT /etc/ldap/cacerts/cacert.asc
Create the directory /etc/ldap/cacerts and copy the cacert to
/etc/ldap/cacerts/cacert.asc
Restart apache.
In redhat based systems:
Install the package: openldap-clients and in the file
/etc/openldap/ldap.conf edit the line:
TLS_CACERT /etc/openldap/cacerts/cacert.asc
Create the directory /etc/openldap/cacerts and copy the cacert to
/etc/openldap/cacerts/cacert.asc
Restart httpd
I was able to get this working properly with openldap on Amazon Linux (Elastic Beanstalk PHP 7.0) with MacOS Server 5 LDAP, with TLS set to demand.
in /etc/openldap/ldap.conf:
TLS_REQCERT demand
TLS_CACERT /etc/openldap/certs/yourcacert.pem
(note that if you are not using openldap, the path will be /etc/ldap/certs/yourcacert.pem). This setup did not work until I placed the certificate inside the certs folder; it did not work from any other path.
The certificate to be placed in that path is NOT the TLS certificate of the server. It is the CA (Certificate Authority) certificate of the authority whom issued the server/domain specific TLS certificate. Only the CA certificate placed in that path will allow TLS to work before attempting an LDAP bind in php. Get the CA certificate from your server or download it from the authority's site, they are freely available.
To test if LDAP bind is even working without TLS, set TLS_REQCERT never temporarily (may need to comment # out TLS_CACERT). If you get "Can't connect to LDAP" it is not a TLS error; it simply cannot connect to the server and you likely need to open port 389 (not 636 for TLS).
Remember to restart your Apache server every time you make a change to the config file or certificate.
Some additional help for others, the certificate solution here solved my ldapsearch command line issue, but still PHP complained **Can't contact LDAP server**
Turned out to be SELinux on RHEL7 ( CentOS7 ) blocks HTTPD from using LDAP ports 389 and 636 by default, you can unblock with:
setsebool -P httpd_can_network_connect 1
Check your SELinux audit log file for things being blocked.

Categories