So here's my problem.
I'm using curl to access my CouchDB by HTTP. I recently updated my WAMP to the WAMP 3 64bit wich comes with PHP 5.6.16 and Apache 2.4.17. Therefore, since this upgrade, I discovered that I couldn't do PUT request anymore.
Env
PHP 5.6.16
Apache 2.4.17
Windows 10 64 bit
Wamp 3 64 bit
Curl --version
curl 7.49.1 (x86_64-pc-win32) libcurl/7.49.1 OpenSSL/1.0.2h nghttp2/1.11.1
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM SSL HTTP2
Code executed
So when I execute this :
<?php
$table="testname";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://localhost:5984/' . $table);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, 'validUser:validPass');
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-type: application/json',
'Accept: */*'
));
$response = curl_exec($ch);
curl_close($ch);
echo $response;
I get a quick response from the server.
Then, I try to create a database :
<?php
$table = "testname";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://localhost:5984/' . $table);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, 'validUser:validPass');
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-type: application/json',
'Accept: */*'
));
$response = curl_exec($ch);
curl_close($ch);
echo $response;
Problem
So, when I execute this code, the request will hang on curl_exec.
What's weird is that, after the timeout, the request will be received by CouchDB but no response will be given. It seems that my "Put" request are stacked in a buffer and they are waiting to be executed.
Verbose curl output
* Hostname in DNS cache was stale, zapped
* Trying ::1...
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 5984 (#0)
* Server auth using Basic with user 'validUser'
> PUT /customers HTTP/1.1
Host: localhost:5984
Authorization: Basic dGVzdEFkbWluOnRlc3RQYXNzd29yZA==
Content-type: application/json
Accept: */*
* Operation timed out after 10000 milliseconds with 0 bytes received
* Closing connection 0
Hints
-I try to install a SSL certificate but It didn't seem to work. Having this certificate still installed can cause problems?
-I can do PUT request with a REST client on my Atom editor without problems.
-I seems like there is a problem in my network route internally. I'm saying this because It affected the PHP-Curl aswell as the Curl CLI. Also, I'm able to do GET request but the PUT request are like "hanging" for no reason and are "Accepted" by my CouchDB when the timeout occurs. It's like if I was sending long poll request.
What have been tested
Execute the same command on the command line -> Same result
Try a REST Client on my Atom editor with success
A friend of mine try to access to my database remotly with success (So CouchDB doesn't seem the problem)
Even if I tested with my Firewall disabled, uninstalling my antivirus ( Bitdefender Total Security 2016) fixed my issue.
Related
I'm using PHP curl library to establish connection and retrieve content from WEB - the usual.
I have multiple SOCKS5 proxy servers running on localhost on ports from 10300 to 10350 and PHP selects a port randomly.
My code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_PROXY, "localhost:".mt_rand(10300, 10350));
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$curl_resp = curl_exec($ch);
curl_close($ch);
/// ... whatever logic
But if I put this code into loop for long time, I get error telling that no more files can be opened. I read online that is because filehandler limit being reached, so I debugged with lsof -p and I noticed that the process has hundreds and even thousands of these lines:
... TCP localhost:43030->localhost:10303 (CLOSE_WAIT)
... TCP localhost:40982->localhost:10341 (CLOSE_WAIT)
... TCP localhost:48718->localhost:10304 (CLOSE_WAIT)
... TCP localhost:41655->localhost:10350 (CLOSE_WAIT)
... TCP localhost:41915->localhost:10310 (CLOSE_WAIT)
... TCP localhost:49746->localhost:10322 (ESTABLISHED)
I was researching why is this that way, but I don't understand, because I am calling curl_close and I would assume that CURL would close PROXY connection as well, or am I missing something?
Versions:
PHP 7.1.4-1+deb.sury.org~trusty+1 (cli) (built: Apr 11 2017 22:45:20) (NTS)
curl 7.52.1 (x86_64-pc-linux-gnu) libcurl/7.52.1 OpenSSL/1.0.1f zlib/1.2.8 libidn2/0.9 libpsl/0.11.0 (+libicu/52.1) libssh2/1.4.3 nghttp2/1.19.0 librtmp/2.3
I also came across this issue. The work around is add
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Connection: close"));
At first I thought is was related with php7, because i can reproduce it on a php7 box, but on php5, it will not happen. Unfortunately, I tried this on another php7 box, it also did not happen again. So I am still confused about the root cause.
I have a curl command that works perfectly from the command line that goes like so :
curl --cert /path/to/cert.pem \
--cert-type PEM \
--form "files[0]=#/path/to/file.csv" \
https://url.com/whatever
I can also run it successfully from php using
exec("curl --cert /path/to/cert.pem --cert-type PEM --form \"files[0]=#/path/to/file.csv\" https://url.com/whatever");
Translating the command to php-curl I got
$url = "https://url.com/whatever";
$certificate = "/path/to/cert.pem";
$filePath = "/path/to/file.csv";
$postData = array("files[0]" => "#".$filePath);
// Upload request
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_SSLCERT, $certificate);
curl_setopt($ch, CURLOPT_SSLCERTTYPE, "PEM");
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
// I have tried WITH or WITHOUT the following options to no avail
// Note that SSL_VERIFYPEER false is not necessary -- I do not have a self-signed cert in the chain
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
$postUploadResponse = curl_exec($ch);
and this does not work. The message I get from the server is that the authentication has failed and that my certificate is not valid (though the exact same certificate works from the cmd line).
An important detail : that function runs perfectly well (with SSL_VERIFYPEER false as I use a self-signed cert locally) from my local XAMPP installation but the same code fails on the server.
The server is on Debian 8.3.
curl --version is
curl 7.38.0 (x86_64-pc-linux-gnu) libcurl/7.38.0 OpenSSL/1.0.1k zlib/1.2.8 libidn/1.29 libssh2/1.4.3 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API SPNEGO NTLM NTLM_WB SSL libz TLS-SRP
What could be causing this ? Is there anything that could be causing my certificate to not be sent properly when using libcurl ?
Here's the BASH command I'm using on my remote server -
curl -i -H "Content-Type: application/json" -H "Authorization: USERNAME:PASSWORD" "https://api7.publicaster.com/Rest/Ping.svc/?format=json"
Here's the code on my PHP script which I run on a WAMP local test environment to request the same information.
header("Content-Type: application/json");
$encrypted_account_id = 'USERNAME';
$api_password = 'PASSWORD';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"http://api7.publicaster.com/Rest/Ping.svc/?format=json");
$headers = array();
$headers[] = 'Content-type: application/json';
$headers[] = 'Authorization: $encrypted_account_id:$api_password';
curl_setopt($ch, CURLOPT_HEADER, $headers);
$server_output = curl_exec($ch);
curl_close($ch);
print_r($server_output);
If on my PHP script I request the HTTP site then it returns -
HTTP/1.1 401 Unauthorized
Cache-Control: private
If on my PHP script I request the HTTPS site then it just returns blank. I do have a self-signed SSL certificate and SSL is enabled, I can access HTTPS portions of my WAMP server but I am given a warning.
I can't figure out if this was an issue with my code or my WAMP server. Any ideas?
If I add the following to my code and try to CURL the HTTPS address I at least receive a response but it's the Unauthorized CC: private error -
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
I think I found the answer, the issue is your curl request could not verify the ssl peer. Windows does not automatically have a CA certificate to do the validation against.
So here is what I did to get my https requests via curl fixed for wamp.
download the cacert.pem file here http://curl.haxx.se/docs/caextract.html and place it in your PHP folder, mine was located at:
C:\wamp\bin\php\php5.5.12
Now open the php.ini file and find the line starting with
; curl.cainfo =
un-comment that line "remove the ; in front" and add the absolute path of the cacert.pem file. Mine looked something like this:
curl.cainfo = C:\wamp\bin\php\php5.5.12\cacert.pem
Now restart wamp and voila! I can happily open https protocols via curl on wamp.
I'm attempting to access the Australia Post API to validate addresses, however I am not able to authorise the call via cURL with PHP or CLI.
The following Wget CLI call works as expected and returns the XML:
wget --user=username#domain.com.au --password=pass https://api.auspost.com.au/ValidateAddress.xml?addressLine1=31+test+street\&suburb=Adelaide\&postcode=5000\&state=SA\&country=Australia
And the CLI log from Wget:
--2014-12-16 00:33:24-- https://api.auspost.com.au/ValidateAddress.xml?addressLine1=31+test+street&suburb=Adelaide&postcode=5000&state=SA&country=Australia
Resolving api.auspost.com.au (api.auspost.com.au)... 54.66.176.138, 54.66.176.138
Connecting to api.auspost.com.au (api.auspost.com.au)|54.66.176.138|:443... connected.
HTTP request sent, awaiting response... 401 Authorization Required
Reusing existing connection to api.auspost.com.au:443.
HTTP request sent, awaiting response... 200 OK
Length: 255 [application/xml]
Saving to: ‘ValidateAddress.xml?addressLine1=31+test+street&suburb=Adelaide&postcode=5000&state=SA&country=Australia.5’
However the below cURL CLI call does not:
curl -u username#domain.com.au:pass https://api.auspost.com.au/ValidateAddress.xml?addressLine1=31+test+street\&suburb=Adelaide\&postcode=5000\&state=SA\&country=Australia -v
cURL log:
* Hostname was NOT found in DNS cache
* Trying 54.66.176.138...
* Connected to api.auspost.com.au (54.66.176.138) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
* Server certificate: api.auspost.com.au
* Server certificate: VeriSign Class 3 Extended Validation SSL SGC CA
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
* Server auth using Basic with user 'user#domain.com.au'
> GET /ValidateAddress.xml?addressLine1=3 HTTP/1.1
> Authorization: Basic auth_hash_here
> User-Agent: curl/7.37.1
> Host: api.auspost.com.au
> Accept: */*
>
< HTTP/1.1 401 Authorization Required
Passing an 'Authorization' header with the auth hash as below also results in a 401 error.
curl -H 'Authorization: Basic auth_hash_here' https://api.auspost.com.au/ValidateAddress.xml?addressLine1=31+test+street\&suburb=Adelaide\&postcode=5000\&state=SA\&country=Australia -v
Ultimately, I need to call this in PHP (ideally via Guzzle but will take cURL if I can get it working!). Below is the PHP code I have tried with no success.
$service_url = 'https://api.auspost.com.au/ValidateAddress.xml?addressLine1=31+test+street&suburb=Adelaide&postcode=5000&state=SA&country=Australia';
$curl = curl_init($service_url);
curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_setopt($curl, CURLOPT_USERPWD, "username#domain.com.au:pass");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_HEADER, true);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, TRUE);
$curl_response = curl_exec($curl);
var_dump($curl_response);
I have successfully ran the call via a Chrome REST client, and can see the correct headers being sent with the call via Chrome's network inspector.
The username and password details are correct, and have verified this numerous times.
Anyone have any suggestions as to how I could get this working in PHP?
How can I convert this php code to curl command? I want to use this code on linux machine by executing single curl command.
$headers = array(
"Content-type: text/xml",
"Content-length: " . strlen($xml)
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $this->url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 10000);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$data = curl_exec($ch);
I tired with this one, but unsuccessful:
curl -X POST -H "Content-type: text/xml" -o output.txt -d "param1=param1&username=username&password=password" https://site.url.com -d #data.xml
Maybe the problem is in the HTTPS because only TLSv1 is allowed on the site.
In php you would use:
curl_setopt($ch, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1_1);
Documentation speaks of more TLS versions:
http://www.php.net/manual/en/function.curl-setopt.php
CURL_SSLVERSION_TLSv1_0
CURL_SSLVERSION_TLSv1_1
CURL_SSLVERSION_TLSv1_2
The TLS versions only work with CURL version 7.34 or newer.
If you want to force curl to use TLSv1, you can use the --tlsv1 option, from the docs:
-1, --tlsv1
(SSL) Forces curl to use TLS version 1.x when negotiating with a remote TLS server. You can use options --tlsv1.0, --tlsv1.1, and --tlsv1.2 to control the TLS version more precisely (if the SSL backend in use supports such a level of control).
-2, --sslv2
(SSL) Forces curl to use SSL version 2 when negotiating with a remote SSL server. Sometimes curl is built without SSLv2 support. SSLv2 is widely considered insecure.
-3, --sslv3
(SSL) Forces curl to use SSL version 3 when negotiating with a remote SSL server. Sometimes curl is built without SSLv3 support.
Problem is not related to TLS/SSL. Client will negotiate TLS automatically.
It seems like you need to make a POST some xml data and specify your credentials as GET parameters.
It can be done by putting your GET parameters to the request URL
Im not sure on syntax, but try this:
curl -X POST -H "Content-type: text/xml" -o output.txt https://site.url.com?param1=param1&username=username&password=password -d #data.xml
Also, (small offtopic for message above, but i cannt comment it) please not force SSL2, SSL3 or TLS1.0 since they have vulnerabilities. Most servers will negotiate best version of TLS automatically.