PHP CURL with proxy causes CLOSE_WAIT on socket - php

I'm using PHP curl library to establish connection and retrieve content from WEB - the usual.
I have multiple SOCKS5 proxy servers running on localhost on ports from 10300 to 10350 and PHP selects a port randomly.
My code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_PROXY, "localhost:".mt_rand(10300, 10350));
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$curl_resp = curl_exec($ch);
curl_close($ch);
/// ... whatever logic
But if I put this code into loop for long time, I get error telling that no more files can be opened. I read online that is because filehandler limit being reached, so I debugged with lsof -p and I noticed that the process has hundreds and even thousands of these lines:
... TCP localhost:43030->localhost:10303 (CLOSE_WAIT)
... TCP localhost:40982->localhost:10341 (CLOSE_WAIT)
... TCP localhost:48718->localhost:10304 (CLOSE_WAIT)
... TCP localhost:41655->localhost:10350 (CLOSE_WAIT)
... TCP localhost:41915->localhost:10310 (CLOSE_WAIT)
... TCP localhost:49746->localhost:10322 (ESTABLISHED)
I was researching why is this that way, but I don't understand, because I am calling curl_close and I would assume that CURL would close PROXY connection as well, or am I missing something?
Versions:
PHP 7.1.4-1+deb.sury.org~trusty+1 (cli) (built: Apr 11 2017 22:45:20) (NTS)
curl 7.52.1 (x86_64-pc-linux-gnu) libcurl/7.52.1 OpenSSL/1.0.1f zlib/1.2.8 libidn2/0.9 libpsl/0.11.0 (+libicu/52.1) libssh2/1.4.3 nghttp2/1.19.0 librtmp/2.3

I also came across this issue. The work around is add
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Connection: close"));
At first I thought is was related with php7, because i can reproduce it on a php7 box, but on php5, it will not happen. Unfortunately, I tried this on another php7 box, it also did not happen again. So I am still confused about the root cause.

Related

cURL PUT Requests hangs

So here's my problem.
I'm using curl to access my CouchDB by HTTP. I recently updated my WAMP to the WAMP 3 64bit wich comes with PHP 5.6.16 and Apache 2.4.17. Therefore, since this upgrade, I discovered that I couldn't do PUT request anymore.
Env
PHP 5.6.16
Apache 2.4.17
Windows 10 64 bit
Wamp 3 64 bit
Curl --version
curl 7.49.1 (x86_64-pc-win32) libcurl/7.49.1 OpenSSL/1.0.2h nghttp2/1.11.1
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM SSL HTTP2
Code executed
So when I execute this :
<?php
$table="testname";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://localhost:5984/' . $table);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, 'validUser:validPass');
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-type: application/json',
'Accept: */*'
));
$response = curl_exec($ch);
curl_close($ch);
echo $response;
I get a quick response from the server.
Then, I try to create a database :
<?php
$table = "testname";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://localhost:5984/' . $table);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, 'validUser:validPass');
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-type: application/json',
'Accept: */*'
));
$response = curl_exec($ch);
curl_close($ch);
echo $response;
Problem
So, when I execute this code, the request will hang on curl_exec.
What's weird is that, after the timeout, the request will be received by CouchDB but no response will be given. It seems that my "Put" request are stacked in a buffer and they are waiting to be executed.
Verbose curl output
* Hostname in DNS cache was stale, zapped
* Trying ::1...
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 5984 (#0)
* Server auth using Basic with user 'validUser'
> PUT /customers HTTP/1.1
Host: localhost:5984
Authorization: Basic dGVzdEFkbWluOnRlc3RQYXNzd29yZA==
Content-type: application/json
Accept: */*
* Operation timed out after 10000 milliseconds with 0 bytes received
* Closing connection 0
Hints
-I try to install a SSL certificate but It didn't seem to work. Having this certificate still installed can cause problems?
-I can do PUT request with a REST client on my Atom editor without problems.
-I seems like there is a problem in my network route internally. I'm saying this because It affected the PHP-Curl aswell as the Curl CLI. Also, I'm able to do GET request but the PUT request are like "hanging" for no reason and are "Accepted" by my CouchDB when the timeout occurs. It's like if I was sending long poll request.
What have been tested
Execute the same command on the command line -> Same result
Try a REST Client on my Atom editor with success
A friend of mine try to access to my database remotly with success (So CouchDB doesn't seem the problem)
Even if I tested with my Firewall disabled, uninstalling my antivirus ( Bitdefender Total Security 2016) fixed my issue.

IPN PayPal - Couldn't resolve host [duplicate]

I'm trying to fetch the contents of a page using CURL. The page that is doing the fetching is https and the page it is trying to fetch is also https. I'm getting an error "Couldn't resolve host" with all of the settings I try.
$c=curl_init();
curl_setopt($c, CURLOPT_URL,$url);
//curl_setopt($c, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:x.x.x) Gecko/20041107 Firefox/x.x");
curl_setopt ($c, CURLOPT_RETURNTRANSFER, TRUE);
//curl_setopt($c, CURLOPT_SSL_VERIFYPEER, TRUE);
//curl_setopt($c, CURLOPT_SSL_VERIFYHOST, TRUE);
curl_setopt($c, CURLOPT_HEADER, FALSE);
$html=curl_exec($c);
if($html === false) {
echo curl_error($c);
}
else {
echo 'Operation completed without any errors';
}
curl_close($c);
Any ideas?
I found that curl can decide to use IPv6, in which case it tries to resolve but doesn't get an IPv6 answer (or something to that effect) and times out.
You can try the curl command line switch -4 to test this out:
curl -4 http://x.com
In PHP, you can configure this line by setting this:
curl_setopt($c, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4);
Official manual page for this option: https://curl.se/libcurl/c/CURLOPT_IPRESOLVE.html
Maybe a DNS issue?
Try your URL against this code:
$_h = curl_init();
curl_setopt($_h, CURLOPT_HEADER, 1);
curl_setopt($_h, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($_h, CURLOPT_HTTPGET, 1);
curl_setopt($_h, CURLOPT_URL, 'YOUR_URL' );
curl_setopt($_h, CURLOPT_DNS_USE_GLOBAL_CACHE, false );
curl_setopt($_h, CURLOPT_DNS_CACHE_TIMEOUT, 2 );
var_dump(curl_exec($_h));
var_dump(curl_getinfo($_h));
var_dump(curl_error($_h));
I had the same problem. Coudn't resolve google.com. There was a bug somewhere in php fpm, which i am using. Restarting php-fpm solved it for me.
Just a note which may be helpful- I was having this trouble with Apache on my laptop (which connects by wifi AFTER startup), and restarting the server (after connect) fixed the issue. I guess in my case this may be to do with apache starting offline and perhaps there noting that DNS lookups fail?
There is a current bug in glibc on Ubuntu which can have this effect:
https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1674733
To resolve it, update libc and all related (Packages that will be upgraded: libc-bin libc-dev-bin libc6 libc6-dev libfreetype6 libfreetype6-dev locales multiarch-support) and restart the server.
We need to add host security certificate to php.ini file. For local developement enviroment we can add cacert.pem in your local php.ini.
do phpinfo(); and file your php.ini path open and add uncomment ;curl.capath
curl.capath=path_of_your_cacert.pem
If you do it on Windows XAMPP/WAMP it probaly won't work as in my case.
I solved the problem setting up Laravel's Homestead/Vagrant solution to create my (Ubuntu) development environment - it has built-in: Nginx, PHP 5.6, PHP 7.3, PHP 7.2, PHP 7.1, MySQL, PostgreSQL, Redis, Memcached, Node... to name just a few.
See here for info how to set up the environment - it's really worth the effort!
Laravel Homestead is an official, pre-packaged Vagrant box that provides you a wonderful development environment without requiring you to install PHP, a web server, and any other server software on your local machine. No more worrying about messing up your operating system! Vagrant boxes are completely disposable. If something goes wrong, you can destroy and re-create the box in minutes!
Then you can easily switch PHP versions or set up more virtual hosts, new databases just in seconds.
After tried all above, still can't resolved my issue yet.
But got new solution for my problem.
At server where you are going to make a request, there should be a entry of your virtual host.
sudo vim /etc/hosts
and insert
192.xxx.x.xx www.domain.com
The reason if you are making request from server to itself then, to resolve your virtual host or to identify it, server would need above stuff, otherwise server won't understand your requesting(origin) host.
Your getting the error because you're probably doing it on your Local server environment. You need to skip the certificates check when the cURL call is made. For that just add the following options
curl_setopt($c, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($c, CURLOPT_SSL_VERIFYHOST, 0);
You may have to enable the HTTPS part:
curl_setopt($c, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($c, CURLOPT_SSL_VERIFYHOST, 2);
And if you need to verify (authenticate yourself) you may need this too:
curl_setopt($c, CURLOPT_USERPWD, 'username:password');
add yourlocalhost
ex. 127.0.0.1 cards.localhost
in the /etc/hosts directory.
Now restart apache server

curl_errno 56 on localhost

I'm having trouble with php curl on my localhost for days. It seems that I'm not reaching the CURLOPT_URL parameter. curl_error and curl_errno says, "Recv failure: Connection was reset". The url is supposed to return a processed value of the passed post data. I'm using windows 7 machine and running XAMPP 3.2.2
Appreciate your help guys. Thanks
Below is my code:
private function getCurl ($credentials = array(), $app_code = "")
{
$ch = curl_init();
$data = array('Code' => $code,'type' => 'credentials');
curl_setopt($ch, CURLOPT_URL,'http://localhost/web/service.php');
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Authorization: Basic ' .
base64_encode($credentials['user'] . ":" . $credentials['pass'])));
$curl_response = curl_exec($ch);
//return array(curl_error($ch),curl_errno($ch));
curl_close($ch);
return $curl_response;
}
Apache error logs: [Thu Apr 14 07:46:54.089533 2016] [ssl:warn] [pid 3372:tid 252] AH01909: www.example.com:443:0 server certificate does NOT include an ID which matches the server name
There could be a number of reasons why this is happening. I think you should use the XAMP Control Panel to view the specific errors.
Quite possible that another application such as Skype is using port 80. In this case you may want to check using netstat -ano and find who is using port 80. Close the Program which uses port 80 and try to start apache again.
It may be due to your rights. You will need to be administrator.
Disable Anti-virus (Try first to disable skype and running again, if it didn't work do this step)
Right click on xampp control panel and run as administrator
server name used in Apache (httpd.conf) must be the same as the server name in apache (httpd-ssl.conf) e.g. in Apache (httpd.conf)ServerName localhost:8080 then in apache (httpd-ssl.conf) should be like this ServerName www.example.com:8080
You can also try to change the Configuration of Apache.
a) Select Apache (httpd.conf)
b) searched for this line: Listen 80
c) changed for this: Listen 8081
d) saved file
Further...
a) Select Apache (httpd-ssl.conf)
b) searched for this line: Listen 443
c) changed for this: Listen 444
d) saved file

cURL Error : Cannot connect: SSL is disabled. Error number 35

I am working on the paypal ipn script in one of my applications hosted on a Digital Ocean's box with Centos 7.
When i try to connect to the paypal sandbox api i get the error "Cannot connect: SSL is disabled."
I have tried several things like adding the path of the curl.cainfo in my php.ini file like so
curl.cainfo = /etc/pki/tls/certs/ca-bundle.trust.crt
this is what my cURL script looks like
// STEP 2: Post IPN data back to paypal to validate
$ch = curl_init('https://www.sandbox.paypal.com/cgi-bin/webscr'); // change to [...]sandbox.paypal[...] when using sandbox to test
curl_setopt($ch, CURLOPT_SSLVERSION, 4);
curl_setopt($ch, CURLOPT_SSL_CIPHER_LIST, 'ecdhe_rsa_aes_128_gcm_sha_256');
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $req);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_FORBID_REUSE, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Connection: Close'));
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
I have not got much experience with Linux server setup so I am learning as i go along.
Any help or guide is much appreciated
UPDATE : when i run the this command in the command line
curl --version https://sandbox.paypal.com/cgi-bin/webscr
i get this error
curl: (1) Protocol "https" not supported or disabled in libcurl
Also the command curl --version
displays curl 7.42.1 (x86_64-unknown-linux-gnu) libcurl/7.46.0
So i am guessing the new question would be how to enable https in libcurl?
You're setting the wrong SSL version:
curl_setopt($ch, CURLOPT_SSLVERSION, 4);
The Paypal sandbox only supports TLS 1.2 (which is CURLOPT_SSLVERSION == 6). The correct SSL version will be used automatically if you use PHP 5.5.19+ and OpenSSL 1.0.1+, or you can force it yourself with the following (still requires OpenSSL 1.0.1+):
curl_setopt($ch, CURLOPT_SSLVERSION, 6);
I would only expect to see Protocol "https" not supported or disabled in libcurl if this were running a libcurl not installed from rpm - or if whoever configured the machine deliberately broke it for a valid reason, or was very incompetent; the Centos 7 rpm for libcurl has openSSL as a requirement.
Understanding what happened here and remediation will require root access to the target host to install software. You didn't say what access you have on the target system, and you are presumably paying Digital Ocean for support. Unless you broke it yourself, you should be asking Digital Ocean to fix it.
I was able to solve this problem with the help of one my friends with a lot of Linux experience. Apparently this was an issue with the version of cURL installed on my Digital Ocean box.
So the solution was to remove the version of cURL i had installed which was not installed correctly because i had built the binaries my self.
I installed the problematic cURL using these commands
wget -O curl.tar.gz http://curl.haxx.se/download/curl-7.42.0.tar.gz
tar -xvzf curl.tar.gz
cd curl-7.42.0
./configure
make
make install
I removed the problematic cURL that had https issues like this
which curl
(To tell me where the executable was located and it was located in usr/local/bin/curl)
Then i did
sudo rm -f /usr/local/bin/curl
(To remove the cURL giving issues)
Then i used the command
sudo yum install curl php5-curl
This installed cURL correctly for me. The i ran
curl --version
To confirm that https is now supported in the cURL version and it was.
I logged out of my droplet and logged back in. Then tried the
curl --version "https://sandbox.paypal.com/cgi-bin/webscr"
command and everything worked. I was able to connect to Paypal.
Thanks to everyone for their help and comments.
I think you need to enable fopen socket.You can ask hosting provider

Install Symfony on Windows behind proxy server with NTLM authentication

I tried to downlaod and install Symfony on a server at my company...
c:\> php -r "readfile('http://symfony.com/installer');" > symfony
..and got the following error (partly German, but you'll get the idea):
PHP Warning: readfile(http://symfony.com/installer): failed to open stream: Es konnte keine Verbindung hergestellt werd en, da der Zielcomputer die Verbindung verweigerte.
in Command line code on line 1
So I downloaded it manually over the browser (which worked fine) and tried to create a new site:
php symfony.phar new blog
Now I got this error:
[RuntimeException]
There was an error downloading Symfony from symfony.com server:
Client error response [url] http://symfony.com/download?v=Symfony_Standard_Vendors_latest.tgz [status code] 407 [re
ason phrase] Proxy Authentication Required
[GuzzleHttp\Exception\ClientException]
Client error response [url] http://symfony.com/download?v=Symfony_Standard_Vendors_latest.tgz [status code] 407 [re
ason phrase] Proxy Authentication Required
So I did a step back to see if I can do a cURL request in a PHP script. I used the following script to try it (my company is using a proxy server with NTLM authentication):
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.google.com');
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXY, 'proxy.company.loc');
curl_setopt($ch, CURLOPT_PROXYPORT, 8080);
curl_setopt($ch, CURLOPT_PROXYTYPE, 'HTTP');
curl_setopt($ch, CURLOPT_PROXYAUTH, CURLAUTH_NTLM);
curl_setopt($ch, CURLOPT_PROXYUSERPWD, 'user:password');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
echo curl_exec($ch);
curl_close($ch);
And it worked fine.
What do I have to do to be able to install and use Symfony. Where do I define the proxy, the credentials and the NTLM method? I tried adding HTTP_PROXY in the env vars as "http://user:password#proxy.company.loc:8080", but that didn't help (maybe I also have to add the auth type NTLM somewhere).
The server is a Windows Server 2012 R2. PHP version is 5.6.5 and Apache is on 2.4.12.
Thanks for any help and sorry for any grammar errors!
I will describe solution which I use myself.
Download cntlm-0.92.3-setup.exe and install it (it will set up system service).
Configure cntlm.ini (C:\Program Files (x86)\Cntlm\cntlm.ini) - proxy address, no proxy, domain account etc.
Restart Cntlm Authentication Proxy service.
Set http_proxy/https_proxy to http://localhost:3128 (system wide method)
Try symfony installer now
Sometimes it could be a little bit slow but it works. I use this solution for all console apps that do not support integration with system wide proxy. Cntlm also works on unix based OS.
Hope that's help.

Categories