cURL (via PHP) on my local machine is working fine, but on a production server, I am getting no response when connecting to some hosts, while others are working Fine. Further, I can use wget to download from the hosts curl cannot.
The problem appears to be - my server is on IPv6, and it cURL is trying to make connections via IPv6, to hosts who do not support it. The solution for PHP is:
CURLOPT_IPRESOLVE = CURL_IPRESOLVE_V4
command line:
curl -4 http://host-that-does-not-have-an-ipv6-address
Also if you are using a SOCKS proxy, in PHP, the following may help:
CURLOPT_PROXYTYPE = 7
Related
Server: Centos 8 (fully updated)
PHP Version: 7.4 (FPM)
I am trying to make a call with cURL (and also tried with Guzzle) and I am getting the same problem with only HTTPS URLs. HTTP urls are fine and no problems.
Here is some sample code:
curl_setopt_array($curl, array(
CURLOPT_RETURNTRANSFER => true,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_VERIFYPEER => false,
CURLOPT_CAINFO => BASEPATH . "/data/cacert.pem",
CURLOPT_URL => $url
));
The exact cURL error is: Received HTTP code 403 from proxy after CONNECT
The Apache virtual host is not using any proxy. I think this is somehow being stopped between Apache and PHP-FPM. The logs out put is not helpful. I have ruled out SELinux and mod_security as being the issue. Disabled both and still get the same result.
Another question mentions adding the following to https_proxy.conf
ProxyRequests On
AllowCONNECT 443 563 5000
But this has not yielded any results. The reason is because I am not using an http proxy in any http configurations.
Well, I found out what the problem was. I had an unused environment variable in my .env file which was setting a proxy that I usually use on my dev machine. The script was failing on the production machine.
Turns out cURL and Guzzle automatically load those environment variables in with their requests, thus causing a proxy connection refusal. I removed the variable and everything works as expected.
Live and learn.
I work remotely and can access internal servers via VPN. When my VPN connection is established, I can reach my webserver via curl:
curl http://sub.mydomain.com
I can also reach the webserver in a browser by going to http://sub.mydomain.com. So this does not seem to be a DNS issue with the webserver itself.
When developing my Laravel 4.2 application (PHP 5.6) served locally via Apache, however, php's curl_exec fails to resolve the host. Oddly, php's gethostbyname($hostname) correctly resolves the host. I have tried forcing IPv4 as I have read IPv6 can result in failures of this type with no success.
// works
$ip = gethostbyname($hostname);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://$ip/path");
curl_setopt($ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4);
curl_exec($ch);
// does NOT work
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://$hostname/path");
curl_setopt($ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4);
curl_exec($ch);
This leaves me at a loss. I don't understand how PHP curl handles DNS resolution (clearly not with gethostbyname). I also don't understand exactly how DNS lookups on private networks work in the first place. So I really don't know where to look to get PHP curl to resolve my private hosts.
Command line curl resolved the host. Browser resolved the host. Only PHP curl failed to resolve it.
Ultimately, the issue came down to the curl configuration. I installed PHP with homebrew and as a dependency it installed curl-openssl to be used by PHP. This install of curl is configured by the brew formula to use c-ares for domain name resolution. I don't know how c-ares works, but this VPN DNS is apparently an edge case it does not handle correctly on OS X (perhaps due to OS X doing a lousy job of keeping /etc/resolv.conf up to date).
/usr/bin/curl on the other hand was configured to use the native OS X resolver. This is the same resolver used by PHP's gethostbyname and the web browser, which explains why both of those work as expected.
$ brew uninstall --ignore-dependencies curl-openssl
This resolved my issue by dumping this "broken" curl installation. I am not sure how the fallback mechanism works, but I believe PHP is now using /usr/bin/curl since I have no other installations of curl (that I know about) and the curl version listed in phpinfo() now matches /usr/bin/curl when before it did not.
Via terminal on a Mac (PHP 7.1.23), I can issue a curl (curl 7.54.0) command via the CLI such as:
curl https://www.SERVER_HOSTNAME:8443
and it works without error.
I SSH into the VPS Linux server running x86_64-redhat-linux-gnu with curl 7.61.1 and PHP 7.3.5 and if I do a curl command without the port:
curl https://www.SERVER_HOSTNAME
it works fine but as soon as I add the port it chokes.
curl https://www.SERVER_HOSTNAME:8443
gives this error:
"SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above.
The actual certificate is not self signed. It is via Godaddy with a Bundle and a good CA.
Any suggestions on what I need to fix or set?
Thanks in advance.
I am trying to use Luminati.oi proxy service to crawl URLs but there seems to be a problem with my server connecting to the proxy and utilizing the CURLOPT_PROXY functionality.
$curl = curl_init('http://lumtest.com/myip.json');
curl_setopt($curl, CURLOPT_PROXY, 'http://example:24000');
curl_exec($curl);
$result = curl_exec($curl); echo $result;
echo 'Curl error: ' . curl_error($curl);
curl_close($curl);
I get no response at all from the target URL and the curl_error() function returns Curl error: Failed to connect to exmaple port 24000: Connection refused there are several variations on how to compose the CURL that Luminati.io provides, none seem to work.
If I remove the CURLOPT_PROXY option and just send the request direct from my server with no proxy, it works just fine and I get the correct response back from the target URL. So my server seems okay with CURL just not the proxy function.
The URL and port for the proxy server at luminati.io seem to work fine when using the desktop proxy manager. So the proxy service seems to work okay, the target URL is good and my server can use CURL with no problem, so it seems the issue is isolated around the CURLOPT_PROXY not working. The good people over at Luminati.io think there is some type of server setting or firewall in my Apache Linux server that wont allow the proxy connection to occur, I can't find this setting anywhere and Hostgator seems useless and apathetic when asked.
So I'm hoping someone can provide some greater insight into why CURLOPT_PROXY function wont work.
You can ask Hostgator to give you access to port 22225. I think this is the correct port used by Luminati.
This is how you can contact Hostgator: http://support.hostgator.com/articles/open-new-ports
I have a website (www.example.com | 123.456.789), in that machine i only have 1 web app running. i am trying to use php file_get_content() to a file which in my own server (located on some file like /var/www/my_site/public_html).
The PHP code i am referring is:
$url = 'http://example.com/282-home_default/short-wallet-tan.jpg';
var_dump($url);
$json = #file_get_contents($url);
var_dump($json);
die();
However it always returns error. When i try to do it manually, using CURL
I can confirm that i can CURL other website such as ~$ curl google.com
Can anyone suggest me what to do to resolve this. Thanks
I am using Ubuntu 14.04 LTS to host my webapp.
Some of the solution regarding this question, points out that i should downgrade my version of curl. Did that but still have the same problem.
curl failed to connect to that host's TCP port number 80. You're either not running the http server on port 80, you've made curl try to connect to the wrong server or there's something else in your network that prevents the connect handshake to succeed.