Here is the code in question:
$ch = curl_init();
curl_setopt(CURLOPT_URL, 'https://api.ipify.org?format=json');
curl_setopt(CURLOPT_PROXY, 'ip:port');
curl_setopt(CURLOPT_PROXYUSERPWD, 'user:pass');
$result = curl_exec($ch);
echo curl_error($ch);
The proxy and proxyauth I'm using most definitely work. I've actually tried multiple proxies from various sources with and without auth, but every time I get a connection timeout when connecting to the proxy.
Is there a config setting preventing proxies from being used or something that I'm not aware of? Any help here will be greatly appreciated.
You need to use CONNECT method for https urls :
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
But some proxies doesn't support this feature.
I solved this issue and just wanted to update this question.
The problem was with the server configuration. Only the most common outbound ports were enabled, and I was using proxies with random ports across the whole spectrum. Enabling the ports that I needed (you could just have the restriction disabled altogether) fixed the issue.
So if you run into this same issue where you're timing out while trying to connect to a proxy, contact your support and ask about any restrictions on the outbound ports.
Sounds like an HTTPS issue to me. Try:
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch,CURLOPT_SSL_VERIFYHOST, false);
It might also just be a slow URL, so try:
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT ,0);
curl_setopt($ch, CURLOPT_TIMEOUT, 400);//max seconds to allow cURL functions to run
If there's any redirects involved, you can also try:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
If that doesn't work, try testing more URLs and see if it's an https issue with every site you try, and whether you can recreate the same issue with http. Also check make sure you have PHP's error reporting enabled, and see if you're getting any errors or warnings. Try deliberately causing PHP to generate an error or warning to see if error reporting is working correctly.
Related
I used cURL to get data from another website. Sometimes it shows data and sometimes empty result
Here is my Code
function get_data($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
$agent=$_SERVER["HTTP_USER_AGENT"];
curl_setopt($ch,CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$returned_content = get_data('www.example.com');
echo $returned_content;
Possibly you call too many connections from your curl_init to one Ip Address, So the server blocks the connection and causes the on/off errors.
Not receiving any content back could be due to one or more out of many different reasons and you need to figure out which.
Use curl_error($ch) in your code after the transfer has been performed to see if curl has told you about a problem.
Investigate the response headers. A HTTP transfer can be successful and not return any error but might only respond with HTTP headers and no body at all. The HTTP headers may then contain clues as to why. Perhaps it is a redirect you should follow, perhaps it requires HTTP authentication or perhaps the server indicates a temporary server glitch.
Curl in PHP returns null on AWS EC2 instance
I had similar issue and I fixed it by making sure versions and settings in php.ini file which is in PHP5 and Apache2 folders were same. If its not then Apache2 tries to execute versions set inside the php.ini settings. Also make sure PHP5-libcurl is installed. Its important too.
Possibly the intermittent failures are due to flaky DNS servers, even if it isn't, you'll still benefit from using Google's DNS servers..
See: https://developers.google.com/speed/public-dns/docs/using
Solution
Make sure you have no space for any variables or in url. Like "$user_name = "Lokman Hosen"". You have to give urldecode "$user_name = "Lokman_Hosen"" or "$user_name = "Lokman%20Hosen"". You can use
urldecode()
Function to remove space. Now cURL dont allow any space.
I did a website which uses Steam as OpenID provider. I firstly hosted it on a shared hosting.
But the traffic grew from 50 users in a day to 1000 in a day. I wasn't expecting that and had to change my host. I took another shared hosting with better performance, etc. to see how it is going to grow. But there's now a problem.
My OpenID login with Steam which worked perfectly on the last host doesn't work anymore. I tried with Google, and it worked. So I don't think my script uses a functionality that isn't enabled on my new host.
So when I put Steam identity, it loads during about 30 seconds and then Chrome returns me an error, ERR_EMPTY_RESPONSE. I tried to activate error_reporting E_ALL, but it does the same.
I am using LightOpenID, and here is the portion of the code incriminated:
$openid->identity = 'http://steamcommunity.com/openid';
header('Location: ' . $openid->authUrl());
Actually, it doesn't work whenever I call $openid->authUrl(). Here is the complete code: http://pastebin.com/rChDzECq
How can I resolve this? Thank you in advance.
I also had problems fighting the LightOpenID code in the last couple of hours. I finally made it work and here is what I learned in the process.
Absolutely no HTML output before the header() command. Even the tiniest space prevented the command from redirecting anything.
My server wouldn't allow the use of file_get_contents() and just returned a 404 error from the URL passed to it. You can solve this problem with a custom file_get_contents_curl() command. Here's the one I used myself:
function file_get_contents_curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
I know this is a late answer but I hope it will help someone.
I send an item code to a web service in xml format using cUrl(php). I get the correct response in localhost, but when do it server it shows
cURL Error (7): couldn't connect to host
And here's my code:
function xml_post($post_xml, $url)
{
$user_agent = $_SERVER['HTTP_USER_AGENT'];
$ch = curl_init(); // initialize curl handle
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_xml);
curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
// curl_setopt($ch, CURLOPT_PORT, $port);
$data = curl_exec($ch);
$curl_errno = curl_errno($ch);
$curl_error = curl_error($ch);
if ($curl_errno > 0) {
echo "cURL Error ($curl_errno): $curl_error\n";
} else {
echo "Data received\n";
}
curl_close($ch);
echo $data;
}
I send the item code to the tally and fetch the details from it. I tried using both the versions php 4+ and php5+, nothing works out Any solution.
CURL error code 7 (CURLE_COULDNT_CONNECT)
is very explicit ... it means Failed to connect() to host or proxy.
The following code would work on any system:
$ch = curl_init("http://google.com"); // initialize curl handle
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$data = curl_exec($ch);
print($data);
If you can not see google page then .. your URL is wrong or you have some firewall or restriction issue.
“CURL ERROR 7 Failed to connect to Permission denied” error is caused, when for any reason curl request is blocked by some firewall or similar thing.
you will face this issue when ever the curl request is not with standard ports.
for example if you do curl to some URL which is on port 1234, you will face this issue where as URL with port 80 will give you results easily.
Most commonly this error has been seen on CentOS and any other OS with ‘SElinux’.
you need to either disable or change ’SElinux’ to permissive
have a look on this one
http://www.akashif.co.uk/php/curl-error-7-failed-to-connect-to-permission-denied
Hope this helps
If you have tried all the ways and failed, try this one command:
setsebool -P httpd_can_network_connect on
In PHP, If your network under proxy. You should set the proxy URL and port
curl_setopt($ch, CURLOPT_PROXY, "http://url.com"); //your proxy url
curl_setopt($ch, CURLOPT_PROXYPORT, "80"); // your proxy port number
This is solves my problem
In my case I had something like cURL Error (7): ... Operation Timed Out. I'm using the network connection of the company I'm working for. I needed to create some environment variables. The next worked for me:
In Linux terminal:
$ export https_proxy=yourProxy:80
$ export http_proxy=yourProxy:80
In windows I created (the same) environment variables in the windows way.
I hope it helps!
Regards!
Are you able to hit that URL by browser or by PHP script? The error shown is that you could not connect. So first confirm that the URL is accessible.
Check if port 80 and 443 are blocked. or enter - IP graph.facebook.com and enter it in etc/hosts file
you can also get this if you are trying to hit the same URL with multiple HTTP request at the same time.Many curl requests wont be able to connect and so return with error
This issue can also be caused by making curl calls to https when it is not configured on the remote device. Calling over http can resolve this problem in these situations, at least until you configure ssl on the remote.
In my case, the problem was caused by the hosting provider I was using blocking http packets addressed to their IP block that originated from within their IP block. Un-frickin-believable!!!
For a couple of days I was totally blocked on this. I'm very very new to networking/vms but was keen to try set it up myself instead of paying a hosting company to do it for me.
Context
I'm rebuilding the server side for an app that uses php routines to return various bits of data from internal sources as well as external APIs for a map based app. I have started an Oracle VM instance and have installed/set up Apache and php. All running totally fine, until one of my php routines tries to execute a cURL. I start implementing error logging to find that I don't even get a message - just '7', despite implementation being very similar to the above. My php routine accessing an internal file for data was running successfully so I was fairly sure it wasn't an Apache or php issue. I also checked my Apache error logs, nothing telling.
Solution
I nearly gave up - there's talk on disabling SELinux above and in other articles, I tried that and it did work for my purposes, but here's a really good article on why you shouldn't disable SELinux https://www.electronicdesign.com/technologies/embedded-revolution/article/21807408/dont-do-it-disabling-selinux
If temporarily disabling it works and like me you don't want to do this (but it confirms that SELinux is blocking you!), I found a neat little command that actually prints out any SELinux issues in a more readable fashion:
sealert -a /var/log/audit/audit.log
This returned the following:
found 1 alerts in /var/log/audit/audit.log
--------------------------------------------------------------------------------
SELinux is preventing php-fpm from name_connect access on the tcp_socket port 443.
Great, I now get a bit more information than just '7'. Reading further down, I can see it actually makes suggestions:
***** Plugin catchall_boolean (24.7 confidence) suggests ******************
If you want to allow httpd to can network connect
Then you must tell SELinux about this by enabling the 'httpd_can_network_connect' boolean.
Do
setsebool -P httpd_can_network_connect 1
This has been mentioned further above but now I have a bit more context and an explanation as to what it does. I run the command, and I'm in business. Furthermore, my SELinux is still set to enforcing, meaning my machine is more secure.
There are many other suggestions logged out, if you're blocked it might be worth logging out/checking out /var/log/audit/audit.log.
I was using cURL to scrape content from a site and just recently my page stated hanging when it reached curl_exec($ch). After some tests I noticed that it could load any other page from my own domain but when attempting to load from anything external I'll get a connect() timeout! error.
Here's a simplified version of what I was using:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,'http://www.google.com');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
$contents = curl_exec ($ch);
curl_close ($ch);
echo $contents;
?>
Here's some info I have about my host from my phpinfo():
PHP Version 5.3.1
cURL support enabled
cURL Information 7.19.7
Host i686-pc-linux-gnu
I don't have access to SSH or modifying the php.ini file (however I can read it). But is there a way to tell if something was recently set to block cURL access to external domains? Or is there something else I might have missed?
Thanks,
Dave
I'm not aware about any setting like that, it would not make much sense.
As you said you are on a remote webserver without console access I guess that your activity has been detected by the host or more likely it caused issues and so they firewalled you.
A silent iptables DROP would cause this.
When scraping google you need to use proxies for more than a few hand full of requests and you should never abuse your webservers primary IP if it's not your own. That's likely a breach of their TOS and could even result in legal action if they get banned from Google (which can happen).
Take a look at Google rank checker that's a PHP script that does exactly what you want using CURL and proper IP management.
I can't think of anything that's causing a timeout than a firewall on your side.
I'm not sure why you're getting a connect() timeout! error, but the following line:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
If it's not set to 1, it will not return any of the page's content back into your $contents.
I use this method to get facebook api data. just a search query. but I find use curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); cost more time during a curl time (over 10+ seconds).
Is there other curl method can run faster?
NOTE: I am now testing in localhost
$url = "https://graph.facebook.com/search?access_token=".$token."&q=dallas&type=post&scope=publish_stream,offline_access,user_status,read_stream";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
//curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 2);
//curl_setopt($ch, CURLOPT_CAINFO, dirname(__FILE__). '/file.crt'); the way as Lumbendil recommend, download a crt file via firefox. still slowly.
$body= curl_exec($ch);
curl_close ($ch);
PS:I do not want to use a SDK, becuase I failed set SDK in localhost test. Although I have read many articles of how to set in localhost. I have set http://127.0.0.1/facebook as my callback url. But just failed. So I still want to get an easy curl way.
Thanks.
You could use a .crt file and verify against that instead of ignoring SSL verification, as explained here.
To keep all the information in one place: In your code, you should write the following:
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_CAINFO, '/path/to/crt/file.crt');
To obtain the certificate, you should go with the browser to the page, and then with "view certificate" you have to export it. Remember that you must export it as X.509 Certificate (PEM) for this to work. For a more detailed guide on how to export the certificate, visit the link provided.
If ignoring to check a certificate takes 10 seconds, the problem is not with the certificate or with the checking and quite frankly, it probably isn't with SSL at all.
Ignoring to check the certificate should be very fast and not be measurable compared to how long the rest of the SSL handshake procedure takes.
To properly track down the problem, I would recommend you use the curl command line tool and its --trace-ascii and --trace-time options to see what seems to take time. You may need to snoop on the network with wireshark or similar to get an even better picture of what's going on.
I can't see how the other suggestions of adding a certificate check to the mix will make anything faster.
Just a side note, but if you do wish to use the SDK you can work around the local issue by editing your hosts file and adding localhost.local for 127.0.0.1. /etc/hosts on a linux machine and C:\WINDOWS\system32\drivers\etc\hosts on a windows machine.
Then in the Facebook app settings, simply set localhost.local as your domain and set your site url accordingly.
You should be ready to go then.