I am using curl function to parse data from backpage.com and oodle.com.In backpage.com, I am using proxy, but it not working on client's server but it perfectly working on our server. But in the oodle.com proxy is not working. Is any other way to parse data from oodle.com? My code is,
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$urls);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
//curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$original_file = curl_exec($ch);
curl_close($ch);
In client's server it shows an error
Warning: curl_setopt() [function.curl-setopt]: CURLOPT_FOLLOWLOCATION cannot be activated when safe_mode is enabled or an open_basedir is set in
I am use phpinfo(); to check the settings , the safe_mod option is off but openbase_dir have value. How can I fix this issue? Any one please help me.
Related
Using curl to make calls to the server. The connection is made on the protocol https using ssl certificate.
Use the following code.
curl_setopt($ch, CURLOPT_URL, $szUrl);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 1);
curl_setopt($ch, CURLOPT_CAINFO, $this->aConfig['certificatePath']);
I realized that despite the certificate has expired curl continues to function properly. why? no way to verify the certificate has expired?
what safety problems exposes this thing?
Support for:
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 1);
removed with cURL 7.28.1.
Use
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
Should do the trick.
I've got the following problem: there is an HTTPS web site, and I need to connect to it through a proxy. Here are my cURL setopts:
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_PROXY, '100.100.100.100:8080');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $ua);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER,false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
echo curl_error($ch);
outputs Failed connect to ######.com:8080; No error
Where 100.100.100.100:8080 is a placeholder for a valid HTTPS proxy. This doesn't work. How do I make cURL connect to an HTTPS website through a proxy? I would really like a soultion that would work through not only HTTPS proxies. Also, I would best prefer a method using cURL, but if there is a better way to do it, without cURL, I could use it instead.
Update:
Add
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
It will prevent your HTTP proxy to parse your request headers and to act more transparently - like a tunnel.
initial answer, not interesting
Your code looks OK, and I assume you checked the trivial issues, so the problem is probably that the SSL certificate verification fails. It's the case if the certificate is self signed by example.
Try
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
to allow a request that allows using a self signed certificate.
cURL + proxy noob here, having a hard time. I'm having trouble trying to retrieve a web page from a remote secure server via a proxy. Everything has apparently been set up correctly by a remote dev, such that the following command line instruction works and returns what we're looking for:
curl -k --socks5-hostname localhost:xxxx https://hostname/
However, the following PHP does not echo the requested webpage. Instead it echoes the error 'Couldn't resolve host name':
$proxy = 'localhost:xxxx';
$url = 'https://hostname/';
//$proxyauth = 'user:password';
$ch = curl_init();
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
//curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_URL, $url);
$curl_scraped_page = curl_exec($ch);
$error = curl_error($ch);
curl_close($ch);
if ($error)
echo $error;
elseif ($curl_scraped_page)
echo $curl_scraped_page;
If the $url is changed to a public page, such as Google, the request is successful and everyone is happy.
The connection requires an SSH tunnel if that changes anything at all. The tunnel is open and functioning, as proven by the command line request succeeding.
Is there something obvious that is being missed here?
You need to set option CURLOPT_PROXYTYPE to CURLPROXY_SOCKS5_HOSTNAME, which sadly wasn't defined in old PHP versions, circa pre-5.6; if you have earlier in but you can explicitly use its value, which is equal to 7:
curl_setopt($ch, CURLOPT_PROXYTYPE, 7);
In the option CURLOPT_PROXYTYPE you need to set CURLPROXY_SOCKS5_HOSTNAME option instead of CURLPROXY_SOCKS5.
In this case, the DNS query (for hostname resolving) will be sent to SOCKS proxy and not resolved in the local network.
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5_HOSTNAME);
This constant available since PHP 5.5.23 and PHP 5.6.7 and cURL 7.18.0, so you can simply use it.
I use this code to login to remote server. Evrything working fine when I don't use proxy. But with proxy it doesn't.
Here is code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_REFERER, $ref_url);
curl_setopt($ch, CURLOPT_HEADER, FALSE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, TRUE);
curl_setopt($ch, CURLOPT_PROXY, $proxy.':'.$port);
ob_start();
return curl_exec ($ch); // execute the curl command
ob_end_clean();
curl_close ($ch);
unset($ch);
I try to disable CURLOPT_HTTPPROXYTUNNEL but it didn't help.
So, without proxy I can login fine.
With proxy not.
Proxy is good ad working.
What's the proxy error you get using the code you posted? Does the proxy need authentication?
I was having a same problem , the code works fine without the proxy but with it its returning nothing
, I have done some debugging and found out that actually on most of the servers only the port 80 is allowed , and many of the proxies we use connect through another port
You can unblock the port no for which you have maximum proxies
You can filter the list with only port 80 proxies
Note : Also only use port http, https and socks4 proxies , socks5 proxies runs on udp not on tcp/ip
i'm trying to make post to an external url using curl, the externa page use https, here is the desc of the server i'm using
Server Apache/2.2.11 (Win32) mod_ssl/2.2.11 OpenSSL/0.9.8k PHP/5.3.0
the external url make a redirect to another url that i send in the post, but everytime i try i get this error
curl_errno=35 (Unknown SSL protocol error in connection to [secure site]:443)
so i check the firebug for the response and it say
Failed to load source for: http://localhost/3Party/PHP_VPC_3Party_Auth_Capture_Order_DO.php
Here is the code I'm using
ob_start();
// initialise Client URL object
$ch = curl_init();
// set the URL of the VPC
curl_setopt ($ch, CURLOPT_URL, $vpcURL);
curl_setopt ($ch, CURLOPT_POST, 1);
curl_setopt ($ch, CURLOPT_POSTFIELDS, $this->postData);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_exec ($ch);
if (curl_error($ch)) {
$this->errorMessage =
"curl_errno=". curl_errno($ch) . " (" . curl_error($ch) . ")";
}
curl_close ($ch);
I think the problem is the fact that you are trying to access an "http" URL (instead of "https") on port 443.
You can also try setting the SSL version manually:
curl_setopt($ch, CURLOPT_SSLVERSION, 3);
Replace 3 with whatever SSL version the remote server is using.
After a few weeks dealing with this issue, i was able to at least establish the connection, i don't know if it is the real answer but it works for me, i just added to the example above, the options to use proxy, just like this
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_PROXYAUTH, CURLAUTH_NTLM );
curl_setopt($ch, CURLOPT_PROXY, 'my.proxy');
curl_setopt($ch, CURLOPT_PROXYPORT, 'my.port');
curl_setopt($ch, CURLOPT_PROXYUSERPWD, 'domain\user:password');
hope this can help
It might also be tls/ssl version preference by the server. In this case, you have to try specify different version constants from here: https://curl.haxx.se/libcurl/c/CURLOPT_SSLVERSION.html
E.g. what worked for me was:
curl_setopt($ch, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1_1);