Once again, another question concerning cURL and SSL, as I cannot find matching answers to my problem.
I have working SSL on my webserver, with trusted cert and green signs on browsers address bar a.s.o., NOT self signed. So good, so far.
Now I want communicate with cURL and use the following function (POST data not added yet):
function ssltest(){
$post_data = '';
$url = 'https://myserver/test.php';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_MUTE, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 1);
//curl_setopt($ch, CURLOPT_CAINFO, 'sslstuff/cacert.pem');
curl_setopt($ch, CURLOPT_CAINFO, 'sslstuff/false.pem');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/x-www-form-urlencoded'));
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
echo ssltest();
As the cacert.pem I use this one, which I found in my browser, which is obviously identical to what I found here http://curl.haxx.se/ca/cacert.pem
In the code shown above there is a false.pem to be seen. Now what ? If this file is empty, there's no response from Server, but I tested to paste the cert from another enterprise from the list on curl.haxx.se I get the same correct answer from the server as result, as when I use my correct .pem
What's the issue ? What I am missing ?
"there's no response from Server"
I think that's very unlikely. I suspect there is no HTTP response from the server, but that the SSL negotiation is failing - but you've got no error checking in your code. If $output===false, have a look at curl_error().
You might want to play around with VERIFYHOST and VERIFYPEER to pin down the exact cause of the problem.
I am trying to use PHP's curl() function and for some reason my code does not return any data.
I am making a request to a URL that is unverified:
Here is my code:
<?php
$ch = curl_init("**SENSITIVE URL**");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
print_r($result);
curl_close($ch);
?>
If I put in www.google.com it does return the google webpage to my site. I apoligize, but I can't give out the URL for my site but I assure you that directly going to the URL does return data.
You need to tell cURL to ignore the (bad) SSL cert. Try adding the following options:
// Do not verify the cert
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
// Ignore the "does not match target host name" error
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
Locked. Comments on this question have been disabled, but it is still accepting new answers and other interactions. Learn more.
I'm trying to connect to a Tor hidden service using the following PHP code:
$url = 'http://jhiwjjlqpyawmpjx.onion/'
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_PROXY, "http://127.0.0.1:9050/");
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
$output = curl_exec($ch);
$curl_error = curl_error($ch);
curl_close($ch);
print_r($output);
print_r($curl_error);
When I run it, I get the following error:
Couldn't resolve host name
However, when I run the following command from my command line in Ubuntu:
curl -v --socks5-hostname localhost:9050 http://jhiwjjlqpyawmpjx.onion
I get a response as expected.
The PHP cURL documentation says this:
--socks5-hostname
Use the specified SOCKS5 proxy (and let the proxy resolve the host name).
I believe the reason it works from the command line is because Tor (the proxy) is resolving the .onion hostname, which it recognizes. When running the PHP code above, my guess is that cURL or PHP is trying to resolve the .onion hostname and doesn't recognize it. I've searched for a way to tell cURL/PHP to let the proxy resolve the hostname, but I can't find a way.
There is a very similar Stack Overflow question, cURL request using socks5 proxy fails when using PHP, but it works through the command line.
You need to set option CURLOPT_PROXYTYPE to CURLPROXY_SOCKS5_HOSTNAME, which sadly wasn't defined in old PHP versions, circa pre-5.6; if you have earlier in but you can explicitly use its value, which is equal to 7:
curl_setopt($ch, CURLOPT_PROXYTYPE, 7);
I use Privoxy and cURL to scrape Tor pages:
<?php
$ch = curl_init('http://jhiwjjlqpyawmpjx.onion'); // Tormail URL
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_PROXY, "localhost:8118"); // Default privoxy port
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_HTTP);
curl_exec($ch);
curl_close($ch);
?>
After installing Privoxy you need to add this line to the configuration file (/etc/privoxy/config). Note the space and '.' a the end of line.
forward-socks4a / localhost:9050 .
Then restart Privoxy.
/etc/init.d/privoxy restart
Try to add this:
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
TL;DR: Set CURLOPT_PROXYTYPE to use CURLPROXY_SOCKS5_HOSTNAME if you have a modern PHP, the value 7 otherwise, and/or correct the CURLOPT_PROXY value.
As you correctly deduced, you cannot resolve .onion domains via the normal DNS system, because this is a reserved top-level domain specifically for use by Tor and such domains by design have no IP addresses to map to.
Using CURLPROXY_SOCKS5 will direct the cURL command to send its traffic to the proxy, but will not do the same for domain name resolution. The DNS requests, which are emitted before cURL attempts to establish the actual connection with the Onion site, will still be sent to the system's normal DNS resolver. These DNS requests will surely fail, because the system's normal DNS resolver will not know what to do with a .onion address unless it, too, is specifically forwarding such queries to Tor.
Instead of CURLPROXY_SOCKS5, you must use CURLPROXY_SOCKS5_HOSTNAME. Alternatively, you can also use CURLPROXY_SOCKS4A, but SOCKS5 is much preferred. Either of these proxy types informs cURL to perform both its DNS lookups and its actual data transfer via the proxy. This is required to successfully resolve any .onion domain.
There are also two additional errors in the code in the original question that have yet to be corrected by previous commenters. These are:
Missing semicolon at end of line 1.
The proxy address value is set to an HTTP URL, but its type is SOCKS; these are incompatible. For SOCKS proxies, the value must be an IP or domain name and port number combination without a scheme/protocol/prefix.
Here is the correct code in full, with comments to indicate the changes.
<?php
$url = 'http://jhiwjjlqpyawmpjx.onion/'; // Note the addition of a semicolon.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_PROXY, "127.0.0.1:9050"); // Note the address here is just `IP:port`, not an HTTP URL.
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5_HOSTNAME); // Note use of `CURLPROXY_SOCKS5_HOSTNAME`.
$output = curl_exec($ch);
$curl_error = curl_error($ch);
curl_close($ch);
print_r($output);
print_r($curl_error);
You can also omit setting CURLOPT_PROXYTYPE entirely by changing the CURLOPT_PROXY value to include the socks5h:// prefix:
// Note no trailing slash, as this is a SOCKS address, not an HTTP URL.
curl_setopt(CURLOPT_PROXY, 'socks5h://127.0.0.1:9050');
Here is a simple function to help you.
Tho to save time first of all you need to make sure that you check if the proxy is valid to not with simple check with fsocketopen()
try {
$fp = fsockopen($ip, $port, $errno, $errstr, 10);
fclose($fp);
return true;
} catch (\Throwable $th) {
return false;
}
If socket returns true then go for requestUrl function
private function requestUrl($url, $proxy)
{
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_PROXY, $proxy);
curl_setopt($curl, CURLOPT_HEADER, 1);
curl_setopt($curl, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($curl, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
curl_setopt($curl, CURLOPT_TIMEOUT, 10);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, FALSE);
$contents = curl_exec($curl);
//Check for errors.
// if (curl_errno($curl)) {
// return new \Exception(curl_error($curl));
// }
curl_close($curl);
return $contents;
}
I am having trouble with this function. It works well with http but doesn't return anything when using https. I am working on facebook's url to fetch all public groups post.
function fetchUrl($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$retData = curl_exec($ch);
curl_close($ch);
return $retData;
}
The only way I've seen is to scrape the url by simple php html dom parser and convert it into array. However, I feel there's a better way, or could make some changes to this existing function. Where am I going wrong?
FYI: I have enabled openSSL in php.ini
Try adding these options
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0);
Quoting PHP Manual
CURLOPT_SSL_VERIFYHOST: 1 to check the existence of a common name in
the SSL peer certificate. 2 to check the existence of a common name
and also verify that it matches the hostname provided.
CURLOPT_SSL_VERIFYPEER: FALSE to stop CURL from verifying the peer's
certificate. Alternate certificates to verify against can be specified
with the CURLOPT_CAINFO option or a certificate directory can be
specified with the CURLOPT_CAPATH option. CURLOPT_SSL_VERIFYHOST may
also need to be TRUE or FALSE if CURLOPT_SSL_VERIFYPEER is disabled
(it defaults to 2).
I'm trying to get response using the Google Safe Browsing Lookup API like this:
$url ="https://sb-ssl.google.com/safebrowsing/api/lookup?client=myappname&apikey=mykey&appver=1.0&pver=3.0&url=".urlencode($myurl);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 120);
$body = curl_exec($ch);
$info = curl_getinfo($ch);
I know that my URL is correct since if I dump it and then paste it to the browser I get the expected result (for example 'malware').
So I am assuming it must be something with cURL. I'm working on localhost and extension=php_curl.dll is un-commented in my php.ini, Php version 5.4.4
The html_code is always 0
Simply set CURLOPT_SSL_VERIFYPEER and CURLOPT_SSL_VERIFYHOST to false.
Anything wrong with my cURL code (http status of 0)?
Also check http://code.google.com/p/twitter-api/issues/detail?id=1291 , it might help. It is different APIs, but with the same problem anyway.
The problem and its solution is very nicely described here:
http://richardwarrender.com/2007/05/the-secret-to-curl-in-php-on-windows/
Basically if you are not using a standalone version of cURL the chances are that the cURL functions do not include a certificate bundle which was needed in my case since I was trying to connect to secure host.
$ch = curl_init();
// Apply various settings
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_CAINFO, "C:/xampp/ca-bundle.crt"); //path to the CA-bundle
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec ($ch);
curl_close($ch);