I am having trouble with this function. It works well with http but doesn't return anything when using https. I am working on facebook's url to fetch all public groups post.
function fetchUrl($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$retData = curl_exec($ch);
curl_close($ch);
return $retData;
}
The only way I've seen is to scrape the url by simple php html dom parser and convert it into array. However, I feel there's a better way, or could make some changes to this existing function. Where am I going wrong?
FYI: I have enabled openSSL in php.ini
Try adding these options
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0);
Quoting PHP Manual
CURLOPT_SSL_VERIFYHOST: 1 to check the existence of a common name in
the SSL peer certificate. 2 to check the existence of a common name
and also verify that it matches the hostname provided.
CURLOPT_SSL_VERIFYPEER: FALSE to stop CURL from verifying the peer's
certificate. Alternate certificates to verify against can be specified
with the CURLOPT_CAINFO option or a certificate directory can be
specified with the CURLOPT_CAPATH option. CURLOPT_SSL_VERIFYHOST may
also need to be TRUE or FALSE if CURLOPT_SSL_VERIFYPEER is disabled
(it defaults to 2).
Related
Once again, another question concerning cURL and SSL, as I cannot find matching answers to my problem.
I have working SSL on my webserver, with trusted cert and green signs on browsers address bar a.s.o., NOT self signed. So good, so far.
Now I want communicate with cURL and use the following function (POST data not added yet):
function ssltest(){
$post_data = '';
$url = 'https://myserver/test.php';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_MUTE, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 1);
//curl_setopt($ch, CURLOPT_CAINFO, 'sslstuff/cacert.pem');
curl_setopt($ch, CURLOPT_CAINFO, 'sslstuff/false.pem');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/x-www-form-urlencoded'));
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
echo ssltest();
As the cacert.pem I use this one, which I found in my browser, which is obviously identical to what I found here http://curl.haxx.se/ca/cacert.pem
In the code shown above there is a false.pem to be seen. Now what ? If this file is empty, there's no response from Server, but I tested to paste the cert from another enterprise from the list on curl.haxx.se I get the same correct answer from the server as result, as when I use my correct .pem
What's the issue ? What I am missing ?
"there's no response from Server"
I think that's very unlikely. I suspect there is no HTTP response from the server, but that the SSL negotiation is failing - but you've got no error checking in your code. If $output===false, have a look at curl_error().
You might want to play around with VERIFYHOST and VERIFYPEER to pin down the exact cause of the problem.
I am trying to fetch a data from a site using curl method. Before i can able to get data by using curl.Recently client taken ssl certificate for that site from then onwards i can't able to get the data using curl. But when i try the URL in browser i can able to see data. Can any one tell me why it is not working.did i need to try in some another way.
Below is my code
// create curl resource
$ch = curl_init();
// set url
curl_setopt($ch, CURLOPT_URL, "https://qa.myhealth.today/myhealth-portal/nirvahak/public/validateSessionttt?username=pavithra#gmail.com");
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// $output contains the output string
$output = curl_exec($ch);
// close curl resource to free up system resources
curl_close($ch);
$output=json_decode($output);
print_r($output);
There are 2 fixes:
1) SSL certified websites do not trust any non-SSL requests. So, we need to win trust of SSL enabled client.
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
2) Provide him the security certificate.
Go the the respective site. Save the SSL certificate and pass the certificate path to the CURL request.
So, the final code should be:
// create curl resource
$ch = curl_init();
$url = "https://qa.myhealth.today/myhealth-portal/nirvahak/public/validateSessionttt?username=pavithra#gmail.com";
// set url
curl_setopt($ch, CURLOPT_URL, $url);
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_CAINFO, getcwd() . "/CAcerts/BuiltinObjectToken-EquifaxSecureCA.crt");
// $output contains the output string
$output = curl_exec($ch);
// close curl resource to free up system resources
curl_close($ch);
$output=json_decode($output);
print_r($output);
Detailed instructions on how to do it are place here:
Set CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER to false.
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
CURLOPT_SSL_VERIFYHOST
1 to check the existence of a common name in the SSL peer certificate. 2 to check the existence of a common name and also verify that it matches the hostname provided. In production environments the value of this option should be kept at 2 (default value).
CURLOPT_SSL_VERIFYPEER
FALSE to stop cURL from verifying the peer's certificate. Alternate certificates to verify against can be specified with the CURLOPT_CAINFO option or a certificate directory can be specified with the CURLOPT_CAPATH option.
Source: http://php.net/manual/en/function.curl-setopt.php
Locked. Comments on this question have been disabled, but it is still accepting new answers and other interactions. Learn more.
I'm trying to connect to a Tor hidden service using the following PHP code:
$url = 'http://jhiwjjlqpyawmpjx.onion/'
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_PROXY, "http://127.0.0.1:9050/");
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
$output = curl_exec($ch);
$curl_error = curl_error($ch);
curl_close($ch);
print_r($output);
print_r($curl_error);
When I run it, I get the following error:
Couldn't resolve host name
However, when I run the following command from my command line in Ubuntu:
curl -v --socks5-hostname localhost:9050 http://jhiwjjlqpyawmpjx.onion
I get a response as expected.
The PHP cURL documentation says this:
--socks5-hostname
Use the specified SOCKS5 proxy (and let the proxy resolve the host name).
I believe the reason it works from the command line is because Tor (the proxy) is resolving the .onion hostname, which it recognizes. When running the PHP code above, my guess is that cURL or PHP is trying to resolve the .onion hostname and doesn't recognize it. I've searched for a way to tell cURL/PHP to let the proxy resolve the hostname, but I can't find a way.
There is a very similar Stack Overflow question, cURL request using socks5 proxy fails when using PHP, but it works through the command line.
You need to set option CURLOPT_PROXYTYPE to CURLPROXY_SOCKS5_HOSTNAME, which sadly wasn't defined in old PHP versions, circa pre-5.6; if you have earlier in but you can explicitly use its value, which is equal to 7:
curl_setopt($ch, CURLOPT_PROXYTYPE, 7);
I use Privoxy and cURL to scrape Tor pages:
<?php
$ch = curl_init('http://jhiwjjlqpyawmpjx.onion'); // Tormail URL
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_PROXY, "localhost:8118"); // Default privoxy port
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_HTTP);
curl_exec($ch);
curl_close($ch);
?>
After installing Privoxy you need to add this line to the configuration file (/etc/privoxy/config). Note the space and '.' a the end of line.
forward-socks4a / localhost:9050 .
Then restart Privoxy.
/etc/init.d/privoxy restart
Try to add this:
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
TL;DR: Set CURLOPT_PROXYTYPE to use CURLPROXY_SOCKS5_HOSTNAME if you have a modern PHP, the value 7 otherwise, and/or correct the CURLOPT_PROXY value.
As you correctly deduced, you cannot resolve .onion domains via the normal DNS system, because this is a reserved top-level domain specifically for use by Tor and such domains by design have no IP addresses to map to.
Using CURLPROXY_SOCKS5 will direct the cURL command to send its traffic to the proxy, but will not do the same for domain name resolution. The DNS requests, which are emitted before cURL attempts to establish the actual connection with the Onion site, will still be sent to the system's normal DNS resolver. These DNS requests will surely fail, because the system's normal DNS resolver will not know what to do with a .onion address unless it, too, is specifically forwarding such queries to Tor.
Instead of CURLPROXY_SOCKS5, you must use CURLPROXY_SOCKS5_HOSTNAME. Alternatively, you can also use CURLPROXY_SOCKS4A, but SOCKS5 is much preferred. Either of these proxy types informs cURL to perform both its DNS lookups and its actual data transfer via the proxy. This is required to successfully resolve any .onion domain.
There are also two additional errors in the code in the original question that have yet to be corrected by previous commenters. These are:
Missing semicolon at end of line 1.
The proxy address value is set to an HTTP URL, but its type is SOCKS; these are incompatible. For SOCKS proxies, the value must be an IP or domain name and port number combination without a scheme/protocol/prefix.
Here is the correct code in full, with comments to indicate the changes.
<?php
$url = 'http://jhiwjjlqpyawmpjx.onion/'; // Note the addition of a semicolon.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_PROXY, "127.0.0.1:9050"); // Note the address here is just `IP:port`, not an HTTP URL.
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5_HOSTNAME); // Note use of `CURLPROXY_SOCKS5_HOSTNAME`.
$output = curl_exec($ch);
$curl_error = curl_error($ch);
curl_close($ch);
print_r($output);
print_r($curl_error);
You can also omit setting CURLOPT_PROXYTYPE entirely by changing the CURLOPT_PROXY value to include the socks5h:// prefix:
// Note no trailing slash, as this is a SOCKS address, not an HTTP URL.
curl_setopt(CURLOPT_PROXY, 'socks5h://127.0.0.1:9050');
Here is a simple function to help you.
Tho to save time first of all you need to make sure that you check if the proxy is valid to not with simple check with fsocketopen()
try {
$fp = fsockopen($ip, $port, $errno, $errstr, 10);
fclose($fp);
return true;
} catch (\Throwable $th) {
return false;
}
If socket returns true then go for requestUrl function
private function requestUrl($url, $proxy)
{
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_PROXY, $proxy);
curl_setopt($curl, CURLOPT_HEADER, 1);
curl_setopt($curl, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($curl, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
curl_setopt($curl, CURLOPT_TIMEOUT, 10);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, FALSE);
$contents = curl_exec($curl);
//Check for errors.
// if (curl_errno($curl)) {
// return new \Exception(curl_error($curl));
// }
curl_close($curl);
return $contents;
}
I am having some issues communicating to certain third party system, through CURL in PHP.
This is part of the code I have to submit come requests
$query = '<tag>some xml content with request data</tag>';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://secure.certainsystem.com/function.php");
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15);
curl_setopt($ch, CURLOPT_TIMEOUT, 15);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_POSTFIELDS, $query);
curl_setopt($ch, CURLOPT_POST, 1);
$data = curl_exec($ch);
The request is the same for all cases, but sometimes I don't get any data from the curl_exec($ch), and instead, when I run curl_error($ch), I get:
SSL peer certificate or SSH remote key was not OK
This is not happening all the times, but it is happening, so I'm not sure what the problem could be, if the problem is in the code, or there is a problem in the third party system I'm comunicating to.
I searched for this message error in other places and here as well, and I found that if CURLOPT_SSL_VERIFYPEER is set to true, then it could be a problem with the third party system's certificate, perhaps a self-signed one. But in my case, that option is set to 0, which I assume is taken as false.
Recently I found there is an option CURLOPT_SSL_VERIFYHOST, but I'm not setting that option to any value, so I think it is taking the default, which according to PHP.net documentation, is 2, which means:
2 to check the existence of a common name and also verify that it matches the hostname provided.
Thank you very much for your help.
add options
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0);
more see here https://answers.launchpad.net/ubuntu/+question/171188
Although I am answering an old post, I think it will help the new viewers-
You can check the problem by adding
curl_setopt($ch, CURLOPT_VERBOSE, 1);
The reason is explained in my post here.
This is my Code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $apiUrl);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postValues);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
Now the $result is "false" and the curl_error() shows me the SSL-Error "Peer's Certificate issuer is not recognized.".
But although there is this error, the post data has been sent to the $apiUrl.
Is this correct? Bug or feature? ;)
How can I improve this to prevent sending data to an insecure service?
Thanks in advance! :)
There is two solution for this...
Solution - 1
If you want to skip check SSL certificate use....
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0);
Solution - 2
If you have certificate with you use....
curl_setopt ($ch, CURLOPT_CAINFO, "PATH_TO/cacert.pem");
Thanks.
Finally I found the reason why the data has been transfered although there has been a curl error!
There was a redirect to another location and with "followlocation" active, the error happened on the redirected site!
So the data has been sent to the $apiUrl and has been processed. After this the curl call has been redirected and the error appeared.
My trust in logic has been restored :D
This is incorrect; cURL does not send any data when there is a problem with the SSL certificate.
(I just tested with a connection to a local script - the second script did not run when the first script encountered an SSL error while connecting to it.)