I'm doing a simple curl on this address: https://github.com/users/davidhariri/contributions_calendar_data
When i grab the result with this function:
function fetch_data($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$result = curl_exec($ch);
curl_close($ch);
print_r($result);
return $result;
}
The strings are correct, but the ints (the contributions) are wrong.
Results from curl
[...["2014/01/04",0],["2014/01/05",0],["2014/01/06",0],["2014/01/07",1],["2014/01/08",0]]
Results from just navigating to the address
[...["2014/01/04",0],["2014/01/05",0],["2014/01/06",1],["2014/01/07",5],["2014/01/08",5]]
Something during the curl process might be transforming ints to binary and back again? I have no idea what's happening here.
Check that you are not logged in in the browser. You may get different results if so.
Related
This really has me perplexed and other examples on Stackoverflow have not helped.
If I type into the browser :
https://api.bitfinex.com/v2/book/tXMRUSD/P0
I will get an long array of data which is correct.
With the following simple code :
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"https://api.bitfinex.com/v2/book/tXMRUSD/P0");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPGET, 1);
$result = curl_exec($ch);
curl_close($ch);
var_dump($result);
I get :
bool(false)
The only difference is my browser fetches the data directly, while the PHP code is on a local website hosted using IIS on windows 10.
I've tried everything and just can't see where I'm going wrong. Any help will be most appreciated. (Ironically, all the authenticated and encrypted POST code works fine and I'm stuck on the easy stuff!)
Try This code it will work!!!!
<?php
$url= "https://api.bitfinex.com/v2/book/tXMRUSD/P0";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
$result = curl_exec($ch);
curl_close($ch);
print_r($result);
?>
i am trying to get the content of this json: http://steamcommunity.com/market/pricehistory/?country=DE¤cy=3&appid=730&market_hash_name=Chroma%20Case
This is my code:
$url = "http://steamcommunity.com/market/pricehistory/?country=DE¤cy=3&appid=730&market_hash_name=Chroma%20Case";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIE, 'steamLogin = 76561198075419487%7C%7C3F1A776553C4BE1D0F6DA83059052E79DB7EB3C7');
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
$json_string = json_encode($output, JSON_PRETTY_PRINT);
When printing out $json_string it results in nothing, $output results in "‹ŠŽÿÿ)»L". I would like to grab the actual content on the website, the steamLogin-Cookie is needed for that. The cookie that's stored in my browser at the moment is the one I hardcoded in the source.
If you need any more info, feel free to ask.
Adding curl_setopt($ch, CURLOPT_ENCODING,""); made it :)
I have a Affiliate URL Like http://track.abc.com/?affid=1234
open this link will go to http://www.abc.com
now i want to execute the http://track.abc.com/?affid=1234 Using CURL
and now how i can Get http://www.abc.com
with Curl ?
If you want cURL to follow redirect headers from the responses it receives, you need to set that option with:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
You may also want to limit the number of redirects it follows using:
curl_setopt($ch, CURLOPT_MAXREDIRS, 3);
So you'd using something similar to this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://track.abc.com/?affid=1234");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_MAXREDIRS, 3);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$data = curl_exec($ch);
Edit: Question wasn't exactly clear but from the comment below, if you want to get the redirect location, you need to get the headers from cURL and parse them for the Location header:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://track.abc.com/?affid=1234");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, true);
$data = curl_exec($ch);
This will give you the headers returned by the server in $data, simply parse through them to get the location header and you'll get your result. This question shows you how to do that.
I wrote a function that will extract any header from a cURL header response.
function getHeader($headerString, $key) {
preg_match('#\s\b' . $key . '\b:\s.*\s#', $headerString, $header);
return substr($header[0], strlen($key) + 3, -2);
}
In this case, you're looking for the value of the header Location. I tested the function by retrieving headers from a TinyURL, that redirects to http://google.se, using cURL.
$url = "http://tinyurl.com/dtrkv";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($ch);
curl_close($ch);
$location = getHeader($data, 'Location');
var_dump($location);
Output from the var_dump.
string(16) "http://google.se"
I am trying to open homepages of websites and extract title and description from it's html markup using curl with php, I am successful in doing this to an extent, but many websites are there I am unable to open. My code is here:
function curl_download($Url){
if (!function_exists('curl_init')){
die('Sorry cURL is not installed!');
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $Url);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
// $url is any url
$source=curl_download($url);
$d=new DOMDocument();
$d->loadHTML($source);
$title=$d->getElementsByTagName("title")->item(0)->textContent)
$domx = new DOMXPath($d);
$desc=$domx->query("//meta[#name='description']")->item(0);
$description=$desc->getAttribute('content');
?>
This code is working fine for most websites but there are many whome it doesn't even able to open. What can be the reason?
When I tried getting headers of those websites using get_headers function, its working fine, but these are not being opened using curl. Two of these websites are blogger.com and live.com.
Replace:
$output = curl_exec($ch);
with
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSLVERSION, 3);
$output = curl_exec($ch);
if (!$output) {
echo curl_error($ch);
}
and see why Curl is failing.
It's a good idea to always check the result of function calls to see if they succeeded or not, and to report when they fail. While a function may work 99.999% of the time, you need to report the times it fails, and why, so the underlying cause can be identified and fixed, if possible.
I'm having dificulties to query a webform using CURL with a PHP script. I suspect, that I'm sending something that the webserver does not like. In order to see what CURL realy sends I'd like to see the whole message that goes to the webserver.
How can I set-up CURL to give me the full output?
I did
curl_setopt($ch, CURLOPT_VERBOSE, TRUE);
but that onyl gives me a part of the header. The message content is not shown.
Thanks for all the answers! After all, they tell that It's not possible. I went down the road and got familiar with Wireshark. Not an easy task but definitely worth the effort.
Have you tried CURLINFO_HEADER_OUT?
Quoting the PHP manual for curl_getinfo:
CURLINFO_HEADER_OUT - The request string sent. For this to work, add
the CURLINFO_HEADER_OUT option to the handle by calling curl_setopt()
If you are wanting the content can't you just log it? I am doing something similar for my API calls
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, self::$apiURL);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_POST, count($dataArray));
curl_setopt($ch, CURLOPT_POSTFIELDS, $dataString);
$logger->info("Sending " . $dataString);
self::$results = curl_exec($ch);
curl_close($ch);
$decoded = json_decode(self::$results);
$logger->debug("Received " . serialize($decoded));
Or try
curl_setopt($ch, CURLOPT_STDERR, $fp);
I would recommend using curl_getinfo.
<?php
curl_exec($ch);
$info = curl_getinfo($ch);
if ( !empty($info) && is_array($info) {
print_r( $info );
} else {
throw new Exception('Curl Info is empty or not an array');
};
?>