I use this function (from here) with cURL in PHP:
<?php
function file_get_contents_curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
This works quite well - - for example, that way I can get the 'body' of a JSON. Like this:
<?php
$crossrefurl = 'https://api.crossref.org/works/10.1163/1871191X-13020004';
$obj = file_get_contents_curl($crossrefurl);
# convert to json
$obj = json_decode($obj, true);
What I don't know is: how can I get the response status code (given that my ´curl`-commands are in a function)?
Specifically, I want to find out whether the response status code is 429 in which case I would let the script sleep for 5 seconds (to avoid being rate-limited).
According to this post, the code should be something like:
<?php
if(($httpcode == 429)) { sleep(5); }
... but how do I get to $httpcode in the first place?
Thanks for your help!
Related
i am having trouble getting a JSON file to a php-array.
i got a json-file as response from an api (request done with curl)
and want to make an array out of it but it won't work.
Here is my code:
<?php
class modExpose{
public static function getFunction($id){
//In my code i am "preparing" the request here
// *********** cURL
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url.$qry_str);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
$response = curl_exec($ch);
curl_close($ch);
return $response;
}
}
$id = $_GET['id'];
$data = modExpose::getFunction($id);
$array = json_decode($data,true);
print_r($array);
?>
the print_r function only delivers: 1. (same does the var_dump() function).
I also tried adding html_entity_decode() but the problem still remains.
Thank's for helping!
That is probably because the return value of your curl_exec() call is true on success and that is all you are returning from your method.
If you want to get the data that was returned by the curl call, you need to set the CURLOPT_RETURNTRANSFER option:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url.$qry_str);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
// Return the result on success
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
// Now response will contain the results of your curl call
$response = curl_exec($ch);
Apart from that I assume you have checked the variables that seem to be undefined in your example code.
i write following code to get html data from url and its working for https site like Facebook but not working for Instagram only.
Instagram returns the blank
<?php
$url = 'https://www.instagram.com';
$returned_content = get_data($url);
print_r($returned_content)
/* gets the data from a URL */
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
?>
The Instagram will return only javascript, that can't be render by your browser because it uses dynamic path, so <script src='/path/file.js'> will try to get localhost/path/file.js instead of instagram.com/path/file.js and in this situation the localhost/path/file.js not will exist, so the page will be blank.
One solution is find a way to give the full HTML instead of the Javascript, in this case you can use the "User-Agent" to do this trick. You might know that JS not handle by the search-engine, so for this situation the Instagram (and many websites) give the page without JS that is supported by the bot.
So, add this:
curl_setopt($ch, CURLOPT_USERAGENT, "ABACHOBot");
The "ABACHOBot" is one Crawler. In this page you can found many others alternatives, like a "Baiduspider", "BecomeBot"...
You can use "generic" user-agent too, like "bot", "spider", "crawler" and probably will work too.
Here try this on
<?php
$url = 'https://www.instagram.com';
$returned_content = get_data($url);
print_r($returned_content);
/* gets the data from a URL */
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
//Update.................
curl_setopt($ch, CURLOPT_USERAGENT, 'spider');
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HEADER, false);
//....................................................
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
?>
You should pass
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false)
and other header info as above.
For more detail,Please see
http://stackoverflow.com/questions/4372710/php-curl-https
I'm trying to load json data from this url =
http://api.opencagedata.com/geocode/v1/json?query=48.84737%2C2.28605&pretty=1&no_annotations=1&no_dedupe=1&key=b61388b5a248b7cfcaa9579ed290485b
Using file_get_contents works with other json urls but this one is strange. It returns only "{" the first line. Strlen gives 1480 which is right.Substr(2,18) gives "documentation" which is right too. But still i can't echo the entire text. Maybe there's some way to read the text line by line and save it in another string ? The entire text is still fully loaded in the textfile
Here's the php code i tried
<?php
$url = file_get_contents("http://api.opencagedata.com/geocode/v1/json?query=48.84737%2C2.28605&pretty=1&no_annotations=1&no_dedupe=1&key=b61388b5a248b7cfcaa9579ed290485b");
$save = file_put_contents("filename.txt", $url);
echo $url;
?>
Also tried this function but still same.
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
You can get return value with json_decode.
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return json_decode($data,true);
}
I am trying to access the Toggl Reporting API.
I tried following in PHP with cURL, which connects to the API but gives the following error message: 'This method may not be used.' Any light on why this is the case would be useful as I'm very new to webservices. I may be missing something obvious or totally going the wrong way about it, so apologies if this is the case.
<?php
$userAgent = 'xxx';//username
$token = 'xxx';//token
$returned_content = get_data('https://toggl.com/reports/api/v2/summary?&workspace_id=[workspaceid]&since=2013-05-19&until=2013-05-20&user_agent=[username here]');
print_r($returned_content);
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_USERPWD, $token.':api_token');
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
?>
Edit: I tried a different approach. If I run the following code, I no longer receive any error messages, so the code seems to be executing but I can't print the response to the screen. Is there something specific I need to do to view the output other than print_r?(Toggl API returns JSON). Thanks.
$json = curl%20-v%20-u%[myapitoken]:api_token%20https://toggl.com/reports/api/v2/weekly?workspace_id=[id]&wsid=282507&since=2012-08-19&until=2013-09-20&user_agent=[user].json;
print_r($json);
Edit: Finally resolved! Code is as follows:
$workspace_id = '[id here]';
$user_agent = '[user agent here]'; // no spaces
$api_token = '[token here]';
$report_url = 'https://toggl.com/reports/api/v2/weekly?user_agent='.$user_agent.'&since=2013-08-01&until=2013-09-01';
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
curl_setopt($ch, CURLOPT_USERPWD, $api_token . ':api_token');
curl_setopt($ch, CURLOPT_URL, $report_url . '&workspace_id=' . $workspace_id);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
$result = curl_exec($ch);
$result = json_encode($result);
Hope this helps someone in the future!
As I understand, you are receiving this message because of CURLOPT_SSL_VERIFYPEER == FALSE.
Try to remove this string from the code:
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
Maybe I wrong, but I think with this option you are receiving "HTTP 501 Not Implemented" error from the Toggl server, which contains exactly the same message, "This method may not be used."
I have a Affiliate URL Like http://track.abc.com/?affid=1234
open this link will go to http://www.abc.com
now i want to execute the http://track.abc.com/?affid=1234 Using CURL
and now how i can Get http://www.abc.com
with Curl ?
If you want cURL to follow redirect headers from the responses it receives, you need to set that option with:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
You may also want to limit the number of redirects it follows using:
curl_setopt($ch, CURLOPT_MAXREDIRS, 3);
So you'd using something similar to this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://track.abc.com/?affid=1234");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_MAXREDIRS, 3);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$data = curl_exec($ch);
Edit: Question wasn't exactly clear but from the comment below, if you want to get the redirect location, you need to get the headers from cURL and parse them for the Location header:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://track.abc.com/?affid=1234");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, true);
$data = curl_exec($ch);
This will give you the headers returned by the server in $data, simply parse through them to get the location header and you'll get your result. This question shows you how to do that.
I wrote a function that will extract any header from a cURL header response.
function getHeader($headerString, $key) {
preg_match('#\s\b' . $key . '\b:\s.*\s#', $headerString, $header);
return substr($header[0], strlen($key) + 3, -2);
}
In this case, you're looking for the value of the header Location. I tested the function by retrieving headers from a TinyURL, that redirects to http://google.se, using cURL.
$url = "http://tinyurl.com/dtrkv";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($ch);
curl_close($ch);
$location = getHeader($data, 'Location');
var_dump($location);
Output from the var_dump.
string(16) "http://google.se"