PHP - file get contents using CURL [duplicate] - php

This question already has answers here:
PHP CURL & HTTPS
(4 answers)
Closed 4 years ago.
i have a problem with curl, i trying to get content using CURL but return just null, i dont know what i miss, please tell me, here is my code :
$url = "https://shopee.co.id/api/v1/search_items/?by=pop&order=desc&keyword=yi 067&newest=0&limit=50&page_type=shop&match_id=16775174";
$html = file_get_contents_curl($url);
var_dump($html);
function file_get_contents_curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_ENCODING, 0);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST , "GET");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json'));
$data = curl_exec($ch);
curl_close($ch);
return $data;
}

With curl, it's often a good idea to use curl_getinfo to obtain additional information about your curl connection before closing it. In your case, the NULL/FALSE/empty result could be due to a number of reasons and inspecting the curl info might help you find more detail. Here's a modified version of your function that uses this function to obtain additional information. You might consider writing print_r($info, TRUE) to a log file or something. It might be empty because the server response is empty. It might be false because the url cannot be reached from your server due to firewall issues. It might be returning an http_code that is 404 NOT FOUND or 5XX.
function file_get_contents_curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_ENCODING, 0);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST , "GET");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
$data = curl_exec($ch);
$info = curl_getinfo($ch);
if(curl_errno($ch)) {
throw new Exception('Curl error: ' . curl_error($ch));
}
curl_close($ch);
if ($data === FALSE) {
throw new Exception("curl_exec returned FALSE. Info follows:\n" . print_r($info, TRUE));
}
return $data;
}
EDIT: I added curl_errno checking too.

Related

PHP curl_error doesn't return an error when using CURLOPT_RETURNTRANSFER

I am using curl in PHP to make a post request to a database. The request looks something like this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, getenv('DATABASE_URL'));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
curl_setopt($ch, CURLOPT_FRESH_CONNECT, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($data, '', '&'));
$output = curl_exec($ch)
where $data is defined earlier. The request uses a token as verification, and if the token is invalid, the above $output is the error message
{"error":"You do not have permissions to use the API"}
which is good. So I want to catch that error to handle it properly further down the program, so I use curl_error($ch) to do so. The problem is, this returns the empty string which according to the documentation corresponds to no errors. I have also tried the curl_errno to count the numbers, but this returns 0 errors.
Can anyone see my mistake(s)?

I can't get the content of a specific URL

I can't get the content of this URL (I get a blank page): https://www.euronews.com/api/watchlive.json
I have always used this function and never had problems:
function file_get_contents_curl($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 3);
curl_setopt($ch, CURLOPT_TIMEOUT, 3);
curl_setopt($ch, CURLOPT_REFERER, $referer);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_HEADER, 5);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$data = json_decode(file_get_contents_curl("https://www.euronews.com/api/watchlive.json"), true); $url = $data['url']; echo $url;
Any idea?
EDIT: Unknown problem caused (possibly) by cURL module in Raspbian (Raspberry)
I think you need to echo the function if you want to see the content. I tried this on my end and I got the content. althought I have some undefined variable error because of $referrer, $header, $useragent and $cookie. but it runs and get the result. here is how I call the function
echo file_get_contents_curl("https://www.euronews.com/api/watchlive.json");
You're getting more than you expect in your function response, so the json_decode is failing. You need to set CURLOPT_HEADER to false instead of 5.
curl_setopt($ch, CURLOPT_HEADER, FALSE);
You can verify what you're getting by doing
$response = file_get_contents_curl("https://www.euronews.com/api/watchlive.json");
var_dump($response);
$data = json_decode($response, true);
$url = $data['url'];
echo $url;

bad url never ending request curl php

I have a curl set up like this:
function file_get_contents_curl($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_TIMEOUT, 400);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch,CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
$data = curl_exec($ch);
return $data;
}
When I call it with the following:
$html = file_get_contents_curl("https://training.sap.com/service-id/module.php/core/as_login.php?AuthId=sf-sp&ReturnTo=%2Fsuccessfactors-community%2Fpermission-check");
You can see that link has an authentication, and I do not want to spend time on it. I just want that curl session to die after maybe 3 seconds or x seconds, or determine before hand if it is going to be a never ending loop on the curl for one reason such a SSL or password requirement etc.

Curl function does not return response

I want to call a Api. I have used Curl for this and made a library. When i call the Curl library function from controller, it returns NULL but if i print the response it displays the data. Can anyone help me?
My Curl Library code :
public function callApi($jwt_tkn, $url, $postdata, $method)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Authorization: Bearer ' . $jwt_tkn));
curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
curl_close($ch);
return $result;
}

get the value of an url response with curl

I am using PHP curl method to get a string type response. To create the request I use:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, $data);
$response = curl_exec($ch);
$err = curl_error($ch);
curl_close($ch);
if($response === false)
throw new Exception(__CLASS__."::".__FUNCTION__."_".$err);
return $response;
Why I always receive a bool(true) response instead of the string I echo from the other side?
Thanks
Since you already have
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
in your code. curl_exec should already returns the content of the page instead of a BOOL.
This is a snippet of a library I use. As pointed out this might not be needed but it helped me out once...
//The content - if true, will not download the contents
curl_setopt($ch, CURLOPT_NOBODY, false);
Also it seems to have some bugs related to CURLOPT_NOBODY (which might explain why you have this issue):
http://osdir.com/ml/web.curl.general/2005-07/msg00073.html
http://curl.haxx.se/mail/curlphp-2008-03/0072.html

Categories