Option of Curl return - php

I'm trying to get the URL stream (.m3u8) from here:
https://chch.cdn.clearcable.net/
I tried with cURL:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://chch.cdn.clearcable.net/');
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
$res = curl_exec($ch);
echo $res;
However the URL stream is not present in the code.
Do I need some extra function?
Thank you in advance.

You need to tell curl to return the data instead of output it. If you use this setting:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
The $res should contain the data you expect. I can confirm this works for me. If it's not for you, try this, after thet call to curl_exec:
if($res === false) {
echo 'Curl error: ' . curl_error($ch);
} else {
echo $res;
}
When running on the web, as you mentioned, you will need to view the source to see any code that is parsed by the browser. Or you can use methods like print_r or var_dump to view it on the page.

You will need to check if its returning any value using print_r() function
Try code below
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://chch.cdn.clearcable.net/");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$res = curl_exec($ch);
$post = json_decode($res ,true);
print_r($post);
curl_close($ch);

You can simply use file_get_contents() to stream the entire file into a string. But using it alone will generate an error: SSL operation failed with code 1..
This link describes why.
Try this code:
// Create a stream
$opts = [
"ssl" => [
"verify_peer" => false,
"verify_peer_name" => false
]
];
$context = stream_context_create($opts);
$res = file_get_contents("https://chch.cdn.clearcable.net/", false, $context);
echo $res;

Related

PHP Curl Post and Get Response

I have a shell code as below."curl -X POST -F 'file=#textomate-api.pdf;type=text/plain' https://textomate.com/a/uploadMultiple"
This code send a file to the URL and get the response below.
{
"countedFiles": [
{
"wordsNumber": 340,
"charsNumber": 2908,
"charsNoSpacesNumber": 2506,
"wordsNumberNN": 312,
"charsNumberNN": 2755,
"charsNoSpacesNumberNN": 2353,
"md5": null,
"fileName": "textomate-api.pdf",
"error": null
}
],
"total": {
"wordsNumber": 340,
"charsNumber": 2908,
"charsNoSpacesNumber": 2506,
"wordsNumberNN": 312,
"charsNumberNN": 2755,
"charsNoSpacesNumberNN": 2353,
"md5": null
}
}
And I want to post a file via PHP and get this response or only value of "charsNoSpacesNumber".
Could you please help me with this?
Thanks a lot
You can do it as follows:
YET be sure to first check their T&C as I am not sure if they provide such a service for free.
Also be sure to include some error / exceptions handling.
<?php
//Initialise the cURL var
$ch = curl_init();
//Get the response from cURL
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//Set the Url
curl_setopt($ch, CURLOPT_URL, 'https://textomate.com/a/uploadMultiple');
$path = '/path/to/your/file.pdf';
$file = curl_file_create($path, 'application/pdf', 'file');
//Create a POST array with the file in it
$postData = array(
'file' => $file,
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
// Execute the request, decode the json into array
$response = json_decode(curl_exec($ch), true);
var_dump($response['total']['charsNoSpacesNumber']);
Thanks for your support Bartosz.
This is my code and I also shared an image of the code and results.
I hope there are enough information.
<?php
if(is_callable('curl_init')){
echo "curl is active";
} else {
echo "curl is passive";
}
//Initialise the cURL var
$ch = curl_init();
//Get the response from cURL
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//Set the Url
curl_setopt($ch, CURLOPT_URL, 'https://textomate.com/a/uploadMultiple');
$path = 'textomate-api.pdf';
$file = curl_file_create($path, 'application/pdf', 'file');
//Create a POST array with the file in it
$postData = array(
'file' => $file,
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
// Execute the request, decode the json into array
$response = json_decode(curl_exec($ch), true);
var_dump($response['total']['charsNoSpacesNumber']);
var_dump($response);
var_dump(file_exists($path));
?>
Updated code is below and you can see the results in the image.
I want to use this API on my website to be able to calculate character count of documents. I am using Wordpress and API provider gives us 3 options, Java, PHP (Wordpress) and Shell. They have something for Wordpress but I don't know how to use it.
You can reach all the files from here.
https://github.com/zentaly/textomate-api
If you take a look there, you can get more information and maybe you can find me a better solution.
And again thank you so much for your support, I appreciate it.
<?php
if(is_callable('curl_init')){
echo "curl is active";
} else {
echo "curl is passive";
}
//Initialise the cURL var
$ch = curl_init();
//Get the response from cURL
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//Set the Url
curl_setopt($ch, CURLOPT_URL, 'https://textomate.com/a/uploadMultiple');
curl_setopt($ch, CURLOPT_VERBOSE, 2);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_exec: var_dump(curl_errno($ch));
var_dump(curl_error($ch));
$path = 'textomate-api.pdf';
$file = curl_file_create($path, 'application/pdf', 'file');
//Create a POST array with the file in it
$postData = array(
'file' => $file,
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
// Execute the request, decode the json into array
$response = curl_exec($ch);
var_dump($response);
$response = json_decode($ch);
var_dump($response['total']['charsNoSpacesNumber']);
var_dump($response);
var_dump(file_exists($path));
var_dump($file);
?>

PHP & CURL scraping

I have a problem when I run this script in Google Chrome I got a blank page. When I use another link of a web site, it works successfully. I do not what is happening.
$curl = curl_init();
$url = "https://www.danmurphys.com.au/dm/home";
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec($curl);
echo $output;
There are some conditions which make your result blank. Such as:
Curl error.
Redirection without response body and the curl doesn't follow the redirection.
The target host doesn't give any response body.
So here you have to find out the problem.
For the first possibility, use curl_error and curl_errno to confirm that the curl wasn't errored when its runtime.
For the second, use CURLOPT_FOLLOWLOCATION option to make sure the curl follows the redirection.
For the third possibility, we can use curl_getinfo. It returns an array which contains "size_download". The size_download shows you the length of the response body. If it is zero that is why you see a blank page when printing it.
One more, try to use var_dump to see the output (debug purpose only). There is a possibility where the curl_exec returns bool false or null. If you print the bool false or null it will show a blank.
Here is the example to use all of them.
<?php
$curl = curl_init();
$url = "https://www.danmurphys.com.au/dm/home";
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
$output = curl_exec($curl);
$info = curl_getinfo($curl);
$err = curl_error($curl);
$ern = curl_errno($curl);
if ($ern) {
printf("An error occurred: (%d) %s\n", $ern, $err);
exit(1);
}
curl_close($curl);
printf("Response body size: %d\n", $info["size_download"]);
// Debug only.
// var_dump($output);
echo $output;
Hope this can help you.
Update:
You can use CURLOPT_VERBOSE to see the request and response information in details.
Just add this
curl_setopt($curl, CURLOPT_VERBOSE, true);
It doesn't need to be printed, the curl will print it for you during runtime.

How to extract data from a decoded JSON object in php

I wanted to try to get data from a JSON string which is loaded from another page. I currently have used Curl to get the data from the webpage but I can't acces the data in it.
I've already tried:
var_dump(json_decode($result->version, true));
var_dump(json_decode($result[3][0]["date"], true));
But this does't seem to work as it always returns NULL
$url="https://roosters.deltion.nl/api/roster?group=AO2B&start=20160125&end=20160201";
// Initiate curl
$ch = curl_init();
// Disable SSL verification
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
// Will return the response, if false it print the response
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// Set the url
curl_setopt($ch, CURLOPT_URL,$url);
// Execute
$result=curl_exec($ch);
// Closing
curl_close($ch);
// Will dump a beauty json :3
var_dump(json_decode($result, true));
First decode the JSON, then get the properties you want. Like this:
$yourObject = json_decode($result);
var_dump($youObject->version);
this is working for me.
<?php
$url = "https://roosters.deltion.nl/api/roster?group=AO2B&start=20160125&end=20160201";
// Initiate curl
$ch = curl_init();
// Disable SSL verification
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
// Will return the response, if false it print the response
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// Set the url
curl_setopt($ch, CURLOPT_URL, $url);
// Execute
$result = curl_exec($ch);
// Closing
curl_close($ch);
// Will dump a beauty json :3
$data = json_decode($result);
//echo $data->data[0]['date'];
echo "<pre>";
print_r($data->data[0]->date);
}
?>
if you want to get date of all index then try this in loop.
Firstly if your using GET there is no need to use CURL,
$result = file_get_contents(https://roosters.deltion.nl/api/roster?group=AO2B&start=20160125&end=20160201);
Will work just as well without any of the overhead. I suspect that your CURL isn't returning the page content so using file_get_contents() will fix it.

Warning: file_get_contents: failed to open stream: Redirection limit reached, aborting

I read over 20 related questions on this site, searched in Google but no use. I'm new to PHP and am using PHP Simple HTML DOM Parser to fetch a URL. While this script works with local test pages, it just won't work with the URL that I need the script for.
Here is the code that I wrote for this, following an example file that came with the PHP Simple DOM parser library:
<?php
include('simple_html_dom.php');
$html = file_get_html('http://www.farmersagent.com/Results.aspx?isa=1&name=A&csz=AL');
foreach($html->find('li.name ul#generalListing') as $e)
echo $e->plaintext;
?>
And this is the error message that I get:
Warning: file_get_contents(http://www.farmersagent.com/Results.aspx?isa=1&name=A&csz=AL) [function.file-get-contents]: failed to open stream: Redirection limit reached, aborting in /home/content/html/website.in/test/simple_html_dom.php on line 70
Please guide me what should be done to make it work. I'm new so please suggest a way that is simple. While reading other questions and their answers on this site, I tried cURL method to create a handle but I failed to make it work. The cURL method that I tried keeps returning "Resources" or "Objects". I don't know how to pass that to Simple HTML DOM Parser to make $html->find() work properly.
Please help!
Thanks!
Had a similar problem today. I was using CURL and it wasn't returning my any error. Tested with file_get_contents() and I got...
failed to open stream: Redirection limit reached, aborting in
Made a few searches and I'v ended with this function that works on my case...
function getPage ($url) {
$useragent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36';
$timeout= 120;
$dir = dirname(__FILE__);
$cookie_file = $dir . '/cookies/' . md5($_SERVER['REMOTE_ADDR']) . '.txt';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt($ch, CURLOPT_ENCODING, "" );
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt($ch, CURLOPT_AUTOREFERER, true );
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout );
curl_setopt($ch, CURLOPT_TIMEOUT, $timeout );
curl_setopt($ch, CURLOPT_MAXREDIRS, 10 );
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_REFERER, 'http://www.google.com/');
$content = curl_exec($ch);
if(curl_errno($ch))
{
echo 'error:' . curl_error($ch);
}
else
{
return $content;
}
curl_close($ch);
}
The website was checking for a valid user agent and for cookies.
The cookie issue was causing it! :)
Peace!
Resolved with:
<?php
$context = stream_context_create(
array(
'http' => array(
'max_redirects' => 101
)
)
);
$content = file_get_contents('http://example.org/', false, $context);
?>
You can also inform if you have a proxy in the middle:
$aContext = array('http'=>array('proxy'=>$proxy,'request_fulluri'=>true));
$cxContext = stream_context_create($aContext);
More details on: https://cweiske.de/tagebuch/php-redirection-limit-reached.htm (thanks #jqpATs2w)
Using cURL you would need to have the CURLOPT_RETURNTRANSFER option set to true in order to return the body of the request with call to curl_exec like this:
$url = 'http://www.farmersagent.com/Results.aspx?isa=1&name=A&csz=AL';
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
// you may set this options if you need to follow redirects. Though I didn't get any in your case
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
$content = curl_exec($curl);
curl_close($curl);
$html = str_get_html($content);
I also needed to add this HTTP context options ignore_errors :
see : https://www.php.net/manual/en/context.http.php
$arrContextOptions = array(
"ssl" => array(
// skip error "Failed to enable crypto" + "SSL operation failed with code 1."
"verify_peer" => false,
"verify_peer_name" => false,
),
// skyp error "failed to open stream: operation failed" + "Redirection limit reached"
'http' => array(
'max_redirects' => 101,
'ignore_errors' => '1'
),
);
$file = file_get_contents($file_url, false, stream_context_create($arrContextOptions));
Obviously, I only use it for quick debugging purpose on my local environment. It is not for production.
I'm not sure exactly why you redefined the $html object with a string from get html, The object is meant to be used for searching the string. If you overwrite the object with a string, the object no longer exists and cannot be used.
In any case, to search the string returned from curl.
<?php
$url = 'http://www.example.com/Results.aspx?isa=1&name=A&csz=AL';
include('simple_html_dom.php');
# create object
$html = new simple_html_dom();
#### CURL BLOCK ####
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
# you may set this options if you need to follow redirects.
# Though I didn't get any in your case
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
$content = curl_exec($curl);
curl_close($curl);
# note the variable change.
$string = str_get_html($content);
# load the curl string into the object.
$html->load($string);
#### END CURL BLOCK ####
# without the curl block above you would just use this.
$html->load_file($url);
# choose the tag to find, you're not looking for attributes here.
$html->find('a');
# this is looking for anchor tags in the given string.
# you output the attributes contents using the name of the attribute.
echo $html->href;
?>
you might be searching a different tag, the method is the same
# just outputting a different tag attribute
echo $html->class;
echo $html->id;

Getting multiple http response codes using cURL?

Please take a look at this sample code:
function http_response($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_NOBODY, TRUE); // remove body
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$head = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
echo $httpCode ;
}
this code will print the httpCode of the given url. I have couple of questions:
Can I get rid of some setopt() lines here and still getting httpCode?
What about if I want to check multiple urls at the same time? Can I modify the code to do that?
Can I do the same functionality in a simpler way using libraries different than cURL?
Thanks :)
You should be able to remove CURLOPT_HEADER and CURLOPT_NOBODY and still get the same result.
You could do that like this:
$urls = array(
'http://google.com',
'http://facebook.com'
);
$status = array();
foreach($urls as $url){
$status[$url] = http_response($url);
}
Try print_r($status); after this and you'll see the result.
You could do this with file_get_contents and $http_response_header, to learn more: http://www.php.net/manual/en/reserved.variables.httpresponseheader.php I would however recommend using cURL anyway.
*2. to check multiple urls you have to use this function in a loop, in any programming language 1 response from a server = 1 connection to that server. If you want to use 1 function to get responses from multiple servers you can always pass an array to the function and do the loop inside the function
*3. you can try this way:
function get_contents() {
file_get_contents("http://example.com");
var_dump($http_response_header);
}
get_contents();

Categories