Hi I'm trying to make this api call from wikipedia using this url, however it says it's null when I dump the variable. This function works for my other json api calls but not for this, I tested it in the broswer manually it gives me a result. Here is my attempt
$url = 'http://en.wikipedia.org/w/api.php?action=query&format=json&titles=Image:Romerolagus diazi (dispale) 001.jpg&prop=imageinfo&iiprop=url';
$result = apicall($url);
var_dump($result);
function apicall($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, 'MyBot/1.0 (http://www.mysite.com/)');
$result = curl_exec($ch);
if (!$result) {
exit('cURL Error: '.curl_error($ch));
}
$var = json_decode($result);
return $var;
}
urlencode() problem, modify like this
<?php
$url = 'http://en.wikipedia.org/w/api.php';
$titles = urlencode('Image:Romerolagus diazi (dispale) 001.jpg');
$queryStr = 'action=query&format=json&titles='.$titles.'&prop=imageinfo&iiprop=url';
$url = $url . '?' . $queryStr;
$result = apicall($url);
var_dump($result);
function apicall($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.172 Safari/537.22');
$result = curl_exec($ch);
if (!$result) {
exit('cURL Error: '.curl_error($ch));
}
var_dump($result);
$var = json_decode($result);
return $var;
}
You should consider using http_build_query() to build the URL:
$url = 'http://en.wikipedia.org/w/api.php?' . http_build_query(array(
'action' => 'query',
'format' => 'json',
'titles' => 'Image:Romerolagus diazi (dispale) 001.jpg',
'prop' => 'imageinfo',
'iiprop' => 'url',
));
Related
I am working on a project which needs to get the data from other webpage:
https://eth.ethfans.org/#/miner?0x2998850087633a4806191960c94ed535d97da598
I am trying to use the function cRUL:
<?php
$url = "https://eth.ethfans.org/#/miner?0x2998850087633a4806191960c94ed535d97da598";
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$contents = curl_exec($ch);
curl_close($ch);
echo $contents;
?>
However, I can only get the layout of the site, but I cannot get the data inside.
Can anyone help for this ?
Thanks in Advance.
Regards,
Alex
Use str_get_html to fetch the data from the layout:
$get_html = str_get_html($contents);
Example:
function check()
{
$url = "https://stackoverflow.com/questions/49248329/cannot-extract-the-data-from-the-website-using-php-curl";
$get_html = $this->get_curl($url);
#print_r($get_html); exit;
$get_html = str_get_html($get_html);
$fb = NULL;
foreach ($get_html->find('a') as $v) { // you can get what data from the layout
if(strpos($v->href, 'facebook'))
{
echo $fb = $v->href;
echo "\n";
break;
}
}
unset($get_html);
}
public function get_curl($url)
{
ob_start();
$ch = curl_init($url);
$headers = [
'Accept-Language: en-US,en;q=0.5',
'Cache-Control: no-cache',
'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:28.0) Gecko/20100101 Firefox/51.0',
];
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch,CURLOPT_URL, $url);
$response = curl_exec ($ch);
curl_close ($ch);
ob_end_flush();
return $response;
}
you're hitting the wrong url, the page you're hitting only contains the layout and the javascript required to fetch the actual data, then the javascript fetch the data from https://eth.ethfans.org/api/page/miner?value=2998850087633a4806191960c94ed535d97da598 , so, do as the javascript does, and fetch that url.
i am new to api so i may be completely wrong.
i was going through some docuementaion in github but could not find some answers so i am here
i want to pass url of these functions to api.php
after validating these key and secret.when i echo these data i get key, secret and url but how to get these details in php as its not a post and i cant use _post function to manipulate data based on url submitted and give the result
public function __construct($key = '', $secret = '', $timeout = 30,
$proxyParams = array()) {
$this->auth = array(
"auth" => array(
"api_key" => $key,
"api_secret" => $secret
)
);
$this->timeout = $timeout;
$this->proxyParams = $proxyParams;
}
public function url($opts = array()) {
$data = json_encode(array_merge($this->auth, $opts));
// echo $data;
$response = self::request($data, 'http://somesite.com/a/api.php', 'url');
return $response;
}
here is request function
private function request($data, $url, $type) {
$curl = curl_init();
if ($type === 'url') {
curl_setopt($curl, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json'
));
}
curl_setopt($curl, CURLOPT_URL, $url);
// Force continue-100 from server
curl_setopt($curl, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.85 Safari/537.36");
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
curl_setopt($curl, CURLOPT_FAILONERROR, 0);
curl_setopt($curl, CURLOPT_CAINFO, __DIR__ . "/cacert.pem");
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, 1);
curl_setopt($curl, CURLOPT_TIMEOUT, $this->timeout);
if (isset($this->proxyParams['proxy'])) {
curl_setopt($curl, CURLOPT_PROXY, $this->proxyParams['proxy']);
}
$response = json_decode(curl_exec($curl), true);
if ($response === null) {
$response = array (
"success" => false,
"error" => 'cURL Error: ' . curl_error($curl)
);
}
curl_close($curl);
return $response;
}
}
output of echo data is sufficient but its not post and i tried json_decode but nothing is coming to api.php
here is output of echo
{"auth":{"api_key":"be8fgdffgrfffrffc4b3","api_secret":"1b59fsfvfrgfrfvfb29d6e555a1b"},"url":"https:\/\/i.ndtvimg.com\/i\/2017-06\/modi-at-kochi-metro-station_650x400_81497685848.jpg","wait":true}
i tried these in api.php to get the data but nothing is working
$gggss['url'] = json_decode($data, true); //this returns an array
or
$gggss=$_POST['data'];
any help will be great
I think you are trying get urlencoded data, while your JSON string located in body of request. Try use this instead:
$entityBody = file_get_contents('php://input');
I am new to programming,
I need to extract the wikipedia content and put it into html.
//curl request returns json output via json_decode php function
function curl($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB5');
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
$search = $_GET["search"];
if (empty($search)) {
//term param not passed in url
exit;
} else {
//create url to use in curl call
$term = str_replace(" ", "_", $search);
$url = "https://en.wikipedia.org/w/api.php?action=opensearch&search=".$search."&limit=1&namespace=0&format=jsonfm";
$json = curl($url);
$data = json_decode($json, true);
$data = $data['parse']['wikitext']['*'];
}
so I basically want to reprint a wiki page but with my styles and do not know how to do.
Any ideas, Thanks
OK, before saying this is a duplicate just read a bit....
I have been trying to echo contents of URL that has allow_url_fopen disabled for HOURS now, I have tried every solution posted on stack overflow. EXAMPLE:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
$result = curl_exec($ch);
curl_close($ch);
Doesn't WORK
function curl_get_contents($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
Doesn't WORK
$url = "http://www.google.com";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$data = curl_exec($ch);
curl_close($ch);
echo $data;
Doesn't WORK
fopen("cookies.txt", "w");
$url="http://adfoc.us/1575051";
$ch = curl_init();
$header=array('GET /1575051 HTTP/1.1',
'Host: adfoc.us',
'Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language:en-US,en;q=0.8',
'Cache-Control:max-age=0',
'Connection:keep-alive',
'Host:adfoc.us',
'User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36',
);
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,0);
curl_setopt( $ch, CURLOPT_COOKIESESSION, true );
curl_setopt($ch,CURLOPT_COOKIEFILE,'cookies.txt');
curl_setopt($ch,CURLOPT_COOKIEJAR,'cookies.txt');
curl_setopt($ch,CURLOPT_HTTPHEADER,$header);
$result=curl_exec($ch);
curl_close($ch);
Doesn't WORK
// create the Gateway object
$gateway = new Gateway();
// set our url
$gateway->init($url);
// get the raw response, ignore errors
$response = $gateway->exec();
Doesn't WORK
$file = "http://www.example.com/my_page.php";
if (function_exists('curl_version'))
{
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $file);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
$content = curl_exec($curl);
curl_close($curl);
}
else if (file_get_contents(__FILE__) && ini_get('allow_url_fopen'))
{
$content = file_get_contents($file);
}
else
{
echo 'You have neither cUrl installed nor allow_url_fopen activated. Please setup one of those!';
}
This doesn't work.
The page I am trying to use file_get_contents on is not on my website. I am trying to use file_get_contents so i can make a simple API for the site owner by reading a page and checking if a certain word is present on the page.
But yeah if anyone has any suggestions PLEASE post below :)
You can check first weather the site is available or not for example a sample code
Code taken from here:
<?php
$cURL = curl_init('http://www.technofusions.com/');
curl_setopt ( $cURL , CURLOPT_RETURNTRANSFER , true );
// Follow any kind of redirection that are in the URL
curl_setopt ( $cURL , CURLOPT_FOLLOWLOCATION , true );
$result = curl_exec ( $cURL );
// Getting HTTP response code
$answer = curl_getinfo ( $cURL , CURLINFO_HTTP_CODE );
curl_close ( $cURL );
if ( $answer == ' 404 ' ) {
echo ' The site not found (ERROR 404)! ' ;
} else {
echo ' It looks like everything is working fine ... ' ;
}
?>
For a full answer you can got to this tutorial Curl IN PHP
I have this code to try and get the pagination links using php but the result is not quiet right. could any one help me.
what I get back is just a recurring instance of the first link.
<?php
include_once('simple_html_dom.php');
function dlPage($href) {
$curl = curl_init();
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_URL, $href);
curl_setopt($curl, CURLOPT_REFERER, $href);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.125 Safari/533.4");
$str = curl_exec($curl);
curl_close($curl);
// Create a DOM object
$dom = new simple_html_dom();
// Load HTML from a string
$dom->load($str);
$Next_Link = array();
foreach($dom->find('a[title=Next]') as $element){
$Next_Link[] = $element->href;
}
print_r($Next_Link);
$next_page_url = $Next_Link[0];
if($next_page_url !='') {
echo '<br>' . $next_page_url;
$dom->clear();
unset($dom);
//load the next page from the pagination to collect the next link
dlPage($next_page_url);
}
}
$url = 'https://www.jumia.com.gh/phones/';
$data = dlPage($url);
//print_r($data)
?>
what i want to get is
mySiteUrl/?facet_is_mpg_child=0&viewType=gridView&page=2
mySiteUrl//?facet_is_mpg_child=0&viewType=gridView&page=3
.
.
.
to the last link in the pagination. Please help
Here it is. Look that I htmlspecialchars_decode the link. Cause the href in curl there shouldn't be an & like in xml. Should the return value of dlPage the last link in Pagination. I understood so.
<?php
include_once('simple_html_dom.php');
function dlPage($href, $already_loaded = array()) {
$curl = curl_init();
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_URL, $href);
curl_setopt($curl, CURLOPT_REFERER, $href);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.125 Safari/533.4");
$htmlPage = curl_exec($curl);
curl_close($curl);
echo "Loading From URL:" . $href . "<br/>\n";
$already_loaded[$href] = true;
// Create a DOM object
$dom = file_get_html($href);
// Load HTML from a string
$dom->load($htmlPage);
$next_page_url = null;
$items = $dom->find('ul[class="osh-pagination"] li[class="item"] a[title="Next"]');
foreach ($items as $item) {
$link = htmlspecialchars_decode($item->href);
if (!isset($already_loaded[$link])) {
$next_page_url = $link;
break;
}
}
if ($next_page_url !== null) {
$dom->clear();
unset($dom);
//load the next page from the pagination to collect the next link
return dlPage($next_page_url, $already_loaded);
}
return $href;
}
$url = 'https://www.jumia.com.gh/phones/';
$data = dlPage($url);
echo "DATA:" . $data . "\n";
And the output is:
Loading From URL:https://www.jumia.com.gh/phones/<br/>
Loading From URL:https://www.jumia.com.gh/phones/?facet_is_mpg_child=0&viewType=gridView&page=2<br/>
Loading From URL:https://www.jumia.com.gh/phones/?facet_is_mpg_child=0&viewType=gridView&page=3<br/>
Loading From URL:https://www.jumia.com.gh/phones/?facet_is_mpg_child=0&viewType=gridView&page=4<br/>
Loading From URL:https://www.jumia.com.gh/phones/?facet_is_mpg_child=0&viewType=gridView&page=5<br/>
DATA:https://www.jumia.com.gh/phones/?facet_is_mpg_child=0&viewType=gridView&page=5