I cannot get html output, what could be the problem? the data coming as label, not an html!
function curl_get_contents($url)
{
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$url = 'https://www.cosmetica.com.tr/tr/arama-sonuclari-8697888010218';
$json = curl_get_contents($url);
preg_match_all('#<a ng-href="(.*?)>#si',$json,$source);
print_r($source[0][0]);
html output: blabla.com.tr/{{slugify(category.name)}}-c-{{category.id}}
links label coming : e.g. {{slugify(subCategory.name)}}
but the links look html e.g. blabla.com//tr/p-art-net-far-karistirma-fircasi-no25-31770
Related
In this script curl is working fine. $output has the page but when I try to echo its source code it displays nothing. I think it has something to do with page headers. Any help!!
header('content-type:text/plain');
$ch = curl_init();
$url = "https://www.amazon.com/gp/css/shiptrack/view.html/ref=TE_SIMP_typ?ie=UTF8&addressID=olqnkqjstjp&latestArrivalDate=1483848000&orderID=103-0753412-5410653&shipmentDate=1483706902&orderingShipmentId=4160817392100&packageId=1";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_BINARYTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 1);
$output = curl_exec($ch);
curl_close($ch);
echo $output;
I have modified your code..
Try this.
Remove
header('content-type:text/plain');
and replace your code with mine.
$ch = curl_init();
$url = "https://www.amazon.com/gp/css/shiptrack/view.html/ref=TE_SIMP_typ?ie=UTF8&addressID=olqnkqjstjp&latestArrivalDate=1483848000&orderID=103-0753412-5410653&shipmentDate=1483706902&orderingShipmentId=4160817392100&packageId=1";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 1);
$output = curl_exec($ch);
curl_close($ch);
echo $output;
EDIT
Since you want ctrl+u
Here is the PHP code to fetch the html source code of any website specified. fopen function is used to open the website URL. stream_get_contents is used to read the opened URL. The fetched code is displayed inside a textarea.
$domain = 'https://www.amazon.com/gp/css/shiptrack/view.html/ref=TE_SIMP_typ?ie=UTF8&addressID=olqnkqjstjp&latestArrivalDate=1483848000&orderID=103-0753412-5410653&shipmentDate=1483706902&orderingShipmentId=4160817392100&packageId=1';
$handle = fopen($domain, 'r');
$content = stream_get_contents($handle);
/**
// alternative to stream_get_contents
$contents = '';
while (!feof($handle)) {
$content .= fread($handle, 8192);
}
**/
fclose($handle);
/**
* print html content on textarea
*/
echo "<textarea rows='20' cols='80'>$content</textarea>";
output will be like this:
OR
Also if you want to manipulate the retrieved page somehow, you might
want to try some php DOM parser. I find PHP Simple HTML DOM
Parser very easy to use.
I used several method to get html content of aptoide.com in php.
1) file_get_contents();
2) readfile();
3) curl as php function
function get_dataa($url) {
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (compatible; Konqueror/4.0; Microsoft Windows) KHTML/4.0.80 (like Gecko)");
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
4)PHP Simple HTML DOM Parser
include_once('simple_html_dom.php');
$url="http://aptoide.com";
$html = file_get_html($url);
But all of them give empty output for aptoide.com
Is there a way to get full html content of that url ?
echo file_get_contents('http://www.aptoide.com/'); works fine for me.
So it's possible that aptoide.com has been blocked you. If you want to change your IP (as you said in comment) you have to use this:
$url = 'http://aptoide.com.com/';
$proxy = '127.0.0.1:9095'; // Your proxy
// $proxyauth = 'user:password'; // Proxy authentication if required
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
//curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
use your curl get_dataa function with this line added:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
because that page is redirecting to www.aptide.com
full function:
function get_dataa($url) {
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (compatible; Konqueror/4.0; Microsoft Windows) KHTML/4.0.80 (like Gecko)");
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
I run below code and it worked, but I see blank screen when I try with this URL : https://www.reddit.com/r/funny/comments/3aguyp/i_told_my_boyfriend_the_new_puppy_wasnt_allowed/
<?php include('simple_html_dom.php');
$html = getSslPage('https://www.reddit.com/r/pics/comments/3a53qw/grandmother_mother_and_daughter/');
function getSslPage($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
$html = str_get_html($html);
echo $html;
The 2 page have the same DOM structure.
Below I have a piece of code to return some information through an API.
$page = 'http://api.duedil.com/sandbox/v2/company/03977902.json? fields=get_all&api_key=***********';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $page);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);
This prints out the following results http://www.visitrack.co.uk/testdata.php
what i am trying to do is split the results in to individual variables.
Thanks
Something like this should do the trick:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $page);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // forces curl_exec() to return page as a string
$json = curl_exec($ch);
curl_close($ch);
$data = json_decode($json, TRUE); // parse json string to array
print_r($data);
I use file_get_contents function to get and show external links on my specific page.
In my local file everything is okay, but my server doesn't support the file_get_contents function, so I tried to use cURL with the below code:
function file_get_contents_curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
echo file_get_contents_curl('http://google.com');
But it returns a blank page. What is wrong?
try this:
function file_get_contents_curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
This should work
function curl_load($url){
curl_setopt($ch=curl_init(), CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
curl_close($ch);
return $response;
}
$url = "http://www.google.com";
echo curl_load($url);
I encountered such a problem accessing Google Drive content via the direct link.
After calling file_get_contents returned 302 Moved temporarily
//Any google url. This example is fake for Google Drive direct link.
$url = "https://drive.google.com/uc?id=0BxQKKJYjuNElbFBNUlBndmVHHAj";
$html = file_get_contents($url);
echo $html; //print none because error 302.
With the code below it worked again:
//Any google url. This example is fake for Google Drive direct link.
$url = "https://drive.google.com/uc?id=0BxQKKJYjuNElbFBNUlBndmVHHAj";
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 3);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
$html = curl_exec($ch);
curl_close($ch);
echo $html;
I tested it today, 03/19/2018
//You can try this . It should work fine.
function curl_tt($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 3);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
echo curl_tt("https://google.com");