Grab redirect Url with file_get_content() - php

Having some trouble to grabbing redirect link with file_get_content I used these codes: the $link has a couple urls.
foreach ($link as $site) {
if (strpos($site, 'https://www.mecsumai.com/') === 0) {
$dom = getSiteContent($site);
$div = $dom->getElementsByTagName('iframe');
if ($div->length > 0) {
$iframeLink[] = $div[0]->getAttribute('src');
}
}
}
I grabbed here iframe urls which is like this:
https://www.mecsumai.com/bkn-summary?bid=16078001&bg1=999999&bg2=dddddd&wdt=100%&aln=center
But when try to enter this with file_get_content. I can't get the content of page. Because it will redirecting to this, when you try to enter:
https://www.mecsumai.com/bkn-summary/?bid=16078001&bg1=999999&bg2=dddddd&wdt=100%25&aln=center&newid=06078001
Last newid=line... is redirecting part. So I try this.
foreach($iframeLink as $results){
$opts = array('https' =>
array(
'follow_location' => 1,
)
);
$context = stream_context_create($opts);
$lastUrl[] = file_get_contents($results, false, $context);
}
print_r($lastUrl);
I'm getting no error, also getting the page content but not the whole of this page. The table is not coming?
Any idea to solve this?

I am not sure if file_get_contents allow such option, however you can use curl which does.
Here is an example.
function curl_get_contents($url) {
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
$data = curl_exec($curl);
curl_close($curl);
return $data;
}

Related

Option of Curl return

I'm trying to get the URL stream (.m3u8) from here:
https://chch.cdn.clearcable.net/
I tried with cURL:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://chch.cdn.clearcable.net/');
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
$res = curl_exec($ch);
echo $res;
However the URL stream is not present in the code.
Do I need some extra function?
Thank you in advance.
You need to tell curl to return the data instead of output it. If you use this setting:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
The $res should contain the data you expect. I can confirm this works for me. If it's not for you, try this, after thet call to curl_exec:
if($res === false) {
echo 'Curl error: ' . curl_error($ch);
} else {
echo $res;
}
When running on the web, as you mentioned, you will need to view the source to see any code that is parsed by the browser. Or you can use methods like print_r or var_dump to view it on the page.
You will need to check if its returning any value using print_r() function
Try code below
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://chch.cdn.clearcable.net/");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$res = curl_exec($ch);
$post = json_decode($res ,true);
print_r($post);
curl_close($ch);
You can simply use file_get_contents() to stream the entire file into a string. But using it alone will generate an error: SSL operation failed with code 1..
This link describes why.
Try this code:
// Create a stream
$opts = [
"ssl" => [
"verify_peer" => false,
"verify_peer_name" => false
]
];
$context = stream_context_create($opts);
$res = file_get_contents("https://chch.cdn.clearcable.net/", false, $context);
echo $res;

Redirect php script using CURL

I am trying to make a redirect php script, I want that script to check if the link exist and then redirect the user to the link, if it doesn't exist then it will get the next link and so on, but for some reason is not working, maybe you could give me some help on this.
<?php
$URL = 'http://www.site1.com';
$URL = 'http://www.site2.com';
$URL = 'http://www.site3.com';
$handlerr = curl_init($URL);
curl_setopt($handlerr, CURLOPT_RETURNTRANSFER, TRUE);
$resp = curl_exec($handlerr);
$ht = curl_getinfo($handlerr, CURLINFO_HTTP_CODE);
if ($ht == '404')
{ echo "Sorry the website is down atm, please come back later!";}
else { header('Location: '. $URL);}
?>
You are overwriting your $URL variable..
$URL = 'http://www.site1.com';
$URL = 'http://www.site2.com';
$URL = 'http://www.site3.com';
Put these urls in an array and go through it with a for each loop.
You have a few issues in your code. For 1, your $URL will overwrite itself, resulting in only 1 url in there. It needs to be an array:
array( 'http://www.site1.com', 'http://www.site2.com', 'http://www.site3.com' );
You can get many responses, not just a 404, so you should tell cURL to follow redirects. If the URL was a redirect itself, could get a 301 that redirects to a 200. So we want to follow that.
Try This:
<?php
function curlGet($url)
{
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$output = curl_exec($ch);
$httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if ( $httpcode == 200 ) {
return true;
}
return false;
}
$urlArray = array( 'http://www.site1.com', 'http://www.site2.com', 'http://www.site3.com' );
foreach ( $urlArray as $url ) {
if ( $result = curlGet($url) ) {
header('Location: ' . $url);
exit;
}
}
// if we made it here, we looped through every url
// and none of them worked
echo "No valid URLs found...";
http://php.net/manual/en/function.file-exists.php#74469
<?php
function url_exists($url) {
if (!$fp = curl_init($url)) return false;
return true;
}
?>
This will give you the url exists check.
to check multiple urls though, you need an array:
<?
$url_array = [];
$url_array[] = 'http://www.site1.com';
$url_array[] = 'http://www.site2.com';
$url_array[] = 'http://www.site3.com';
foreach ($url_array as $url) {
if url_exists($url){
// do what you need;
break;
}
}
?>
PS - this is completely untested, but should theoretically do what you need.

Detect if the url is redirecting before iframing it

i am building an app and i am stuck at some point, i am having an iframe that it's source is dynamically determined according to :
<?php
ini_set('display_errors','off');
$file = $_GET['file'];
$url = ('remotedomain. com/' .$file);
?>
now here is the iframe:
<iframe id="embed" src="<? echo $url; ?>"></iframe>
the point i am stuck at is sometime the url(that's the iframe source) is redirecting back to home page, i want to detect this redirect and if it will happen then the source of the iframe should be certain url for example mysite .com/page1.html
Use curl and the option CURLOPT_HEADER. It's something like this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.example.com/"); //your url
curl_setopt($CURL, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
if (preg_match('#HTTP.{4} 30[34]#m', $result)) {
if (preg_match('#\vLocation: (.*)\v#m', $result, $matches)) {
echo "Found redirect: ", $matches[1], "\n";
}
}
You can instead use Guzzle, it will follow redirects.
$response = $client->get('http://github.com');
echo $response->getStatusCode();
// 200
echo $response->getEffectiveUrl();
// 'https://github.com/'
getEffectiveUrl(); will here print where it redirected to.
Something like this:
function getFinalUrl($url) //in your library
{
$client = new GuzzleHttp\Client();
$response = $client->get($url);
if (in_array($response->getStatusCode(), [303, 304], true)) {
$url = $response->getEffectiveUrl();
}
return $url;
}
And iframe:
<iframe id="embed" src="<?=getFinalUrl($url);?>"></iframe>
thanks every one who discussed this with me i have managed to do and here is the answer so if anyone need it:
$url = (remote site .com/' .$file);
$start_time = microtime(TRUE);
$handle = curl_init($url);
curl_setopt($handle, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($handle, CURLOPT_NOBODY, 1);
$response = curl_exec($handle);
$httpCode = curl_getinfo($handle, CURLINFO_HTTP_CODE);
if($httpCode == 302) {
$final_url = 'mysite .com/page1.php';
}
else {
$final_url = $url;
}
You can lookup the current URL of an iframe with the following javascript:
document.getElementById("iframe_id").contentWindow.location.href
You can compare this url to that of the homepage you want to avoid. If they match redirect them to your alternate URL

Get Final URL From Double Shortened URL (t.co -> bit.ly -> final)

I couldn't convert a double shortened URL to expanded URL successfully using the below function I got from here:
function doShortURLDecode($url) {
$ch = #curl_init($url);
#curl_setopt($ch, CURLOPT_HEADER, TRUE);
#curl_setopt($ch, CURLOPT_NOBODY, TRUE);
#curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
#curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$response = #curl_exec($ch);
preg_match('/Location: (.*)\n/', $response, $a);
if (!isset($a[1])) return $url;
return $a[1];
}
I got into trouble when the expanded URL I got was again a shortened URL, which has its expanded URL.
How do I get final expanded URL after it has run through both URL shortening services?
Since t.co uses HTML redirection through the use of JavaScript and/or a <meta> redirect we need to grab it's contents first. Then extract the bit.ly URL from it to perform a HTTP header request to get the final location. This method does not rely on cURL to be enabled on server and uses all native PHP5 functions:
Tested and working!
function large_url($url)
{
$data = file_get_contents($url); // t.co uses HTML redirection
$url = strtok(strstr($data, 'http://bit.ly/'), '"'); // grab bit.ly URL
stream_context_set_default(array('http' => array('method' => 'HEAD')));
$headers = get_headers($url, 1); // get HTTP headers
return (isset($headers['Location'])) // check if Location header set
? $headers['Location'] // return Location header value
: $url; // return bit.ly URL instead
}
// DEMO
$url = 'http://t.co/dd4b3kOz';
echo large_url($url);
Finally found a way to get the final url of a double shortened url. The best way is to use longurl api for it.
I am not sure if it is the correct way, but i am at last getting the output as the final url needed :)
Here's what i did:
<?php
function TextAfterTag($input, $tag)
{
$result = '';
$tagPos = strpos($input, $tag);
if (!($tagPos === false))
{
$length = strlen($input);
$substrLength = $length - $tagPos + 1;
$result = substr($input, $tagPos + 1, $substrLength);
}
return trim($result);
}
function expandUrlLongApi($url)
{
$format = 'json';
$api_query = "http://api.longurl.org/v2/expand?" .
"url={$url}&response-code=1&format={$format}";
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $api_query );
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, 0);
curl_setopt($ch, CURLOPT_HEADER, false);
$fileContents = curl_exec($ch);
curl_close($ch);
$s1=str_replace("{"," ","$fileContents");
$s2=str_replace("}"," ","$s1");
$s2=trim($s2);
$s3=array();
$s3=explode(",",$s2);
$s4=TextAfterTag($s3[0],(':'));
$s4=stripslashes($s4);
return $s4;
}
echo expandUrlLongApi('http://t.co/dd4b3kOz');
?>
The output i get is:
"http://changeordie.therepublik.net/?p=371#proliferation"
The above code works.
The code that #cryptic shared is also correct ,but i could not get the result on my server (maybe because of some configuration issue).
If anyone thinks that it could be done by some other way, please feel free to share it.
Perhaps you should just use CURLOPT_FOLLOWLOCATION = true and then determine the final URL you were directed to.
In case the problem is not a Javascript redirect as in t.co or a <META http-equiv="refresh"..., this is reslolving stackexchange URLs like https://stackoverflow.com/q/62317 fine:
public function doShortURLDecode($url) {
$ch = #curl_init($url);
#curl_setopt($ch, CURLOPT_HEADER, TRUE);
#curl_setopt($ch, CURLOPT_NOBODY, TRUE);
#curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
#curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$response = #curl_exec($ch);
$cleanresponse= preg_replace('/[^A-Za-z0-9\- _,.:\n\/]/', '', $response);
preg_match('/Location: (.*)[\n\r]/', $cleanresponse, $a);
if (!isset($a[1])) return $url;
return parse_url($url, PHP_URL_SCHEME).'://'.parse_url($url, PHP_URL_HOST).$a[1];
}
It cleans the response of any special characters, that can occur in the curl output before cuttoing out the result URL (I ran into this problem on a php7.3 server)

How to get Google +1 count for current page in PHP?

I want to get count of Google +1s for current web page ? I want to do this process in PHP, then write number of shares or +1s to database. That's why, I need it. So, How can I do this process (getting count of +1s) in PHP ?
Thanks in advance.
This one works for me and is faster than the CURL one:
function getPlus1($url) {
$html = file_get_contents( "https://plusone.google.com/_/+1/fastbutton?url=".urlencode($url));
$doc = new DOMDocument(); $doc->loadHTML($html);
$counter=$doc->getElementById('aggregateCount');
return $counter->nodeValue;
}
also here for Tweets, Pins and Facebooks
function getTweets($url){
$json = file_get_contents( "http://urls.api.twitter.com/1/urls/count.json?url=".$url );
$ajsn = json_decode($json, true);
$cont = $ajsn['count'];
return $cont;
}
function getPins($url){
$json = file_get_contents( "http://api.pinterest.com/v1/urls/count.json?callback=receiveCount&url=".$url );
$json = substr( $json, 13, -1);
$ajsn = json_decode($json, true);
$cont = $ajsn['count'];
return $cont;
}
function getFacebooks($url) {
$xml = file_get_contents("http://api.facebook.com/restserver.php?method=links.getStats&urls=".urlencode($url));
$xml = simplexml_load_string($xml);
$shares = $xml->link_stat->share_count;
$likes = $xml->link_stat->like_count;
$comments = $xml->link_stat->comment_count;
return $likes + $shares + $comments;
}
Note: Facebook numbers are the sum of likes+shares and some people said plus comments (I didn't search this yet), anyway use the one you need.
This will works if your php settings allow open external url, check your "allow_url_open" php setting.
Hope helps.
function get_plusones($url) {
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, "https://clients6.google.com/rpc");
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, '[{"method":"pos.plusones.get","id":"p","params":{"nolog":true,"id":"' . $url . '","source":"widget","userId":"#viewer","groupId":"#self"},"jsonrpc":"2.0","key":"p","apiVersion":"v1"}]');
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Content-type: application/json'));
$curl_results = curl_exec ($curl);
curl_close ($curl);
$json = json_decode($curl_results, true);
return intval( $json[0]['result']['metadata']['globalCounts']['count'] );
}
echo get_plusones("http://www.stackoverflow.com")
from internoetics.com
The cURL and API way listed in the other posts here no longer works.
There is still at least 1 method, but it's ugly and Google clearly doesn't support it. You just rip the variable out of the JavaScript source code for the official button with a regular expression:
function shinra_gplus_get_count( $url ) {
$contents = file_get_contents(
'https://plusone.google.com/_/+1/fastbutton?url='
. urlencode( $url )
);
preg_match( '/window\.__SSR = {c: ([\d]+)/', $contents, $matches );
if( isset( $matches[0] ) )
return (int) str_replace( 'window.__SSR = {c: ', '', $matches[0] );
return 0;
}
The next PHP script works great so far for retrieving Google+ count on shares and +1's.
$url = 'http://nike.com';
$gplus_type = true ? 'shares' : '+1s';
/**
* Get Google+ shares or +1's.
* See out post at stackoverflow.com/a/23088544/328272
*/
function get_gplus_count($url, $type = 'shares') {
$curl = curl_init();
// According to stackoverflow.com/a/7321638/328272 we should use certificates
// to connect through SSL, but they also offer the following easier solution.
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
if ($type == 'shares') {
// Use the default developer key AIzaSyCKSbrvQasunBoV16zDH9R33D88CeLr9gQ, see
// tomanthony.co.uk/blog/google_plus_one_button_seo_count_api.
curl_setopt($curl, CURLOPT_URL, 'https://clients6.google.com/rpc?key=AIzaSyCKSbrvQasunBoV16zDH9R33D88CeLr9gQ');
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, '[{"method":"pos.plusones.get","id":"p","params":{"nolog":true,"id":"' . $url . '","source":"widget","userId":"#viewer","groupId":"#self"},"jsonrpc":"2.0","key":"p","apiVersion":"v1"}]');
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Content-type: application/json'));
}
elseif ($type == '+1s') {
curl_setopt($curl, CURLOPT_URL, 'https://plusone.google.com/_/+1/fastbutton?url='.urlencode($url));
}
else {
throw new Exception('No $type defined, possible values are "shares" and "+1s".');
}
$curl_result = curl_exec($curl);
curl_close($curl);
if ($type == 'shares') {
$json = json_decode($curl_result, true);
return intval($json[0]['result']['metadata']['globalCounts']['count']);
}
elseif ($type == '+1s') {
libxml_use_internal_errors(true);
$doc = new DOMDocument();
$doc->loadHTML($curl_result);
$counter=$doc->getElementById('aggregateCount');
return $counter->nodeValue;
}
}
// Get Google+ count.
$gplus_count = get_gplus_count($url, $gplus_type);
Google does not currently have a public API for getting the +1 count for URLs. You can file a feature request here. You can also use the reverse engineered method mentioned by #DerVo. Keep in mind though that method could change and break at anytime.
I've assembled this code to read count directly from the iframe used by social button.
I haven't tested it on bulk scale, so maybe you've to slow down requests and/or change user agent :) .
This is my working code:
function get_plusone($url)
{
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, "https://plusone.google.com/_/+1/fastbutton?
bsv&size=tall&hl=it&url=".urlencode($url));
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
$html = curl_exec ($curl);
curl_close ($curl);
$doc = new DOMDocument();
$doc->loadHTML($html);
$counter=$doc->getElementById('aggregateCount');
return $counter->nodeValue;
}
Usage is the following:
echo get_plusones('http://stackoverflow.com/');
Result is: 3166
I had to merge a few ideas from different options and urls to get it to work for me:
function getPlusOnes($url) {
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, "https://plusone.google.com/_/+1/fastbutton?url=".urlencode($url));
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
$html = curl_exec ($curl);
curl_close ($curl);
$doc = new DOMDocument();
$doc->loadHTML($html);
$counter=$doc->getElementById('aggregateCount');
return $counter->nodeValue;
}
All I had to do was update the url but I wanted to post a complete option for those interested.
echo getPlusOnes('http://stackoverflow.com/')
Thanks to Cardy for using this approach, then I just had to just get a url that worked for me...
I've released a PHP library retrieving count for major social networks. It currently supports Google, Facebook, Twitter and Pinterest.
Techniques used are similar to the one described here and the library provides a mechanism to cache retrieved data. This library also have some other nice features: installable through Composer, fully tested, HHVM support.
http://dunglas.fr/2014/01/introducing-the-socialshare-php-library/

Categories