Get the real video URL from a url - php

I'm trying to get the real file URL from a url that doesn't show up the real file name.
My url is like this http://video.premium.com/file/ee7bfec921cfbe16e6f08e282992b99670a00ca3/3
If I could get the real file url I could stream it directly online through a web player, but it needs .mp4 or other file format to play, just the url http://video.premium.com/file/ee7bfec921cfbe16e6f08e282992b99670a00ca3/3 doesnt work.
but when I open the URL using VLC media player, it works. doesn't work with online flash or other players..
Is this even possible? Anyway to do this?

With curl, use this: It follows the redirect until it finds the endpoint. In the included code I used goo.gl to shorten the url to a rando image. You will see the output is the original link (and it would output whatever number of redirects and their URLs), and the final redirect to the actual file. I think this is what you are looking for. I did not write this originally, but found it somewhere some time ago and reused it many times again with tweaking when needed. It seems to fit well in many places. Glad to pass it on. I think it might help to achieve what you are looking for.
<?php
function follow_redirect($url){
$redirect_url = null;
if(function_exists("curl_init")){
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
curl_close($ch);
}
else{
$url_parts = parse_url($url);
$sock = fsockopen($url_parts['host'], (isset($url_parts['port']) ? (int)$url_parts['port'] : 80));
$request = "HEAD " . $url_parts['path'] . (isset($url_parts['query']) ? '?'.$url_parts['query'] : '') . " HTTP/1.1\r\n";
$request .= 'Host: ' . $url_parts['host'] . "\r\n";
$request .= "Connection: Close\r\n\r\n";
fwrite($sock, $request);
$response = fread($sock, 2048);
fclose($sock);
}
$header = "Location: ";
$pos = strpos($response, $header);
if($pos === false){
return false;
}
else{
$pos += strlen($header);
$redirect_url = substr($response, $pos, strpos($response, "\r\n", $pos)-$pos);
return $redirect_url;
}
}
$url = 'http://goo.gl/66VJB';
echo '<ol>';
while(($newurl = follow_redirect($url)) !== false){
echo '<li>', $url, '</li>';
$url = $newurl;
}
echo '</ol>';
echo '', $url, '';
?>
Output:
http://goo.gl/66VJB
http://1.bp.blogspot.com/_XE0TDW07Noo/TOSVQXZtgAI/AAAAAAAAELo/aG80jZ7u_fo/s1600/aptitude_test.gif

Related

Difficulties using PHP to authenticate with CAS

I'm currently trying to get PHP to login and authenticate with a CAS single sign on server which is proving difficult.
The official site has some basic source code here which is supposed to handle the authentication and log in a user. As far as I can see and in my testing it completes steps 1 and 2 in the process (see this diagram for the basic process). Once I've logged into the test server I can complete step 3 and retrieve the service ticket from the URL that sent me back to my page. There doesn't seem to be any examples anywhere to complete steps 4 and 5 of the process. Is it correct that I need to write my own code to do that?
I have attempted to get the ticket back and then send it off to the validation service using some of my own code with cURL or fsockopen with no luck.
if (isset($_GET['ticket']))
{
$currentProtocol = (isset($_SERVER['HTTPS']) && $_SERVER['HTTPS'] != 'off') ? 'https://' : 'http://';
$requestUri = explode('?', $_SERVER['REQUEST_URI']);
$requestUri = $requestUri[0];
$ticket = $_GET['ticket'];
$port = ($_SERVER['SERVER_PORT'] != 80) ? ':8080' : '';
$currentUrl = urlencode($currentProtocol . $_SERVER['SERVER_NAME'] . $port . $requestUri);
$validateUrl = 'ssl://server.com/cas/serviceValidate?service=' . $currentUrl . '&ticket=' . $ticket;
$errno = 0;
$errstr = '';
$fp = fsockopen($validateUrl, 443, $errno, $errstr, 30);
if (!$fp) {
echo "$errstr ($errno)<br />\n";
}
else {
var_dump($fp);
$out = "GET / HTTP/1.1\r\n";
$out .= "Host: www.example.com\r\n";
$out .= "Connection: Close\r\n\r\n";
fwrite($fp, $out);
while (!feof($fp)) {
echo fgets($fp, 128);
}
fclose($fp);
}
}
I can get a legitimate response from the service if I access it through the browser directly e.g:
https://server.com/cas/serviceValidate?service=http%3A%2F%2Flocalhost%3A8080%2Ftestcas%2Fcas-client.php&ticket=ST-35717-XLiWQ2ucCCuks2wsVNMJ-cas
Which returns an XML response containing the Active Directory User ID:
<cas:serviceResponse xmlns:cas='http://www.server.com/cas'>
<cas:authenticationSuccess>
<cas:user>c314317</cas:user>
</cas:authenticationSuccess>
</cas:serviceResponse>
But I really think I need to be able to access that URL directly from the server side with PHP, then once I have the user ID I can link that back with our systems and log them into the site.
My problem is there doesn't seem to be any code to handle the ticket and validation side of things. Can anyone point me in the right direction?
Thanks very much.
OK I think I solved the problem with cURL. I didn't have the CURLOPT_SSL_VERIFYPEER set to false and that's why it was failing. I can now get the XML response with PHP, process the XML response and retrieve the user ID. Here's the code:
// Get the current server address we are executing the PHP from
$currentProtocol = (isset($_SERVER['HTTPS']) && $_SERVER['HTTPS'] != 'off') ? 'https://' : 'http://';
$requestUri = explode('?', $_SERVER['REQUEST_URI']);
$requestUri = $requestUri[0];
$ticket = $_GET['ticket'];
$port = ($_SERVER['SERVER_PORT'] != 80) ? ':' . $_SERVER['SERVER_PORT'] : ''; # Don't need the port if it's 80, but needed if for example test server is running port 8080
$currentUrl = $currentProtocol . $_SERVER['SERVER_NAME'] . $port . $requestUri;
// Setup the validation URL
$validateUrl = 'https://sso.server.com/cas/serviceValidate?service=' . strtolower(urlencode($currentUrl)) . '&ticket=' . $ticket;
// Send request to validate the URL
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $validateUrl); # The URL to get the data from
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); # Return the value of curl_exec() instead of outputting it out directly.
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 120); # The number of seconds to wait while trying to connect
curl_setopt($ch, CURLOPT_TIMEOUT, 120); # The maximum number of seconds to allow cURL functions to execute
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2); # Check the existence of a common name and also verify that it matches the hostname provided
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); # Stop cURL from verifying the peer's certificate
curl_setopt($ch, CURLOPT_HEADER, false); # Don't include the header in the output
// Execute the request and close the handle
$xml = curl_exec($ch);
curl_close($ch);
// Get the user ID from the XML using XPath
$xml = new SimpleXMLElement($xml);
$result = $xml->xpath('cas:authenticationSuccess/cas:user');
$userId = null;
while(list( , $node) = each($result))
{
$userId = (string) $node;
}
echo 'user: ' . $userId . "<br>";

Retrieving contents of re-directed url | curl vs. contexts

I'm using file_get_contents as such
file_get_contents( $url1 ).
However the actual url's contents are coming from $url2.
Here is a specific case:
$url1 = gmail.com
$url2 = mail.google.com
I need a way to grab $url2 progrmatically in PHP or JavaScript.
I believe you can do this by creating a context with:
$context = stream_context_create(array('http' =>
array(
'follow_location' => false
)));
$stream = fopen($url, 'r', false, $context);
$meta = stream_get_meta_data($stream);
The $meta should include (among other things) the status code and the Location header used to hold the redirection url. If $meta indicates a 200, the you can fetch the data with:
$meta = stream_get_contents($stream)
The down side is when you get a 301/302, you have to set up the request again with the url from the Location header. Lather, rinse, repeat.
If your looking to pull the current url, in JS you can use window.location.hostname
I don't get why you would want either PHP or JavaScript. I mean... they are kind of different in approaching the problem.
Assuming you want a server-side PHP solution, there's a comprehensive solution here. Too much code to copy verbatim but:
function follow_redirect($url){
$redirect_url = null;
//they've also coded up an fsockopen alternative if you don't have curl installed
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
curl_close($ch);
//extract the new url from the header
$pos = strpos($response, "Location: ");
if($pos === false){
return false;//no new url means it's the "final" redirect
} else {
$pos += strlen($header);
$redirect_url = substr($response, $pos, strpos($response, "\r\n", $pos)-$pos);
return $redirect_url;
}
}
//output all the urls until the final redirect
//you could do whatever you want with these
while(($newurl = follow_redirect($url)) !== false){
echo $url, '<br/>';
$url = $newurl;
}

Get the filesize of a js file on another domain using php

How do I get the filesize of js file on another website. I am trying to create a monitor to check that a js file exists and that it is more the 0 bytes.
For example on bar.com I would have the following code:
$filename = 'http://www.foo.com/foo.js';
echo $filename . ': ' . filesize($filename) . ' bytes';
You can use a HTTP HEAD request.
<?php
$url = "http://www.neti.ee/img/neti-logo.gif";
$head = get_headers($url, 1);
echo $head['Content-Length'];
?>
Notice: this is not a real HEAD request, but a GET request that PHP parses for its Content-Length. Unfortunately the PHP function name is quite misleading. This might be sufficient for small js files, but use a real HTTP Head request with Curl for bigger file sizes because then the server won't have to upload the whole file and only send the headers.
For that case, use the code provided by Jakub.
Just use CURL, here is a perfectly good example listed:
Ref: http://www.php.net/manual/en/function.filesize.php#92462
<?php
$remoteFile = 'http://us.php.net/get/php-5.2.10.tar.bz2/from/this/mirror';
$ch = curl_init($remoteFile);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); //not necessary unless the file redirects (like the PHP example we're using here)
$data = curl_exec($ch);
curl_close($ch);
if ($data === false) {
echo 'cURL failed';
exit;
}
$contentLength = 'unknown';
$status = 'unknown';
if (preg_match('/^HTTP\/1\.[01] (\d\d\d)/', $data, $matches)) {
$status = (int)$matches[1];
}
if (preg_match('/Content-Length: (\d+)/', $data, $matches)) {
$contentLength = (int)$matches[1];
}
echo 'HTTP Status: ' . $status . "\n";
echo 'Content-Length: ' . $contentLength;
?>
Result:
HTTP Status: 302
Content-Length: 8808759
Another solution. http://www.php.net/manual/en/function.filesize.php#90913
This is just a two step process:
Crawl the the js file and store it to a variable
Check if the length of the js file is greater than 0
thats it!!
Here is how you can do it in PHP
<?php
$data = file_get_contents('http://www.foo.com/foo.js');
if(strlen($data)>0):
echo "yay"
else:
echo "nay"
?>
Note: You can use HTTP Head as suggested by Uku but then if you are seeking for the page content if js file has content then you would have to crawl again :(

PHP: Check if URL redirects?

I have implemented a function that runs on each page that I want to restrict from non-logged in users. The function automatically redirects the visitor to the login page in the case of he or she is not logged in.
I would like to make a PHP function that is run from a exernal server and iterates through a number of set URLs (array with URLs that is for each protected site) to see if they are redirected or not. Thereby I could easily make sure if protection is up and running on every page.
How could this be done?
Thanks.
$urls = array(
'http://www.apple.com/imac',
'http://www.google.com/'
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
foreach($urls as $url) {
curl_setopt($ch, CURLOPT_URL, $url);
$out = curl_exec($ch);
// line endings is the wonkiest piece of this whole thing
$out = str_replace("\r", "", $out);
// only look at the headers
$headers_end = strpos($out, "\n\n");
if( $headers_end !== false ) {
$out = substr($out, 0, $headers_end);
}
$headers = explode("\n", $out);
foreach($headers as $header) {
if( substr($header, 0, 10) == "Location: " ) {
$target = substr($header, 10);
echo "[$url] redirects to [$target]<br>";
continue 2;
}
}
echo "[$url] does not redirect<br>";
}
I use curl and only take headers, after I compare my url and url from header curl:
$url="http://google.com";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_TIMEOUT, '60'); // in seconds
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_NOBODY, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$res = curl_exec($ch);
if(curl_getinfo($ch)['url'] == $url){
echo "not redirect";
}else {
echo "redirect";
}
You could always try adding:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
since 302 means it moved, allow the curl call to follow it and return whatever the moved url returns.
Getting the headers with get_headers() and checking if Location is set is much simpler.
$urls = [
"https://example-1.com",
"https://example-2.com"
];
foreach ($urls as $key => $url) {
$is_redirect = does_url_redirect($url) ? 'yes' : 'no';
echo $url . ' is redirected: ' . $is_redirect . PHP_EOL;
}
function does_url_redirect($url){
$headers = get_headers($url, 1);
if (!empty($headers['Location'])) {
return true;
} else {
return false;
}
}
I'm not sure whether this really makes sense as a security check.
If you are worried about files getting called directly without your "is the user logged in?" checks being run, you could do what many big PHP projects do: In the central include file (where the security check is being done) define a constant BOOTSTRAP_LOADED or whatever, and in every file, check for whether that constant is set.
Testing is great and security testing is even better, but I'm not sure what kind of flaw you are looking to uncover with this? To me, this idea feels like a waste of time that will not bring any real additional security.
Just make sure your script die() s after the header("Location:...") redirect. That is essential to stop additional content from being displayed after the header command (a missing die() wouldn't be caught by your idea by the way, as the redirect header would still be issued...)
If you really want to do this, you could also use a tool like wget and feed it a list of URLs. Have it fetch the results into a directory, and check (e.g. by looking at the file sizes that should be identical) whether every page contains the login dialog. Just to add another option...
Do you want to check the HTTP code to see if it's a redirect?
$params = array('http' => array(
'method' => 'HEAD',
'ignore_errors' => true
));
$context = stream_context_create($params);
foreach(array('http://google.com', 'http://stackoverflow.com') as $url) {
$fp = fopen($url, 'rb', false, $context);
$result = stream_get_contents($fp);
if ($result === false) {
throw new Exception("Could not read data from {$url}");
} else if (! strstr($http_response_header[0], '301')) {
// Do something here
}
}
I hope it will help you:
function checkRedirect($url)
{
$headers = get_headers($url);
if ($headers) {
if (isset($headers[0])) {
if ($headers[0] == 'HTTP/1.1 302 Found') {
//this is the URL where it's redirecting
return str_replace("Location: ", "", $headers[9]);
}
}
}
return false;
}
$isRedirect = checkRedirect($url);
if(!$isRedirect )
{
echo "URL Not Redirected";
}else{
echo "URL Redirected to: ".$isRedirect;
}
You can use session,if the session array is not set ,the url redirected to a login page.
.
I modified Adam Backstrom answer and implemented chiborg suggestion. (Download only HEAD). It have one thing more: It will check if redirection is in a page of the same server or is out. Example: terra.com.br redirects to terra.com.br/portal. PHP will considerate it like redirect, and it is correct. But i only wanted to list that url that redirect to another URL. My English is not good, so, if someone found something really difficult to understand and can edit this, you're welcome.
function RedirectURL() {
$urls = array('http://www.terra.com.br/','http://www.areiaebrita.com.br/');
foreach ($urls as $url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// chiborg suggestion
curl_setopt($ch, CURLOPT_NOBODY, true);
// ================================
// READ URL
// ================================
curl_setopt($ch, CURLOPT_URL, $url);
$out = curl_exec($ch);
// line endings is the wonkiest piece of this whole thing
$out = str_replace("\r", "", $out);
echo $out;
$headers = explode("\n", $out);
foreach($headers as $header) {
if(substr(strtolower($header), 0, 9) == "location:") {
// read URL to check if redirect to somepage on the server or another one.
// terra.com.br redirect to terra.com.br/portal. it is valid.
// but areiaebrita.com.br redirect to bwnet.com.br, and this is invalid.
// what we want is to check if the address continues being terra.com.br or changes. if changes, prints on page.
// if contains http, we will check if changes url or not.
// some servers, to redirect to a folder available on it, redirect only citting the folder. Example: net11.com.br redirect only to /heiden
// only execute if have http on location
if ( strpos(strtolower($header), "http") !== false) {
$address = explode("/", $header);
print_r($address);
// $address['0'] = http
// $address['1'] =
// $address['2'] = www.terra.com.br
// $address['3'] = portal
echo "url (address from array) = " . $url . "<br>";
echo "address[2] = " . $address['2'] . "<br><br>";
// url: terra.com.br
// address['2'] = www.terra.com.br
// check if string terra.com.br is still available in www.terra.com.br. It indicates that server did not redirect to some page away from here.
if(strpos(strtolower($address['2']), strtolower($url)) !== false) {
echo "URL NOT REDIRECT";
} else {
// not the same. (areiaebrita)
echo "SORRY, URL REDIRECT WAS FOUND: " . $url;
}
}
}
}
}
}
function unshorten_url($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_URL, $url);
$out = curl_exec($ch);
$real_url = $url;//default.. (if no redirect)
if (preg_match("/location: (.*)/i", $out, $redirect))
$real_url = $redirect[1];
if (strstr($real_url, "bit.ly"))//the redirect is another shortened url
$real_url = unshorten_url($real_url);
return $real_url;
}
I have just made a function that checks if a URL exists or not
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
function url_exists($url, $ch) {
curl_setopt($ch, CURLOPT_URL, $url);
$out = curl_exec($ch);
// line endings is the wonkiest piece of this whole thing
$out = str_replace("\r", "", $out);
// only look at the headers
$headers_end = strpos($out, "\n\n");
if( $headers_end !== false ) {
$out = substr($out, 0, $headers_end);
}
//echo $out."====<br>";
$headers = explode("\n", $out);
//echo "<pre>";
//print_r($headers);
foreach($headers as $header) {
//echo $header."---<br>";
if( strpos($header, 'HTTP/1.1 200 OK') !== false ) {
return true;
break;
}
}
}
Now I have used an array of URLs to check if a URL exists as following:
$my_url_array = array('http://howtocode.pk/result', 'http://google.com/jobssss', 'https://howtocode.pk/javascript-tutorial/', 'https://www.google.com/');
for($j = 0; $j < count($my_url_array); $j++){
if(url_exists($my_url_array[$j], $ch)){
echo 'This URL "'.$my_url_array[$j].'" exists. <br>';
}
}
I can't understand your question.
You have an array with URLs and you want to know if user is from one of the listed URLs?
If I'm right in understanding your quest:
$urls = array('http://url1.com','http://url2.ru','http://url3.org');
if(in_array($_SERVER['HTTP_REFERER'],$urls))
{
echo 'FROM ARRAY';
} else {
echo 'NOT FROM ARR';
}

What is the fastest way to determine if a URL exists in PHP?

I need to create a function that returns if a URL is reachable or valid.
I am currently using something like the following to determine a valid url:
static public function urlExists($url)
{
$fp = #fopen($url, 'r');
if($fp)
{
return true;
}
return false;
}
It seems like there would be something faster, maybe something that just fetched the page header or something.
You can use curl as follows:
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_NOBODY, true); // set to HEAD request
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); // don't output the response
curl_exec($ch);
$valid = curl_getinfo($ch, CURLINFO_HTTP_CODE) == 200;
curl_close($ch);
You could check http status code.
Here is a code you could use to check that an url returns 2xx or 3xx http code to ensure the url works.
<?php
$url = "http://stackoverflow.com/questions/1122845";
function urlOK($url)
{
$url_data = parse_url ($url);
if (!$url_data) return FALSE;
$errno="";
$errstr="";
$fp=0;
$fp=fsockopen($url_data['host'],80,$errno,$errstr,30);
if($fp===0) return FALSE;
$path ='';
if (isset( $url_data['path'])) $path .= $url_data['path'];
if (isset( $url_data['query'])) $path .= '?' .$url_data['query'];
$out="GET /$path HTTP/1.1\r\n";
$out.="Host: {$url_data['host']}\r\n";
$out.="Connection: Close\r\n\r\n";
fwrite($fp,$out);
$content=fgets($fp);
$code=trim(substr($content,9,4)); //get http code
fclose($fp);
// if http code is 2xx or 3xx url should work
return ($code[0] == 2 || $code[0] == 3) ? TRUE : FALSE;
}
echo $url;
if (urlOK($url)) echo " is a working URL";
else echo " is a bad URL";
?>
Hope this helps!
You'll likely be limited to sending some kind of HTTP request. Then you can check HTTP status codes.
Be sure to send only a "HEAD" request, which doesn't pull back all the content. That ought to be sufficient and lightweight enough.

Categories