I am visiting one site, which already provide following functionality: When user browser visits this url, it will automatically prompt window to download containing file (video). In my script I need to download this attached file from this site to disk using php. I tried to use curl:
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
And also similar file_get_contents but it doesnt work for me. It get me only containing HTML on that site. Have you any idea how to save this file on the disk using php?
One standard way that developers do this in PHP is to use the "header" method. After calling header, the page can output the video contents, which will prompt a user download.
See this example: PHP header attach AVI-file
Related
I am using CURl to retrieve a page for a small search engine project I am working on, but on some pages it's not retrieving the entire page.
The function that I have setup is:
public function grabSourceCode($url) {
// Try and get source code using #file_get_contents
$ch = curl_init();
$timeout = 50;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_USERAGENT,'NameBot/0.2');
$source_code = curl_exec($ch);
curl_close($ch);
return $source_code;
}
and I am retrieving the page using:
$Crawler->grabSourceCode('https://sedo.com/search/searchresult.php4?keyword=cats&language_output=e&language=e')
on this page I get everything, but on this page, I only get part of the page.
I have tried using file_get_contents() but that has the same results.
It seem's to be an issue with dynamic loading of the page, when I run the browser in JavaScript blocking mode it shows the same results as the CURl function.
Is there anyway to do this in PHP, or would I have to look at another language, such as JavaScript?
Thanks, Daniel
I'm trying to create a program using PHP where I can load a full webpage and navigate the site while still staying in a different domain. The problem I'm having is that I can't load things like stylesheets and images because they are relative links. I need a way to make the relative links in to absolute links.
Right now I can get just plain HTML from the page using this handy bit of code:
echo file_get_contents('http://tumblr.com');
I can't use an iframe to display the webpage.
Your code should work, but you must set allow_url_fopen to on before running it.
echo file_get_contents('http://othersiteurl.com');
You may also use cURL. Example:
function get_data($url, $timeout = 5) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
Slightly modified code from: https://davidwalsh.name/curl-download
How does one download a file from a web page without a direct path to the file. For example a URL with GET information instead of the path. The code below seems to be downloading the actual page html instead of the file...
Not sure what I'm doing wrong. I also would like to augment this to also perform on sites that require logins but I think I would just have to add
curl_setopt($ch, CURLOPT_USERPWD, "$username:$password")
to the code?
$output_filename = "advanced.exe";
$host = "http://download.cnet.com/Advanced-SystemCare-Free/3001-2086_4-10407614.html?hlndr=1";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $host);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_AUTOREFERER, false);
curl_setopt($ch, CURLOPT_REFERER, "http://download.cnet.com");
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_setopt($ch, CURLOPT_HEADER, 0);
$result = curl_exec($ch);
curl_close($ch);
$fp = fopen($output_filename, 'w');
fwrite($fp, $result);
fclose($fp);
The link you have there isn't the actual link to the file, only the page that initiates the download. By the looks of it, the page uses JavaScript to trigger the download, so you would want to dig through their code to find out exactly how they do it. Then you can find the real URL to the file.
A simple way, if you are doing this only for one file, would be to download the file in your browser, and then access the URL it used from the browser's download manager. (In Firefox, for example, right click the file and choose "Copy Download Link")
I also would like to augment this to also perform on sites that require logins but I think I would just have to add ...
That would work only for HTTP based authentication. If the site uses a traditional login form, this will not work. You'd have to submit several, sequential HTTP requests via CURL, using cookies to store the session state.
I want to be able to allow user to enter in variable URL which file they would like to download from remote server URL e.g /download.php?url=fvr_anim_foxintro_V4_01.jpg
<?php
$url = $_GET['url'];
header("Location: http://fvr.homestead.com/files/animation/" . $url);
?>
The above is purely an example I grabbed from google images. The problem is I do not want the end user to be allowed to see where the file is originally coming from so it would need to get the file download to the server and the server passes it along to the end user. Is there a method of doing this?
I find many examples for files hosted on the server but no examples for serving files hosted on a remote server. In other words I would be passing them along. The files would be quite large (up to 100MB)
Thanks in advance!
You can use cURL for this:
<?php
$url = "http://share.meebo.com/content/katy_perry/wallpapers/3.jpg";
$ch = curl_init();
$timeout = 0;
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
// Getting binary data
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1);
$image = curl_exec($ch);
curl_close($ch);
// output to browser
header("Content-type: image/jpeg");
echo $image;
?>
Source: http://forums.phpfreaks.com/topic/120308-solved-curl-get-image/
Of course, this example is just for an image (as you've suggested) but you can use cURL for all kinds of remote data retrieval via HTTP GET, POST, PUT, DELETE, etc. Search around the web for "php curl" and you'll find an endless supply of information.
The ideal solution would be to use PHP's cURL Library, but if you're using shared hosting keep in mind this library may be disabled.
Assuming you can use cURL, you simply echo the Content-type header with the appropriate MIME Type and echo the results from curl_exec().
To get a basic idea of how to use the cURL library, look at the example under the curl_init() function.
With permission from the other site, I am supposed to periodically download an image hosted on another site and include it in a collection of webcam photos on our site with an external link to the contributing site.
This hasn't been any issue for any of the other sites, but with this particular one, I can't open the image to resize it and save it to our server. It's not a hotlinking issue because I can create a plain ole' <img src="http://THEIRIMAGE /> on a page on our site and it works fine.
I've tried using $img = new Imagick($sourceFilePath) directly as with all the others, as well as trying to use PHP's copy and also trying to copy the image using cURL but when doing so, the page just times out with no results at all.
Here's the image in question: http://island-alpaca.selfip.com:10202/SnapshotJPEG?Resolution=640x480&Quality=Standard
Like I've said, I'm able to do this sort of thing with several other webcams, but it isn't working with this one, and I am stuck as to why it isn't. Any help would be greatly appreciated.
Thank you.
In order to reduce bandwidth and server load some sites block certain bots from accessing their content. Your cURL request needs to more closely mimic an actual browser, which would include a referrer (usually), user agent, etc. It could also be that there is a redirect and you haven't told cURL to follow redirects.
Try setting more opts like this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch,CURLOPT_ENCODING , "gzip");
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_USERAGENT, 'PHP');
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
If that doesn't work, get the user agent from an actual browser and put it in there and see if that makes a difference.
Try using file_get_contents().
for example:
$url = 'http://island-alpaca.selfip.com:10202/SnapshotJPEG?Resolution=640x480&Quality=Standard';
$outputfile = "tmp/image" . date("Y-m-d_H.i.s");
$cmd = "wget -q \"$url\" -O $outputfile";
exec($cmd);
$temp_img = file_get_contents($outputfile);
$img = new Imagick($temp_img);
Can you try this and get back to me?