I am using fopen to check the existence of an image file (and as a precursor for extracting the image from the external url).
It is working perfectly fine for most images, for example,
SY300.jpg">http://ecx.images-amazon.com/images/I/51DbiFInDUL.SY300.jpg
But it is not working for images from a website like Victoria's Secret, for example:
http://dm.victoriassecret.com/product/428x571/V360249.jpg
Is this a permissions problem? And if so, is there any work around?
$url = "http://dm.victoriassecret.com/product/428x571/V360249.jpg";
$handle = #fopen($url,'r');
if($handle !== false){
return true;
}
For successful link, $handle returns "Resource ID #11", but for unsuccessful link like Victoria's Secret, $handle returns nothing.
Additionally, exif_imagetype is not returning anything for the images (we have the exif extension installed).
Is there any work around for this? We are building a bookmarklet that allows users to extract pictures from sites. We noticed that other bookmarklets are able to get around this (i.e. Pinterest) and are able to get the pictures from Victoria's Secret.
It's don't show a data due to hotlink protection defined in .htaccess file. You need to grab a data as a client. I tried you can using CURL if you put HTTP header information of user agent read contents and save to file.
In my solutions your problem is solved.
Note: Be note for filetype on remote server that are using in header, there are for an example GIF file image/gif so you can put another filetype example for PNG.
Example of solution that WORKS:
error_reporting(E_ALL);
ini_set('display_errors', '1');
$url = "http://dm.victoriassecret.com/product/428x571/V360249.jpg";
function getimg($url) {
$headers[] = 'Accept: image/gif, image/x-bitmap, image/jpeg, image/pjpeg';
$headers[] = 'Connection: Keep-Alive';
$headers[] = 'Content-type: application/x-www-form-urlencoded;charset=UTF-8';
$user_agent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)';
$process = curl_init($url);
curl_setopt($process, CURLOPT_HTTPHEADER, $headers);
curl_setopt($process, CURLOPT_HEADER, 0);
curl_setopt($process, CURLOPT_USERAGENT, $user_agent);
curl_setopt($process, CURLOPT_TIMEOUT, 30);
curl_setopt($process, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($process, CURLOPT_FOLLOWLOCATION, 1);
$return = curl_exec($process);
curl_close($process);
return $return;
}
$imgurl = $url;
$imagename= basename($imgurl);
if(file_exists($imagename)){continue;}
$image = getimg($imgurl);
file_put_contents($imagename,$image);
Note: If you are on Linux filesystem be sure that root folder is writeable (CHMOD) otherwise will not save a file in a path.
And so you are talking about EXIF data, and how is CURL downloaded image is identical to orignal I've checked with md5sum between original image on victoriasecret server and downloaded using CURL. However, a results are SAME, IDENTICAL so you can grab and analyzing downloaded data for future... and delete if you don't need anymore.
On a Linux platform you can use for testing identical files by sum of md5 result using md5sum:
md5sum V360249.jpg V360249_original.jpg
893a47cbf0b4fbe4d1e49d9d4480b31d V360249.jpg
893a47cbf0b4fbe4d1e49d9d4480b31d V360249_original.jpg
A result are same and you can be sure that exif_imagetype information is correctly and identical.
By removing the # symbol, I was able to get a more meaningful error:
Warning: fopen(http://dm.victoriassecret.com/product/428x571/V360249.jpg) [function.fopen]: failed to open stream: HTTP request failed! in [removedSomedatahere]/test.php on line 5
It does similar in curl, wget, and fopen with no other options set. I would hypothesize that this has something to do with cookies or other setting not being set, but I don't have a direct answer for you. Hopefully that helps a little.
[Edited - Solution based on comments]
So it appears that using curl may be a better option in this case if you also set the user agent. The site was blocking based on the user agent. So the solution is to set a commonly used browser as the agent.
Here is an example of setting the user agent:
curl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
Please see this link to understand how to set the user agent in curl.
Related
I have a PHP script that works just fine downloading most remote images to my file system, but when I try to download a Facebook or Instagram image I get an error that says "failed to open stream: No error in" followed by the line of my fopen function and two additional errors "fwrite() expects parameter 1 to be resource, bool given in" which is obviously due to the Facebook image not downloading.
My code is as follows:
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_BINARYTRANSFER,1);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_REFERER, "https://www.facebook.com/");
curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)');
$page = curl_exec($curl);
if(curl_errno($curl)):
echo 'Erro: ' . curl_error($curl);
exit;
endif;
curl_close($curl);
// Use basename() function to return the base name of file
$file_name = basename($url);
if(file_exists($file_name)){
unlink($file_name);
}
$fp = fopen($file_name,'x');
fwrite($fp, $page);
fclose($fp);```
There also seems to be an error when I try to add the first line of code to this post which is:
```$url = 'https://scontent-sea1-1.xx.fbcdn.net/v/t1.0-0/p180x540/119041795_176402070632464_6192328410277888324_o.jpg?_nc_cat=111&ccb=2&_nc_sid=825194&_nc_ohc=O7khk9mGFO4AX86nd5X&_nc_ht=scontent-sea1-1.xx&tp=6&oh=ff35f5eaf960fa7bd30ab1d549f0d817&oe=6045A0D1';```
Screenshot APIs are the workaround for this. There are many, but to spare the one I am using from any scrutiny for how it is being used by now I will not name the specific program. Most are basically the same, but you should choose one that allows you to install it on your own system rather than being dependent on simply getting a screenshot from a URL of theirs. That way you can encode the Facebook/Instagram image URL that you want to take a screenshot of, send the encoded URL to the API, and save the result on your file system.
The ability to encode the URL is key because of all the query strings in Facebook/Instagram image URLs.
I have a php page with a text input where the user is supposed to paste a remote URL of an image, and I will have to store it in the server and display it to the user. Now the problem is, I don't trust a user will always provide a proper image url, and I don't want them to upload a pdf or other file, or a huge, few gb worth of file. Now I can check the extension, but that isn't very helpful, and I hear I can check the mime-type, but I don't know how I can open the file once and check all the validations like mime-type and file size in one go, and then copy the file over. Moreover, since the file will be pretty much served as it is(with a minor name change), I would like to know if it is possible to make sure that the file doesn't have any injected virus or problematic code.
Any suggestions appreciated.
You can use exif_imagetype() to see if its an image.
If you want to be 100% sure that its not malware or something weird. its a good idea to use the GD library and save it via the GD library. So there is no dangerous code inside.
Well there are really multiple things that can be done here. I would suggest using cURL as your mechanism for transferring the file (rather than file_get_contents() or similar). The reason for this is that you can first send a HEAD request against the resource to just get the header information before committing to actually download it. From the headers, you should be able to evaluate the file name, file size, mime-type information, etc. Note that NONE of this information should be trusted, but it at least gives you a sanity check before committing to the file download.
Once you have done the sanity check, you can download the file into a local snadbox directory. This should not be a web-accessible directory. You could use exif_imagetype() to determine if the file is indeed an image of the type you are interested in.
Assuming this all looks good, I would just do the last bit of cleanup-and renaming in GD library (perhaps use imagecreatefrom*() functions to make final image from the temp download file).
With Curl you have no problem with https, you may store a file and check it.
Here is the code to check content-type for image then file is checked with exif_imagetype() (enable php_mbstring and php_exif extentions).
$url = 'https://www.google.com/images/icons/ui/doodle_plus/doodle_plus_google_logo_on_grey.gif';
$userAgent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT,60);
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_exec( $ch ) ;
if(!curl_errno($ch))
{
$type = curl_getinfo($ch, CURLINFO_CONTENT_TYPE);
if ( stripos($type, 'image') !== FALSE )
{
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_HEADER, false);
$filename = tempnam('/path/to/store/file/', 'prefix');
$fp=fopen($filename,'wb');
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_exec($ch);
fclose($fp);
if ( exif_imagetype($filename) !== FALSE )
{
echo "100% IMAGE!";
// take it!
}
unlink($filename);
}
}
curl_close($ch);
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
save image from php url using php
How can i use php to save/grab an image from another domain(not on my domain/server) and save it in a directory of my site.
The URL of the image for example , will be :
http://anothersite/images/goods.jpg
How can i use PHP to grab "good.jpg" ,and save it in my directory which is www.mysite.com/directory/
I hope someone could guide me.
Thanks!
You should be able to use file_get_contents for this one. In order to use an URL with file_get_contents make sure allow_url_fopen is enabled in you php.ini file.
define('DIRECTORY', '/home/user/uploads');
$content = file_get_contents('http://anothersite/images/goods.jpg');
file_put_contents(DIRECTORY . '/image.jpg', $content);
Make sure that you have write permission to the directory where you want to store the image; to make the folder writable you could do this:
chmod +w /home/users/uploads
References
file_get_contents
allow_url_fopen
chmod command
This link may be answer your question:
http://stackoverflow.com/questions/909374/copy-image-from-remote-server-over-http
to me the following code should serve ur need:
$url = "http://other-site/image.png";
$dir = "/my/local/dir/";
$lfile = fopen($dir . basename($url), "w");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)');
curl_setopt($ch, CURLOPT_FILE, $lfile);
fclose($lfile);
curl_close($ch);
I have a website to show opensource movies and videos.
I have saved urls in mysql and linked both videos as well as the images to the content server.
But users are complaining of slow website as images are getting fetched from outside and most of time Internet Explorer is not even displaying the image.
I just learnt about cURL and would like to save images as well as videos to my own server and provide mirror to original website.
I got " curl -O ('') ; " syntax at many places to do the task but don't know how to use it inside my php script.
In short:
I already have my form for url saving in mysql. I wish it to also save save file to a directory on my webserver and save file path to another column in mysql.
Any sort of help is welcome.
Thanx in Advance
$local_file = "/tmp/filename.flv";//This is the file where we save the information
$remote_file = "http://www.test.com/filename.flv"; //Here is the file we are downloading
$ch = curl_init();
$fp = fopen ($local_file, 'w+');
$ch = curl_init($remote_file);
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_ENCODING, "");
curl_exec($ch);
curl_close($ch);
fclose($fp);
I've decided to update this answer almost 7 years later.
For those who have copy() enabled for remote hosts, you can simply use:
copy("http://www.test.com/filename.flv", "/some/local/path/filename.flv");
I used this function for downloading SWF files(flash games) sucessfully. When I use this script for one particular site it downloads all games(I told the script to download 4 games from a list) with exact size of 299bytes? I tried downloading these games with Google Chrome and the download is sucessfull. Is there something missing in the CURL functions I use or the download algorithm is not good enough? Any help will be greatly appreciated.
function saveFlash($fullPaths,$folder,$gamenames,$i){
$curl = curl_init($fullPaths[$i]);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
//Create a new file in the given folder
$fp = fopen($folder."/".$gamenames[$i].".swf", 'w');
if ($fp == FALSE){
echo "File not opened<br>";}
//Ask cURL to write the contents to a file
curl_setopt($curl, CURLOPT_FILE, $fp);
//Execute the cURL session
curl_exec ($curl);
//Close cURL session and file
curl_close ($curl);
fclose($fp);
}
Text editor gives the following
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved here.</p>
<hr>
<address>Apache/2.2.3 (CentOS) Server at freeonlinegames.com Port 80</address>
</body></html>
You'll want to set CURLOPT_FOLLOWLOCATION to allow it to follow the redirects.
You may also want to set a CURLOPT_MAXREDIRS so it doesn't redirect out of control.
That error you're getting is a common way of telling you that no hotlinking is allowed. If you simply want to download the SWF, you need to set the referrer.
curl_setopt($ch, CURLOPT_REFERER, 'http://urlofpagetheswfwasfoundon');
If it still doesn't work after that, you might need to set an appropriate user-agent string.
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 Something Something');
Also, be very sure that you are allowed to do what you are trying to do. Ripping stuff off others' sites is very frowned upon, and usually illegal.