Collecting file with PHP CURL after validating request downloads an empty file - php

I am doing a system where one of my sites goes to the other to get documents.
On the first site I am using Curl to make a request to get the file wanted:
I am using the solution from Download file from URL using CURL :
function collect_file($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_AUTOREFERER, false);
curl_setopt($ch, CURLOPT_REFERER, "http://example.com");
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_setopt($ch, CURLOPT_HEADER, 0);
$result = curl_exec($ch);
curl_close($ch);
echo $result;
return($result);
}
function write_to_file($text,$new_filename){
$fp = fopen($new_filename, 'w');
fwrite($fp, $text);
fclose($fp);
}
$curlUrl = 'http://site2.com/file-depository/14R4NP8JkoIHwIyjnexSUmyJibdpHs5ZpFs3NLFCxcs54kNhHj';
$new_file_name = "testfile-new.png";
$temp_file_contents = collect_file($curlUrl);
write_to_file($temp_file_contents,$new_file_name);
I am testing downloading an image. If i use a direct URL into $curlUrl , for instance http://site2.com/file-depository/image.png it works perfect.
What I am doing is that the URL http://site2.com/file-depository/14R4NP8JkoIHwIyjnexSUmyJibdpHs5ZpFs3NLFCxcs54kNhHj is then parsed and checked against a database to match the document requested, once there is a document matched I need to provide this document to the Curl response.
I have tried many ways to read the file but everytime i am getting a file on the other end but it is only 1kb in size (45 expected) and when trying to open it i get an error unkown file type etc.
On the second site, once the URL is validated here is what I have:
$file = readfile('some-image.png');
echo $file;
I am guessing there is part of the information which belongs to the file missing but can't figure it out, any pointers appreciated!

I have replaced
function write_to_file($text,$new_filename){
$fp = fopen($new_filename, 'w');
fwrite($fp, $text);
fclose($fp);
}
by file_put_contents($new_file_name,trim($temp_file_contents));
Please note the trim(), the issue was that I was apparently collecting some empty space in front of the file content.

Related

Curl and php download file from url not working

I'm trying do retrieve and download a file (image) from a remote location.
Inside the php.ini the allow_url_fopen is enabled, but i can't download the image.
Code i'm using is described below
$local_file = "test.jpg";
$remote_file = "http://somehost:6346/cgi-bin/CGIProxy.fcgi?cmd=snapPicture2&usr=xxxx&pwd=xxxx";
$ch = curl_init();
$fp = fopen ($local_file, 'w+');
$ch = curl_init($remote_file);
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_ENCODING, "");
curl_exec($ch);
curl_close($ch);
fclose($fp);
with any other url that contains a real jpg file, it's working perfectly, i suppose that the issue is that the url use some special characters that doesn't like to curl.
If i try to execute the php snippet above,page load for almost 1 minute,and it seems that no error are displayed,the image test.jpg is created, but it's empty.
Do you have any suggestion?
Thanks!
Try this
$local_file = "test.jpg";
$remote_file = "http://somehost:6346/cgi-bin/CGIProxy.fcgi?cmd=snapPicture2&usr=xxxx&pwd=xxxx";
function getPage($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
function saveToFile($base, $decode=false, $output_file)
{
$ifp = fopen($output_file, "wb");
if ($decode){
fwrite($ifp, base64_decode($base));
}else{
fwrite($ifp, $base);
}
fclose($ifp);
return($output_file);
}
$remote_page = getPage($remote_file);
$saved_file = saveToFile($remote_page , false, $local_file);
when debugging issues like this, set CURLOPT_VERBOSE, it will probably reveal why the page loaded for almost 1 minute, with no apparent output.
i suppose that the issue is that the url use some special characters - this is fully possible, for example your username and password, they're supposed to be urlencoded. urlencoding is binary safe, meaning you can have any special characters you'd like, you just need to encode it properly. use urlencode() or http_build_query() for that, eg
$remote_file = "http://somehost:6346/cgi-bin/CGIProxy.fcgi?" . http_build_query ( array (
'cmd' => 'snapPicture2',
'usr' => 'username',
'pwd' => 'password'
) );
now http_build_query will properly urlencode any special characters in your username and password (for example, if your username is an email address, the # becomes %40).
if that doesn't fix it, what does CURLOPT_VERBOSE say?
also, final note, here you're sending the download request with credentials in a GET request. that's very unusual, the vast majority of websites want you to login with a POST request, and there are good security-related reasons for that, are you sure your website allows sending credentials in GET parameters? the vast majority of websites doesn't allow it... (and the best way to find out, is to record a browser logging in, does the browser use GET parameters, or POST parameters?)

PHP copy() doesn't work for random URLS which redirect to files

trying to copy() .MP3 file from remote url but it always fails.
$link = str_replace(' ','%20','http://mp3hungama.com/music/download.php?song_id=80522');
if (!copy($link,'/home2/muser/tmp/newname.mp3')) {
echo 'copy failed !';
}
$link url redirects to http://mp3hungama.com/music/audio//Indian%20Movies/Indian%20Movies%20Hindi%20Mp3%20Songs/Singh%20Is%20Bling%20(2015)/songs/Cinema%20Dekhe%20Mamma%20#%20Mp3HunGama.Com.mp3
same code works for others random urls like www.example.com/download.php?id=2332. what's the specifically problem here or any other way to do this job ?
I've tested your code and I also couldn't download the file, then, I've used curl an it work as expected:
$local_file = "/home2/muser/tmp/newname.mp3";//This is the file where we save the information
$remote_file = "http://mp3hungama.com/music/download.php?song_id=80522"; //Here is the file we are downloading
$ch = curl_init();
$fp = fopen ($local_file, 'w+');
$ch = curl_init($remote_file);
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_ENCODING, "");
curl_exec($ch);
curl_close($ch);
fclose($fp);
NOTE:
Make sure /home2/muser/tmp/ has write permissions.
TIP:
In the future, if you need to encode/decode a url, use urlencode or urldecode instead of str_replace
This link
already redirects to second link. So it's working already.

How to grap this specific url?

The code below(learned from Save image from url with curl PHP) works fine when try to grap image from internet. But when come to the url below, I only got a "test.jpg" which is actually a 404 error page("test.jpg" could be opened by notepad). PS: And i could open the url with browser and could see the image. Thanks to Mike,Problem solved and code updated.
$url = 'https://spthumbnails.5min.com/10368406/518420256_c_570_411.jpg';
$reffer="http://www.sohu.com";
$user_agent="Baiduspider+(+http://www.baidu.com/search/spider.htm)";
$saveto="test.jpg";
grab_image($url,$saveto);
function grab_image($url,$saveto,$reffer,$user_agent){
$ch = curl_init ($url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
curl_setopt($ch,CURLOPT_REFERER,$reffer);
curl_setopt($ch,CURLOPT_USERAGENT,$user_agent);
$raw=curl_exec($ch);
curl_close ($ch);
$fp = fopen($saveto,'w');
fwrite($fp, $raw);
fclose($fp);
}
Thanks to Mike. This site do need "CURLOPT_REFERER" option(which i ignored) to grap the image. And I also add the useragent option to make sure it work on other situation.

Saving an image from a PHP URL using curl and PHP

I have tried to download an image from a PHP link. When I try the link in a browser it downloads the image. I enabled curl and I set “allow_url_fopen” to true. I’ve used the methods discussed here Saving image from PHP URL but it didn’t work. I've tried "file_get_contents" too, but it didn't work.
I made few changes, but still it doesn’t work. This is the code
$URL_path='http://…/index.php?r=Img/displaySavedImage&id=68';
$ch = curl_init ($URL_path);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
$raw=curl_exec($ch);
curl_close ($ch);
$fp = fopen($path_tosave.'temp_ticket.jpg','wb');
fwrite($fp, $raw);
fclose($fp);
Do you have any idea to make it works? Please help. Thanks
<?php
if( ini_get('allow_url_fopen') ) {
//set the index url
$source = file_get_contents('http://…/index.php?r=Img/displaySavedImage&id=68');
$filestr = "temp_ticket.jpg";
$fp = fopen($filestr, 'wb');
if ($fp !== false) {
fwrite($fp, $source);
fclose($fp);
}
else {
// File could not be opened for writing
}
}
else {
// allow_url_fopen is disabled
// See here for more information:
// http://php.net/manual/en/filesystem.configuration.php#ini.allow-url-fopen
}
?>
This is what I used to save an image without an extension (dynamic image generated by server). Hope it works for you. Just make sure that the file path location is fully qualified and points to an image. As #ComFreek pointed out, you can use file_put_contents which is the equivalent to calling fopen(), fwrite() and fclose() successively to write data to a file. file_put_contents
You can use it as a function :
function getFile($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$tmp = curl_exec($ch);
curl_close($ch);
if ($tmp != false){
return $tmp;
}
}
And to call it :
$content = getFile(URL);
Or save its content to a file :
file_put_contents(PATH, getFile(URL));
You're missing a closing quote and semicolon on the first line:
$URL_path='http://…/index.php?r=Img/displaySavedImage&id=68';
Also, your URL is in $URL_path but you initialise cURL with $path_img which is undefined based on the code in the question.
Why use cURL when file_get_contents() does the job?
<?php
$img = 'http://…/index.php?r=Img/displaySavedImage&id=68';
$data = file_get_contents( $img );
file_put_contents( 'img.jpg', $data );
?>

Updating Twitter background via API

I'm having a little trouble updating backgrounds via Twitter's API.
$target_url = "http://www.google.com/logos/11th_birthday.gif";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch, CURLOPT_URL,$target_url);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$html = curl_exec($ch);
$content = $to->OAuthRequest('http://twitter.com/account/update_profile_background_image.xml', array('profile_background_image_url' => $html), 'POST');
When I try to pull the raw data via cURL or file_get_contents, I get this...
Expectation Failed The expectation given in the Expect request-header
field could not be met by this server.
The client sent
Expect: 100-continue but we only allow the 100-continue expectation.
OK, you can't direct Twitter to a URL, it won't accept that. Looking around a bit I've found that the best way is to download the image to the local server and then pass that over to Twitter almost like a form upload.
Try the following code, and let me know what you get.
// The URL from an external (or internal) server we want to grab
$url = 'http://www.google.com/logos/11th_birthday.gif';
// We need to grab the file name of this, unless you want to create your own
$filename = basename($url);
// This is where we'll be saving our new file to. Replace LOCALPATH with the path you would like to save the file to, i.e. www/home/content/my_directory/
$newfilename = 'LOCALPATH' . $filename;
// Copy it over, PHP will handle the overheads.
copy($url, $newfilename);
// Now it's OAuth time... fingers crossed!
$content = $to->OAuthRequest('http://twitter.com/account/update_profile_background_image.xml', array('profile_background_image_url' => $newfilename), 'POST');
// Echo something so you know it went through
print "done";
Well, given the error message, it sounds like you should load the URL's contents yourself, and post the data directly. Have you tried that?

Categories