I have sites where stored some xml files, and I want to download to our server, we don't have ftp connection, so we can download through http. I always used file(url) is there any better way to download files through php
If you can access them via http, file() (which reads the file into an array) and file_get_contents() (which reads content into a string) are perfectly fine provided that the wrappers are enabled.
Using CURL could also be a nice option:
// create a new CURL resource
$ch = curl_init();
// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://www.server.com/file.zip");
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
set_time_limit(300); # 5 minutes for PHP
curl_setopt($ch, CURLOPT_TIMEOUT, 300); # and also for CURL
$outfile = fopen('/mysite/file.zip', 'wb');
curl_setopt($ch, CURLOPT_FILE, $outfile);
// grab file from URL
curl_exec($ch);
fclose($outfile);
// close CURL resource, and free up system resources
curl_close($ch);
Related
Which way is fastest to save external file to my server. why and how ?
Using Curl :
$ch = curl_init();
$fp = fopen ($local_file, 'w+');
$ch = curl_init($remote_file);
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_ENCODING, "");
curl_exec($ch);
curl_close($ch);
fclose($fp);
Using Copy:
copy($extFile, "report.csv");
it mostly depends on the protocol (for instance, if it was a local file, copy() would be faster), but since you're saying "remote file", curl will probably be faster. you're using CURLOPT_ENCODING and CURLOPT_FOLLOWLOCATION, i guess that means it's transferred over http, where curl is generally much faster than copy, for at least 2 reasons:
1: PHP's fopen http wrappers doesn't use compression, but when you set CURLOPT_ENCODING to emptystring here, you tell curl to use compression if possible. (and while it depends on how libcurl is compiled, gzip and deflate compression is usually compiled in with libcurl.)
2: copy() keeps reading from the socket until the remote server closes the connection, which may be much later than when the file is completely downloaded. meanwhile, curl will only read until it has read bytes equal to the Content-Length:-http header, then close the connection itself, which is often much faster than stalling on read() until the remote server closes the connection (which copy() does, but curl_exec() doesn't.)
but the only way to know for sure ofc, TIAS.
$starttime=microtime(true);
$ch = curl_init();
$fp = fopen ($local_file, 'w+');
$ch = curl_init($remote_file);
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_ENCODING, "");
curl_exec($ch);
curl_close($ch);
fclose($fp);
echo "used ".(microtime(true)-$starttime)." seconds.\n";
vs
$starttime=microtime(true);
copy($extFile, "report.csv");
echo "used ".(microtime(true)-$starttime)." seconds.\n";
gives you roughly microsecond precision (IEEE 754 double floating point precision probably corrupts it somewhat, but probably not enough to matter.)
I have the following PHP script that works perfectly 99% of the time. But it will not download an image from this one server which I think is running Varnish Cache.
<?php
$imglink = 'http://www.dirtrider.com/wp-content/uploads/2014/10/WD-10_1_14-001.jpg';
$ch = curl_init($imglink);
$fp = fopen('/home/path/www/tmp/test.jpg', "w");
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_TIMEOUT, 5);
curl_exec($ch);
fclose($fp);
You get a 403 Forbidden error if you use CURL to load that image. You can work around this error very easily. Just add an alternate user agent for your CURL request:
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
And et voila! It works like a charm. Seems like Varnishe Cache blocks CURL requests which use a CURL default user agent.
So far I want a curl command to download a file into a specific directory , So I have done this
system('curl -o hi.txt www.site.com/hi.html');
It doesn't work because my directory isn't writable , I need to find a way to set curl to download that file to a writable directory of mine .
Instead of curl you could use file_get_contents and file_put_contents
$file = file_get_contents('http://www.site.com/hi.html');
file_put_contents('/htdocs/mysite/images/hi.txt', $file);
This method will work even if the curl module isn't installed. To do this using cURL (which will allow more control over the actual http request):
// create a new cURL resource
$ch = curl_init();
// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://www.site.com/hi.html");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
// grab URL and pass it to the browser
$out = curl_exec($ch);
// close cURL resource, and free up system resources
curl_close($ch);
// Save the file
$fp = fopen('/htdocs/mysite/images/hi.txt', 'w');
fwrite($fp, $out);
fclose($fp);
EDIT
system('curl -o /htdocs/mysite/images/hi.txt www.site.com/hi.html');
I wish to download files from my web server with download progress information. For that purpose, PHP cURL seems to be the best choice.
However, I have difficulties that the downloaded files are not placed into Downloads folder, where all the web files are normally downloaded. I use the following file download routine:
$fp = fopen(dirname(__FILE__) . 'uploaded.pdf', 'w+');
$url = "file:///D:/WEB/SAIFA/WWW/PickUpTest.pdf";
$ch = curl_init(str_replace(" ","%20", $url));
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_BUFFERSIZE, 1024*8);
curl_setopt($ch, CURLOPT_NOPROGRESS, false );
curl_setopt($ch, CURLOPT_PROGRESSFUNCTION, 'progressCallback' );
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 0);
curl_setopt( $ch, CURLOPT_FILE, $fp );
curl_exec( $ch );
curl_close($ch);
fclose($fp);
unset($fp);
My problem is, that instead Downloads folder, the file is silently downloaded into my WWW folder, where the my PHP scripts including this cURL one reside. I get no File Download Save As dialog box neither.
To force Save As dialog box, I added the following header, at the beginning of the script:
header("Content-Disposition: attachment; filename=\"uploaded.pdf\"");
$fp = fopen(dirname(__FILE__) . 'uploaded.pdf', 'w+');
...
After using the header, I get the Save As dialog box however, the file is still silently download into the folder with my PHP scripts. In the Downloads folder, a file 'uploaded.pdf' with filesize 0 is saved.
My question is, how to make PHP cURL to download files properly and place them into Downloads folder and offer Save As dialog box?
I use:
WAMP
Windows 7
PHP Version 5.4.12
Curl Version 7.29.0
By using the file functions you're actually asking your server to save the file so it makes sense that the results of the cURL call end up in your PHP folder.
What you really want, if I understand the problem, is to send the results of the cURL back to the browser. You're halfway there by sending the header(...) - which lets the user's browser know a file is coming and should be downloaded, the step you've missed is sending the cURL data with the header.
You could echo the contents of the file after you've saved it or, more efficiently (assuming you don't want an extra copy of the file), remove the code to save the file locally and remove the cURL option CURLOPT_RETURNTRANSFER. That will tell cURL to send the output directly so it will become the data for the download.
Hope that helps!
EDIT A simple example that grabs a local file (C:\test.pdf) and sends it to the user's browser (as uploaded.pdf).
<?php
header("Content-Disposition: attachment; filename=\"uploaded.pdf\"");
// Get a FILE url to my test document
$url = 'file://c:/test.pdf';
$url= str_replace(" ","%20", $url);
$ch= curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_exec( $ch );
curl_close ($ch);
Hope that helps a bit more!
I am a little new to PHP and curl. I setup a script in WAMP on my local machine. It will perform a curl and save the results to a .csv file. This same curl script is not working on a LAMP development environment that was recently setup for me. Instead, the resulting JSON is printed in the browser. Is there some kind of php extension I need to enable in the LAMP stack? here is the code:
$reportfileurl = 'https://www.googleapis.com/analytics/sampleurl';
$path = 'temp/temp.csv';
$fp = fopen($path, 'w');
$request = curl_init();
curl_setopt($request, CURLOPT_URL, $reportfileurl);
curl_setopt($request, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($request, CURLOPT_HTTPHEADER, array($authorization_header));
//curl_setopt($request, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($request, CURLOPT_FILE, $fp);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, TRUE);
curl_setopt($request, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($request, CURLOPT_CAINFO, getcwd() . "/CAcerts/BuiltinObjectToken-EquifaxSecureCA.crt");
$report_file = curl_exec($request);
curl_close($request);
fclose($fp);
On another note, since the curl request returns JSON i know i need to decode it before I can save the contents as a csv. I've found some posts about how to create an array from a JSON object, but I can't seem to be able to reference the contents inside the JSON object in an array so I wonder if part of the problem is with the difference in the curl handling?
cool. I figured out how to make a user group with permissions to add files/folders to /var/www and then i added the apache2 user 'www-data' to the group. instrcutions here
https://serverfault.com/questions/6895/whats-the-best-way-of-handling-permissions-for-apache2s-user-www-data-in-var
i am now able to write the .csv to the server. now I just need to decode the JSON into actual comma separated values!