I have a php file that acts as a gatekeeper for all the files I want people to download, who ahve sufficient privilages.
The code I use throw the file to the user is
header('Content-Description: File Transfer');
header('Content-Type: application/octet-stream');
header("Content-disposition: attachment; filename=\"".$public_filename."\"");
header("Content-Transfer-Encoding: Binary");
header('Expires: 0');
header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
header('Pragma: public');
header("Content-length: ".$f_filesize);
readfile($file_path);
Most files are fairly large.... 400mb-10GB.
What would be a good way to do this, and keep the true locations + filenames secret, so people cant just link to the files directly, but HAVE to link thru my download.php?file=ID gatekeeper?
Thanks
EDIT: Im not asking how to do user authentication, all that is done. Im just asking if my way of doing it, is a good idea on a large scale. Seems like it could cause memory problems if I keep reading 10GB files.
Ok, having php send files of around 400Mb–10Gb is not good. You need to somehow let whatever webserver you're using actually serve the files.
This really comes down to how secure you need it to be. The easiest solution that comes to mind (but far from the most secure) is using symbolic links with long random names that link to the original file. After a certain time the symbolic links expire and are removed. Each user get their own symbolic link (or "token") to the file they're downloading. I'm not sure how this plays out in Windows-environment, but on unix it's fairly straightforward anyway.
Here's some pseudo code:
if($user->isAllowedToDownload($file)){
$token = md5($user->name . $file->name . time() . $someGoodRandomValue);
symlink($file, $download_path . $token);
header("Location: $download_url$token");
}
Then you need a cron job that cleans out old symbolic links. You also need to make sure the webserver is set to follow symbolic links, preferably only for that folder where these download tokens are created.
So when the user maybe requests domain.com/download?file=bigfile.mp4 a symbolic link is created in the webservers public space that points to the real file outside the webservers public space. The user gets redirected to maybe domain.com/getFile/ab739babec890103bdbca72 which in turn causes the webserver to serve the file. Now it's very hard for users to try and guess what an URL is for a file, and that's the "security".
You're already doing that - the $public_filename is what you want it called, the readfile($file_path) part is the file - it's location isn't made public. Past that, it could be above the document root.
Put the files somewhere that is not accessible via HTTP.
Create a database table of file IDs with file paths.
Link to the files via file ID (as you noted above, download.php?fileID=0000).
???
Profit.
As someone who did this previously (many years ago), you need to consider the memory impact this will have on your server. The readfile function was not available then, so it is possible you may not need to do anything special for memory considerations.
You'll want to somehow authenticate them (an HTML form, HTTP basic auth, whatever), then set a session flag, which your download.php script can check. Note that this doesn't prevent people from downloading the file, then distributing it themselves.
You should configure your web server so the real files are not directly accessible.
It's not going to cause memory problems per se. readfile does not read the file into memory. However, using PHP will create overhead. You can eliminate some of this delay by using X-Sendfile.
Your method will cause memory problems, however it is possible to read and output the file in chunks. You will need to use flush() function after you echo each chunk of file. You can also make resuming downloads to work with a little more effort. Still this is an CPU hungry approach.
The easier and better solution is to use "x-sendfile" header tag supported by both apache and lighttpd through their modules. All you'll have to do is just specify file name in your header, similar to this:
header('X-Sendfile: filename-on-your-file-system');
Link for lighttpd:
http://redmine.lighttpd.net/projects/lighttpd/wiki/X-LIGHTTPD-send-file
Related
I need to deliver big files like file.zip (~2 GB) to customers, with a unique URL for each customer. Then I will redirect (with .htaccess) a customer download link example.com/download/f6zDaq/file.zip to something like
example.com/download.php?id=f6zDaq&file=file.zip
But as the files are big, I don't want the fact that PHP processes the downloading (instead of just letting Apache handle it) to be a CPU / RAM performance issue for my server. After all, asking PHP to do it involves a new layer, so it might cause such an issue, if not done properly.
Question: among the following solutions, which one(s) are the best practice? (in particular, in terms of CPU/RAM)?
1: PHP solution with application/download
header('Content-Type: application/download');
header('Content-Disposition: attachment; filename=file.zip');
readfile("/path/to/file.zip");
CPU usage measured while downloading: 13.6%.
1bis: PHP solution with application/octet-stream (coming from Example #1 of this page)
header('Content-Description: File Transfer');
header('Content-Type: application/octet-stream');
header('Content-Disposition: attachment; filename=file.zip');
header('Expires: 0');
header('Cache-Control: must-revalidate');
header('Pragma: public');
header('Content-Length: ' . filesize('file.zip'));
readfile("/path/to/file.zip");
1ter: PHP solution with application/octet-stream (coming from here):
header('Content-Description: File Transfer');
header('Content-Type: application/octet-stream');
header('Content-Disposition: attachment; filename=file.zip');
header('Content-Transfer-Encoding: binary'); // additional line
header('Connection: Keep-Alive');
header('Expires: 0');
header('Cache-Control: must-revalidate, post-check=0, pre-check=0'); // additional line
header('Pragma: public');
header('Content-Length: ' . filesize('file.zip'));
readfile("/path/to/file.zip");
1quater: Another PHP variant with application/force-download (edited; coming from here):
header("Content-Disposition: attachment; filename=file.zip");
header("Content-Type: application/force-download");
header("Content-Length: " . filesize($file));
header("Connection: close");
2: Apache solution, no PHP involved: let Apache serve the file, and use .htaccess to provide different URL for the same file (many ways to do it can be written). In terms of performance, it's similar to let the customer download example.com/file.zip, served by Apache server.
3: Another PHP solution. This would probably work:
$myfile = file_get_contents("file.zip");
echo $myfile;
but wouldn't this ask PHP to load the whole content in memory? (which would be bad in terms of performance!)
4: Just do a header("Location: /abcd/file.zip"); redirection as explained in File with a short URL downloaded with original filename.
Problem with this solution: this discloses the actual location of the file
example.com/abcd/file.zip
to the end user (who can then use or share this URL without authentification) which is not wanted...
But on the other hand, it is much lighter for the CPU since PHP just redirects the request and doesn't deliver the file itself.
CPU usage measured while downloading: 10.6%.
Note: the readfile doc says:
readfile() will not present any memory issues, even when sending large files, on its own. If you encounter an out of memory error ensure that output buffering is off with ob_get_level().
but I wanted to be 100% sure that it won't be slower / more CPU/RAM hungry than pure Apache solution.
You could use .htaccess to redirect the request to the file while keeping the permalink structure:
RewriteEngine On
RewriteBase /
RewriteRule ^download\/([^\/]+)\/file.zip download.php?id=$1 [L,NC]
Then in your download.php, you can check if the provided id is valid:
// Path to file
$file = 'file.zip';
// If the ID is valid
if ($condition) {
header("Content-Disposition: attachment; filename=\"" . basename($file) . "\"");
header("Content-Type: application/force-download");
header("Content-Length: " . filesize($file));
header("Connection: close");
} else {
// Handle invalid ids
header('Location: /');
}
When the user visits a valid url http://example.com/download/f6zDaq/file.zip, the download will start and the connection will be closed.
If the user visits an invalid url, they will be redirected to the home page.
The biggest problems you're going to face with files of those sizes are the following:
people downloading it with a download manager
interrupted connections
Normally, keep-alive can be a bad idea, as it dedicates a connection to a download, which can bog down your network connections instead of allowing them to be freed up easily. However, if you're expecting all of your files to be large, this is your friend, because you don't want people re-starting those downloads. And those downloads will make reliable connections with keep-alive, and be easier for the client to resume which helps reduce people trying to re-download massive files.
As such, of your presented options, I recommend
1ter
However, as others on here, I still recommend you test your solutions, and preferably from a location separate than you're serving the files from.
Addendum:
This said, serving with PHP isn't the best idea unless you have to get the header control features and .htaccess control in, because it's just adding more processing power. By far the better path would be simply to have the files in an accessible directory. .htaccess can rewrite access to files and folders, not just PHP scripts.
To create Apache-based protected download folders instead:
Options +FollowSymLinks
RewriteEngine On
RewriteRule ^/user/files/folder1.*$ http://example.com/userfiles/ [R=301,L]
Then, if you need to password-protect it, instead of using PHP, use Apache (which is already installed with most PHP installations). You do this by including a .htaccess file in the targeted folder (if you're dynamically making users, you might need to create a script to generate these for each new user) and making sure apache is prepped to handle passwords:
AuthType Basic
AuthName "Authentication Required"
AuthUserFile "/user/password/.htpasswd"
Require valid-user
(See here for more detail: Setting up Apache Passwords)
After this point, you make sure to have an .htpasswd file in the password directory with the format username:password/hashedpassword.
e.g.:
andreas:$apr1$dHjB0/..$mkTTbqwpK/0h/rz4ZeN8M0
john:$apr1$IHaD0/..$N9ne/Bqnh8.MyOtvKU56j1
Now, assuming you're not wanting them to pass in the password every single time, in the download link, include the access
Link (hopefully behind a password-protected interface.)
[Note: Do NOT use the direct password link method if passwords are not randomly assigned per file.]
OR if you're populating based off of the root apache password management AND your site is utilizing apache for it's login process, they might not need the user:pass part of the link at all, having already logged in with Apache.
NOTICE:
Now, this said, the files will be be accessible by people that the full link (with username/password) are shared with. So they'll be as secure (or as unsecure) as your server's https (or http if you allow) protocols, as well as your users sharing or not-sharing links.
Doing it this way, the files will be open to the users it's meant for with the full capabilities of the web accessible to them, meaning download helpers, browser-plugins that help, REST calls, and more, depending on your user's use cases. This can reduce security, which may or may not be a big deal depending on what you're hosting. If you're hosting private medical data (few users, high security, lower speed demands), I wouldn't do it this way. If you're hosting music albums, I'd totally do it this way (many users, lower security, high speed demands).
I would go with readfile. I used it for years, and never got memory issues, even running on a 128MB VPS.
Using PHP means you can easily handle authentication, authorization, logging, adding and removing users, expiring URL and so on. You can use .htaccess to do that, but you will have to write a rather large structure to handle this.
You can use X-Accel-Redirect when your webserver is Nginx. For Apache it's mod_xsendfile with X-Sendfile header.
<?php
header('X-Accel-Redirect: /download/f6zDaq/file.zip');
It costs less, also have a better performance, because web server handles file.
Memory & CPU wise you should probably go with readfile() or write some custom code using fopen() and fread() with custom buffer size.
Regarding the headers you send they do not impact the performance of the script, they will just instruct the client what to do with the server response (in your case, the file). You can Google each header and see what exactly it does.
You should probably have a look over this: Is there a good implementation of partial file downloading in PHP?. The things that might be interesting for you there: download range and download resuming support, ways to do this using web server plugins, PEAR packages or libraries that offer the functionality you need.
As mentioned in Fastest Way to Serve a File Using PHP, I finally did this:
apt-get install libapache2-mod-xsendfile
a2enmod xsendfile # (should be already done by previous line)
Then I added this in apache2.conf:
<Directory />
AllowOverride All
Require all granted
XSendFile on
XSendFilePath /home/www/example.com/files/
</Directory>
I then did service apache2 restart and included this in .htaccess:
RewriteRule ^(.*)$ download.php?file=$1 [L,QSA]
and this in the download.php:
header("X-Sendfile: /home/www/example.com/files/hiddenfolder_w33vbr0upk80/" . $file);
header("Content-type: application/octet-stream");
header('Content-Disposition: attachment; filename="' . $file . '"');
NB: strangely, even I have AllowOverride All enabled in the apache2.conf VirtualHost, doing this:
XSendFile on
XSendFilePath /home/www/example.com/files/
just in the /home/www/example.com/.htaccess or /home/www/example.com/files/.htaccess file didn't work (it fails with xsendFilePath not allowed here).
Benchmark:
10.6% CPU when downloading, exactly like if I do a direct download of the file with Apache (and no PHP at all), so it's all good!
Suppose I have a device that knows the name of a file they want to download from my server.
How can I transfer said file to that device without giving the device access to the file system?
For example, suppose I have a page ping.php which receives a get request for "something.zip"
ping.php knows the location of something.zip (somewhere on the server's file system), but I can't allow the user access to the file system, or allow them to know the location of the file (it even needs to be hidden from somebody using something like wireshark).
How can I solve this problem?
It might be an easy solution, I'm just not extremely well versed in these matters.
If it makes any difference I'll be using an Apache server on a Linux box.
You can create a PHP script to facilitate the file transfer while the file is sitting in a folder that is not accessible via the Web. This is how I commonly handle file downloads on my system.
There are any number of sample scripts that you may use to do the actual file transfer. The key is to put that file outside the web-accessible file system.
For completeness, here's some code I've used in the past to do a file download in PHP:
$filename="filetodownload.xyz";
$cf = realpath("/non-webaccessible-folder/".$filename);
$file=$cf;
header('Content-Disposition: attachment; filename="' . basename($cf) . '"');
header("Content-Length: " . filesize($cf));
header("Content-Type: application/octet-stream");
readfile(realpath($cf));
I have .mp3 files in my website and I want to set my site so that after my users have logged in they can download files. If users are not logged in they won't be able to download files. I do not want anyone to be able to find the path of the files.
I'd make the file impossible to access via an HTTP request alone, and with PHP, just print it out:
<?php
session_start();
if (isset($_SESSION['logged_in'])) {
$file = '/this/is/the/path/file.mp3';
header('Content-type: audio/mpeg');
header('Content-length: ' . filesize($file));
readfile($file);
}
?>
You can create a token based on something like the user session id and some random value. Then, the logged in user urls would be like :
/download.php?token=4782ab847313bcd
Place the MP3 files above your docroot, or if that is impossible, deny access to them with .htaccess (if using Apache).
Verify a user is logged in.
Send the appropriate headers and readfile() on the MP3 when the user requests the file.
From wordpress.stackexchange.com/a/285018
Caution: Be wary of using this PHP-driven file download technique on larger files (e.g., over 20MB in size). Why? Two reasons:
PHP has an internal memory limit. If readfile() exceeds that limit when reading the file into memory and serving it out to a visitor, your script will fail.
In addition, PHP scripts also have a time limit. If a visitor on a very slow connection takes a long time to download a larger file, the script will timeout and the user will experience a failed download attempt, or receive a partial/corrupted file.
Caution: Also be aware that PHP-driven file downloads using the readfile() technique do not support resumable byte ranges. So pausing the download, or the download being interrupted in some way, doesn't leave the user with an option to resume. They will need to start the download all over again. It is possible to support Range requests (resume) in PHP, but that is tedious.
In the long-term, my suggestion is that you start looking at a much more effective way of serving protected files, referred to as X-Sendfile in Apache, and X-Accel-Redirect in Nginx.
X-Sendfile and X-Accel-Redirect both work on the same underlying concept. Instead of asking a scripting language like PHP to pull a file into memory, simply tell the web server to do an internal redirect and serve the contents of an otherwise protected file. In short, you can do away with much of the above, and reduce the solution down to just header('X-Accel-Redirect: ...').
I'm developing a web service. With this service, user's will upload their .php files, and service will remove UTF8 BOM characters from php file. And then, There will be a link like this :
Download Your File
But when i click this link, browser browsing to this file. I don't want browse it, i want to download it. So , when user click this link, downloading will start.
Any ideas ?
(P.S. I don't want modify uploadedfile.php file, also i read 5 questions about this, but still i have problem.)
You need to supply this HTTP header:
Content-Disposition: attachment; filename=example.txt
You can usually specify this for entire directories at a time by configuring your web server appropriately. If you mention which web server you are using, somebody may be able to suggest how to do this.
The problem is that you're allowing people to upload PHP files on your server, then giving them a link to execute that PHP file. The web server is automatically treating those uploaded PHP files like any other PHP file, i.e. executing it, which opens you up to a massive security hole.
Whatever purpose your web service has, I'd suggest renaming the file on your server when it is uploaded (something 'random' is best, without an extension), then having a PHP script feed it back out with the appropriate headers set when it is requested.
The URL for such a script would look like:
http://www.example.com/get_uploaded_file.php?id=jgh3h8gjdj2389
It would link the value in id with the file on the server, and if you've saved the original filename somewhere (flat file, DB), you can serve it out using its original name, so long as you set the right HTTP headers.
Linking directly to the PHP file may end up executing it. One way is (like somebody above suggested) to rename it. Or, you can have a downloader.php which does below:
<?php
header('Cache-Control: no-cache, must-revalidate');
header('Expires: Mon, 01 Jan 2000 01:00:00 GMT'); // some date in past
header('Content-type: text/plain');
header('Content-Disposition: attachment; filename='.basename($filepath));
header('Content-Length: ' . filesize($filepath));
flush(); // or any other flush function/mechanism you use.
readfile($filepath);
and link it something like:
Download
This method will let you retain the .php extension. Also, if the PHP file is big and connection is slow, they progress-bar would be accurate (because you've flushed the content length upfront.
Firstly: I'm a lowly web designer who knows just enough PHP to be dangerous and just enough about server administration to be, well, nothing. I probably won't understand you unless you're very clear!
The setup: I've set up a website where the client uploads files to a specific directory, and those files are made available, through php, for download by users. The files are generally executable files over 50MB. The client does not want them zipped, as they feel their users aren't savvy enough to unzip them. I'm using the php below to force a download dialogue box and hide the directory where the files are located.
It's Linux server, if that makes a difference.
The problem: There is a certain file that becomes corrupt after the user tries to download it. It is an executable file, but when it's clicked on, a blank DOS window opens up. The original file, prior to download opens perfectly. There are several other similar files that go through the same exact download procedure, and all of those work just fine.
Things I've tried: I've tried uploading the file zipped, then unzipping it on the server to make sure it wasn't becoming corrupt during upload, and no luck.
I've also compared the binary code of the original file to the downloaded file that doesn't work, and they're exactly the same (so the php isn't accidentally inserting anything extra into the file).
Could it be an issue with the headers in my downloadFile function? I really am not sure how to troubleshoot this one…
This is the download php, if it's relevant ($filenamereplace is defined elsewhere):
downloadFile("../DIRECTORY/files/$filenamereplace","$filenamereplace");
function downloadFile($file,$filename){
if(file_exists($file)) {
header('Content-Description: File Transfer');
header('Content-Type: application/octet-stream');
header('Content-Disposition: attachment; filename="'.$filename.'"');
header('Content-Transfer-Encoding: binary');
header('Expires: 0');
header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
header('Pragma: public');
header('Content-Length: ' . filesize($file));
# flush();
readfile($file);
exit;
}
}
ETA Additonal Info:
- Tests for working/non-working files have been done on the same machine
- If it makes any difference, the original file has a custom icon. After download, the file has a generic blank document icon.
Additonal Info: I THINK THIS ONE'S IMPORTANT!
I just tried downloading the file directly (to bypass the download link that triggers the download function above). If I download the file by just going to its url and downloading it that way, the downloaded file WORKS. So I'm thinking it must have something to do with the download function. But what??
3/17 MAJOR CORRECTION —AND RESOLVED—
So I woke up this morning and it dawned on me that maybe I was comparing the files wrong. (I had re-saved them as binary text, and then compared them. I didn't realize the comparison program would take and compare actual exe files). This morning I tried comparing the actual exe files and there is a difference. There was one line of php code that was being injected into the first line of the file. I adjusted the php, and the problem was fixed. (It was from the if/else statement that defined teh $filenamereplace variable in the code I'd cited). Thanks again for all your help, and sorry for misleading you in insisting that the files' contents were identical!
"I've also compared the binary code of the original file to the downloaded file that doesn't work, and their exactly the same (so the php isn't accidentally inserting anything extra into the file)."
If that's really true, then the problem must be in how the exe is started after it has been downloaded. It should certainly not be a problem with your PHP code.
Perhaps they were corrupted on upload. This can happen if you transfer them via FTP in ASCII mode instead of BINARY.