I allow PDF files to be uploaded to my site (PHP).
I would like to offer the ability to also allow .zip files which contain PDF files in directories so it is easier for users to simply zip a directory and upload one file instead of uploading multiple zip files individual.
For those of you who offer a .zip file upload feature to your (PHP) website, what are the technical, security, and other issues you have faced?
Be careful how you unpack the zip, you could find yourself consuming far more resources than you expected. Perhaps some setrlimit(2) resource limits before unpacking would be wise.
The unzip(1) utility has several nice safety features built in; the -^ command line option turns off control-character filtering, so make sure you don't touch this :) and the -: command line option allows stupid pathnames like ../../../../etc/passwd. Make sure you're on at least version 5.50, so that those stupid pathnames are forbidden by default. (And don't use that command line option. I mention the options just so you can more easily find the documentation for them. :)
If you use an API, make sure it has options to prevent both kinds of silly filenames.
Assuming the .zip gets unpacked eventually you would have to make sure the directory they get unpacked in is unreachable by the the clients' browsers (with .htaccess or by placing it outside the web root directory), and even in that case I'd still monitor the contents of the unpacked .zip to make sure they didn't contain anything that might prove harmful (php or other files run by the server, html spoofs).
Another issue is i guess the upload_max_filesize set in php.ini, you can make sure it can be set big enough to suit your purposes before you start coding.
edit: also read sarnold's answer ;)
AFAIK, php can handle zip files pretty efficiently. Difficulties/Issues that I can think of is, while accessing the file where We need to extract the zip first, and then retrieve the actual needed file. Due to that reason, extracting a zip, might consume additional amount of server time, depending on the size of the file itself.
Where As, during uploads, I do not suppose there is any difficulties or issues specially emphasized on zip types.
Related
i am trying to implement a user system (php, apache) where the user can upload several files and download a zipped version of them. (or uploading a zipped file and download the uncompressed files).
question: is there any risk to zip/unzip those unknown files?
in other words, are unix/php zip/unzip operations treat files as text only or some execution can occur?
This question is relevant to all compression methods, zip is just an example.
EDIT: #Alex Brown AND #Parallelis wrote 2 risks so obviously it is not safe.
any one can post a short explanation on how to implement a safe compress/uncompress of unknown files?
As It seems reasonable to me. You cant do this because of some issues, what if those files are bootstrap scripts? (Refer comments of Alex and Parallelis for 2 more issues).
Solutions :
If you going to store the zip files as zip files after being uploaded, Doing so you will face additional issues since zips can contain lots of files that may or may not be appropriate. In that case you may want to get a list of the contents of the file to automatically include in your field so people downloading them will know if the file contents are valid. You could also integrate with something like ClamAV to scan all the files that are uploaded.
Note: Google is doing same thing, they use their anti virus scanner programs (which offcourse are not available for public use).
Also you can place the file in a temporary directory first and then use zip_open on it in that location. Their you will be able to use OS level commands (which come with their own risks) to verify the integrity of the file without actually unzipping it. You can also refer this tool for same thing.
There are several potential issues:
Zip bomb - this is generally not that much of an issue any more because most decompression tools / languages will prevent nested levels of decompression.
Relative paths: This in my mind is your biggest concern - the zip is decompressed, but it includes the file: ../../../../../../vendor/autoload.php for example.. this then overwrites your autoload.php file and is executed whenever someone views your website. Game over.
A zillion inodes. A zip file may contain millions of 0 byte files, using up all your available inodes on the system. This would stop the hard disk being able to create new files on that partition. This could be medium-bad.
You also should know that zip archives could contain symlinks. If user can read files after unzip, it is possible to read arbitrary files on your filesystem.
zip utility has --symlinks option for storing symlinks.
Im planning to add file manager (very basic once) because I never used FTP functions, and it looks easier (FTP connection loses when scripts is done). I would simply use POST request (or what should I?) instead of FTP functions. Is it good idea? Anyone knows restrictions?
As far as I can see only FTP functions are to post and receive files.
What you need to do is add dynamic form where you can select multiple files and upload them to specific directory of your chose.
You will need to get all available directories and files in them, probably with some kind of recursive function. More optimal way is to get directories/files of current folder and when you click on folder it will get files/folder for it.
Can it be done - sure. Is it a good idea - no. People will have access for uploading malicious files, we are not talking about images here, php scripts, shell scripts, executable viruses and so on...
If you are doing this only for yourself, for file posting and receiving I suggest you to use FTP clients for that.
I wouldn't recommend it, but it's probably best to use a 3rd party tool, rather than to write your own.
PHP File Manager
PHPfileNavigator2
FileManager
...
Keep in mind that both PHP and your webserver can put certain restrictions on the size of files that you can transfer, it is of course possible to change these in the configuration files.
im trying to implement on my site a system who let the user download a file that have to be change before the download.
I have a master file (a .exe program), that inside have a variable who has to be change for every different user.
The most simple solution is to change a variable inside a xml file every time the user want his personalized exe and then make the exe file to read the external file. BUT i dont want the user to download more than one file.
Is this possible? using php can i change a parametter inside a compiled program? Thanks for any help and suggestions!
If you really really know what you're doing and you know exactly the bits that need to be flipped inside the file, then yes, it's possible to modify the .exe file with PHP. Otherwise you have to make changes to the source or other files the .exe is built with and compile the program on the server before sending it to the user.
In theory it's certainly possible (PHP is turing complete), but as stated in other answers it will be hardly worth the hassle (considering the fact that you have to ask whether it is possible shows you'd have to investigate at last for days into the standard exe-format).
I'd recommend one of the following:
1) Zip the program with the configuration file; either use a separate launcher (e.g. Java [a JAR is a ZIP file]) or add a configuration file that is read by the program itself. There is a number of PHP libraries for generating ZIP files.
2) compile the program with the changed source on the server itself; however this can also become quite complicated depending on your server configuration and the programming environment you use. If you have never administered a virtual server I would not even slightly recommend that as an option.
3) If you can assume that the user got somewhat stable Internet access you might also consider to let hir download a standard executable, where additional configuration will be downloaded later on by the program itself (e.g. by transmitting the username to the server). However this creates dependencies you might want to avoid (your user probably can't use it on machines without Internet access and you should assert that your server is up most of the time).
While it's probably possible, I doubt it's worth the hassle. Unless you're trying to fight piracy or something. Why don't you just serve the user a custom .zip file with the .exe and a config .xml?
[edit after OP commented]
I presume what you're trying to edit is the facebook ID/username? Well, try to compile the base file with some unique string like "THISNEEDSTOBEREPLACED", then use some binary safe function to replace it. Though remember things can and will get tricky if the string lengths don't match.
I want to allow members the option of uploading content using a zip file. Once uploaded, I want to use PHP's ZipArchive class to decompress the zip file contents to a directory, and then move the files into our system.
I'm concerned about the potential security risks though, and I can't find any documentation on php.net. The first (Well, the only) risk that comes to mind, is someone creating a zip file with relative paths like "../../etc/passwd" (If they assume I decompress the file in /tmp/somedir).
I'm actually having a hard time creating a relative path in a zip file, so I can't test if such a thing would be possible. I also can't find any way to extract the contents of the zip file using ZipArchive, and have it ignore directories (Decompress all the files, but don't create the directory structure inside the zip).
Can anyone tell me if such an exploit is possible, and/or how to ignore the directory structure in a zip file using ZipArchive?
Interesting question, but I urge you to go about this a different way. I would highly recommend you run your web process with least privileges in a chroot jail. Assuming you do that, the WORST thing that can happen is your website get's defaced, and then you restore a backup and do some forensics to plug that specific hole.
New holes are discovered constantly, you will have a very difficult time completely securing your website going after hunches like these. Minimizing the attacker's sandbox really goes a long way.
I had the same concerns and had a look at the PHP 5.3 source code where I found this:
/* Clean/normlize the path and then transform any path (absolute or relative)
to a path relative to cwd (../../mydir/foo.txt > mydir/foo.txt)
*/
virtual_file_ex(&new_state, file, NULL, CWD_EXPAND TSRMLS_CC);
path_cleaned = php_zip_make_relative_path(new_state.cwd, new_state.cwd_length);
if(!path_cleaned) {
return 0;
}
Looks fine to me. Checkout PHP and see ./ext/zip/php_zip.c for details.
You need to make sure that the extracted contents are not served directly by your application server. So if someone has a php file in his archive that he cant execute it via your webserver.
Another thing is you should keep things safe from being included in user generated content. But this should be considered also without having zip archives in place.
In the end I'm going with Pekka's solution, of using the command line unzip utility. It provides switches to ignore directories in the zip file. The concerns others have pointed out aren't an issue here. Once the files are unzipped, we add them to the system using the same process as our regular uploads, which means each file is scrutinized using the security measures we already have in place.
I wrote a PHP script to dynamically pack files selected by the client into zip file and force a download. It works well except that when the number of files is huge (like over 50000), it takes a very long time for the download dialog box to appear on the client side.
I thought about improving this using cache (these files are not changed very often), but because the selection of the files are totally decided by the user, and there are tens of thousands of combinations on the selection, it is very hard to cache combinations. I also thought about generating zip archives for individual files first, and then combining the zip files on-the-fly. But I did't find a way to concatenate zip files in PHP. Another way I can think of is sending (i.e., reading) the zip file at the same time as generating it. I also don't know if this is supported.
If someone could help me on this, I would really appreciate your help.
To extened Mike Sherov's answer, try using a combination of Tar and Gzip/Zip. Individually pre-compress all the files using Gzip/Zip, Then when the client makes their selection, you simply Tar those files together. That way you still get the benefit of compression and the simplicity of downloading one file, but none of the overheads and delays associated with compressing large files in real time.
While not a silver bullet, you can try tar'ing the files instead. The resulting file is larger, but compression time is much shorter. See here for more info: http://birdhouse.org/blog/2010/03/08/zip-vs-tar-gzip/
Check out mod_zip for Nginx:
https://github.com/evanmiller/mod_zip
It streams a ZIP file to the client dynamically and can include very large (2GB+) files while using very little RAM.