I want to allow members the option of uploading content using a zip file. Once uploaded, I want to use PHP's ZipArchive class to decompress the zip file contents to a directory, and then move the files into our system.
I'm concerned about the potential security risks though, and I can't find any documentation on php.net. The first (Well, the only) risk that comes to mind, is someone creating a zip file with relative paths like "../../etc/passwd" (If they assume I decompress the file in /tmp/somedir).
I'm actually having a hard time creating a relative path in a zip file, so I can't test if such a thing would be possible. I also can't find any way to extract the contents of the zip file using ZipArchive, and have it ignore directories (Decompress all the files, but don't create the directory structure inside the zip).
Can anyone tell me if such an exploit is possible, and/or how to ignore the directory structure in a zip file using ZipArchive?
Interesting question, but I urge you to go about this a different way. I would highly recommend you run your web process with least privileges in a chroot jail. Assuming you do that, the WORST thing that can happen is your website get's defaced, and then you restore a backup and do some forensics to plug that specific hole.
New holes are discovered constantly, you will have a very difficult time completely securing your website going after hunches like these. Minimizing the attacker's sandbox really goes a long way.
I had the same concerns and had a look at the PHP 5.3 source code where I found this:
/* Clean/normlize the path and then transform any path (absolute or relative)
to a path relative to cwd (../../mydir/foo.txt > mydir/foo.txt)
*/
virtual_file_ex(&new_state, file, NULL, CWD_EXPAND TSRMLS_CC);
path_cleaned = php_zip_make_relative_path(new_state.cwd, new_state.cwd_length);
if(!path_cleaned) {
return 0;
}
Looks fine to me. Checkout PHP and see ./ext/zip/php_zip.c for details.
You need to make sure that the extracted contents are not served directly by your application server. So if someone has a php file in his archive that he cant execute it via your webserver.
Another thing is you should keep things safe from being included in user generated content. But this should be considered also without having zip archives in place.
In the end I'm going with Pekka's solution, of using the command line unzip utility. It provides switches to ignore directories in the zip file. The concerns others have pointed out aren't an issue here. Once the files are unzipped, we add them to the system using the same process as our regular uploads, which means each file is scrutinized using the security measures we already have in place.
Related
i am trying to implement a user system (php, apache) where the user can upload several files and download a zipped version of them. (or uploading a zipped file and download the uncompressed files).
question: is there any risk to zip/unzip those unknown files?
in other words, are unix/php zip/unzip operations treat files as text only or some execution can occur?
This question is relevant to all compression methods, zip is just an example.
EDIT: #Alex Brown AND #Parallelis wrote 2 risks so obviously it is not safe.
any one can post a short explanation on how to implement a safe compress/uncompress of unknown files?
As It seems reasonable to me. You cant do this because of some issues, what if those files are bootstrap scripts? (Refer comments of Alex and Parallelis for 2 more issues).
Solutions :
If you going to store the zip files as zip files after being uploaded, Doing so you will face additional issues since zips can contain lots of files that may or may not be appropriate. In that case you may want to get a list of the contents of the file to automatically include in your field so people downloading them will know if the file contents are valid. You could also integrate with something like ClamAV to scan all the files that are uploaded.
Note: Google is doing same thing, they use their anti virus scanner programs (which offcourse are not available for public use).
Also you can place the file in a temporary directory first and then use zip_open on it in that location. Their you will be able to use OS level commands (which come with their own risks) to verify the integrity of the file without actually unzipping it. You can also refer this tool for same thing.
There are several potential issues:
Zip bomb - this is generally not that much of an issue any more because most decompression tools / languages will prevent nested levels of decompression.
Relative paths: This in my mind is your biggest concern - the zip is decompressed, but it includes the file: ../../../../../../vendor/autoload.php for example.. this then overwrites your autoload.php file and is executed whenever someone views your website. Game over.
A zillion inodes. A zip file may contain millions of 0 byte files, using up all your available inodes on the system. This would stop the hard disk being able to create new files on that partition. This could be medium-bad.
You also should know that zip archives could contain symlinks. If user can read files after unzip, it is possible to read arbitrary files on your filesystem.
zip utility has --symlinks option for storing symlinks.
Im planning to add file manager (very basic once) because I never used FTP functions, and it looks easier (FTP connection loses when scripts is done). I would simply use POST request (or what should I?) instead of FTP functions. Is it good idea? Anyone knows restrictions?
As far as I can see only FTP functions are to post and receive files.
What you need to do is add dynamic form where you can select multiple files and upload them to specific directory of your chose.
You will need to get all available directories and files in them, probably with some kind of recursive function. More optimal way is to get directories/files of current folder and when you click on folder it will get files/folder for it.
Can it be done - sure. Is it a good idea - no. People will have access for uploading malicious files, we are not talking about images here, php scripts, shell scripts, executable viruses and so on...
If you are doing this only for yourself, for file posting and receiving I suggest you to use FTP clients for that.
I wouldn't recommend it, but it's probably best to use a 3rd party tool, rather than to write your own.
PHP File Manager
PHPfileNavigator2
FileManager
...
Keep in mind that both PHP and your webserver can put certain restrictions on the size of files that you can transfer, it is of course possible to change these in the configuration files.
Is there any tool out there which could tell the useless files in the code base?
We have a big code base (PHP, HTML, CSS, JS files) and I want to be able to remove the not needed files. Any help would be appreciated.
I'm guessing deleting files and running your phpunit tests is a none starter.
If your files are not already in a version-control system - add them. Having the files in a version control system (such as svn or git) is crucial to allow you to recover from deleting any files that you thought were not being used but you later find out were.
Then, you can delete anything you think may not be being used, and if it doesn't affect the running of your application you can conclude that the files aren't used. If adverse effects show up - you can restore them from your repository with ease.
The above is most appropriate (probably) for frontend files (css, js, images). Any files you delete that are requested will show up in your webserver error log giving you a quick reference for files that nolonger exist that you need to restore.
For your php files, that's quite a bit more tricky, How did you arrive at a position where you have php files which you aren't using? Anyway you could for example:
Use xdebug
Enable profiling
Use append mode (one profile)
Use all the functions of your application
and you would then have a profile which includes all files you loaded. Scanning the generated profile for each php file in your codebase will give you some indication of which files you didn't use.
If you are only looking for unused files, don't be tempted to use code coverage analysis - it is very intensive and not the level of detail you're asking for.
A slightly less risky way would be to log whenever a file is loaded. e.g. put this at line one of each file:
<?php file_put_contents('/some/location/fileaccess.log', __FILE__, FILE_APPEND); ?>
and simply leave your application to be used for a while (days, weeks). Thereafter just scan that log, for any file that is named - remove the above line of code. For any that are not - delete (preferably after looking for the filename in your whole sourcecode and confirming it's nowhere).
OR: you could use a shutdown function which dumps the response of get_included_files() to a log file. This would allow you to achieve the same without editing all php files in your source tree.
Caveat: Be careful deleting your php files. Whereas a missing css/js/image will probably mean your application still works, a missing php file of course will have rather more impact :).
If it is in Git why not delete the local file and then do a git rm <file name> to remove it from that branch.
Agree with everything said by #AD7six.
What you might like to try with PHP is to log the use of the files in someway (logging to flat file or database).
This technique does not have to be in place for long you can do it with an include and require_once at the top of each file.
That technique also works for javascript functions you can just print to the console each function, and then unit test your site. You can probably clean out a lot of redundant code that way.
The rest is not so easy, but version tracking is the way to go.
Ok this might seems a bad idea or an obvious one. But let's imagine a CMS like PHPBB. And let's imagine you'd build one. I'd create just 1 file called PHPBB.install.php and running it it will create all folders and files needed with PHP. I mean, the user run it just once and every file and folder of the app is created via the PHP file.
Why to do this?
Well mostly because it's cleaner and you are pretty much sure it creates everything as you wish (obliviously checking everything about the server first). Also, having all the files backed-up inside a file you would be able to restore it very easily by deleting everything and reinstalling it running again PHPBB.install.php. Backing-up files like this will allow you to also prevent errors: How? When an error occurred in a file, this file is restored as it was and automatically re-run.
It would be too heavy!
The installation would happen only once and you'd be sure the user will not forget to place the files correctly. The error-preventing will worth the cause and it would also happen only once.
Now the questions:
Does this technique exists? If so, What's its name?
Why would you discourage it?
As others have said, an installer.
It requires the web server to have permission to write to the filesystem, and ends up having the files owned by the user the web server runs as. Even when one has the ability to change filesystem permissions, it's usually a longer process than just extracting an archive and having the initial setup verify permissions.
Does this technique exists? If so, What's its name?
I'd advise to read about __halt_compiler(). It allows you to mix PHP code with non-php data which is not parsed, so you may have PHP code ("installer") and binary data (e.g., compressed contents of all the files) in single PHP file.
1 - Yes, there is a single install file in PHPBB. You run through an online wizard defining your settings and then it installs automatically.
http://www.phpbb.com/support/documents.php?mode=install&version=3&sid=908f5766fc04868ccb985c1b1e6dee4b#quickinstall
2 - The only reason to discourage it would be if you want the user to understand exactly how the system works. Automatically installing it means the user has no need to understand the nitty gritty of it all - of course, many see this as a good thing.
I allow PDF files to be uploaded to my site (PHP).
I would like to offer the ability to also allow .zip files which contain PDF files in directories so it is easier for users to simply zip a directory and upload one file instead of uploading multiple zip files individual.
For those of you who offer a .zip file upload feature to your (PHP) website, what are the technical, security, and other issues you have faced?
Be careful how you unpack the zip, you could find yourself consuming far more resources than you expected. Perhaps some setrlimit(2) resource limits before unpacking would be wise.
The unzip(1) utility has several nice safety features built in; the -^ command line option turns off control-character filtering, so make sure you don't touch this :) and the -: command line option allows stupid pathnames like ../../../../etc/passwd. Make sure you're on at least version 5.50, so that those stupid pathnames are forbidden by default. (And don't use that command line option. I mention the options just so you can more easily find the documentation for them. :)
If you use an API, make sure it has options to prevent both kinds of silly filenames.
Assuming the .zip gets unpacked eventually you would have to make sure the directory they get unpacked in is unreachable by the the clients' browsers (with .htaccess or by placing it outside the web root directory), and even in that case I'd still monitor the contents of the unpacked .zip to make sure they didn't contain anything that might prove harmful (php or other files run by the server, html spoofs).
Another issue is i guess the upload_max_filesize set in php.ini, you can make sure it can be set big enough to suit your purposes before you start coding.
edit: also read sarnold's answer ;)
AFAIK, php can handle zip files pretty efficiently. Difficulties/Issues that I can think of is, while accessing the file where We need to extract the zip first, and then retrieve the actual needed file. Due to that reason, extracting a zip, might consume additional amount of server time, depending on the size of the file itself.
Where As, during uploads, I do not suppose there is any difficulties or issues specially emphasized on zip types.