I have a server that has some very sensitive information on it, so security is a big issue. The user needs to be able to upload a video. I know allowing users to upload files poses a security threat because there is no 100% way to keep them from uploading non-videos. But I obviously can choose which files the server will keep.
I know that checking the file-extension won't suffice. Checking the MIME type is better but it can still be faked. So how do I go about checking if the file is a video?
Play it!
Only way to be sure is to have some code that decodes videos of the type in question, take a look at it (and check there's sensible results, like a non-zero duration).
Otherwise though, your risks are low:
Non-malicious scenario:
Uploader uploads video, with a video/* content-type.
You store the octets and content-type.
Downloader downloads video, and you use the content-type you received.
Downloader watches video (or grumbles about codecs, etc.)
Malicious scenario 1:
Uploader uploads a nasty trojan, with a video/* content-type.
You store the octets and content-type.
Downloader downloads nasty trojan, and you use the content-type you received.
Downloader opens nasty trojan in video player. Nasty trojan does nothing because it's not a video. User grumbles about codecs. Worse case scenario is, they write rant on ubuntu forums about lack of support for proprietary formats, adds ill-spelt comments on your page about how the site sucks because the video didn't work, etc.
Malicious scenario 2:
Uploader uploads nasty trojan that is written into a video that exploits some buffer-overflow issue with a popular video player.
You store...
Downloader...
Could just be like one of the above, but it could also be that they get hit by the exploit (if they're using the affected player).
Three things to note about scenario 2 though:
Testing it's a video won't guarantee safety, as it could well have also work fine in some players.
Testing it's a video could make your server vulnerable to the exploit, if the vulnerability is in ffmpeg!
Exploits of this type are both rare, and hard to do. The general risk is the same as uploading and downloading jpegs or pngs. Indeed it's a bit smaller (there was indeed an exploit of this type affecting commonly-used jpeg libraries for a while).
In all, just make sure you only output with the content-types you accept, and force file-extensions to match them; if the user uploads a video/mpeg called hahaha.exe, then rename it hahaha.mpg
Edit: Oh, also:
Malicious scenario 3:
Uploader uploads video that exploits some players in a way that uses a lot of resources. In this case a downloader will just kill-9/ctrl-alt-delete/your-OSs-kill-them-all-of-choice, but if your server is testing it's a video, then it could end up in trouble as there's no one on hand to step in and kill the 200 (and growing as the script-kiddies's script keeps uploading more) "videos" it's trying to interpret.
Just doing normal video-processing could be enough to introduce the ability to DoS you (video processing is relatively heavy after all), so testing the file could introduce more dangers than it saves you from.
You can call ffmpeg via a php extension:
https://github.com/char0n/ffmpeg-php/
which essentially wraps the output of ffmpeg, which you then can check in php. However, you should familliarize yourself with ffmpeg first, which is a whole topic on its own. If you don't want to use the library, you can execute ffmpeg on your own via exec.
Additionally, I would check the mimetype. You can also check the file on the client side within a file input via JS (not in all browsers and this is no replacement for a true validation).
lg,
flo
Users can safely upload anything as long as it goes to the right directory and nothing on the server tries to run it (and if it's supposed to be a video, nothing will try). Malware can't do anything unless the victim somehow activates it.
I agree that unless video players have a issue that can be exploited via some corrupted video files I won't worry much. But say for not necessarily security reasons one had to check whether the file you have is a video file and all of it is valid you could execute the following steps
Run ffprobe on the file without arguments. It will give some information about the file. The codecs, containers, duration, frame rate, bitrate.
Now run ffprobe with -show_packets. It should give out frame by frame information about the file. You should get as many video frames as duration*frame_rate given by the first command. This can be checked because someone could just give a crafted header or initial section of the file could be valid video (ffmpeg and ffprobe will only test the first few seconds of the video to verify its type) the rest could be corrupted.
Run ffprobe with -show_frames. This will try and decode headers of each frame to ensure each one is a valid video frame. This is an additional step because some container simply have a table that can be used for show_packets. Hence ffprobe could have simply read that table and the data it is pointing to may be corrupt.
Now it is theoretically possible that a file has all valid headers for every frame but wrong data but without decoding the actual content and viewing it on a player this is the best you can do afaik. And I would say that is good enough and it is very fast.
Related
Writing a secure file upload PHP Script from the bottom sounds like hell to me.
The basic rules to uploading a file in no particular order:
1) Create a new file, something random, and give the new uploaded file that name
2) Check the extension
3) Check for the exif trick
4) Store all uploaded files off the web root, and give that directory no permissions to execute files.
5) make sure that the file upload function is PHP does not execute the code while uploading the file
6) Check the file size
7) Do some malware scan
8) limit filesize
So i am thinking thats a lot :)
I havent even begun writing a script for all this, because i have 3 basic questions.
1) Is my list complete, if something are missing please state which
2) is there some sort of framework that can do all this for me? Something simple, not a big huge one that can do multible other things.
3)
Is this a guide good?
http://www.sitepoint.com/file-uploads-with-php/
I would love to post code, but since this subject is big, i feel its better to ask larger.
Thanks in advance.
The “exif trick” and other measures in that article to sniff file contents are of little use in themselves. (OK, it's worth checking uploaded images are of the expected pixel size, but that's application-specific rather than a security problem.)
The article doesn't say what the threat model is that it's trying to address with filetype sniffing, but what this is commonly trying to do is prevent cross-site scripting attacks, where the attacker includes some active content in the file. Usually this is with HTML in files, which browsers (especially IE) sniff and decide to interpret as HTML even though that's not how the file is being served. Unfortunately, checking that a file begins with a PDF header, or represents a valid GIF image does not help you here because it's possible to make “chameleon” files that can be interpreted as different filetypes simultaneously.
This attack can be blocked in modern browsers by serving the files with a specific non-HTML Content-Type and an X-Content-Type: nosniff header. However there are more obscure attacks involving getting content into Flash or Java plugins that are not affected by this header, and it's not watertight against older browsers.
The really-safe way to stop XSS attacks on uploaded files is simply to serve them from a different hostname (ideally, a different domain name and IP address, but a simple subdomain is at least mostly-effective). Then you can let an attacker XSS the user-uploads-hosting site as much as they like without it having a negative effect on your main site.
Virus scanning is unlikely to prove useful for general-purpose file upload functions. If you are expecting people to use the site to exchange Windows executables then it can be worth scanning those for traditional malware, but for the general case you're typically concerned about attacks against the website itself—server exploitation, XSS, browser exploits—and those kind of attacks are not detected by AV scanners.
Your step (1) of creating a new random filename is a much better approach than “sanitising” user-supplied filenames as the linked article tries to do. Its “safe filename” function is not directly vulnerable to directory traversal, but it does still allow oddnesses like .. (on its own), the empty string, .htaccess, and filenames that would confuse a Windows server, like trailing dots, reserved names and over-long names.
You are right that secure file upload is much trickier than it initially seems, and unfortunately most tutorial code out there (especially for PHP) is pretty disastrous.
What are the security considerations when a server fetches a file from an untrusted domain?
What are the security considerations when resizing an image that you don't trust with PHPs GD2 library?
The file will be stored on the server machine, and will be offered for download. I know I can't trust the MIME-Type header. Is there anything else I should be aware of?
I have a webservice that looks like this:
input
An http-URL (or a String that is expected to be a URL)
output
A meta description of the file, or an error if there was one.
The meta description has one of two forms:
It's an image + a URL to the image on my domain + a thumbnail of the image (generated on and hosted by my server)
It's not an image + a URL to the file on my domain
update
Concerns that I can come up with:
The remote server is a malicious server that will send tiny bits of information, enough to keep the socket open, but doesn't do anything useful - like slowloris. I don't know how real of a threat this is. I suppose it could be easily avoided with timeout + progress check.
The remote server serves something that looks like an image (headers, mime-type) but causes PHP to crash when I load it with GD2.
The server sends a useless or bad MIME-type header. Like text-plain for binary files.
The remote server serves an image with a virus in it. I assume that resizing the image will get rid of the virus, but I will serve the original image if there is no reason to scale.
The remote server serves a file with a virus in it. The file will not be treated as an image so my server will do nothing with it. Nothing will happen until the user downloads, and runs it.
Also, I assume I can trust the users of my service. This is a private application in a situation where users can be held accountable for bad behavior. I assume they wont intentionally try to break it.
What are the security considerations when a server fetches a file from an untrusted domain?
The domain (host) and the file is not to be trusted. This spreads over two points:
Transport
Data
To transport the data safely, use a timeout and a size limit. Modern HTTP client libraries offer both of that. If the file could not be requested in time, drop the connection. If the file is too large, drop the data. Tell the user that there was a problem getting the file. Alternatively let the user handle the transport to that server by using the users browser and javascript to obtain the file. Then post it. Set the post limit with your script.
As long as the data is untrusted you need to handle it with caution. That means, you implement yourself a process that is able to run different security checks on the file before you mark it as "safe".
What are the security considerations when resizing an image that you don't trust with PHPs GD2 library?
Do not pass untrusted data to the image library then. See the step above, bring it into a safe state first.
The file will be stored on the server machine, and will be offered for download. I know I can't trust the MIME-Type header. Is there anything else I should be aware of?
I think you're still at the point above. How to come to safe from untrusted. Sure you can't trust the Content-Type header, however it's good to understand it as well.
You want to protect against the Unrestricted File Upload VulnerabilityOWASP.
Check the filename. If you store the data on your server, give it a safe temporary name that can not be guessed upfront and that is not accessible via the web.
Check the data associated with the filename, e.g. the URL information of the source of that file. Properly handle encoding.
Drop anything that does not meet your expectations, so check the pre-conditions you formulate strictly.
Validate the file data before you continue, for example by using a virus checker.
Validate the image data before you continue. This includes file-headers (magic numbers) as well as that the file-size and file-content is valid. You should use a library that has specialized for the job, e.g. an image-file-format-malformation-checker. This is specialized software, so if this part of your business get into business. Many free software image file code exists, I leave this just for the info, you can't trust any recommendation anyway and need to get into the topic.
If you plan to resize the image yourself, you need to make everything double-safe, because next to hosting you plan to process the data. So know what you do with the data first to locate potential fields of problems.
Do logging and monitoring.
Have a plan for the case that everything get's wrong.
Consider to repeat the process for already existing files, so if you change your procedure, you are able to automatically apply the principles to uploads that were done in the past as well.
Create a system for each type of work that is able to be cleaned after the work has been done. One system to do the download, one system to obtain the meta data etc.. After each action, restore the system from an image. If a single components fails, it won't be left over in an exploited state. Additionally if you detect a fail, you can take your whole system out of business until you have found the flaw.
All this depends a bit how much you want to do, but I think you get the idea. Create a process that works for you knowing where improvement can be added, but first create an infrastructure that is modular enough to deal with error-cases and which probably encapsulates the process enough to deal with any outcome.
You could delegate critical parts to a system that you don't need to care about, e.g. to separate processing from hosting. Additionally, when you host the images the webserver must not be clever. The more stupid a system is, the less exploitable it is (normally).
If hosting is not part of your business, why not hand it over to amazon s3 or similar stores? Your domain can be preserved via DNS settings.
Keep the libraries you use to verify images with up-to-date (which implicates you know which libraries are used and their versio, e.g. the PHP exif extension is making use of mbstring etc. pp. - track the whole tree down). Take care you're in the position to report flaws to the library maintainers in a useful way, e.g. with logging, storing upload data to reproduce stuff etc..
Get knowledge about which exploits for images did exist in the past and which systems/components/libraries (example, see disclaimer there) were affected.
Also get into the topic which are common ways to exploit something, to get the basics together (I'm sure you are aware, however it's always good to re-read some stuff):
Secure file upload in PHP web applications (Alla Bezroutchko; June 13, 2007; PDF)
Some related questions, assorted:
Is it important to verify that the uploaded file is an actual image file?
PHP Upload file enhance security
What you're describing basically comes down to an input validation problem; you don't trust what your application is reading in as input and processing.
To address this, what you should do is to download the resource in question and then attempt to determine a true file type. There are multiple ways to attempt this, but basically you will want to use either some custom-code or a library to parse through the file and look for the tell-tail signs of a certain type. There is a good SO discussion on how to do this in PHP here - How can I determine a file's true extension/type programatically? - I would check the second answer that lists some PHP-specific functions to do this. When your application receives a file, it should perform some true file typing like this and then compare the result to what the specified MIME type from the remote server is; if they match accept the file and if they do not, drop it.
I would also suggest using a whitelist of allowable filetypes (a list of everything your service will support and then ONLY accept files of those types). If you have a very general-purpose service, then you should at least do a blacklist of disallowed filetypes (a list of everything your service absolutely will not support and drop those immediately based on the outcome of your MIME type compares). Again, the use of these is entirely dependent on your use-cases.
Once you've got a type, the concern becomes if what the remote server has sent you is a bad file that targets your server (contains malicious code, buffer overflow designed to make the GD2 library blow up and run arbitrary code, etc). Basically, you are relying on the GD2 library to not contain bugs that would lead to such a successful exploit. There's not much you can do here, short of running security audit on the library yourself and I'm going to assume that's out-of-scope. Basically, keep up on any reported security bugs with the library and patch as soon as you can; as a consumer of the library, you are really relying on the maintainers to find and remedy security vulnerabilities like this.
Next, the concern is that the remote server has sent you a bad file that targets your users/clients (contains malicious code, buffer overflows, viruses, etc). Here, if there is corrupted data that is really malware in the image, it will most likely either (1) break or exploit GD2 when it is read (see above for that scenario) or (2) be eliminated when the resize operation is performed by the library if GD2 can successfully process it. There is still a chance it will remain despite the processing, but there's not much you can do there either. If you're really concerned about this, you can apply a virusscan using an external product designed for that; I would suggest that if you're doing that to do so both (1) after the download and before GD2 processing and then (2) on the manipulated file before you serve it out. Personally, I don't think you get much by doing this, but if you want to provide an additional check / warm fuzzies to your users, it cannot hurt.
To address the slow-feeding of data to keep a connection open, put a timeout on any connection to deal with this problem; unless you are dealing with a specific threat to your use-case here, I do not think this is a huge concern.
1) My primary concern with blindly fetching a file from an untrusted domain would be how to verify that the file is, in fact, what you expected to get.; could the untrusted server trick your script into downloading a harmful file (like a virus) or possibly a script that would allow a backdoor into your system?
2) I haven't read any security issues with resizing an image with the GD2 library. If it's not an image to begin with, the GD2 functions would throw an error. I don't think you have much to worry about with this part.
3) I (personally) would not ever do this without reviewing every single file that my script downloaded first. If you want to partially automate this, you might consider running magic number tests on all the files as a pre-filter. But a human look is the safest way to serve random files. When you finish this project - before you make it live - try to break / trick / hack it as hard as you can. Get some knowledgeable friends involved to help.
when it is not an image you store the file any way regardless what kind of file? so they can upload and php file and browse to it to execute php code on your server?
I am writing a small PHP application and I am not sure whether I have a security issue. So this is what the application does:
the user can upload either image files (png, gif, jpg, jpeg, tiff and a few others) or zip files
I check for mime-type and extension and if it's not an allowed I don't allow the upload (this is not the part I am worried about).
Now once uploaded I rename the file to a unique hash and store in a folder outside root access.
The user can now access the file through a short URL. I make the file accessible by setting the right mime-type for the header and then I just use readfile().
My question is whether the exploit where a jar file is included inside the image file works here? I am serving the image as a pure image.
If it does what are ways to prevent this?
Thanks.
MIME type checks will not solve the GIFAR issue. 2009's JREs are already patched, but if you want to solve the issue you can either
Serve your images from a different domain
Run a server side code to check if an image contains a valid JAR, like mentioned here
Anything else (short of denying the file to any Java enabled browser with an old enough JRE) may fail on specific cases.
Also remember that to perform a good attack with this technique your server infrastructure would have to be somewhat open (the fact that a request comes from the same domain doesn't mean that you should give any information it asks for.)
Checking the mime-type is not sufficient because that (or any other) HTTP header field can be forged. The best way to confirm that a file is a valid image is to attempt to read it as an image programatically. If it can be parsed as an image, you can be reasonably confident that it's not malicious code.
Related: ensuring uploaded files are safe
Any kind of hidden exploit like you describe should not affect the server because of the way you handle it. You're simply storing binary information, and retrieving binary information, without processing it in anyway. Browsers attempting to display exploited content might be at risk, but standard image types are fairly safe.
If you'd like to be safer, you could run an anti-virus on each uploaded file. If you're on a *nix platform, you can use the industry-standard ClamAV.
I'd be more worried of someone trying to upload a very large image file.
You can do 2 things. Serve your images from images.domain.com. this would have to be on another physical/virtual server, or firewall'd such that no open ports on the server can be accessed from that domain.
Or you can run the image file thru a java script (not javascript) like the one here. This will tell you if there is a jar file embedded in the image.
More info on this issue here:
http://www.gnucitizen.org/blog/java-jar-attacks-and-features/
I didn't actually even hear about this attack before your question, so first off, thanks for enlightening me! Googling around, it seems that there are basically two different attack vectors here. Both include the attacker luring "regular" users to a malicious site pointing to the masqueraded JAR file, and both have to do with the fact that the JAR will be executing in the "context" of your site (i.e. the origin will be your site).
First attack has to do with the applet being able to read user cookies, which basically means it'll be able to steal the user's login information for your domain.
The second one has to do with the fact that the applet is now allowed to open connections to other sockets within your domain, which is pretty bad if one of the users behind your server's firewall visits the malicious page (enabling the attacker to effectively bypass your firewall).
So this attack does not necessarily harm your server directly, but it does harm your users - and hopefully you care about your users. The two things you can do ensure their safety have already been mentioned in most of the other answers and are summarized on this page.
i check the file for its extension, and mime type - is there anything else i can be doing to help make file uploads safer?
its for an avatar (so all the images are in one folder). i was thinking about using htaccess to forbid any php execution just incase some php file found its way in there. what do you think?
Neither file extension nor mime type can give you 100% security that you are dealing with a image file. But as long as you're not going to execute the file (e.g. by using include()), that is not a problem and you do not need to check for PHP code or anything else. The only security breach imaginable using a forged image file would be something that exploits the browser's rendering engine. This is impossible to protect effectively against from server side and is the browser vendor's responsibility.
So, as long as you make sure you use is_uploaded_file() and move_uploaded_file() when handling the upload, you should be fine, at least on the image format front. Make sure you read #bobince's post below and follow the link, it contains a bunch of great information on other security aspects when dealing with files.
You could however, to provide totally maximum security, of course copy the image into a new image container using GD's imagecopy. This would erase any ID3 and other header information contained in the file, and probably destroy any exploit attempts (GD would probably choke on such a file and return an error). This works for GIF, JPEG, and PNG only, of course, and you may run into some issues like alpha channel and colour profile problems.
Never use user-submitted filenames at all; make up new ones like «random number».jpeg. ‘Sanitising’ filenames is harder than you think, especially if the app needs to be able to run on a Windows server.
For images, use the PHP getimagesize function to determine the filetype of an image, rather than looking at the highly-unreliable filename and mimetype submissions. Disallow uploads that don't parse as images.
For files that are intended to be downloaded, use the Content-Disposition: attachment header to stop IE sniffing for HTML content and displaying it in the browser.
For files that must display inline you'll have to serve them from a different hostname to your main site, otherwise HTML content inside them can cross-site-script into your security context.
Making a file upload feature secure is hard. More discussion.
I need to stream an flv file.
Streaming should look like live streaming,
and I there should be a way to change the target file.
Sorry for my poor English.
If by "streaming" you mean "showing a flash-video clip", flv-streaming isn't really streaming, but it's a normal file transfer, with the flash player starting playback even if the file isn't completely downloaded to the client. Time seeking is implemented in a standard HTTP way, with a file download offset - if you scroll the video halfway on the timeline and it hasn't downloaded yet, the current download is (most often) aborted, and restarted with an offset of filesize/2 bytes.
So, you could be fine with just using plain old readfile() and fixing the HTTP mime-type headers appropriately with header(). But, since you'd need seeking, you need to parse the HTTP request and instead of readfile(), use fpassthru(), seeking the right place, instead.
tl;dr: Avoid handling this on the PHP side, and let Apache (or the httpd of your choice) handle this instead.
I'm not quite sure what you're asking about, but if you have a bunch of media files and want to play them with a Youtube-esque media player on your website, I suggest checking out Flowplayer - It has an amazing Javascript API that lets you do all kinds of cool stuff to the player, including handling stuff like playlists and programmatically changing the current video.
I recently used it on a project with great results.