I want to be able to read an uploaded file, but I don't want the file to be saved on the server because of security concerns... Is it possible just to directly read a file into a variable?
If not, how does the Temp file thing work, how secure is it to save a temp file on the server, and when is it deleted?
You're going to have to save the upload to a file, otherwise any large file will cause an error because it'll overload available memory pretty easily.
Temp is VERY insecure, generally anybody on the system can read/write/delete your temp file.
The best way to go about this is just do a normal file upload, and in your submission script either read the file and process it or move it to a more permanent location. You can now issue a delete command to the tmp copy.
Since you may not have delete permission and/or the file could be auto deleted, it's best to issue the command this way (the # symbol suppresses any errors, because you don't really care if the file is already deleted or what not, this is a "just in case" scenario)
#unlink($filename);
So far as I'm aware it has to be stored somewhere in order to interact with it, but it's deleted as soon as your script finishes executing. See PHP: When does the temporary uploaded files get deleted?
Related
I have a cron script that compresses images. It basically iterates over folders and then compresses the files in the folder. My problem is that some images are getting processed halfway. My theory is that users are uploading a image, and before the image has finished uploading the file, the compressor tries to compress the file. Thus compressing a half-uploaded image, and resulting in half an image being displayed.
Is there a way in PHP to confirm that a file has finished uploading? So that I can only do the compression once i know the file has been fully written?
Or alternatively, is there a way to check if a file is being used by another process?
Or alternatively, would it be reliable enough to look at when the file was "written to disk" and not process it until 10 minutes has gone by?
PHP doesn't trigger your action until the files are fully uploaded, but it is possible for your cron job to start interacting with files before they're fully saved.
When saving something from $_FILES, save it to a version with a . prefix on it to tag it as incomplete. Make sure your cron job skips any such files.
Then, once the save operation is complete, rename the file without the . prefix to make it eligible for processing.
There are two ways to handle the scenario
Flags
Set flag that files before modify/write it.
Our App handles lots of files, we set flags before taking them to process once it's done we remove the flag, as it runs on cron flag is the best way to process files.
Usually, you can an extra column on the table on each file. or you can have an array where you can store all currently handling files.
filemtime()
As you mentioned you can check like if file mtime is more than 10 min that current time, then you compress them but if some other processes are using the file opened the at the same time. it causes the problem again.
So its better to go with flag. If other processes never modify the files often.
You can use flock to ensure file is not in use, see here for example. Alternatively you can check whether an image is broken or corrupted see here.
I'm reading about security stuff for PHP and my biggest concern now is the users file upload form. I've read a lot that some users may upload files that seems to be something else by changing the extension or even manipulating the header and the mimetype. I understand this.
But my question is how will this be an issue if I rename any uploaded file and move it to a directory that they do not know.
Please let me know if this will be enough or not, and if not, just give me some headline of what extra security checks should I perform
Thanks a lot
It really depends on what your online application is looking to achieve. If you wish to limit access directly to files which are uploaded, then you should set the folder permissions for the parent folder of the uploaded area to block user access. Then in your database you can record to path and only host the files through the http response. This will ensure that no files are accessed which could be potentially harmful, and also that users can still upload what they feel. As an extra step, you could add an erroneous file extension to each file while it is hosted and then remove it when it is served.
You might run an antivirus scan daemon in the background like avscand, configured. for scanning and moving infected files to a quarantaine directory. This ought to prevent delivering infected files later back to the people. Configure automatic virus database updating. A bit back that I did do such things, so investigate.
A simple renaming of the file name to one with safe characters should be sufficient; per user separated of course.
To have a more secure site the following needs to happen:
Due to the nature of security, this list will need be updated every so often.
Set the upload_max_filesize to something sensible
Install an Antivirus on the server
Set the upload_tmp_dir to something sensible, that the user may not access. See Setting PHP tmp dir - PHP upload not working
Have your form you upload files (which you already have done)
Your form handler should:
Run a file command to get the type of the data without executing it
Reject random files
The PHP interpreter will validate the file size
Run the virus scanner on the file
Do a file rename to ensure the filename is clean (if you need to reference things, it is convenient to rename the file to the primary key of your attachments table)
Move the file to a location that isn't accessible by the client (but move it, so if a later upload comes in with the same name nothing happens)
When you move the files, ensure they don't have execute permissions
We have a FreeBSD server with samba that employees copy image files on to, which then get uploaded to our web servers (this way they don't have to mess with ftp). Sometimes, if the upload script is running at the same time as files are being copied, it can upload an incomplete file.
We fixed this by getting the list of files along with the file sizes, then waiting 5 seconds and rechecking the file sizes. If the sizes match then its save to upload, if they don't match it checks again in another 5 seconds.
This seems like an odd way to check if the files are being written to. is there a better, more simple way of doing this?
Use a flock function http://php.net/flock - when writing a file obtain an exclusive lock flock($handle, LOCK_EX), after it is written release the lock flock($handle, LOCK_UN).
The upload script could try to obtain the exclusive writing lock too, if it succeeds it is Okay to move the file, otherwise no.
EDIT: Sorry, I forgot about the users copying the files to the server through samba... So there is no space to use flock while copying... But the upload script could still use flock($handle, LOCK_EX) to see, if it is successful or not.
I recommend to shell_exec() smbstatus(1), e.g. smbstatus -LB to check for locked files
Write a script to copy the files to a temp folder on the Samba server and then, when fully copied and flushed, move, (ie, unlink/link, not copy again), them to the upload folder.
I have a form page that posts to another page where multiple fields as well as file uploads are processed. Just wondering what happens to the 'tmp_name' files when/if the user enters some incorrect info and I send them back to the form page with a meta refesh?
If successful, I move the file to a new location. But if not successful, do the files get unset or erased if the user gets redirected? If they don't, can I reaccess them again so the user doesnt have to reupload? OTOH if there is a problem with the file, say it is not the expected MIME type, should I unlink($_FILES['userFile']['tmp_name']? Its easy to force the user to re-upload again, i think, but I dont want the server being filled up with files that will never be used? If the form passes inspection, and I use rename() to move the file, is the temp file really gone? Did it ever exist on the server's hard drive, or was it only in RAM? Whats the best practice here?
do the files get unset or erased if the user gets redirected?
The uploaded files are stored in the /tmp directory (or whatever is specified as PHP's temporary location). Once your script has run, files left there are subject to deletion any time. I don't think they usually get deleted straight away, but the contents of /tmp will be automatically purged by the OS when necessary.
/tmp is usually located on a hard drive, not in RAM.
Managing this is usually nothing you need to worry about.
If the form passes inspection, and I use rename() to move the file, is the temp file really gone?
Yes, but you must use move_uploaded_file() on uploaded files instead of rename() for security reasons.
The file is stored in the tmp folder and if you don't move it elsewhere it will stay there. It will be removed automatically by the OS on next cleanup.
Edit:
Please look at Marc's comment below.
http://www.php.net/manual/en/features.file-upload.post-method.php
The file will be deleted from the
temporary directory at the end of the
request if it has not been moved away
or renamed.
If I send a big (200mb) file to the browser with readfile(), and while the user is downloading I modify the file, will the download finish succesfully? If not - how do I modify large files that are constantly being downloaded?
No, it will not finish successfully if you edit the file data directly. A corrupted file would be received.
However...
In Linux, if you rm or mv a file, any program that already has that file open will continue to access it. It is only once the file is closed that the file is completely released. Therefore, you could safely edit your large file with these steps:
Copy the file away to a temporary name.
Edit the file.
rm file_being_downloaded; mv new_file file_being_downloaded;
I have not tested this, but that should allow all people downloading the 200mb file to receive their copy completely in tact, and new downloaders will get your updated version.
I can't test this but most probably the download will get corrupted and readfile() will return false and throw an error. Another option, iff readfile() locks the file is that you won't be able to write to the file until it has finished reading it - I don't think this is the case though.
If I were you I would duplicate the file, serve it and then delete it. You might also want to load the file contents into memory and serve that, but for a 200MB file this would be unpractical...
$file = '/path/to/file.200mb';
$temp = tempnam(sys_get_temp_dir(), time() . '_');
copy($file, $temp);
readfile($temp);
unlink($temp);
I would say it depends on how the download is being handled.
If the file is saved to a separate memory position on the server, and you modify the actual file. Seeing as the user is trying to download the file in the alternate memory position, it shouldnt affect it.
Your best bet is to put up a new file, and then update the code to load the new file. That way users who are downloading the old file will continue to do so, but you will be able to transition to a new file.