I am trying to upload a file via HTTP and run it through a virus scan and immediately move to a ftp location. Here are the basic steps,
Upload via HTTP
Do the virus scan on the php tmp directory (upload_tmp_dir) using php command line functions.
Move the file directly to ftp from the tmp directory after the virus check.
You might notice I am not doing a move_upload_file(), is this the best way to do it ? Or should I do steps 2 & 3 after I do a move_upload_file().
If I should not do it before move_upload_file() then what would be the reason ?
Thanks
I think that doing it before move_upload_file is appropriate. If you move it first, then you've put a file into your FTP directory which may have a virus; for the length of time that you scan, an unsafe file is available.
Also, if you do it before move_upload_file and find a virus, you can just leave it there and PHP will tidy up for you; no need to call unlink yourself :)
Related
My application is keeping watch on a set of folders where users can upload files. When a file upload is finished I have to apply a treatment, but I don't know how to detect that a file has not finish to upload.
Any way to detect if a file is not released yet by the FTP server?
There's no generic solution to this problem.
Some FTP servers lock the file being uploaded, preventing you from accessing it, while the file is still being uploaded. For example IIS FTP server does that. Most other FTP servers do not. See my answer at Prevent file from being accessed as it's being uploaded.
There are some common workarounds to the problem (originally posted in SFTP file lock mechanism, but relevant for the FTP too):
You can have the client upload a "done" file once the upload finishes. Make your automated system wait for the "done" file to appear.
You can have a dedicated "upload" folder and have the client (atomically) move the uploaded file to a "done" folder. Make your automated system look to the "done" folder only.
Have a file naming convention for files being uploaded (".filepart") and have the client (atomically) rename the file after upload to its final name. Make your automated system ignore the ".filepart" files.
See (my) article Locking files while uploading / Upload to temporary file name for an example of implementing this approach.
Also, some FTP servers have this functionality built-in. For example ProFTPD with its HiddenStores directive.
A gross hack is to periodically check for file attributes (size and time) and consider the upload finished, if the attributes have not changed for some time interval.
You can also make use of the fact that some file formats have clear end-of-the-file marker (like XML or ZIP). So you know, that the file is incomplete.
Some FTP servers allow you to configure a hook to be called, when an upload is finished. You can make use of that. For example ProFTPD has a mod_exec module (see the ExecOnCommand directive).
I use ftputil to implement this work-around:
connect to ftp server
list all files of the directory
call stat() on each file
wait N seconds
For each file: call stat() again. If result is different, then skip this file, since it was modified during the last seconds.
If stat() result is not different, then download the file.
This whole ftp-fetching is old and obsolete technology. I hope that the customer will use a modern http API the next time :-)
If you are reading files of particular extensions, then use WINSCP for File Transfer. It will create a temporary file with extension .filepart and it will turn to the actual file extension once it fully transfer the file.
I hope, it will help someone.
This is a classic problem with FTP transfers. The only mostly reliable method I've found is to send a file, then send a second short "marker" file just to tell the recipient the transfer of the first is complete. You can use a file naming convention and just check for existence of the second file.
You might get fancy and make the content of the second file a checksum of the first file. Then you could verify the first file. (You don't have the problem with the second file because you just wait until file size = checksum size).
And of course this only works if you can get the sender to send a second file.
I have a directory on a remote machine in which my clients are uploading (via different tools and protocols, from WebDav to FTP) files. I also have a PHP script that returns the directory structure. Now, the problem is, if a client uploads a large file, and I make a request during the uploading time, the PHP script will return the file even if it's not completely uploaded. Is there a way to check whether a file is completely uploaded using PHP?
Setup your remote server to move uploaded files to another directory, and only query the directory files are moved to for files.
AFAIK, there is no way (at least cross-machine) to tell if a file is still being uploaded, without doing something like:
Query the file's length
Wait a few seconds
Query the file's length
If it's the same, its possibly completed
Most UNIX/Linux/BSD-like operating systems has a command called lsof (lsof stands for "list open files") which outputs a list of all currently open files in the system. You can run that command to see if any process is still working with the file. If not, your upload has finished. In this example, awk is used to filter so only files will show that are open with write or read/write file handlers:
if (shell_exec("lsof | awk '\$4 ~ /.*[uw]/' | grep " . $uploaded_file_name) == '') {
/* No file handles open for this file, so upload is finished. */
}
I'm not very familiar with Windows servers, but this thread might help you to do the same on a Windows machine (if that is what you have): How can I determine whether a specific file is open in Windows?
I would think that some operating systems include a ".part" file when downloading a file, so there may be a way to check for the existence of such a file. Otherwise, I agree with Brian's answer. If you were using the script on the same system it is simple enough to tell using move_uploaded_file()'s return if it was being uploaded by a PHP script, but it does become a challenge pulling from a remote directory that can be added to with different protocols.
So a PHP program I built to download files from one server and upload them to another is now needed to rename the files.
For some reason I'm getting a 'invalid argument' warning on the rename() function which seems impossible because the file exists and there is 0 white space.
So now I was wondering if it was possible to rename a file as I download it from an ftp using ftp_get()?
http://php.net/manual/en/function.ftp-get.php
Would suggest so. Give it a local file name.
Also make sure you are using rename() correctly
http://uk.php.net/manual/en/function.rename.php
I'm uploading files via JS and storing the temp path in the session.
Than i use the following code to move the files.
if(move_uploaded_file($_SESSION['temp_img'][$key]['path'], $dest.$bigimg)){
$dest and $bigimg are defined earlier in the script with the id from the database.
Any Ideas or alternatives ?
MANCHUCK's answer was close but not quite there. You must call move_uploaded_file within the script where the file was uploaded. You cannot do what you're doing, that is, "storing temp path in the session" because that path is only valid for one request.
From the PHP manual:
The file will be deleted from the
temporary directory at the end of the
request if it has not been moved away
or renamed.
(Emphasis mine)
move_uploaded_file checks that a file has been uploaded to that page. You are actually uploading the file to a different PHP script then storing in a session. Instead of using move_upload_file use rename.
What is the output of $_SESSION['temp_img'][$key]['path'], also do you have permission to write to the web directory your placing the files. You may need to set it to 777 for some hosts to allow the webserver to write there.
I've seen ways to upload files directly to S3 and I've seen ways to upload a file already on the server. I need to modify the file's name before sending it on to S3, but I don't want to store it locally and then upload it, if I can help it. Is there a way to put it in a buffer or something? Thx.
The file will always end up in the temporary directory first while the upload completes, even before you're able to work with the uploaded file. You get all the file's chunks before, and then it get rebuilt in the /tmp directory by default. So no, there's no "pass-through". But I guess you could re-upload directly from the temporary directory afterwards instead of moving it to another working directory.