I'm using Blueimp jQuery File Upload with chunked uploads to handle large files, and everytime file upload are completed or file are removed, I update the client's disk space used in database.
My problem is: when the upload process is interrupted, I just can delete incomplete file from client-side by using fail callback to send a DELETE request to server, like this guide : https://github.com/blueimp/jQuery-File-Upload/wiki/Chunked-file-uploads#deleting-aborted-chunked-uploads.
But if clients' internet connections has been lost or the browser is closed, the server will not get any request, there's no update in database too, and the corrupted file still exists. When the client re-connects to the site and remove file, it will cause wrong amount of disk space used in database.
Is there any way to remove imcomplete file of chunked upload from server-side or another solution for my case ? Please help me.
Related
Hi I am wondering if the input field with type "file" is automatically uploading files to the server when there is a file selected by the user.
The thing I want is that a user can select files to submit it and that a PHP FTP connection is established to upload the files.
I am not sure if the browser pre-uploads the files to a temp directory on the server or does it do that only when you hit the submit button?
Because if the file is already uploaded to the server it would be unnecessary to use FTP to upload the file again only to a different location on the server.
Basically what I want to accomplish is something similar to what we transfer does. I believe they are not uploading before the user hits the button.
Can anybody point me in the right direction and provide me with some extra info about this matter. Haven't found the desired information yet.
Thanks in advance.
The file is uploaded only once the form is submitted (in the same HTTP POST request as all other form fields). Why don't you try it, to see yourself?
And you cannot use PHP to upload a file via FTP from a client to a server. That's not possible. PHP runs on the server, it cannot access files on a client's machine. See also:
Upload a local file from application via web server with PHP code to FTP server
PHP uploading file from browser to FTP.
I am gonna to save file when the file is still uploading to the server!
For example, the client uploads a video to the server and I want to get that file and save it when client waits to finish it. in other words, I want to save uploaded file every 100kb.
Maybe you ask me, why I am going to do this! Here is why:
I am creating a video conferencing stream by PHP, I want to save uploading video and get it in another clients device!
Can I use php:// to get that file?
reference
after strugelling for about a week to get an old ASP.net site to accept a multipart form Post request I finally got it to work with a bit of a work around the trick is that Jmeters multipart form functionallity wasnt working properly for this site, so now i have to write the Post request in the Body manually. But the problem is again on the upload file part. I am able to upload empy .txt files but i am unable to upload .png .jpeg i susspect that i am doing something wrong. I read on a site that i am supposed to use _FileToString somehow to upload an existing file but i have no idea how i am supposed to do that.
Full POST request image
WebKitFormBoundaryBbo97BWEj5UehRpp
Content-Disposition: form-data; name="ctl00$ContentPlaceHolder1$8$9$fu_appDocument$fu_document";
filename="StacktraceKnijki.png"
Content-Type: image/png
Can someone help me turn this into a working upload file request? Thank you.
Why don't you just record the file upload request using JMeter's HTTP(S) Test Script Recorder? JMeter should be smart enough to produce the relevant HTTP Request sampler from the captured request.
The only pre-requisite is having your StacktraceKnijki.png file in JMeter's "bin" folder during the recording process, see Recording File Uploads with JMeter guide for more details.
My application is keeping watch on a set of folders where users can upload files. When a file upload is finished I have to apply a treatment, but I don't know how to detect that a file has not finish to upload.
Any way to detect if a file is not released yet by the FTP server?
There's no generic solution to this problem.
Some FTP servers lock the file being uploaded, preventing you from accessing it, while the file is still being uploaded. For example IIS FTP server does that. Most other FTP servers do not. See my answer at Prevent file from being accessed as it's being uploaded.
There are some common workarounds to the problem (originally posted in SFTP file lock mechanism, but relevant for the FTP too):
You can have the client upload a "done" file once the upload finishes. Make your automated system wait for the "done" file to appear.
You can have a dedicated "upload" folder and have the client (atomically) move the uploaded file to a "done" folder. Make your automated system look to the "done" folder only.
Have a file naming convention for files being uploaded (".filepart") and have the client (atomically) rename the file after upload to its final name. Make your automated system ignore the ".filepart" files.
See (my) article Locking files while uploading / Upload to temporary file name for an example of implementing this approach.
Also, some FTP servers have this functionality built-in. For example ProFTPD with its HiddenStores directive.
A gross hack is to periodically check for file attributes (size and time) and consider the upload finished, if the attributes have not changed for some time interval.
You can also make use of the fact that some file formats have clear end-of-the-file marker (like XML or ZIP). So you know, that the file is incomplete.
Some FTP servers allow you to configure a hook to be called, when an upload is finished. You can make use of that. For example ProFTPD has a mod_exec module (see the ExecOnCommand directive).
I use ftputil to implement this work-around:
connect to ftp server
list all files of the directory
call stat() on each file
wait N seconds
For each file: call stat() again. If result is different, then skip this file, since it was modified during the last seconds.
If stat() result is not different, then download the file.
This whole ftp-fetching is old and obsolete technology. I hope that the customer will use a modern http API the next time :-)
If you are reading files of particular extensions, then use WINSCP for File Transfer. It will create a temporary file with extension .filepart and it will turn to the actual file extension once it fully transfer the file.
I hope, it will help someone.
This is a classic problem with FTP transfers. The only mostly reliable method I've found is to send a file, then send a second short "marker" file just to tell the recipient the transfer of the first is complete. You can use a file naming convention and just check for existence of the second file.
You might get fancy and make the content of the second file a checksum of the first file. Then you could verify the first file. (You don't have the problem with the second file because you just wait until file size = checksum size).
And of course this only works if you can get the sender to send a second file.
I have a site that is receiving 30-40k photo uploads a day and I've been seeing an issue pop up with more frequency now. This issue is this:
Our upload script receives (via $_FILES['name']['tmp_name']) a file (photo) that was NOT uploaded by the user & the majority of the time the file received is a "partial" upload.
Of course at first I thought it was my PHP code making a simple mistake and I've spent days looking over it to make sure, but after placing checks in the code I've found that the file received via a HTTP POST upload to PHP is actually the wrong file. So the issue is happening before it reaches my code. The tmp file (phpxxxx) received by the script is sometimes incorrect, as if it was somehow being overwritten by another process and its usually overwritten by a file that was partially uploaded.
Has anyone every seen an issue like this? Any help is greatly appreciated. I'm turning to this as a last resort after days of searching/asking other PHP devs
So to recap:
User uploads a photo
PHP script receives a file that was not uploaded by the user (pre code, via $_FILES in /var/tmp)
Usually the incorrect file received is a partial upload or a broken upload
It seems to happen randomly and not all the time
First off, check you PHP version.
Second, check your file upload limits and POST_MAX_SIZE in php.ini
It might just be that someone tries to upload a file that's too large :-)
Can you try different names for the temp file to avoid its being overwritten? Can you identify the origin of the new, incorrect and incomplete file?
Is this a development environment? Is it possible that more than one user is uploading files at the same time?
Try your program with very small images to check if SchizoDuckie is correct about filesize problems.
Try with different navigators to eliminate the admittedly remote possibility that it is a local problem.
Check permissions on the directory where the temp file is stored.
PHP's built-in file handling does not support partial uploads.
Turn off KeepAlives and/or send a 'Connection: close' header after each upload.
Configure your webserver to send the header 'Allow-Ranges: none'.