I need to write a script to upload big Files (~2GB+) to a server.
I don't think HTTP is the right way to do this so I want to use (S)FTP.
There are several tutorials about this (using cURL or ftp_connect) and I understand that i have to set several things in the php.ini.
But all this tutorials upload the file to a remote Server, what I want to do is upload it to the Server the script is running on, without having to upload the file to the Server over HTTP first.
Is this possible? If so, how would I do that?
HTTP can be the right way to upload large files. You can use resumable.js or similar library to split the file in "chunks" and then reassemble the file on the server.
If you decided not to go with HTTP and have shell access, I recommend you use rsync (with the --partial flag) which will do the heavy lifting for you.
Related
I have a standard HTML form, including a file input, allowing users of a web application to upload files (pictures, documents or videos).
It technically works, except when it comes with a large file...
I have a personal dedicated server on which I can change the PHP configuration to handle larger files ;
But this is not a reliable solution as my client has a shared hosting.
I know HTTP has different limits and is definitely not the best protocol to handle files.
So my objective is to avoid HTTP upload.
I was wondering if there is any way to rely on FTP to upload the selected file.
Any solution to upload files through FTP, directly from client's browser ?
EDIT :
I've read some solutions about Java applets, however this is not
really something I can, or even want to, provide.
And as the files are confidential, using a third-party service is
not possible neither.
I'm going to be dealing with large files (max of 2GB) that clients will use to upload raw images/long pdfs/etc. I've read online that PHP isn't the best solution but I can't have every single client download an FTP program, have me make a username/directory for them, and give them a password just to upload a file once.
What I am asking is what is a better, faster function to use, or does it not matter? ftp_get() or move_uploaded_file() ? Is there a better way I can accomplish this using PHP?
I was thinking since it's ftp that ftp_put would be faster and more viable, but i'm not really sure which is why I am asking this question.
Thanks for your input.
You will always have to use move_uploaded_file() after you have a file upload. This is to move the file from the temporary storage to wherever you want to store it (usually a uploads/ dir in your application's tree).
ftp_put() is to move that file again off site to another FTP server.
This is not the same job.
First case, your user upload a file directly to your server, and then you process it using move_uploaded_file.
Pros: User does not require to previousely send a file on a ftp server.
Cons: You're directly involved with the user who generally have a slow connection, this require that you allow very long connection which is potentially dangerous.
Second case, your user upload a file to an FTP server, built for such uploads. Then, you'll download a file that is generally on a remote server with a good bandwidth and upload speed.
Pros: You don't need to configure your web server to handle very-long connections and very large inputs, better for security. Download will be generally faster, so the user will feel your services aren't that slow.
Cons: Your user require to send a file to a ftp, that's not very convenient for someone who don't even know what it is.
A better solution should be to provide both options, according to the size of the file.
And you put your FTP in the same server as the web's one, in such a way, you can access the file directly in the filesystem without ftp_get, this should be even faster.
Is it possible to upload files (even big files) to a ftp using PHP?
Been reading about ftp_connect() and it looks like I can, or can't I?
I had a look at this example, it's in Italian but you can read the code anyway, if that does what I'm asking, will I have to add an html form? I need to be able to pick up a file form my computer via a web page and upload it to an ftp basically.
Anyone?
Thanks
Especially on large files, you should make sure that the maximum execution time for the script is big enough to complete the transfer before the script is aborted. You can choose the maximum execution time in the php.ini file.
You will have to use an HTML form if you want to pick up the file from your computer via a web page.
As soon as the form is submitted, you can access the file using the $_FILES array. You can use this information to get a temporary path to where the file is stored, and can read it from there to upload it to a remote server using the FTP functions.
You could also split the two processes by using the PHP script only to drop the file into the local file system, and then use a second program which runs locally to do the upload. This has the advantage that you won't run into problems when multiple users upload simultaneously and your FTP is set up in a way that it allows only 1 simultaneous connection. You could program the second script also in PHP and run it using a Cronjob for example once per 30min.
If your goal is a direct stream from your computer to the FTP server, this is not easily possible using a pure PHP / HTML solution since the PHP script is only invoked when the file transfer from your computer to the machine serving the PHP script is complete.
After tearing my hair out for the last week, I am looking for some sort of web uploader that allows my customers to upload a bunch of files (often up to 200) and store them to a remote FTP server. What I am looking for is something similar to uploadify, swfupload etc. but has the possibility to upload files via my web page (at my hosting company) and stored to my local ftp server.
I am looking for something similar to uploadify, swfupload and such, but it is absolutely critical that it has the possibility to store the files on my local server.
If this is somehow impossible to do, it could also just upload the files to my website via html (which uploadify etc. does) and after completion copy the files from the web server to my local ftp.
The closest thing i found was something called filechunker and it looked like the perfect solution, BUT it wont let me add multiple files, just one by one.
All help would be greatly apreciated!
Unfortunately I can't give you a concrete answer, but let me say that it should be theoretically possible to do for a Flash or Java application since they can use raw TCP sockets and implement the FTP protocol (but I am not aware of any Flash-based implementation).
If I'm not wrong all major browsers offer native file upload via FTP by browsing to the FTP directory itself (but you can't influence the visual appearance), just like Windows Explorer can access FTP servers and use them like a network drive.
However, I discourage you from using a FTP server at all. That protocol with it's double connection and that passive/non-passive modes often causes problems. It's usually much better to upload via HTTP and implement a HTTP-based file server yourselves, which is rather easy after all (but be very careful not to expose too much of your server's file system).
I see no real reason for using FTP unless you really want to allow your users to use their FTP client of choice, but that is contrary to your question.
Hope this helps.
Update: I just noticed the sentence "copy the files from the web server to my local ftp". In case you are really talking about two different servers I would still suggest a HTTP upload and then forward the file to the FTP server via the PHP script (your web server acting as a proxy).
I don't think it's feasible to upload directly from the browser to your FTP as you would have to have your credentials more or less visible on the website (e.g. in your javascript source).
I once created something similar, but because of that issue I decided to upload via plupload to Amazon S3 and sync the files afterwards via s3sync. The advantages were
Large filesizes (2GB+)
One time Tokens for upload, no need to send credentials to the client
no traffic to your web server (The communication runs client->s3)
Take a look at this thread for an implementation: http://www.plupload.com/punbb/viewtopic.php?id=133
After a wild search i finally found something that I could use. This java applet lets me upload endless amounts of files, zips them down and i managed to pass a php variable into the applet so the zip file is stored with the users e-mail adress as the filename. Cost me $29 though, but well worth it since I now have full control of where the files go, and who uploadeded them.
am working on a website and i have a big problem when i tried to upload files, i increase upload_max_filesize and post_max_size and the code still understand only as a max. 10M. for any different folder php accepts 100M. but inside the site folder ( which am working inside) it doesn't understand it. i check for local php.ini or .htaccess.
note: am running a linux server.
For uploading bigger files I would suggest a dedicated uploader plug-in.
Like a SWF of Java. For these reasons:
Security - you can easily encode the sent data (encoding ByteArray in AS3.0 is very easy, can be even tokenized so it is hard to intercept the stream)
Reliability - with simple HTTP requests it is hard to actually monitor the upload progress, so the user might choose to close the uploaded (because he thinks it got stuck)
User friendly - again, progress bar
Not limited by server - if you accept it directly with PHP custom code, you won't need any configuring for annoying things like max file size on upload.
On server-side you will need either a Socket listener, or an HTTP tunnel if unavailable.
You can use JumpLoader, which is a Java applet, and it is able to split large files into partitions, and upload them one by one. Then a PHP script rebuilds the original file from the uploaded partitions on the server.
Plupload can split large files into smaller chunks. See the documentation.
Do you run Apache with mod_security? Then check if the LimitRequestBody is in affect.
Here is a good tutorial about Settings for uploading files with PHP.
Thanks guys,
I found the problem. i don't know why is the file is not visible for me /public_html/site/.htaccess
i tried to overwrite it, and it's seems to be working.
Thanks a lot for efforts.