I'm looking at building a web app that includes a file upload element. I'd like users to be able to upload files of any type and of fairly large size (say, up to 100MB). This will be a publicly accessible site, so security is obviously very important.
I've done a decent amount of googling in search of answers, but it's difficult when I don't really know exactly what I'm searching for.
My experience is mainly with PHP, but I realise that PHP is not considered to be the best when it comes to file uploading, so I'm happy to look at other languages if necessary. Although, if a decent solution using PHP can be acheived, that would be preferable.
As I have no experience with this kind of project, I'm also fairly in the dark on what kind of server setup is required for such an app.
I have braistormed a few ideas, but am willing to budge on them if unreasonable:
I'd like to use Amazon S3 to store the files if possible (to reduce the load on the server)
I'd like to be able to rename the files after upload
I'm considering Uploadify (uploadify.com) for the client side
Basically, imagine I was looking to build a file-sending app like wetransfer.com or yousendit.com and you'll get the general idea.
I'm familiar with all the usual PHP file upload issues (checking mime-types, upload_max_filesize, memory_limit, etc, etc) covered by 99% of posts on the internet on this topic, but obviously this project goes a fair bit beyond your average, run-of-the-mill avatar upload script.
I know this is a massive topic and I'm obviously not expecting anyone to present me with a magic solution, but basically I'm looking for some pointers on where to start. Can anyone recommend any good books, articles or websites where I can gain a better understanding of the requirements of the task? Covering everything from the programming to the server requirements? Even if it's just a list of keywords or phrases that I should be googling.
Thanks in advance!
P.S. I wasn't 100% sure if this was the right StackExchange site to post this question on. I also considered serverfault.com and webmasters.stackexchange.com. If you think this question would be better asked elsewhere, please let me know.
If you funnel the upload through your PHP you need to make sure that it accepts those large files. Especially upload_max_filesize, post_max_size and max_input_time. See POST method uploads for a general how to.
With Resumable.js you could circumvent above limitations quite nicely. It uploads small chunks of your 100MB at a time. This allows it to keep track of what's been uploaded to allow pause/resuming uploads.
While I've never worked with Amazon S3, I do not believe you can upload data from any client - at least not without some sort of authentication. You'll probably have to funnel the upload through your own server in order to push it to S3.
Related
I made a basic upload script by using the w3schools tutorial (here) and I was basically wondering if there was anyway to increase the upload size to above 5Gb. I tried changing the script to upload large video files but it errors telling me that the file is too big.
Would anyone be able to help me with the issue that I am having
To try the uploader for yourselves (uploader), it accepts .mov formats, but like I say my error lies within the upload file size.
As others mentioned in the comments you can 'chunk' the file into slices, and upload them piece-by-piece. The server will merge the files after all slices were uploaded.
A working demo for this is http://dnduploader.filkor.org.
You can learn from it's source code, what you can find on Github (see the FAQ section on the page).
In general, it's not an easy task to create such an uploader, but from it you can learn some novel ideas about the current 'trends' of file uploading..
As mentioned by Critical Point, you will most likely need to chunk the upload.
There are a number of other additional factors to be aware of - for example error recovery and connection interruption.
It may not be suitable for your uses, but in the past I have had success using a commercial component:
http://www.aurigma.com/UploadSuite/
There is a fairly steep entry price, but provided are multi-platform components for both client and server side. In terms of comparison to building and troubleshooting a home-made solution, Aurigma provides great ROI.
I'm looking for a tool to facilitate mulitple webpage file uploads from a single file browse dialogue. I know this has been asked previously, but I can't find anything current.
I'd like to check file size prior to upload, and I gather Flash is still the only way to do that cross-browser?
Ideally, I'd like an upload progress metre. I'll be using Linux and Apache servers, but don't have access to install add-ons such as PHP APC. Again, I assume something flash-based is the only option there?
I've looked at SWFUpload, but that appears to be another of these projects where the developers have become quite zealous and turned a simple concept into a full suite of tools for the masses. It seems quite cumbersome and I don't think I want to use it for my purpose.
I'd prefer not to have to write something from scratch for this. Could someone recommend me something or perhaps suggest a non-Flash alternative if there is one? I do need full cross-browser compatibility without too many layers of degradation, so anything HTML 5 probably isn't what I want.
Thanks
As I mentioned earlier today ( Multiple file upload (client side) )
I am a big fan of Plupload which can check file size, show progress bar, single dialog for multiple files, and supports things other than Flash if needed.
I am interested in finding the most reliable method for uploading files in PHP. I need a progress bar with the upload.
I have tried SWFUpload but it randomly issues an I/O Error. Even if the same file is uploaded sometimes there is an error and sometimes there is not. I have configured all the necessary INI/Mysql/Apache directives to accept large file uploads.
So, I am looking for alternatives as a Flash based solution has not worked. Would Java be more reliable? I have also looked into PHP with APC.
I definitely cannot afford these random errors, so any help on reliable software / suggestions on how to minimize them would be appreciated.
Thank you.
There are other flash based solutions other than SWFupload. Have a look at uploadify.com
I haven't come around to try this myself yet but http://www.plupload.com/ might be what you're looking for on. But otherwise PHP + APC works good as well.
I am assuming 2 things here:
1) Some kind of client will be doing the file upload
2) You get some kind of say on what the client installs on their computer to help make this happen.
If this is the case, my first suggestion would be:
Give them FTP or SFTP client software to upload files. The php page you make can have a link to Filezilla, along with instructions on how to use it. ftp and sftp are THE protocols to use for transferring files. HTTP is just not designed(well) for it, nor are browsers.
Before anyone rips me a new one...I HAVE PERMISSION to hotlink images from an external site. It works all good, however I don't like that everytime i refresh the page it pulls the images again. My server is running PHP, is there a way to cache the images once, then display them via some local code. I'm really just looking for a way to speed up the page, and not waste anyones bandwidth. Thanks in advance.
I was looking for an answer to this myself and didn't find anything that fit my needs perfectly. TimThumb came close (you'll have to Google it; I'm a newbie and can thus only post one hyperlink), but it was a little overkill (it has all kinds of image manipulation stuff built-in) and couldn't handle some of the image types I was interested in using (specifically *.ico files). So I wrote my own quick-n-dirty PHP script that should handle any image type and is only concerned with caching the images alone and passing them through without any modifications.
I'm a bit concerned my script may have glaring security flaws or could be more efficient. Also, it's not very smart the way it caches. It never bothers to check later to see if the image has been updated, and it never bothers to clean up its own cache. If anyone has suggestions for improvements to my code, I'm open to feedback.
Here's the script: Warm linker - RefactorMyCode.com
You might consider using a proxying CDN like CoralCDN.
I have a problem with maintenance of my php based website. My website is built on the Zend Framework. When I wish to upload a new copy or version online - during the time of upload especially when crucial files like models and controllers are being uploaded and rewritten - the site won't run understandably.
Is there a way to upload a website without having to go through this issue?
My updates are really quite regular. Let's say like once or twice a week in this case.
You can make use of the fact that renaming directories is quick and easy even through FTP. What I usually do is:
Have two directories, website_live and website_upload
website_live contains the live website (obviously)
Upload contents to website_upload
Rename website_live to website_old (or whatever)
Rename website_upload to website_live
done! Downtime less than two seconds if you rename quickly.
It gets a bit more complex if you have uploaded content in the old version (e.g. from a CMS) that you need to transfer to the new one. It's cumbersome to do manually every time, but basically, it's just simple rename operations too (renaming works effortlessly in FTP as well).
This is a task that can be automated quite nicely using a simple deployment script. If you're on Linux, setting up a shell script for this is easy. On Windows, a very nice tool I've worked with to do automated FTP synchronizing, renaming and error handling - even with non-technical people starting the process - is ScriptFTP. It comes with a good scripting language, and good documentation. It's not free, though.
If you're looking to get into hard-core automated PHP deployment, I've been doing some research in that field recently. Maybe the answers to my recent bounty question can give you inspiration.