I have seen many questions concerning the storage of user uploaded image files onto a web application, but most of these are dealing with the following:
Indexing of the images, so as to retrieve them later
How to store them (on the server itself as a file or in the database)
I have a question in regards to this subject, but the question is:
In what directory do I put the uploaded image file? (or other file type, for that matter)
I have a small group I am running php apps for. There is very little files that get uploaded, but nontheless, they get uploaded.
I currently have them in my public html document root under /var/www/images/* , however I am told that it is not smart to store your user uploaded content straight to the /var/www/* directory and that it should be stored elsewhere.
However I cannot find a straightforward statement of where "elsewhere" is.
Keep in mind I do not have a server farm where I can establish certain servers for specific purposes (such as uploaded user files).
Therefore, on a single webserver that hosts usual scripting files, etc. what is the best storage practice for such content?
Thank you.
I don't think there's necessarily a 'best practice' per se; anywhere on your server will be fine, so long as you're able to retrieve the images later on. Typically they'd go inside a folder under /var/www/images/.
Personally I'd recommend creating an individual folder to store these user-uploaded images in (as something like /var/www/images/user_uploads), so that they don't get confused with other images you might have uploaded directly to /var/www/images/ (such as backgrounds or core imagery).
Related
I'm facing a dilemma on how to implement file upload and download in a PHP website.
I have these criteria:
Performance - does not give performance issues to the website
File size - around 2GB and up.
Authorization - I want to be able to change who can access the files in PHP. Allow multiple users to gain access to a single file.
User friendly - no additional tools to use.
So here are the methods I'm currently looking at and how I assess them based on my criteria:
Database BLOB
Writing the file data into the output stream will take time and blocks other requests (is this correct?)
I read somewhere that there's a size limit for BLOB.
OK - I can easily control who can download the files here.
OK - No additional tools, just the website.
FTP
OK - since it is designed to store files.
OK - file system is the limit.
I need to create another credentials for each user aside from the username and password for the website. I assume I have to move the file from one location to another to update authorization, but how if multiple users can access one file? Shared directory? It looks messy.
Need another tools/program for accesing their files, need to remember another username and password.
My questions:
Based on my assumptions, do you thimk I understand the methods correctly?
If my assumptions are wrong, is there a way I can do this functionality while meeting my criteria?
PS Please excuse my English.
Why not just use the file system to store the files and store the path to the given file (+ permissions, if needed) in addition in a database.
The upload folder isn't accessible from the public and an wrapper script serves the content to the user.
Performance shouldn't be a problem as you just move/copy the uploaded file to a dedicated data directory.
File size isn't a problem (as long, as you have enough disk space)
The wrapper script handles permissions and serves files to the users
It's as friendly, as you design your ui
for me the usage of BLOB is not the best. I thought about BLOB to upload pictures in my own website, but the best way to upload file is to put them directly on ur server locally.
As the title says, i am concerned about when i should upload a file on a database rather than the server.
In these days i first encountered the necessity of uploading .pdf files into my web application in order to make it accessible for everyone who needs it.
But my first approach at it was uploading a file into a database.
But that wasn't my case; i just needed to upload a file and, with a download link, make it downloadable from the user.
But on the other side, i wondered, since the big limitations of "blob" type: when should i upload a file into a database rather then the server?
I have always preferred to store items on the filesystem and store the paths in the database.
I think the only advantage to putting it in a BLOB is that you can move the database around and not have to worry about distributing files. For normal websites this wouldn't really matter but for some kind of product like a CMS or website addon it would be a handy way to include a resource file with the distribution...
The reason I prefer the filesystem is because that is what the filesystem is designed for - serving files. I don't see any valid reason to put all that stress into a bottleneck by pushing all the data through the database.
On the other hand the only time I reccomend storing files as a BLOB in SQL is when the images or files contain sensitive data (i.e. Medical Images, etc.) where you need the security that SQL Server provides. Otherwise storing and reading the file from a filesystem is much faster; you can keep a reference to the file's location in SQL Server for the best method.
I have made a database with tables like projects, employees.
I have some fotos and pdf files (for instance, scan of certificates) related to the entries in my DB, that should be accessible via our internal web site.
Does anyone have any suggestions on how I should manage this?
I was thinking on setting up a subdomain "files.ourdomain.com", and create subdirectories there for each table. And make a directory for each record? Should i create a DB field for "employee.certificates" with the entire path/filename of the certificate foto?`
Or should i actually store the files in the database? (MySql INNODB)
In a comment, Peter said that it's easier and better to store the files in a database for security reasons and there is a case for this - only DBA's and the webservers will have access to the files and they could be encrypted. I don't agree with the "easier" bit though as you would need to encode the image before placing it into the DB and decode again when you want to display it.
You have a choice:
you store the files actually in the database itself
you store the file in a folder (outside of the web tree) and only store the
filename to the file
Each method has pros and cons:
If you store the files actually within the database you could take the backup file and put it onto another server and everything will be present. However, if you're using replication or clustering then each server will have a copy of the file and so storage requirements increase. Obvious, you will also need more space for backups, and backing up and restoration will take proportionally longer.
If you store the file in a central location and only record the location, your DB storage requirements are lower and multiple DB servers can be confident that there is only ever one copy of a file. Again, the files can be backed up separately. The downside is "what happens if your file storage server fails". However, with mirroring and backups, this can be mitigated against.
In both cases, you would need to store a filename for each file so that the webservers can use them.
Have a look at this Stack Overflow question which does a much better job of the pros and cons than I ever could.
Speaking personally, I store a filename which links to an external file.
I have a site that allows people to upload files to their account, and they're displayed in a list. Files for all users are stored on different servers, and they move around based on how popular they are (its a file hosting site).
I want to add the ability for users to group files into folders. I could go the conventional route and create physical folders on the hard drive, for each user on the server, and transverse them as expected. The downside to that is the user's files will be bound to a single server. If that server starts running of space (or many files get popular at the same time), it will get very tricky to mitigate it.
What I thought about doing is keeping the stateless nature of files, allowing them to be stored on any of the file servers, and simply storing the folder ID (in addition to the user id who owns the file) with each file in the database. So when a user decides to move a file, it doesn't get physically moved anywhere, you just change the folder ID in the database.
Is that a good idea? I use php and mysql.
Yes, it is.
I don't see any downside, except maybe more queries to the database, but with proper indexing of the parent folder id, this will probably be faster than accessing the filesystem directly.
Forget about folders and let the users tag their files, multiple tags per file. Then let them view files tagged X. This isn't much different to implement than virtual folders but is much more flexible for the users.
A while a go I had to developed a music site that allowed audio files to be uploaded to a site and then converted in to various formats using ffmpeg, people would then download the uploaded audio files after purchasing them and a tmp file would be created and placed at the download location and was only valid for each download instance and the tmp file would then get deleted.
Now I am revisiting the project, I have to add pictures and video as upload content also.
I want to find the best method for storing the files,
option 1 : storing the files in a folder and reference them in the database
option 2 : storing the actual file in the database(mysql) as blob.
I am toying around with this idea to consider the security implications of each method, and other issues I might have not calculated for.
See this earlier StackOverflow question Storing images in a database, Yea or nay?.
I know you mentioned images and video, however this question has relevance to all large binary content media files.
The consensus seems to be that storing file paths to the images on the filesystem, rather then the actual images is the way to go.
I would recommend storing as files and storing their locations in the database.
Storage the files in a database requires more resources and makes backing up/restoring databases slower.
Do you really want to have to transfer lots of videos every time you do a database dump?
File systems work very well for dishing out files, and you can back them up/sync them very easily.
I would go for the database option. I've used it on a number of projects, some very larger 100+GB. The storage implementation is key, design it poorly and your performance will be punished. See this example for some good implementation ideas:
Database storage allows more scalability and security.
I would go for storing files directly on the disk, and database holding only their ID/url.
This way accessing those files (that can be large, binary files) doesnt require any php/database operation, and it's done by the webserver directly.
Also it will be easier to move those files to another server if you'd want to.
Actually only one upside I can see atm of storing them in database is easier backup - you wanna backup your DB anyway, this way you'll have all data in one place and you can be sure that each backup is full (i.e. you don't have files on disk that aren't used by database entries; and you don't have image IDs in your database that point to nowhere)
I asked a similar question using Oracle as the backend for a Windows Forms application.
The answer really boils down to your requirements for backing up and restoring the files. If that requirement is important then use the database as it'll be easier (as you're backing up the database anyway, right? :o)