First sorry I'm a big beginner.
Would like to ask a more experienced developers opinion.
I have small website a really small social network for sports, and I would like to allow the users to create image folders and upload multiple images there.
As I was reading through the internet, they say that it is bad to store the images in the database, only save the location of the image.
But this is the part what I don't understand, more precisely the logic what I don't really understand.
I go to the users profile select the folder, but what is the way to show the images?
Is it a good idea to select the folder and use scandir to get the images? And if I'm scanning the folder is it possible to limit it to, for example, limit it to only one image at first what points to the gallery?
And I was thinking to separate the location path table in the database tied to the user ID, won't that be a problem? Won't that make the database really large?
Sorry if these are stupid questions, I would just really like to know.
Now I don't want anybody to write this for me, just give an opinion about the logic.
Thank you.
I would recommend you to store all information about the folder and pictures in the database.
For example a schema like the one in the diagram below. The advantage of having the data in a database is that you can get as many pictures as you want. And later you can add additional information to the table (e.g., permission for other users, comments, etc.)
Related
I'm implementing profile pictures in my android application. I came to a decision that I don't know which road to take. The problem is that I have to store the images that users upload and view them also to their contacts / friends, as in telegram I see the images of others and they see mine.
1) First hypothesis:
I know that MySQL can store images in a field with the "blob" (tell me if I'm wrong), so you just have to upload and download images from grabbing this field.
2) Second hypothesis:
The user loads the image, I memorize in a folder (a bit 'like a cloud), unless its path on the server in a field in the database (for example in the field photo I put "http:site/project/image.jpg ") and I download the photos to his friends in this way.
I do not know how to do, which one should I choose? The first method (to store the file directly in mysql) does not make me become the database slower and less accurate?
The application handles messages, post etc. (as in a social network), the pictures can give problems ??
The second method seems faster though less elegant. I trust in your wise advice, thanks for the reply: D
IMHO - store images as files and store only their paths to database.
Since I am just experimenting on this, (only localhost) I may like to ask for some ideas(since nothing is really coming out of my mind) about letting a user, who is going to, for example, register to a mini-social-networking site, with a corresponding username/password, personal details, etc. I would upload the image, and save it to a folder(ON MY HARD DRIVE be it Drive C:\ or D:), for example '/images/username' and the full path of the folder would be the one inserted to a row named img_dir (of course it is a string, instead of putting it as a BLOB, so later i would just use img src="path"). I would not mind where it will be going to be saved. But since I am new to cakephp i haven't really grasped the idea of what I am going to do. I have no problems about registering/login sessions. This was easy in C# but I am too stupid for PHP maybe? :P
While this may not give you a direct solution in CakePHP, you had asked for some ideas.
I've outlines some pros and cons of storing a file on the filesystem (along with some other approaches) in this post.
Hope that helps...
I've written a complete plugin for that kind of task and it's more thought through then just the idea of saving the file path.
A file has some more meta data like it's size and mime type which is useful when the file is served. So an uploaded file should be handled as an entity of it's own. I personally think it is a bad idea to directly save the path to a file within the record it belongs to. What happens if you need two images later? Adding incrementing fields like path1, path2?
It is IMO better to have a separate table for files and associate records with these file records. Expressed in CakePHP associations: User hasOne Avatar or Gallery hasMany Image for example.
Also saving files in path like this uploads\username1\pic.jpg can result in slowing down the app because of file system performance issues if you get a lot directories and files within the same level of the file system.
However, check my plugins readme.md out, there is more about why it does things like it does to solve different kinds of issues you can run into.
I'm coding a basic gallery for a website with around 40.000 online people at any given time. Users will be able to create galleries and upload images.
My question is, should I make a seperate folder for each gallery and put the images in them, or make a single folder and put all images in it, but keep the gallery_id for each image in the database? Or, should I make a directory for every user, then another directory inside them for the gallery names?
How would you do this?
Ps. I need it to be as light as it can.
I would store them by id
and i would split them into folders (dependant of filesystem, some don't perform well with lots of files in 1 folder), plus it is easier to find them if you have to manually look at something
Give each file an id, then using the first 3 digits of the file name, split them into folders. (you could start your auto-increment counter at 100000 or zero pad the id, so there is at least 3 levels
/photos/1/0/3/103456.jpg
/photos/9/4/1/941000.jpg
/photos/0/0/0/000001.jpg
You can store the relationship of photo to user / gallery / etc in the database
Or if you want to see how the big boys do it
Needle in a haystack: efficient storage of billions of photos
Typically web servers don't want you to have more than a few thousand images in a single folder (I recently had to deal with 70,000 images causing super slow reads and sorts so trust me on this) so certainly not a single folder if you think you will have thousands of images. I would suggest the best solution would be to host off of amazon's S3 connected to their CDN CloudFront but if that isn't realistic you can still do several things just on your own server.
Make a separate folder for each gallery like you suggest only if you know some bounds on how large a gallery can get and have an idea of how many galleries will be created. (This is what I would suggest for your specific problem right now)
Put the image name through a hash function then use the first 1-3 characters of the hash to name folders to put the images into. The hash ensures that the images are roughly equally split among the folders and you can decide how many folders you need.
At any rate having the information of what gallery and the image id in the actual path will probably be useful to you moving forward both in code and whenever a human has to hunt bugs on the server. I would probably name the folders based on the gallery id and just make sure that no gallery has more than a few thousand images in it.
I store mine like this:
images/userid/photoid
This way I can quickly isolate user images if I need to inspect anything at a later date. It seems more organized than dropping them all in one central directory.
I have no idea how the big websites save the pictures on their servers. Could any one tell me how do they save the pictures that are uploaded by the users in their database?
I was thinking, maybe they would just save the file(the picture) in some path and just save that path in the databse is that right?
But I want to do it this way. Is this right? For example, a website named www.photos.com. When a user uploads a picture I would create a folder of the user name and save those pictures in that folder.
I believe we can create a directory using php file concepts. So when a new user uploads his picture or file, I want to create a directory with his name.
Example: if user name is john, I would create a directory like this on photos.com www.photos.com/john/ and then save all his pictures to this directory when he uploads a picture. Is this the right way to do this?
I have no one here that has good knowledge of saving the files to servers so please let me know how to do this? I want to do it the correct and secure way.
All big websites don't save pictures to the database they store them in the disk.
They save a reference to the picture's position in a table. And then link from there.
Why? Performance.
Pulling heavy content from a database is a huge performance bottleneck. And databases don't scale horizontally that well, so it would mean even a bigger problem. All big sites use static content farms to deal with static content such as images. That's servers who won't care less about your identity.
How do they keep the pictures really private you might ask? They don't.
The picture's link is, in itself, the address and the password. Let's take Facebook, for example. If I store a private picture on my account you should not be able to open it. But, as long as you have the correct address you can.
This picture is private. Notice the filename
10400121_87110566301_7482172_n.jpg
(facebook changes the url from time to time so the link may be broken)
It's non sequential. The only way to get the picture is to know it's address.
Based on a previous user photo you can't guess the next one.
It has a huge entropy so even if you start taking random wild guesses you'll have an extensive amount of failures and, if you do get to a picture, you won't be able to, from there, realize the owners identity which, in itself, is protection in anonymity.
Edit (why you should not store images in a "username" folder:
After your edit it became clear that you do intent to put files on disk and not on the database. This edit covers the new scenario.
Even though your logic (create a folder per user) seams more organized it creates problems when you start having many users and many pictures. Imagine that your servers have 1T disk space. And lets also imagine that 1T is more or less accurate with the load the server can handle.
Now you have 11 users, assume they start uploading at the same time and each will upload more than 100GB of files. When they reach 91GB each the server is full and you must start storing images on a different server. If that user/folder structure is followed you would have to select one of the users and migrate all of his data to a different server. Also, it makes a hard-limit on a user who can't upload more than 1T in files.
Should I store all files in the same folder, then?
No, big-sites generally store files in sequential folders (/000001/, /000002/, etc) having an x defined number of files per folder. This is mainly for file-system performance issues.
More on how many files in a directory is too many?
It is usually a bad idea to store images in your database (if your site is popular). Database is, traditionally, one of main bottlenecks in most any application out there. No need to load it more than necessary. If images are in the filesystem, many http servers (nginx, for example) will serve them most efficiently.
The biggest social network in Russia, Vkontakte does exactly this: store images in the filesystem.
Another big social network implemented a sophisticated scalable blob storage. But it's not available to the public, AFAIK.
Summary of this answer: don't store blobs in the database.
is this the right way to do
Yes.
The only thing I'd suggest to use not name but id.
www.photos.com/albums/1234/ would be okay for starter.
Image management may best be achieved by physically uploading images to the server and then recording file location and image details in a database. Subsequently, a Search Form could be configured to permit the user to do a text search, part number search, or other queries. A PHP script could be written to produce a valid HTML image tag based on data found in the table.
uploading images into a MySQL™ BLOB field is such a bad idea such image data is generally problematic if the images are much larger than thumbnails. If the images are large, you can end up having to copy/paste one SQL INSERT statement at a time (into phpMyAdmin). If the images are large and the SQL INSERT statement is broken into two lines by your text editor, you'll never be able to restore the image.
I am planning to do a photo album website, So each user may upload as many number of images. What is the best way to keep track of images for an individual user. What should be the server configuration to handle this part.
-Lokesh
Depending on the amount of images, you will probably want to store them on a static domain. Then, have a table in whatever database you are using to store the paths to each of the images for each user.
Well like many design topics there are lots of different ways to go about it. Two ways that come to mind right now are as follows.
you could simply have a directory created on the server for each user and then have the images each use uploads saved into that directory. Ofcourse you'd want to make sure they didn't over write any existing images with images of the same name. You could do this by warning them about conflicting names or by adding some sort of noce string (like a time stamp) to the end of of the file name. This is a pretty straight forward solution and means that you can login to your server and see all the images each user has uploaded right there for you to do anything you like with.
Another idea would be to save the images in a database. This can be done by serializing the images to a string and storing it in a database. This is nice becaues it means you don't have to worry about handling directories and duplicate file names. You will have to deserialize each image when you want to display it which will put your DB under load so for a very high traffic volume site this might not really be the way to go.
There are ofcourse combinations of these ideas and many others. It really comes down to working out which solution best fits your exact needs.