I'm implementing profile pictures in my android application. I came to a decision that I don't know which road to take. The problem is that I have to store the images that users upload and view them also to their contacts / friends, as in telegram I see the images of others and they see mine.
1) First hypothesis:
I know that MySQL can store images in a field with the "blob" (tell me if I'm wrong), so you just have to upload and download images from grabbing this field.
2) Second hypothesis:
The user loads the image, I memorize in a folder (a bit 'like a cloud), unless its path on the server in a field in the database (for example in the field photo I put "http:site/project/image.jpg ") and I download the photos to his friends in this way.
I do not know how to do, which one should I choose? The first method (to store the file directly in mysql) does not make me become the database slower and less accurate?
The application handles messages, post etc. (as in a social network), the pictures can give problems ??
The second method seems faster though less elegant. I trust in your wise advice, thanks for the reply: D
IMHO - store images as files and store only their paths to database.
Related
I have a website where users each have their own profile page. Here they can upload a single image that acts as their avatar. This is the only image users can upload to the server across the whole site. There is no archive so it can be overwritten if a user wishes to update their avatar.
I have never had to do anything like this before so I would like to open it up and ask for a suitable, scalable option for this website.
My initial thought is to give each user's image a random name, a string 6-12 characters long. This way you couldn't build a script that just pulls every user's profile pic from a directory (eg 001.png, 002.png etc). My second thought is that there should be only be a certain amount of images per directory to make sure they can be retrieved quickly by the server.
There may well be other things I'm missing here, I'm not sure on exact details hence why I'm asking.
I would recommend storing the images on something like Amazon S3. Depending on how many pictures you're storing, serving images can really take a tow on your web server. S3 is scalable and with multi-zone deployments through CloudFront (Amazon's version of a CDN), you can really speed up this part of your service.
It's good idea to not overload single directory. Very often you can see that images are stored in hierarchy of folders according to theirs first few letters. An example of this is
b5dcv5.jpg -> /b/5/b5dcv5.jpg
bsgb0g.jpg -> /b/s/bsgb0g.jpg
a5dcbt.jpg -> /a/5/a5dcbt.jpg
and so on. I thing you got the principle. Advantage of this is to have access to and image in O(log N) when filenames are uniformly distributed instead of O(N) as it would be in single folder solution.
I've been using base64 to store them within an SQL database. No need to manage files. It works well for relatively low resolution options.
How about not storing them as images at all?
You could leverage an external placeholder for each user, you could cache a random image from lorempixel.com: http://lorempixel.com/100/100. Use an MD5 hash of the user's name or ID. You could also just save the image using the user's ID, for example 442.jpg.
Lets say I'm building a image gallery using PHP, where users would be able to upload their photos.
Every user would have 1 folder on server side with all their images there.
Now lets say I need to provide information in browser. Users would be able to browse images and should see lots of information about them, like image size, image dimensions, even EXIF information etc.
I could do this in 2 ways:
Save all information about image into database when uploading image.
Use PHP functions to browse through folder, and get information from every image.
I have something like file manager class, that can do all manipulations with files on server side, like deleteDir, deleteFile, countItems, getFileSize, getDirSize.
And it would be easy to only write one more class that would inspect images, and then I could just upload images, and get their information right from the folders without a need for relation database.
And now the question you all been waiting for is: ... :)
What would be faster, first or second solution? Lets say that site gets loads of traffic.
What solution would be better if I want it to be fast, and not to stress server to much?
actually, I got this situation like yours, this is my solution:
Save all information about image into database when uploading image.
Why?
I tested 2 ways:
Using php to get the image info for 1000 times.
Getting image info from database for 1000 times.
And the result is :
Getting image info from database is faster and faster.
Last but not least:
What would you do if you want to do a image info analystics?
If you save all info in database ,you can easily get them and analyse them ,but if you using php to get the info? it's hard to image.
So, just save all information about image into database when uploading image.
Good luck.
storing it in the database once
reading the data from the database and store it in cache,
redoing things always costs especially if it happens all the time
Depending on the size of these images, you probably want to show thumbnails instead of the original when people are browsing, which means you need to generate them. I would generate the thumbnail on upload and grab all the file info. Then save the file info in the database and put the original and thumbnail in the file system. If you get a lot of traffic, throw memcache on there too.
Storing data in separate places has a way of creating maintenance headache. I would just serialize the metadata for images in each folder and dump it to a file there. If you use gzip compression on the file, retrieval and storage should be very fast.
First sorry I'm a big beginner.
Would like to ask a more experienced developers opinion.
I have small website a really small social network for sports, and I would like to allow the users to create image folders and upload multiple images there.
As I was reading through the internet, they say that it is bad to store the images in the database, only save the location of the image.
But this is the part what I don't understand, more precisely the logic what I don't really understand.
I go to the users profile select the folder, but what is the way to show the images?
Is it a good idea to select the folder and use scandir to get the images? And if I'm scanning the folder is it possible to limit it to, for example, limit it to only one image at first what points to the gallery?
And I was thinking to separate the location path table in the database tied to the user ID, won't that be a problem? Won't that make the database really large?
Sorry if these are stupid questions, I would just really like to know.
Now I don't want anybody to write this for me, just give an opinion about the logic.
Thank you.
I would recommend you to store all information about the folder and pictures in the database.
For example a schema like the one in the diagram below. The advantage of having the data in a database is that you can get as many pictures as you want. And later you can add additional information to the table (e.g., permission for other users, comments, etc.)
I have no idea how the big websites save the pictures on their servers. Could any one tell me how do they save the pictures that are uploaded by the users in their database?
I was thinking, maybe they would just save the file(the picture) in some path and just save that path in the databse is that right?
But I want to do it this way. Is this right? For example, a website named www.photos.com. When a user uploads a picture I would create a folder of the user name and save those pictures in that folder.
I believe we can create a directory using php file concepts. So when a new user uploads his picture or file, I want to create a directory with his name.
Example: if user name is john, I would create a directory like this on photos.com www.photos.com/john/ and then save all his pictures to this directory when he uploads a picture. Is this the right way to do this?
I have no one here that has good knowledge of saving the files to servers so please let me know how to do this? I want to do it the correct and secure way.
All big websites don't save pictures to the database they store them in the disk.
They save a reference to the picture's position in a table. And then link from there.
Why? Performance.
Pulling heavy content from a database is a huge performance bottleneck. And databases don't scale horizontally that well, so it would mean even a bigger problem. All big sites use static content farms to deal with static content such as images. That's servers who won't care less about your identity.
How do they keep the pictures really private you might ask? They don't.
The picture's link is, in itself, the address and the password. Let's take Facebook, for example. If I store a private picture on my account you should not be able to open it. But, as long as you have the correct address you can.
This picture is private. Notice the filename
10400121_87110566301_7482172_n.jpg
(facebook changes the url from time to time so the link may be broken)
It's non sequential. The only way to get the picture is to know it's address.
Based on a previous user photo you can't guess the next one.
It has a huge entropy so even if you start taking random wild guesses you'll have an extensive amount of failures and, if you do get to a picture, you won't be able to, from there, realize the owners identity which, in itself, is protection in anonymity.
Edit (why you should not store images in a "username" folder:
After your edit it became clear that you do intent to put files on disk and not on the database. This edit covers the new scenario.
Even though your logic (create a folder per user) seams more organized it creates problems when you start having many users and many pictures. Imagine that your servers have 1T disk space. And lets also imagine that 1T is more or less accurate with the load the server can handle.
Now you have 11 users, assume they start uploading at the same time and each will upload more than 100GB of files. When they reach 91GB each the server is full and you must start storing images on a different server. If that user/folder structure is followed you would have to select one of the users and migrate all of his data to a different server. Also, it makes a hard-limit on a user who can't upload more than 1T in files.
Should I store all files in the same folder, then?
No, big-sites generally store files in sequential folders (/000001/, /000002/, etc) having an x defined number of files per folder. This is mainly for file-system performance issues.
More on how many files in a directory is too many?
It is usually a bad idea to store images in your database (if your site is popular). Database is, traditionally, one of main bottlenecks in most any application out there. No need to load it more than necessary. If images are in the filesystem, many http servers (nginx, for example) will serve them most efficiently.
The biggest social network in Russia, Vkontakte does exactly this: store images in the filesystem.
Another big social network implemented a sophisticated scalable blob storage. But it's not available to the public, AFAIK.
Summary of this answer: don't store blobs in the database.
is this the right way to do
Yes.
The only thing I'd suggest to use not name but id.
www.photos.com/albums/1234/ would be okay for starter.
Image management may best be achieved by physically uploading images to the server and then recording file location and image details in a database. Subsequently, a Search Form could be configured to permit the user to do a text search, part number search, or other queries. A PHP script could be written to produce a valid HTML image tag based on data found in the table.
uploading images into a MySQL™ BLOB field is such a bad idea such image data is generally problematic if the images are much larger than thumbnails. If the images are large, you can end up having to copy/paste one SQL INSERT statement at a time (into phpMyAdmin). If the images are large and the SQL INSERT statement is broken into two lines by your text editor, you'll never be able to restore the image.
I am planning to do a photo album website, So each user may upload as many number of images. What is the best way to keep track of images for an individual user. What should be the server configuration to handle this part.
-Lokesh
Depending on the amount of images, you will probably want to store them on a static domain. Then, have a table in whatever database you are using to store the paths to each of the images for each user.
Well like many design topics there are lots of different ways to go about it. Two ways that come to mind right now are as follows.
you could simply have a directory created on the server for each user and then have the images each use uploads saved into that directory. Ofcourse you'd want to make sure they didn't over write any existing images with images of the same name. You could do this by warning them about conflicting names or by adding some sort of noce string (like a time stamp) to the end of of the file name. This is a pretty straight forward solution and means that you can login to your server and see all the images each user has uploaded right there for you to do anything you like with.
Another idea would be to save the images in a database. This can be done by serializing the images to a string and storing it in a database. This is nice becaues it means you don't have to worry about handling directories and duplicate file names. You will have to deserialize each image when you want to display it which will put your DB under load so for a very high traffic volume site this might not really be the way to go.
There are ofcourse combinations of these ideas and many others. It really comes down to working out which solution best fits your exact needs.