I need to store images data in a database. I'm uploading images to a server and then, of course, I need to retrieve them from the database. I know that in the database I should only store just datas leaving the actual files to the filesystem so I'm trying to think which one may be the best way to save those datas.
What's bothering me the most is choosing a naming scheme. In the database I'll just have a filename column, where to store just the filename plus the extension. No path, so I can config it in my application. So: what shall I use for the filename? I'd like to keep a meaningful name, so it would be cool to keep the photo title, or at least a part of it. But to ensure uniqueness which one could be the best approach?
I know Codeigniter has a string helper, but I also read about using the php function uniqid. Then I can attach the id to the picture name.
What do you suggest?
Feel free to store the entire image in your database, as long as there won't be hundreds of thousands of them (or you just like paying for expensive SCSI drives :) ). If you're unsure, the file system is fine too.
For the image's unique id, the best approach is probably to just use the DB insert id (guaranteed unique, no threading/clashing/crying). So here's a good approach, assuming you want to keep the images in the file system:
create table images (
image_id int(20) primary key auto_increment, /* assumes mysql */
file_path varchar(2500) not null, /* the actual file name and full path */
file_label varchar(255), /** user's memorable name for the file */
description varchar(2500), /** user provided description of contents */
media_type varchar(100), /** something like image/jpeg; privided by php in upload */
);
Then you'll probably want to connect image_id with some sort of categorization or tags, a user_id, etc.
EDIT
Added for comments:
You can always build your links using the real file name if you like, even if you store them as simply the image id in your file system. You just need to utilize search friendly urls to do something like this (where 1234 is the image id and /images/ redirects to your php script):
mysite.com/images/db/1234/picture_of_a_cat.jpg)
Then you just redirect /images/db/ to your php script. Alternately, if you don't want to try rewriting them, you can just use $PATH_INFO.
For example, if you have mysite.com/images/render.php/1234/picture_of_cat.jpg:
<?php
$parts = explode("/",$PATH_INFO);
$image_id = $parts[3];
$image_from_db = null; //fetch the row from your dabatase here!
header("Content-type: ".$image_from_db['media_type']);
echo file_get_contents($image_from_db['file_path']);
Good luck! ;)
Well, I prefer that you use a hash in the name of the image, as can happen in case there are two images of the same name, and will generate a future problem.
Another tip, never want to stores the contents of a file in the database, the cost is much higher than just store the path to that file.
Storing images in a database is not a good idea, instead store the links to the images in the database.
For example;
create a table with the following fields
image_id //unique id of the image
image_type // type of image("png","jpg","gif".....)
image_link // path to folder where the image is located
upload date // date of upload of image(usefull for ordering in database queries)
You can add any other field you want
Related
I am developing a chat application for iOS.
In this app, the users can set up an image as their profile picture.
So my question is, how can i be able to store images in mysql ?
I have seen that many people say, just store the link to the image(on device) in mysql, but how will the images be available on different iOS Devices, from a database right ?
I have also tried using BLOB, but when the table rows are displayed(json encoded), the value for BLOB field comes out to be NULL.
Please answer in brief.
Thanks,
This is fairly simple :) The physical image will be stored on a server and you will store in your DB only the image name, or the relative path to the image, or however you want it. So, if you like, you will have to store in your DB a "pointer" to that image.
So:
- Image in folder on the server
- In DB -> path/to/file or file_name.format OR if you know you path, and you know your format just file_name
Hope this helps! :D
May be late But I just found way to do that.
Way 1: Get image from user and rename with the id (Pref. Primary key) and in Your public_html create folder for images. Set proper permissions for that folder. and save image in that folder and the url of that image will be stored DB column . Like
https://www.example.com/images/user1.png do in that style.
Way 2: use other things to rename that image like user's email,username etc
way 3: In above 2 cases it may happen that other users,hackers may try to download images by using IDs,emails etc.
That's why , another way you can do is you can generate a hash for profile and check if the hash already exists in column if exits then generate another one. Likely I don't think there is possibility of generating similar hashes.but you may check to avoid error in future. and now rename image using that generated hash.
You may also reduce size of that link column using only storing generated hash and in your app declare some variables and achieve
https://www.example.com/images/generated_hash.png
here the url will be same in all columns excluding that hash key.
you may only store that hashes.
Hope it will help
Reference from : https://www.techupdates.live/how-to-save-profile-picture-in-php-in-5-simple-steps/
I store files of users in their own name directory something like
/username/file01.jpg
/username/file02.mp4
/username/file03.mp3
But if more users come and upload more files then this creates problem because this will lead to migration of some or many users to another drive.I choose username directory solution first because i dont want filenames to be mixed. I dont want to change filename too. Also if another user upload same filename then it creates problem ,if the files are stored with original name.
What could be the best way to do this. I have one solution but want to ask community is this the best way .
i will use sequential folders and then hash the file name to some thing very unique and store into the directory.
What i will do is store the original name of file and username into database and hashvalue of filename which is stored in Disk.
When anyone want to access that file,I will read that file through php either replace the name or will do something at that point so that the file is downloaded as original filename.
I have only this proposed solution in mind. Do you guys have any other better than this one.
Edit:
I use folder system too, and possibly for 2nd way i will use virtual folders.
My database is MongoDB
Guys all your answers were awesome and really helpful. i wanted to give bounty to everyone, thats why i left it so that community can provide automatically.
Thanks all for your answers.I really appreciate it.
Could you create relational MySQL tables? e.g.:
A users table and a files table.
Your users table would keep track of everything you are (I assume) already tracking:
id, name, email, etc.
Then the files table would store something like:
id, fileExtension, fileSize, userID <---- userID would be the foreign key pointing to the id field in the files table.
then when you save your file you could save it as it's id.fileExtension and use a query to pull the user associated with that file, or all files associated with a user.
e.g.:
SELECT users.name, files.id, files.extension
FROM `users`
INNER JOIN `files` on users.id = files.userID;
I handle file metadata on the database and retrive the files with a UUID. What i do is:
Content based identification
MD5 from file's content
Namespaced UUID:v5 to generate unique identifier based on user's uuid and file's md5.
Custom function to generate path based on 'realname'.
Save on the database: uuid, originalname (the uploaded name), realname (the generated name), filesize, and mime. (optional dateAdded, and md5)
File retrival.
UUID to retrive metadata.
regenerate filepath based on realname.
Originalname is used to show a familiar name to the user that downloads the file.
I process the file's name assigning it a namespaced UUID as the database primary key, and Generate the path based on User and filename. The precondition is that your user has a uuid assigned to him. The following code will help you avoid id collisions on the database, and help you identify files by its contents (If you ever need to have a way to spot duplicate content and not necesarily filenames).
$fileInfo = pathinfo($_FILE['file']['name']);
$extension = (isset($fileInfo['extension']))?".".$fileInfo['extension']:"";
$md5Name = md5_file($_FILE['file']['tmp_name']); //you could use other hash algorithms if you are so inclined.
$realName = UUID::v5($user->uuid, $md5Name) . $extension; //UUID::v5(namespace, value).
I use a function to generate the filepath based on some custom parameteres, you could use $username and $realname. This is helpful if you implement a distributed folder structure which you might have partitioned on file naming scheme, or any custom scheme.
function generateBasePath($realname, $customArgsArray){
//Process Args as your requirements.
//might as well be "$FirstThreeCharsFromRealname/"
//or a checksum that helps you decide which drive/volume/mountpoint to use.
//like some files on the local disk and some other from an Amazon::S3 mountpoint.
return $mountpoint.'/'.$generatedPath;
}
As an added bonus this also:
helps you maintain a versioned file repository if you add an attribute on the file's record of which file (uuid) it has replaced.
create a application Access Control List if you add an attributes of 'owner' and/or 'group'
also works on a single folder structure.
Note: I used php's $_FILE as an example of the file source based on this question's tags. It can be from any file source or generated content.
Since you already use MongoDB, I would suggest checking out GridFS. It's a specification that allows you to store files(even if they are larger than 16mb) into MongoDB collections.
It is scalable, so you'll have no problems if you add another server, it also stores metadata, it is possible to read files in chunks and it also has built in backup functions.
Another tactic is to create a 2-dimensional structure where the first level of directories are the first 2 characters of the username, then the second level is the remaining characters (similar to how Git stores its SHA-1 object IDs). For example:
/files/jr/andomuser/456.jpg
for user 'jrandomuser'.
Please note that as usernames will likely not be distributed as randomly as SHA-1 values, you may need to add another level later on. Doubt it, though.
I would generate a GUID based on a hash of the filename, Date and Time of the Upload and username for the Filename, save those values, as well as the path to the file in a database for later use. If you generate such a GUID, the filenames can not be guessed.
As example lets take user Daniel Steiner (me) uploads a file called resume.doc on the 23rd of april 2013 at 37 past twelve am to your server. this would give a base value of
Daniel_Steiner+2013/23/04+00:37+resume.doc which then would be as MD5 hash 05c2d2f501e738b930885d991d136f1e. to ensure that the file will be opened in the right programm, we will afterwards add the right file ending and thus will get something like http://link.to/your/site/05c2d2f501e738b930885d991d136f1e.doc If your useraccounts already have a user id, you could add those to the URL, for example, if my User ID would be 123145, the url would be http://link.to/your/site/123145/05c2d2f501e738b930885d991d136f1e.doc
If you save the original filename to the database, you can later also offer a downloadscript that provides the file with its original filename for download, even tough it has another filename on your server.
In case you can use symbolic links, relocating the files on another harddisk shouldn't be a problem either.
If you want to, I could come up with an PHP example as well - shouldn't be too much code.
Since filesystem is a tree, not a graph (faceted classification), its hard to come up with some way for it to easily represent multiple entities, like users, media types, dates, events, image crop types etc. Thats why using relational database is easier - it is convertible to graph.
But since its another level of abstraction, you need to write functions that do low-level synchronization yourself, including avoiding name collisions, long path names, large file count per folder, ease of transfer per-entity, horizontal scaling etc. So it depends how complex your application needs to be
I suggest to use following database structure:
Where File table has at least:
IDFile is an auto_increment column / primary key.
UserID is nullable foreign key.
For FK_File_User I suggest:
ON UPDATE NO ACTION -- IDUser is auto_increment too. No changes need to be tracked.
ON DELETE SET NULL -- If user deleted, then File is not owned. Might be deleted
-- with CRON job or something else.
Still, another columns might be added to the File table:
Actual upload date and time
Actual mime-type
Actual storage place (for distributed storage systems)
Download count (another table might be a better solution)
etc...
Some benefits:
You don't need to calculate file size, hash, extension or any file meta, because you might obtain it with one database operation.
You can obtain statistics for each user of a file count / space used / whatever you wrote to File table by single SELECT ... GROUP BY ... WITH ROLLUP statement, and it would be faster, than analysis of actual files, which may be spread across multiple storage devices.
You may apply file access permissions for different users. It will cost not significant change of table structures database.
I don't consider as an option, that original filenames needed at storage, because of two reasons:
File may have name, which not correctly supported by Server OS filesystem, like Cyrillic ones.
Two different files may have completely identical names, so one of them might be overwritten by another.
So, there is a solution:
1) Rename files when they are uploaded to IDFile from INSERT into File table. It's safe and there are no dublicates.
2) Restore name of the file, when it's needed / downloaded, like:
// peform query to "File" table by given ID
list($name, $ext, $size, $md5) = $result->fetch_row();
$result->free();
header('Content-Length: ' . $size);
header('Content-MD5: ' . $md5);
header('Accept-Ranges: bytes');
header('Connection: close');
header('Content-Type: application/force-download');
header('Content-Disposition: attachment; filename="' . $name . '.' . $ext . '"');
// flush file content
3) Actual files may be stored within single directory (because IDFile is safe) and IDUser-named subdirectory - depends on a situation.
4) As IDFile is a direct sequence, if some of files are gone missing, you may obtain their database meta by evaluating missing segments of actual filenames sequence. Then, you may "inform owners", "delete file meta" or both of this actions.
I'm against the idea of storing large actual files in DBMS itself as a binary content.
DBMS is about data and analysis, it's not a FileSystem, and should never be used in that way, if my humble opinion matters.
You can install a LDAP server. LDAP lookup is very fast since it is highly optimized for heavy read operations. You can even query for data
LDAP organizes the data in a tree like fashion.
You can organize data as following example "user->IP address->folder->file name". This way file could be physically/geographically spread out and you can fetch the location very quickly.
You can query too using standard LDAP query for e.g. get all the list of file for a particular user or get the list of files in the folder etc.
Mongodb to store the actual filename (eg: myImage.jpg) and other attributes (eg: MIME types), plus $random-text.jpg from 2. & 3. below
Generate some $random-text, eg: base_convert(mt_rand(), 10, 36) or uniqid($username, true);
Physically store the file as $random-text.jpg - always good to maintain same extension
NOTE: Use filter_var() to ensure the input filename doesn't pose security risk to Mongodb.
Amazon S3 is reliable and cheap, be aware of "Eventual Concurrency" with S3.
Assuming users have a unique ID (Primary Key) in the database, if a user with ID 73 uploads a file, save it like this:
"uploads/$userid_$filename.$ext"
For example, 73_resume.doc, 73_myphoto.jpg
Now, when fetching files, use this code:
foreach (glob("uploads/$userid_*.*") as $filename) {
echo $filename;
}
This can be combined with hashing solutions (stored in the DB), so that a user who gets a download path as 73_photo.jpg does not randomly try 74_photo.jpg in the browser address bar.
This question already has answers here:
Closed 11 years ago.
Possible Duplicates:
Images in MySQL
Storing images in MySQL
I'm trying to develop a website where users upload their images as part of registration. I want it that for each image, there should be a thumb created with PHP (which is not that difficult). I want to save the thumbs (since they are very small) in the database and I use MySQL. (I don't want to save the thumbs as physical files on the drive.)
Does MySQL allow saving and retrieving image data and how do I go about it? If it doesn't support image data, is there any free database that does? I will be happy if a link can be provided.
Thanks.
Yes, you can store images in the database, but it's not advisable in my opinion, and it's not general practice.
A general practice is to store images in directories on the file system and store references to the images in the database. e.g. path to the image,the image name, etc.. Or alternatively, you may even store images on a content delivery network (CDN) or numerous hosts across some great expanse of physical territory, and store references to access those resources in the database.
Images can get quite large, greater than 1MB. And so storing images in a database can potentially put unnecessary load on your database and the network between your database and your web server if they're on different hosts.
I've worked at startups, mid-size companies and large technology companies with 400K+ employees. In my 13 years of professional experience, I've never seen anyone store images in a database. I say this to support the statement it is an uncommon practice.
You'll need to save as a blob, LONGBLOB datatype in mysql will work.
Ex:
CREATE TABLE `test`.`pic` (
`idpic` INTEGER UNSIGNED NOT NULL AUTO_INCREMENT
PRIMARY KEY,
`caption` VARCHAR(45) NOT NULL,
`img` LONGBLOB NOT NULL
)
As others have said, it's bad practice but it can be done. Not sure if this code would scale well, though.
You can store images in MySQL as blobs. However, this is problematic for a couple of reasons:
The images can be harder to manipulate: you must first retrieve them from the database before bulk operations can be performed.
Except in very rare cases where the entire database is stored in RAM, MySQL databases are ultimately stored on disk. This means that your DB images are converted to blobs, inserted into a database, and then stored on disk; you can save a lot of overhead by simply storing them on disk.
Instead, consider updating your table to add an image_path field. For example:
ALTER TABLE `your_table`
ADD COLUMN `image_path` varchar(1024)
Then store your images on disk, and update the table with the image path. When you need to use the images, retrieve them from disk using the path specified.
An advantageous side-effect of this approach is that the images do not necessarily be stored on disk; you could just as easily store a URL instead of an image path, and retrieve images from any internet-connected location.
You will need to store the image in the database as a BLOB.
you will want to create a column called PHOTO in your table and set it as a mediumblob.
Then you will want to get it from the form like so:
$data = file_get_contents($_FILES['photo']['tmp_name']);
and then set the column to the value in $data.
Of course, this is bad practice and you would probably want to store the file on the system with a name that corresponds to the users account.
I'm making an image gallery script with PHP which is including a comment system.
The basic function of the script is to read the folder's name and to show a gallery with the folder's
name as title and it's images.
The problem is, that the comment system, which is made with mysql saves a comment like this:
ID | Folder | Image
This works great, but when I change the name of the folder, the script isn't working because there is set the wrong foldername in the Mysql table.
Also when I just save the image's name there is the possibillity of having two pictures in diffrent
folders with the same name.
Is it possible to identify a folder without using it's name?
//Note:
Thank you for your help, I think I found a solution. Now I just save the md5 hash of the image and save it in the Database instead of the foldername.
I guess it's better use another table where you'll save folders ids and their names
folder_id | folder_name
Something like that. Relying on folder names is not a good practice
No. You need to synchronize your filesystem with your database. Whenever you rename/remove/edit a folder, update its entry in the mysql database.
One possibility could be to look at using a checksum of the image contents as a unique identifier. As long as the content doesn't change then you should be OK. As far as identifying changing names of the folders I can't think of any cunning way of detecting this unless you look at using a hash that 's a function of the files contained in it.
I think to be honest that it would be good to re-think the way you're storing/identifying these items in the database.
You could always just do it this way:
Create 3 tables:
galleries -> with the columns
(id,title)
files -> with the columns (id,
gallery, name, filename)
comments -> with the columns (id,
file_id, comment)
You would then create all your galleries with their name in the "galleries" table. When you created the gallery you would put all the images that you want to display in a folder, for example uploads/. You would then insert the name of the file (Maybe generate random filenames too) in the database along with it's real name. For example "Dolphins at Beach" could point to dolphins1234142.png that's in the uploads folder. All your comments would then be in the comments table where you would setup a relationship to the files table. This will allow you to have comments for a certain image.
Just my view on a gallery system, hope it makes sense.
I have a form where an admin will upload three pictures with different dimensions to three different designated directories. now to make sure that i don't get into the problem of duplicate file names i implemented something like the php will compare the uploaded file name and it will check if that file name exist in the designated directory if yes then it will echo an error and stop the script execution.
Now one of my friend suggested me that it is very bad asking the admin to manually rename the picture file and asking them to take care of the file duplication problem. the solution he suggested was to rename the file automatically and then store it in the database and then direct it to directory.
I am confused about what combination should i give to the renamed file and also make sure it will remain unique file name to be more precise i would like you to understand my directory structure
as i said there will be three pictures files the admin will be uploading namely
a) Title Picture b) Brief Picture c)
Detail Picture
and all the three picture files will be moved to the different respective directory, like title picture goes to title directory and so on.
i am using to script below currently just to move and store the file name with path using varchar in the database.
$ns_pic_title_loc= $_FILES["ns_pic_title"]["tmp_name"];
$ns_pic_title_name = $_FILES["ns_pic_title"]["name"];
move_uploaded_file($ns_pic_title_loc, $ns_title_target.$ns_pic_title_name) or die(mysql_error());
that is just the sample code i havent included the validation function which i am using. i was thinking like i want to rename all the files like
a) In title directory the file should be stored as.
title_1.jpg
title_2.jpg
title_3.jpg
title_4.jpg
and so on
and the same way to rest of the pictures. how do i do that? what function do i use to achieve my target. and if this is not the good way to rename the file i would appreciate any suggestion followed to rename the file.
thanks in advance
Well, here's a possible solution:
Get uploaded filename from $_FILES["ns_pic_title"]["name"] and separate extension OR if we are only talking about image files get the image type with getimagesize($_FILES["ns_pic_title"]["tmp_name"]);
Check your database for the maximum id of the image records and make the the $file_name variable 'title_'.($max_id + 1)
At this point you should have $file_name and $file_extension so do move_uploaded_file($_FILES["ns_pic_title"]["tmp_name"], $ns_title_target.$file_name.'.'.$file_extension)
Hopefully this makes sense and helps.
There are a couple of good options with various pros and cons.
Use php's tempnam when moving the file, and store the path in your mysql database. tempnam generates a unique filename.
Use mysql to store the image content in a blob. This way you will access the image content via an id instead of a pathname.
Instead of having logic to figure out what the latest picture name is and calculate the next number increment, why not just use PHP's tempnam() function? It generates an unique name with a prefix of your choice (i.e., "title", "brief", "detail"). You could also simply prepend a timestamp to the file name -- if you don't have a whole lot of admins uploading pictures at the same time, that should handle most name conflicts.
Since your pictures are going to be sorted into title, brief and detail directories already, it's not really necessary to name each picture title_*, brief_*, and detail_*, right? If it's in the title directory, then it's obviously a title picture.
Also, you're going to be putting the file names in the database. Then elsewhere in the app, when you want to display a picture, I assume you are getting the correct file name from the database. So it isn't really important what the actual file name is as long as the application knows where to find it. If that's correct, it's not necessary to have a very friendly name, thus a tempnam() file name or a timestamp plus the original file name would be acceptable.
Because you are storing references into the DB, I would prefer to just md5 the datetime and use that for the filename and store the disk filename to the DB also. It doesn't matter what name it is written to disk with as long as you can point to it with the unique name into the DB.
I use this methodology, and in none of my testing does the disk name (md5 from the datetime) ever require multiple tries.