What I know, in Database Context, BLOB or Binary Large OBject is nothing but actually a stored binary code for a given data. Can Reserves spaces in GBs and can be used to store virtually any data type. But What's actually a use of it?
My major is Computer Vision and I'm fairly novice at databases and web development. Currently, I'm working on a sentiment analysis project and want to collect a large dataset for this purpose i.e. huge number of images and also want to keep record of whether a image has been used for the analysis purpose or not. I thought storing images in database with separate column for access record is the best thing I can do to have an organized and systematic approach. But Everyone I talked with recommends not to store image as a blob in database but just have its URL or name there and should have images in a dedicated folder.
Moreover, since BLOB is just binary encoding of a file how would we decode it into an image file? I found codes like following to convert a BLOB value into an image:
echo '<img src="data:image/png;base64,' . base64_encode($image->getimageblob()) . '" />';
echo '<img src="data:image/jpg;base64,' . base64_encode($image->getimageblob()) . '" />';
But these codes are specific to the extension (And personally I haven't been successful with any such codes). As all extensions for sure have some different schemes and thus a code cannot be used for image of all those extensions. My dataset targets visuals of an image and not on extension thus contains images of various extensions so how can one deal with them using a BLOB?
So the approach of storing just names in database and and images in a dedicated folder sounds good but then what is the use of database itself? Can not we have some renaming mechanism for images via PHP and store them directly into that folder. Why use database when we can rename images like img_1_accesses_5.png and split image name to get the ID and number of times it accessed?
If BLOB can store virtually every type of data, why the use of BLOB is such horrible and everyone recommends not to use it? And what is the problem if we directly inject images into database as BLOB? And finally If its suitable for images then how to deal with it?
So my question is How to effectively use BLOB and for which purposes it is suitable?
So my question is How to effectively use BLOB and for which purposes it is suitable?
Quick and dirty answer
The simple answer is: BLOBs smaller than
256KB are more efficiently handled by a database,
while a filesystem is more efficient for those greater
than 1MB. Of course, this will vary between
different databases and filesystems
There is a microsoft technical report here : Compare blob and ntfs filesystem . The report is quite old (2006) but i think there isn't any much change from there.
Imaging when you want to read file which stored in blob. you have send request to your database software, then the software controller will read blob data which is stored in filesystem. Instead of directly read from file-system, you have to go through 2 steps processes. So when the size of your file become bigger, blob will slow down your database a lot. And we all know that speed is the main key for database.
Hope that help
Related
I'm making an android application which takes a photo and push the image (as a base64 encoded string) to a PHP script, from here I'll be storing data about the image inside a MySQL database.
Would it be wise to store the image inside the database (since it's passed as a base64 string), would it be better to convert it back to an image and store it on the filesystem?
A base64 encoded image takes too much place (about 33% more than the binary equivalent).
MySQL offers binary formats (BLOB, MEDIUM_BLOB), use them.
Alternatively, most people prefer to store in the DB only a key to a file that the filesystem will store more efficiently, especially if it's a big image. That's the solution I prefer for the long term. I usually use a SHA1 hash of the file content to form the path to the file, so that I have no double storage and that it's easy to retrieve the record from the file if I want to (I use a three level file tree, first two levels being made respectively from the first two characters and the characters 3 and 4 of the hash so that I don't have too many direct child of a directory). Note that this is for example the logic of the git storage.
The advantage of storing them in the DB is that you'll manage more easily the backups, especially as long as your project is small. The database will offer you a cache, but your server and the client too, it's hard to decide a priori which will be fastest and the difference won't be big (I suppose you don't make too many concurrent write).
I've done it both ways, and every time I come back to code where I stored binary data in a MySQL table I always switch it to filesystem with a pointer in the MySQL table.
When it comes to performance, you're going to be much better off going to the FS as pulling multiple large BLOBs from a MySQL server will tend to saturate its pipe quickly. Usually it's a pipe you don't want clogged.
You could always save the base64_encode($image) in a file and only store the file path in the database, then use fopen() to get the encoded image.
My apologies if I didn't understand the question correctly.
"wise" is pretty subjective, I think. I think it would be wise from a "keep people from directly linking to my images" perspective. Also, it may be helpful as far as if you decide you need to change up dir structures etc.. it might make it easier on you (but this really depends on how you wrote your scripts to begin with..) but other than that... offhand I can't really think of any benefits to doing this.
I'm creating a blog with a featured image on each post. I have a dilemma, I'm unsure what to do with my image data...
Should I insert image data into my MYSQL database using BLOB?
Or should I just create an uploader which makes a directory into the users images folder and upload the photo that way...then just reference it directly in the image field when adding a Blog Post?
Is there a standardised way?
Kind regards,
adam
Upload the files to your server and save the location of the file in your database. Less strain on your DB and your HTTP daemon is better at serving images than MySQL.
The general approach is not to store files in DB, unless you understand why do you need it to be stored there. So, since you are not sure, it's much simplier storing them in upload folder.
But, just in case you decide you need storing files (no matter images or some other) in DB, you have to declare BLOB field and then save it using some BLOB-supporting DB mechanism. 'PHP's MySQLi extension: Storing and retrieving blobs' is a good example of how it can be made
You should store images in folder. Click on below link from where you can get idea how to crop different-different size images and store images name in to database table:
How can I upload images in a normal insert form (MySql)? after upload the image should have three versions of different sizes and different names
convert the image data to base64. This can be done within PHP:
<?
$image=file_get_contents("image.png");
$image=base64_encode($image);
?>
Storing images in a DB is a good idea for secure images.
Always store images, music files etc in system files on disk, and then store the urls to them in the database. That will make it
1) faster
2) easier to configure security settings
3) better in any ways I can imagine
Disadvantage
If file system is corrupted you will have hard time recovering.
You can also use third party Image hosting sites too, you can use Amazon S3 or Mosso Cloud Files.
Problem with file system is it is difficult to scale.
Facebook uses cassandra to store images.
Since it is blog you can store images in filesystem.
Both are valid approaches.
They have different advantages/disadvantages.
Storing it in the database means you need to add extra code to change the image to a representation which will fit inside a INSERT/UPDATE statement (base64 is one approach, and requires equivalent decode, but you could just use mysql_real_escape_string()). Although you can't query the image directly (other than finding exact matches) it may reduce the number of seek and I/O operations required to retrieve the data compared with looking up the path in the database then retrieving the file.
It's also a lot simpler to set up replication of a database compared with setting up replication of the database AND the filesystem if you run on multiple nodes. And there's the issue og keeping filesystem and database backups synchronized.
OTOH, using a filesystem makes your data tables much smaller, and therefore faster to retrieve records from.
which makes a directory into the users images folder
You certainly don't want to allow users to upload content directly into your webserver's document tree - regardless of which route you take, the data should be stored in a location not directly accessible by the webserver but accessible by your code.
I have 500k unique images in my folder/directory. I want call it by name and all names are stored in Mysql database. but I heard that images can be stored in a database. So my question is which is more fastest option to display image faster. do I need to store in mySQl or can I keep same method which I am following?
If I need to store in mySQL then how do I create a table for it, and how do I store all these images?
This has been answered quite a few times. But you haven't talked about what type of application that you are building, if you are building a web application then storing the images on the file system has advantages re: caching.
Storing Images In Filesystem As Files Or In BLOB Database Field As Binaries
Storing Images in DB - Yea or Nay?
It's easy enough to store the images in a table, I would definitely avoid it though if your images are quite large.
I do not think 500k entries in a single directory will go over very well: How many files can I put in a directory?
Once upon a time, Linux ext3 started running slowly for very simple operations once a directory accumulated around 2000 files. (O(N) sorts of slowly!) After the htree patches were merged, large directory performance improved drastically, but I'd still recommend partitioning your image store based on some very easy criteria. (Squid web cache, for example, creates directory trees like 01/, 02/, and places files based on the first two characters in the filename.)
Do not store so many data in a db like mysql especially if you are not so familiar like you sound. Keep the images on the fs
I have 500k unique images in my
folder/directory. I want call it by
name and all names are stored in Mysql
database. but I heard that images can
be store in database. so my question
is which is more fastest option to
display image faster.
You should use the file system. Storing in database is not going to work very well. You should read Facebook Photo Storage Architecture to learn how facebook does it. They have the most photos in the world.
Haystack file storage:
Also interesting:
http://www.fredberinger.com/high-performance-at-massive-scale-lessons-learned-at-facebook/
Storing images into the database (in a blob datatype) in much more inefficient than keep those images stored on the file system.
BTW here is explained how to insert binary data into a mysql table
If you have Redis, you could put all the images in memory, that would be the quickest way
which is a better place to upload images to? A database or in the web directory? And why?
You should only store images in your database if you have a specific need to, like security, or like an absolute to-die-for need to keep all custom data in a database.
Other than that, getting large files into databases usually isn't worth the trouble. Storing and retrieving the file get that much more complicated to implement, and database updates/upgrades/conversions have that many more things that can go wrong.
I don't see that there is an advantage storing images in a database. There is certainly no inherent security in this. Files are for the filesystem so store your images in there.
I don't think you can "upload" an image to a database. You can store the image's string value in the database and stream it via "header("Content-Type")" later on. That saves space in your web server, but obviously takes space on your database.
If I were you, I'd upload to a web directory, that way you have the image for a regular URL request later on. If you don't have it in a regular directory, you'll have to connect to the database every time the image is requested, and stream it then.
Well It depends on your requirement.
If you are considering security as a major issue then definitely you should store it in db other wise nothing will leads you to store images in db.
Also retieving images from database is quite complicated as in database images are stored as binary data. So if you have specific need then only store images in database other wise storing images in directory would be fine.
As you can see there are many reasons why to use/why not to use the database for image storage. Personally I prefer not to use the database for storage of files (images, documents etc), except when I'm ordered to store them.
-Sometimes you're tired and screw up a query, something like "SELECT * FROM images", this will kill the server if there are too many images with huge size (2MB and more) in the database.
-The security issue: you can still save the files in the disk and still be secure, how? Well save the files outside the web directory, whenever the file is requested read the file and give it to the user.
-If by any chance you are using MySQL: if your database has got to big (say 2-3 GB), and you are using a shared hosting, well good luck making that backup or trying to restore that image database.
It's just my point of view
The best way to store images into MySQL is by storing the image location as a character string.
If you need to manipulate the image, then, the best way is to copy the image as a binary.
How one can store images into binary form and how we can retrieve them back? I don’t know anything about this technique. Please tell me how we can do this.
Don't store images in the database. Store them in the filesystem, then store their relative paths in the database.
I've written some blogs on this (and have some data from SQL Server)
http://www.atalasoft.com/cs/blogs/loufranco/archive/2007/12/03/images-in-databases-part-i-what-to-store.aspx
http://www.atalasoft.com/cs/blogs/loufranco/archive/2007/12/04/images-in-databases-part-ii-web-images-are-random-access.aspx
http://www.atalasoft.com/cs/blogs/loufranco/archive/2009/10/26/more-on-images-in-databases.aspx
Basically,
Small images are ok to put in a blob
Large images are much better to put on the filesystem
Images in a blob are much easier to manage (transactions, backup, simpler code, access control)
Images on the filesystem will perform much better
Think about pulling some meta-data out of the image and storing in separate columns for filtering and sorting purposes.
Almost every professional enterprise system that needs to deal with a lot of large blobs has some way of putting them on the filesystem. The latest SQL Server even has a field type that will do it automatically (and then it's as easy to program and manage as a blob)
You can use the BLOB data type. Although I agree with #Ignacio Vazquez-Abrams, there are times where storing the image in the DB is best. I have done so in past with great results. As long as the files are not large then this is a good solution.