I happen to have a database with pictures stored as blob fields. Can't help it, it was the previous developer's choice.
Now I need that data in a new site and the provider won't let me copy the data the easy way (file has become 11Mb big - won't upload that and I don't have shell access).
So I thought I'd write a script that opens a connection in db1, selects all the records, and then copies each into a table in the new db2.
Works all fine if I exclude the blobs. If I want to copy them too, it won't insert.
Anyone had something similar before?
Should I treat the blobs differently when it comes to inserting?
Thanks for any ideas or help.
11MB isn't a huge file, I'm surprised your host has such a low max upload size.
Have you thought about exporting as SQL, splitting the file in two (in Notepad++ or something) then uploading it in smaller sections? Wouldn't take long.
Perhaps check to see if you can increase the max_allowed_packet setting on your mysql DB. I'm not sure if it affects inserts, but I remember having to adjust this setting when I worked on a web-app that allowed users to download 3-5MB binaries from blob fields in the DB.
This link may be helpful, from a quick google search: http://www.astahost.com/info.php/max_allowed_packet-mysql_t2725.html
Related
I am creating an application where users will post texts. Currently I am creating the text files of that texts post but i am not sure if storing it to MySql table is good or creating a txt file is good. .
Note:
1) There is no limit of the size of text post. Users may post a very big text or user me post containing only a letter.
2) There is no limit of post a user can upload.
Please Help
Thank You. .
Saving data into a file is more cost operation rather than in database.So its better to go for database.Because saving in file will do IO operation and this file I/O operation is really very costly in terms of computation and from OS perspectives.
You should store it in the database instead of text file. There are many reasons for it. Go through this question for more descriptive answer. In short, you should choose database over files for security reason, speed, code maintainability and concurrency.
You can use LOAD DATA INFILE to read the contents of a file and store it in a table in the database in a structured format.
LOAD DATA INFILE 'data.txt' INTO TABLE db2.my_table;
You can also use largetext in mysql.But it would be slower and non-resilient.
Please check here
See there are some major things you should know before taking decision on where to save things
If the file is an image/Video/PDF
For this you should save the FILE NAME into the database and the image on server
If the file is a large text
For this you should save the file to the server as the database will not be able to take loads of text and many requests for long text at the same time and this will also save you space for more things in database
For any other thing, if you find it space consuming just save it on the server as this will give you space for database and more advantages
Well,I am learning the web development and currently working on PHP and mySQL,I am just totally a newbee to database concepts.To be honest Iam not sure when to create a table or when to create a database.I need some suggestions and help from you.
Okay I have these doubt kindly clear me this.I am not much aware of this but how much security does php file concepts provide us. Is there any harm in using file concepts of php?
Okay let me tell you these I want to save some data that user has entered into a text file on the server, the data might be like some message or something like his information,I wanted to save the data in a file and then save its directory path in the database. and while retrieving the data just get the file path from the database then retrieve it from the text file. Is this a good or bad idea of doing it? or should I need to save the user data in the database itself?
Similarly I also want to save the path of the images or pictures in the database and then just put the path in <img>tag.I got no one here to help me with this questions.So please help me with this,Any help is greatly appreciated.
Kindly let me know what is the way I should choose to do ?
For images and other file-bound resources, it makes sense to store the image on the file system and the path to it in the database. After all, file systems are great for storing files. You can store the file as a binary field in the database, so that's certainly an option. There are pros and cons either way. Personally, I prefer to keep the files on the file system.
I'm not sure where you're going with this:
the data might be like some message or something like his information,I wanted to save the data in a file and then save its directory path in the database
Is there a particular reason why this data needs to be in a file? Or are you just not sure how to store it in a database? If the data is structured and consistently organized then I can't imagine a reason not to keep it in a relational database. (And even if it isn't as structured, I'd probably still look into a database over the file system for this sort of thing.)
Images and other non-relational resources are generally file-bound, so keep them in files. But if you're just storing text in a text file for no other reason than that's what you've always done, I'd recommend using a database.
PHP provides only as much security as the underlying filesystem. But putting files on disk and saving the path in the db is the traditional method.
Files in a database are generally not the best solution. But that's mostly because people talk about storing binaries (e.g. images, zip files, etc...) which would be an opaque blob as far as the database is concerned.
With text files, especially small ones, it'd still be at least somewhat useable by the DB, even if only via SUBSTR() type matching/searching, so this is one case where storing in the DB could make sense.
There is a good rule-of-thumb here:
If it is something, the DB "understands" (such as text), store it in
the DB - you might later want fulltext indexing, search, text transformation, whatever.
If not (e.g. Images), store it in files
as all rules of thumb, this might or might not fit your index finger
which is a better place to upload images to? A database or in the web directory? And why?
You should only store images in your database if you have a specific need to, like security, or like an absolute to-die-for need to keep all custom data in a database.
Other than that, getting large files into databases usually isn't worth the trouble. Storing and retrieving the file get that much more complicated to implement, and database updates/upgrades/conversions have that many more things that can go wrong.
I don't see that there is an advantage storing images in a database. There is certainly no inherent security in this. Files are for the filesystem so store your images in there.
I don't think you can "upload" an image to a database. You can store the image's string value in the database and stream it via "header("Content-Type")" later on. That saves space in your web server, but obviously takes space on your database.
If I were you, I'd upload to a web directory, that way you have the image for a regular URL request later on. If you don't have it in a regular directory, you'll have to connect to the database every time the image is requested, and stream it then.
Well It depends on your requirement.
If you are considering security as a major issue then definitely you should store it in db other wise nothing will leads you to store images in db.
Also retieving images from database is quite complicated as in database images are stored as binary data. So if you have specific need then only store images in database other wise storing images in directory would be fine.
As you can see there are many reasons why to use/why not to use the database for image storage. Personally I prefer not to use the database for storage of files (images, documents etc), except when I'm ordered to store them.
-Sometimes you're tired and screw up a query, something like "SELECT * FROM images", this will kill the server if there are too many images with huge size (2MB and more) in the database.
-The security issue: you can still save the files in the disk and still be secure, how? Well save the files outside the web directory, whenever the file is requested read the file and give it to the user.
-If by any chance you are using MySQL: if your database has got to big (say 2-3 GB), and you are using a shared hosting, well good luck making that backup or trying to restore that image database.
It's just my point of view
I want to save an uploaded image to database. Now my question is:
What should be the data type for SQL Server 2000 for image?
a. image
b. varbinary
what is the code for saving image in database?
How to retrieve the image from database and show?
Please refer me any tutorial or guidline. Or if you can please share with me.
Thanks in advance.
The data type should be text because the best way to save an image to a database is to save its path. Let your OS do the job of storing the actual files.
Typically on SQL Server, you would use a BLOB, Binary Large OBject, to store images. We used it for Word documents on a previous project, and it worked just fine. See this article on Database Journal for more info, although a quick Google for the BLOB type will throw up lots more examples.
I wrote this article a while back on this subject. It should help you with #2 and #3. It uses MySQL, but the basics are the same, you just need to replace the MySQL calls with the MSSQL calls.
As to #1, I would go with the obvious choice: "image".
I am not 100% sure of the differences between the two, however. It just seems obvious :)
Edit, according to this, it appears that the image datatype is deprecated. It will be removed in future versions. Not sure how much this affects you, seeing as you are using a 10 year old version, but it is worth keeping in mind.
In SQLServer IMAGE or VARBINARY are allmost the same. IMAGE is 2GB but VARBINARY() needs a length argument.
It's not good idea to store images in a database, the size increases a lot, with each Backup you need to save all images, with increasing size also increases the time to perform a Backup and Restore. You also need to change the network packet size (at the server properties, advanced, network, network packet size) In recents SQLServer versions the most adecuate datatype is varbinary(MAX)
I want to upload a large file of maximum size 10MB to my MySQL database. Using .htaccess I changed PHP's own file upload limit to "10485760" = 10MB. I am able to upload files up to 10MB without any problem.
But I can not insert the file in the database if it is more that 1 MB in size.
I am using file_get_contents to read all file data and pass it to the insert query as a string to be inserted into a LONGBLOB field.
But files bigger than 1 MB are not added to the database, although I can use print_r($_FILES) to make sure that the file is uploaded correctly. Any help will be appreciated and I will need it within the next 6 hours. So, please help!
You will want to check the MySQL configuration value "max_allowed_packet", which might be set too small, preventing the INSERT (which is large itself) from happening.
Run the following from a mysql command prompt:
mysql> show variables like 'max_allowed_packet';
Make sure its large enough. For more information on this config option see
MySQL max_allowed_packet
This also impacts mysql_escape_string() and mysql_real_escape_string() in PHP limiting the size of the string creation.
As far as I know it's generally quicker and better practice not to store the file in the db as it will get massive very quickly and slow it down. It's best to make a way of storing the file in a directory and then just store the location of the file in the db.
We do it for images/pdfs/mpegs etc in the CMS we have at work by creating a folder for the file named from the url-safe filename and storing the folder name in the db. It's easy just to write out the url of it in the presentation layer then.
Some PHP extensions for MySQL have issues with LONGBLOB and LONGTEXT data types. The extensions may not support blob streaming (posting the blob one segment at a time), so they have to post the entire object in one go.
So if PHP's memory limit or MySQL's packet size limit restrict the size of an object you can post to the database, you may need to change some configuration on either PHP or MySQL to allow this.
You didn't say which PHP extension you're using (there are at least three for MySQL), and you didn't show any of the code you're using to post the blob to the database.
The best answer is to use an implementation that is better and also works around that issue.
You can read an article here. Store 10MB, 1000MB, doesn't matter. The implementation chunks/cuts the file into many smaller pieces and stores them in multiple rows.. This helps with load and fetching so memory doesn't also become an issue.
You could use MySQL's LOAD_FILE function to store the file, but you still have to obey the max_allowed_packet value and the fact that the file must be on the same server as the MySQL instance.
You don't say what error you're getting (use mysql_error() to find out), but I suspect you may be hitting the maximum packet size.
If this is the case, you'd need to change your MySQL configuration max_allowed_packet
You don't say what error you're getting (use mysql_error() to find out), but I suspect you may be hitting the maximum packet size.
If this is the case, you'd need to change your MySQL configuration max_allowed_packet
Well I have the same problem. And data cannot be entered in the mysql database chunck by chunck in a "io mode"
loop for :
read $data from file,
write $data to blob
end loop
close file
close blob
A solution seems to create a table with multi-part blobs like
create table data_details
(
id int pk auto_increment,
chunck_number int not null,
dataPart blob
);
???