Problems loading bulk data in mysql using php - php

I have a script that gets the raw binary image data via url request. It then takes the data and puts it into mysql.
Pretty simple right? Well It's I'm inserting some 8,000 decent sized 600x400 jpegs and for some odd reason some of the images are getting cut off. Maybe the part of my script that iterates through each image it needs to get is going to fast?
When I do a straight request to the URL I can see all the raw image data, but on my end, the data is cut off some way down the line.
Any ides why?

Is something in the chain treating the binary data as a string, in particular a C style null-terminated string? That could cause it to get cut off at the first null byte ('\0').

Have you tried simply call your script that pulls the binary image, and dump it out. If you see the image correctly then its not pulling part, might be something to do with inserting.

Are you setting the headers correctly?
ie:
header('Content-Length: '.strlen($imagedata));
header('Content-Type: image/png');
...

A string datatype would definitely not be the optimum for storing images in a DB.
In fact I've seen several recommendations that the image should go in a folder somewhere in your filesystem and the DB contains only the address/file path.
This is a link to a page about inserting images.
It contains the suggestion about the filepath and that a blob datatype is better if the images must go in the database.
If it's a blob, then treating it as a string won't work.

If you make repeated requests to the same url, does the image eventually load?
If so that points to a networking issue. Large packet support is enabled in your kernal (assuming linux) which doesn't work correctly for a lot of windows clients. I've seen a similar issue with large(1+MB) javascript libraries served from a linux machine.
http://en.wikipedia.org/wiki/TCP_window_scale_option
http://support.microsoft.com/kb/314053

Related

Is it possible to translate an image into raw text and then reverse it?

I have a picture. For whatever reason, I need that picture to be sent to an environment that can only receive text and not images. Images and other files must be sent through their filter and I want to get around this. I calculated that there would be 480,000 independent hex values being manipulated but this is really the only option I have. Also, is it possible to compress and uncompress it for less pixels being sent? I will need to send the picture from a PHP web server [lets say, mysite.com/image.php] and receive it in Lua, and my only connection to the server is over a web request. No ftp, no even loading image files. Just setting 480,000 variables to the different id's
Oh, one more thing: it needs to not crash my server when I run it. ;)
Convert your image to base64 (Eg: Can pass to the variable).
Eg: I converted PNG image
Base 64 image will look like this.
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAcAAAAHCAYAAADEUlfTAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAE9JREFUeNpiYMADGLEJKssrCACp+Uw4JPYD8QdGHBIP7j58EMgCFDAAcvqBOBGI64FYAMpmYIFqAilYD6Udgbo+IBvXAMT/gXg9sjUAAQYAG6IS47QjgzEAAAAASUVORK5CYII="
You can use it in image source to display.
Hope this helps!

Is it wise to store base64 encoded images inside a database?

I'm making an android application which takes a photo and push the image (as a base64 encoded string) to a PHP script, from here I'll be storing data about the image inside a MySQL database.
Would it be wise to store the image inside the database (since it's passed as a base64 string), would it be better to convert it back to an image and store it on the filesystem?
A base64 encoded image takes too much place (about 33% more than the binary equivalent).
MySQL offers binary formats (BLOB, MEDIUM_BLOB), use them.
Alternatively, most people prefer to store in the DB only a key to a file that the filesystem will store more efficiently, especially if it's a big image. That's the solution I prefer for the long term. I usually use a SHA1 hash of the file content to form the path to the file, so that I have no double storage and that it's easy to retrieve the record from the file if I want to (I use a three level file tree, first two levels being made respectively from the first two characters and the characters 3 and 4 of the hash so that I don't have too many direct child of a directory). Note that this is for example the logic of the git storage.
The advantage of storing them in the DB is that you'll manage more easily the backups, especially as long as your project is small. The database will offer you a cache, but your server and the client too, it's hard to decide a priori which will be fastest and the difference won't be big (I suppose you don't make too many concurrent write).
I've done it both ways, and every time I come back to code where I stored binary data in a MySQL table I always switch it to filesystem with a pointer in the MySQL table.
When it comes to performance, you're going to be much better off going to the FS as pulling multiple large BLOBs from a MySQL server will tend to saturate its pipe quickly. Usually it's a pipe you don't want clogged.
You could always save the base64_encode($image) in a file and only store the file path in the database, then use fopen() to get the encoded image.
My apologies if I didn't understand the question correctly.
"wise" is pretty subjective, I think. I think it would be wise from a "keep people from directly linking to my images" perspective. Also, it may be helpful as far as if you decide you need to change up dir structures etc.. it might make it easier on you (but this really depends on how you wrote your scripts to begin with..) but other than that... offhand I can't really think of any benefits to doing this.

Server request performace: Get a lot of images with one call

I have a tableview, that contain an image for each cell. Since there will be quite a lot of cell, is it possible to package all the images needed, and retrive them with one request, or do you have to request each and every image separate? Trying to reduce the stress for on server. Or is this the normal way to do it?
Thanks in advance
I think you should just load the images asynchronously, more than dealing with such an optimization. As you ask, yes, this is the normal way to do that, unless your server has really huge traffic (huge = google.com, facebook.com, etc.).
I use the SDWebImage open source library and it works really well, making it transparent to you. After importing it, for each cell, you should do:
[cell.myImageView setImageWithURL:[NSURL URLWithString:imageURL] placeholderImage:[UIImage imageNamed:#"MyPlaceholderImage"]];
and it will try to load (and cache) dynamically each image, placing a placeholder image while it's loading.
I use it in a table with 100+ rows and it works like a charm.
Hope this helps!
If you need, you can serve a collection of images as a zip file.
(And how do you do that? It depends. I don't know if the images are static or dynamic.)
try converting the image in base64 formate and then send it

PHP / Python script that base64encodes images in css

I find myself manually encoding background images in the css in base64 often.
When I mean manually, I mean that I encode the image, copy the resulting string, paste it into the css file and so on. This is stupid!
I came to the conclusion that writing a script in PHP or Python that does it automatically would not be difficult, it's just a matter of parsing the css, finding the image on the HD, encoding it in base64, replace the result with the original string in the css file and save a new file.
Then I thought: "how come nobody has already done this? Maybe it would be better to ask before doing it."
So here I am, does a similar solution exist?
Thanks
Well, Chris Coyer # CSS-Tricks published an article talking about Data URIs, where he explains how to use them and how they are useful. Near the end, he states that's it's very easy to generate those on the fly with PHP, like so
if you are using PHP (or PHP as CSS), you could create data URIs on the fly like this
<?php echo base64_encode(file_get_contents("../images/folder16.gif")) ?>
However, take not that you shouldn't use base64_encode on all images on a website. the size of the string generated by base64_encode is larger by about 33% of the original image. Data URIs are great when you have small pictures and you don't want to waste requests on them.

Image save in database

I want to save an uploaded image to database. Now my question is:
What should be the data type for SQL Server 2000 for image?
a. image
b. varbinary
what is the code for saving image in database?
How to retrieve the image from database and show?
Please refer me any tutorial or guidline. Or if you can please share with me.
Thanks in advance.
The data type should be text because the best way to save an image to a database is to save its path. Let your OS do the job of storing the actual files.
Typically on SQL Server, you would use a BLOB, Binary Large OBject, to store images. We used it for Word documents on a previous project, and it worked just fine. See this article on Database Journal for more info, although a quick Google for the BLOB type will throw up lots more examples.
I wrote this article a while back on this subject. It should help you with #2 and #3. It uses MySQL, but the basics are the same, you just need to replace the MySQL calls with the MSSQL calls.
As to #1, I would go with the obvious choice: "image".
I am not 100% sure of the differences between the two, however. It just seems obvious :)
Edit, according to this, it appears that the image datatype is deprecated. It will be removed in future versions. Not sure how much this affects you, seeing as you are using a 10 year old version, but it is worth keeping in mind.
In SQLServer IMAGE or VARBINARY are allmost the same. IMAGE is 2GB but VARBINARY() needs a length argument.
It's not good idea to store images in a database, the size increases a lot, with each Backup you need to save all images, with increasing size also increases the time to perform a Backup and Restore. You also need to change the network packet size (at the server properties, advanced, network, network packet size) In recents SQLServer versions the most adecuate datatype is varbinary(MAX)

Categories