I am working on a real estate website and i would like to write a program that
can figure out(classify) if an image is a floor plan or a company logo.
Since i am writing in php i will prefer a php solution but any c++ or opencv solution will be fine as well.
Floor Plan Sample:
alt text http://www.rentingtime.com/uploads/listing/l0050/0000050930/68614.jpg
alt text http://www.rentingtime.com/uploads/listing/l0031/0000031701/44199.jpg
Logo Sample:
alt text http://www.rentingtime.com/uploads/listing/l0091/0000091285/95205.jpg
As always, there is a built-in PHP function for this. Just joking. =)
All the floor plans I've seen they are pretty monochromatic, I think you can play with the number of colors and color saturation to have a pretty good guess is the image is a logo or a floor plan.
E.g.: is the image has less than 2 or 3 colors is a floor plan.
E.g.: if the sum / average of the saturation is less than X it's a floor plan.
Black and white (and other similar colors that are used in floor plans) have a saturation that is zero, or very close to zero, while logos tend to be more visually attractive, hence use more saturated colors.
Here is a simple function to compute the saturation of a Hex RGB color:
function Saturation($color)
{
$color = array_map('hexdec', str_split($color, 2));
if (max($color) > 0)
{
return (max($color) - min($color)) / max($color);
}
return 0;
}
var_dump(Saturation('000000')); // black 0.0000000000000000
var_dump(Saturation('FFFFFF')); // white 0.0000000000000000
var_dump(Saturation('818185')); // grey 0.0300751879699249
var_dump(Saturation('5B9058')); // green 0.3888888888888889
var_dump(Saturation('DE1C5F')); // pink 0.8738738738738738
var_dump(Saturation('FE7A15')); // orange 0.9173228346456692
var_dump(Saturation('FF0000')); // red 1.0000000000000000
var_dump(Saturation('80FF80')); // --- 0.4980392156862745
var_dump(Saturation('000080')); // --- 1.0000000000000000
Using imagecolorat() and imagecolorsforindex() you can implement a simple function that loops trough all the pixels of the image and sums / computes the average of the saturation. If the image has a saturation level above of a custom threshold you define you can assume that the image is a logo.
One thing you shouldn't forget is that images that have a higher resolution will normally have more saturation (more pixels to sum), so for the sake of this algorithm and also for the sake of your server performance it would be wise to resize all the images to a common resolution (say 100x100 or 50x50) to classify them and once classified you could use the original (non-resized) images.
I made a simple test with the images you provided, here is the code I used:
$images = array('./44199.jpg', './68614.jpg', './95205.jpg', './logo.png', './logo.gif');
foreach ($images as $image)
{
$sat = 0;
$image = ImageCreateFromString(file_get_contents($image));
for ($x = 0; $x < ImageSX($image); $x++)
{
for ($y = 0; $y < ImageSY($image); $y++)
{
$color = ImageColorsForIndex($image, ImageColorAt($image, $x, $y));
if (is_array($color) === true)
{
$sat += Saturation(dechex($color['red']) . dechex($color['green']) . dechex($color['blue']));
}
}
}
echo ($sat / (ImageSX($image) * ImageSY($image)));
echo '<hr />';
}
And here are the results:
green floor plant: 0.0151028053
black floor plant: 0.0000278867
black and white logo: 0.1245559912
stackoverflow logo: 0.0399864136
google logo: 0.1259357324
Using only these examples, I would say the image is a floor plant if the average saturation is less than 0.03 or 0.035, you can tweak it a little further by adding extra examples.
It may be easiest to outsource this to humans.
If you have a budget, consider Amazon's Mechanical Turk. See Wikipedia for a general description.
Alternatively, you could do the outsourcing yourself. Write a PHP script to display one of your images and prompt the user to sort it as either a "logo" our "floorplan." Once you have this running on a webserver, email your entire office ånd ask everyone to sort 20 images as a personal favor.
Better yet, make it a contest-- the person who sorts the most images will win an ipod!
Perhaps most simply, invite everyone you know over for pizza and beers and setup a bunch of laptops and get everyone to spend a few minutes sorting.
There are software ways to accomplish your task, but if it is a one-off event with less than a few thousand images and a budget of at least a few hundred dollars, than I think your life may be easier using humans.
One of the first things that comes to mind is the fact that floor plans tend to have considerably more lines oriented at 90 degrees than any normal logo would.
A fast first-pass would be to run Canny edge detection on the image and vote on angles using a Hough transform and the rho, Theta definition of a line. If you see a very strong correspondence for Theta=(0, 90, 180, 270) summed over rho, you can classify the image as a floor plan.
Another option would be to walk the edge image after the Canny step to only count votes from long, continuous line segments, removing noise.
I highly doubt any such tool already exists, and creating anything accurate would be non-trivial. If your need is to sort out a set of existing images ( e.g., you have an unsorted directory ), then you might be able to write a "good enough" tool and manually handle the failures. If you need to do this dynamically with new imagery, it's probably the wrong approach.
Were I to attempt this for the former case, I would probably look for something trivially different I can use as a proxy. Are floor plans typically a lot larger then logos ( in either file size or image dimensions )? Do floor plans have less colors then a logo? If I can get 75% accuracy using something trivial, it's probably the way to go.
Stuff like this - recoginition of patterns in images - tends to be horribly expensive in terms of time, horribly unreliable and in constant need of updating and patching to match new cases.
May I ask why you need to do this? Is there not a point in your website's workflow where it could be determined manually whether an image is a logo or a floor plan? Wouldn't it be easier to write an application that lets users determine which is which at the time of upload? Why is there a mixed set of data in the first place?
Despite thinking this is something that requires manual intervention, one thing you could do is check the size of the image.
A small (both in terms of MB and dimensions) image is likely to be a logo.
A large (both in terms of MB and dimensions) image is likely to be a floorplan.
However, this would only be a probability measurement and by no means foolproof.
The type of image is also an indicator, but less of one. Logos are more likely to be JPG, PNG or GIF, floorplans are possibly going to be TIFF or other lossless format - but that's no guarantee.
A simple no-brainer attempt I would first try would be to use SVM to learn the SIFT keypoints obtained from the samples. But before you can do that, you need to label a small subset of the images, giving it either -1 (a floor plan) or 1 (a logo). if an image has more keypoints classified as a floor plan then it must be a floorplan, if it has more keypoints classified as a logo then it must be a logo. In Computer Vision, this is known as the bag-of-features approach, also one of the simplest methods around. More complicated methods will likely yield better results, but this is a good start.
As others have said, such image recognition is usually horribly complex. Forget PHP.
However, looking over your samples I see a criteria that MIGHT work pretty well and would be pretty easy to implement if it did:
Run the image through good OCR, see what strings pop out. If you find a bunch of words that describe rooms or such features...
I'd rotate the image 90 degrees and try again to catch vertical labels.
Edit:
Since you say you tried it and it doesn't work maybe you need to clean out the clutter first. Slice the image up based on whitespace. Run the OCR against each sub-image in case it's getting messed up trying to parse the lines. You could test this manually using an image editor to slice it up.
Use both color saturation and image size (both suggested separately in previous answers). Use a large sample of human-classified figures and see how they plot in the 2-D space (size x saturation) then decide where to put the boundary. The boundary needs not be a straight line, but don't make too many twists trying to make all the dots fit, or you'll be "memoryzing" the sample at the expense of new data. Better to find a relatively simple boundary that fits most of the samples, and it should fit most of the data.
You have to tolerate a certain error. A foolproof solution to this is impossible. What if I choose a floorplan as my company's logo? (this is not a joke, it just happens to be funny)
Related
I have a library of like 1 million images, and roughly half of these are watermarked with the same, half transparent watermark in the same spot.
Where do I begin, detecting the images with the watermarks? Is there some standard tools for this purpose?
If according to your question, you just want to detect the images that are watermarked, you can use the following algorithm:
Extract a sample of the watermarking image Scan the watermark image
pixel by pixel and store the first pixels in an array. Scan each
image pixel by pixel and store in an array. Whenever a row matrix
from the image being scanned contains elements of the array in the
same order, it's most likely a match.
The code could be something like this:
$no_of_pixels = what_you_got;
$matched = 0;
$thumbpixels = array();
$wmark = imagecreatefrompng("watermark.png");
list($width, $height) = getimagesize("watermark.png");
$tesimage = imagecreatefrompng("test.png");
for($h = 0; $h < $height; $h++){
for($w = 0; $w < $width; $w++){
if(imagecolorsforindex($testimage, imagecolorat($testimage, $w, $h)) == $thumbpixels[0]){
while($thumbpixels[$i++] === imagecolorsforindex($tesimage, imagecolorat($wmark, $w, $h)) && $no_of_pixels != $matched){
$matched++;
}
if($matched == $no_of_pixels) echo "Voila, we found it!";
}
}
}
EDIT
Just seeing your thumbnail example. If you just want to detect text, you can try tesseract-ocr or PhpOCR.
You may also consider PHPSane
Detecting almost any feature in an image is called Object Detection. There is a widely used libray called OpenCV. It has a very simple SDK, although setting up can be a real pain. It is well supported for C/C++ and (nearly well supported for) Python. It took me arnd 3 weeks to train my own Classfier (training), first time I started using OpenCV.
But I would not really depend on this solution entirely and consider my priorities. Also, it is very hard to achieve good rate with custom classifier. Other methods are more time consuming.
In short, not with complete accuracy.
At best, you could only apply heuristics on the image to see if it matches an exact watermark, and get a confidence rating -- for example, if the watermark if a 50% white overlay, then a scene that was predominantly white could give a false positive, and of course the inverse is true.
There are also problems that could arise if the images use a lossy compression, such as JPEG, as the edges, and the saturation may result in a watermark that isn't as saturated as expected, or as exactly positioned as expected.
Because you know where the watermark always is it is possible that you could use imagecolorat and imagecolorsforindex to get the alpha value for pixels both inside and outside of the watermark. I would expect the alpha values to be similar when there is no watermark, and different when there is (within some threshold that you would need to determine). Of course, this may not work on all images so if you need 100% accuracy you would probably need something more reliable.
In your case, where you are looking for the same logo in a predictable location, it is relatively simple. However it's much, much simpler and faster (as per my comment elsewhere) to match a copyright notice in the meta data!
A watermark is not going to produce fixed changes to the content - each modified pixel will obtain a new value based on the watermark and the image itself. Hence you need to extract this information - I'd go with diferentiating the image and just looking at the magnitude of the derivative (not the phase).
Then it's simply a matter of correlating the differential with one of just the watermark (or lots with the watermark and other content).
You really don't want to be doing this kind of image processing in PHP unless you're happy writing your own extensions. Most image processing toolkits will support differentiation and correlation.
BTW: if you don't know how to differentiate an image, and/or can't understand how to correlate an image, please don't ask - this is not the right forum for that discussion
Well if there is no tool to do this, you could try the following:
identify where the watermark appears as a percentage of pixels, eg bottom right 40px x 100px
For each image, make a temp copy and crop out the location of where the watermark would appear. This should leave both the watermarked version and the non watermarked version as the same
compare the images - e.g. combination of width x height, filesize, CRC or actual pixel comparison, though for a million images you'd need some serious CPU power.
I am working on a PHP image processing library that works with rendered images from a game, being processed on a PHP webhost. In order to process the image, I need to find islands of red(where the hue is within x amount of 0 or 360), and get a point within them(not necessarily the center, but preferably near the center). There are about 100 such islands, of all different sizes. They are trapezoids or near that. Since the image is a PNG, and is uncompressed without antialiasing, the edges are crisp to the pixel, but some areas may be darker than the others.
Currently, what I have tried is using imagecolorat( in GD and testing the HSV value after conversion, followed by trying points around it until I reach a non-red pixel, but that process seems take some time to complete, and appears to encounter PHP timeouts due to the 5 second limit, if I am processing a large image. Is there a more efficient way to detect said islands and get a point not necessarily at the center, but preferable near?
I've also tried, where I know the size of the trapezoids, assuming that none come within a certain distance and skipping that distance to save time.
I don't necessarily need code, just a pointer in the right direction.
My current code:
function RGBToHSL($RGB) {
$r = 0xFF & ($RGB >> 0x10);
$g = 0xFF & ($RGB >> 0x8);
//SNIP
}
$image=imagecreatefrompng($filename);
$redislands=[];
for($xpos=0; $xpos<=imagesx($image); $xpos++){
for($ypos=0; $ypos<=imagesy($image); $ypos++){
if (RGBToHSL(imagecolorat(xpos, ypos)->saturation<=20||RGBToHSL(imagecolorat(xpos, ypos)->saturation>=350){
$redislands[]=[xpos, ypos]
}
}
}
I haven't really ever used gd, but I just looked briefly and I see
imagefilter() w/ the IMG_FILTER_EDGEDETECT flag.
seems like after that, you could then identify all the objects with a single simple loop(if objects have a min size, you can make the loop use this stripe size and still be guaranteed you'll encounter each object). Just look for a pixel w/ the edge color, and when you find one, recursively explore neighbor pixels w/ the edge color. For example, if an object is guaranteed to be at least 7 pixels high, after edges are highlighted, you only need to loop over the pixels in rows 0, 7, 14, 21 etc...
Once you've extracted the component by using the edge around it, I think you could identify it's shape, color, and center fairly easily.
You might need to play with thresholding the image(gd can do this) if edges don't get detected robustly.
Another option to seriously consider is to not use php for this, but instead execute an external program using php's exec() function. This opens up the possibility of using all kinds of libraries(like opencv) and programs meant for this stuff. It also lets you write your code in something like c...you'd be amazed at how fast you can loop over a c array that holds a few million ints representing pixel colors. But, I have a feeling this isn't a viable option for you.
Something else you might investigate is getting access to a php array(or a string) of pixels. function call overhead is large, and calling a function like imagecolorat() millions of times does add up, compared to native php array access like $pixels[$i]. I don't know anything about image formats, but maybe you could save as bitmap and then load it into a string, and then just use string offsets. I bet that would fly if a bitmap has a suitable array-like binary representation.
I am creating business card maker online, now so far everything was easy until i came to card rendering. In my html document i have:
width = 517px which is equals to 90mm
height = 287px which is equals to 50mm
And according to Photoshop 90mm -> 255px so I need to somehow convert 517px to 255px.
After some googling best solution i came up with so far is (517 / 5.7) * 3 which gives me 272 and it's far from correct answer.
Any suggestions would help a lot :)
P.S I am using php GD.
The correlation between pixels and physical measurements depends entirely on the display and how many physical pixels there are per (square) millimeter. 72ppi (pixels per inch) used to be a typical resolution, but with pixel density increasing across many devices, that's not a given anymore.
There's simply no standard formula.
The resulting pixel number depends on the DPI (resolution) of the output media.
Starting with 90mm equaling ~3,54in you get:
72dpi: 255px
96dpi: 340px
120dpi: 425px
146dpi: 517px
If you don't want to do all the calculation there is a PHP Class which can do this for you.
Here are some informatiosn what the Library can do:
When layouting things one often has to compute various positions, zoom factors, scalings and the like. When the specific sizes are of dynamical nature the layout code quicky gets a confusing endless series of mathematical equations. You name it: spaghetti code. This library tries to offer two approaches to solve this problem:
by taking handling of units from explicit recomputation to implicit
consideration, so all you have to do is name what unit a specific
value is to be interpreted as
by offering well known arithmetical concepts as types which allows to
think in a more high level way about layouting instead of just
simple numeric values.
https://github.com/arkascha/php-urithmetic
I think px divided by 2.02 works perfect in my case at any size of page. In php code i use :
<?php $mm = $px/2.02 ?>
I've got a security camera set up, and i'm comparing 1 minute batches of images to check if there's been any motion. I have 10 coordinates that I check between each image. If any pixels don't match the previous image, it triggers a warning message.
Problem is, it works too well.
The logic is basically if imagecolorat() is greater or less than a 10% difference from the previous imagecolorat(), it triggers. So, if a cloud comes over the house, it triggers. Basically any change in light triggers it. So, I've moved the threshold from 10% to 30% and it triggers less but I'm worried now that if I move any past that, real motion wont be detected.
Note: I'm using the raw output of imagecolorat(), not the RGB values. I'm not sure if this would have an impact.
You are looking for larger discontinuities - things like noise and slow variation should be discounted (unless this motion detection is for very slow moving small things, like garden gnomes).
Therefore, do a histogram equalize or something similar before image subtraction to account for global shifts in light, and do some filtering / edge enhancement before differencing to enhance the changes. Using more of the image will be better I would think, than just 10 points.
Histogram equalization entails looping through the image, and counting bins for each brightness value - so you have a resulting data set that says how many pixels are in a set of tonal ranges. In other words, say you divvy up into 16 bins - pixels that have a greyscale value (or alternately, the Brightness in an HSB model) in the value of 0..15 (assuming an 8 bit image here in this channel) are all in bin 1. Then you go forth and compute a series of linear stretches to apply to each bin so that it occupies an output range in proportion to its population. So if ALL of your pixels are in the 0.. 15 bin, you would just multiply everything by 16 to stretch them out. The goal is that your histogram of your equalized image is flat - equal amounts of pixel in every bin.
Edge enhancement can be simply done with the application of a Sobel filter.
I have a whole bunch of images of illustrations that I would like to crop to a smaller preview size.
The problem is that I want to crop them to show an "interesting" part of the illustration (ie avoid areas of whitespace).
The images typically have a flat color or a subtle gradient for the background. They are mostly vector style artwork with fairly distinct shapes.
Here are some examples: link ;-)
I've been thinking about using some sort of image feature detection algorithm with a sliding window to find the area with the greatest number of features.
I'm implementing this in PHP, but I don't mind implementing it myself if there isn't a library or extension available.
Ideas?
ImageMagick has a trim operation. It's available as a library but I don't know how hard it is to use from PHP. There are some PHP interfaces.
OK, so here's what I would've done, after looking at the examples:
Sum all rows and all columns of each image. You'll get two arrays, both looking like this:
/-----\ /--\
_/ -- |
___- \_________
By looking at these arrays for a few images, find a suitable threshold (probably something just above zero). Then the leftmost and the rightmost crossing of this threshold is where you have to crop. I hope I've managed to make it clear enough, if not -- ask!
Here's a fairly simple approach using an edge-detection filter, and then cropping around the center-of-edginess of the image to generate a thumbnail. It works pretty well on most images, but not if there are more than one subject. I'm open to suggestions on other ways of identifying the "interesting" points in a source image.
Well, you might want to consider just using an edge detection algorithm. Pick the area with the largest number of edges. Give higher weight to edges that are not blurry (as they may be from the background).
ImageMagick for PHP has automated generation of thumbnails. This SO question has a link to an ImageMagick auto-crop operator, and I'm not sure, but I think this is the PHP interface to it.
From the link:
bool Imagick::trimImage ( float
$fuzz )
Remove edges that are
the background color from the image.
For more general "interestingness", maybe try an inverse of seam carving (to find the highest energy, rather than lowest energy areas).
A CLI program using http://pecl.php.net/package/imagick:
<?php
dl('imagick.so');
$img = new Imagick();
$img->readImage($argv[1]);
# (* 0.0: exact match; * 1.0: crop entire image)
$fuzz = current($img->getQuantumRange()) * 0.25;
$img->trimImage($fuzz);
$img->writeImage($argv[2]);
?>
It should work good enough, as long as the image doesn't have a frame around its border.
Drupal has a project called smartcrop, which has PHP code to find highest entropy and "interesting" areas in images. See the output examples.
You should be able to use the functions in the module and the library in none-Drupal projects too.