Let me explain what I am trying to do, I am building a command line tool with PHP that uses these programs to optimize images for the web...
imagemagick to determine file type and convert non-animated gif's to png
gifsicle to optimize Animated Gif images
jpegtran to optimize jpg images
pngcrush to optimize png image
pngquant to optimize png images to png8 format
pngout to optimize png images to png8 format
This is a pretty heavy process to be running, luckily it is done very infrequently but I would still like to optimize as much as I can.
Right now to process around 12 images takes roughly 76 seconds. So you can see it is a slow process, imagine 100 images.
I would really like to somehow, mark an image as optimized somehow, so when I am loading a batch of images, the first thing it does is run through ImageMagick to get the exact file type, would be nice if I could somehow embed a message that says this image is already optimized as much as it can be, and then when I am reading images in, if it detects a message it will know to not waste valuable time running that particular image through all the other programs, if this is possible, it could greatly increase speed.
Please help me, I am not used to working with images like this, is this possible even, if it is what is it called and how could I achieve it?
Thanks for any help
If you were to include a flag in the image itself then that would be served to the clients. It would add to the filesize of your images thus negating some of your optimisation.
Suggestions
Keep a reference of the status
Keep a catalog in a file in the same directory - much like the Windows Thumbs.db file.
Another option would be to keep the record in a database or datastore such as Redis or Memcached.
Move after processing
You could move the files to a different directory once they are processed (as #Jordan mentions).
Change the filename to indicate it is processed
Another option would be to append an extra "extension" onto the file name for example:
my_image.processed.jpg
Embedding data in images
Steganography
Usually this is used for attempting to hide covert data in an image and it is called Steganography. It is not really suited to this use case however.
EXIF data
You could write it into the EXIF data of an image, but this would be JPEG and TIFF only as far as I am aware. There is a PHP library available called PEL for writing and reading EXIF data.
You could use the Comment field to tag your image as already optimised, like this:
convert x.jpg -set comment "Optimised" x.jpg
Then, when you are processing, you can extract it like this:
identify -format "%c" x.jpg
Optimised
Related
I'm currently trying to speed up the websites we develop. The part I'm working on now is to optimise the images so that they are as small (filesize, not dimensions) as possible without losing quality.
Our customers can upload their own images to the website through our custom CMS, but images aren't being compressed or optimised at all. My superior explained this is because the customers can upload their own images, and these images could be optimised beforehand through Photoshop or tools like it. If you optimise already optimised images, the quality would get worse. ...right?
We're trying to find a solution that won't require us to install a module or anything. We already use imagejpeg(), imagepng() and imagegif(), but we don't use the $quality parameter because of reasons previously explained. I've seen some snippets, but they all use imagejpg() and the like.
That being said, is there a sure-fire way of optimising images without the risk of optimising previously optimised images? Or would it be no problem at all to use imagejpeg(), imagepng() and imagegif(), even if it would mean optimising already optimised images?
Thank you!
"If you optimise already optimised images, the quality would get worse. "
No if you use a method without loose.
I don't know for method directly in php but if you are on linux server you can use jpegtran or jpegoptim ( with --strip-all) for jpeg and OptiPNG or PNGOUT for png.
Going from your title, I am going to assume compression
So, lets say a normal jpg of 800x600 is uploaded by your customers.
The customers jpg is 272kb because it has full details and everything.
You need to set tresholds for filesizes at dimensions what is acceptable.
Like:
if $dimensions->equals(800,600) and file_type($image) =='jpg' and file_size($image) > 68kb
then schedule_for_compression($image)
and that way you set up parameters for what is acceptable as an upper limit of file size. If the dimensions match, and the filesize is bigger, then its not optimised.
But without knowing more details what exactly is understood about optimising, this is the only thing I can think of.
If you are using a low number of images to compress, you might find an external service, such as: https://tinypng.com/developers might be of assistance.
I've used their on-line tools for reducing filesize on both JPG and PNG file manually but they do appaear to offer a free API service for the first 500 images per month.
Apologies if this would be better as a comment than an answer, I'm fairly new to stackoverflow and haven't got enough points yet, but felt this may be a handy alternative solution.
Saving a JPEG with the same or higher quality setting will not result in a noticeable loss in quality. Just re-save with your desired quality setting. If the file ends up larger, just discard it and keep the original. Remove metadata using jpegran or jpegoptim before you optimize so it doesn't affect the file size when you compare to the original.
PNG and GIF wont't lose any quality unless you reduce the number of colors. Just use one of the optimizers Gyncoca mentioned.
I've been doing some speed optimization on my site using Page Speed and it gives recommendations like so:
Optimizing the following images could reduce their size by 35.3KiB (21% reduction).
Losslessly compressing http://example.com/.../some_image.jpg could save 25.3KiB (55% reduction).
How are the calculations done to get the file size reduction numbers? How can I determine if an image is optimized in PHP?
As I understand it, they seem to base this on the image quality (so saving at 60% in Photoshop or so is considered optimized?).
What I would like to do is once an image is uploaded, check if the image is fully optimized, if not, then optimize it using a PHP image library such as GD or ImageMagick. If I'm right about the number being based on quality, then I will just reduce the quality as needed.
How can I determine if an image is fully optimized in the standards that Page Speed uses?
Chances are they are simply using a standard compression or working on some very simple rules to calculate image compression/quality. It isn't exactly what you were after but what I often use on uploaded images etc etc dynamic content is a class called SimpleImage:
http://www.white-hat-web-design.co.uk/blog/resizing-images-with-php/
this will give you the options to resize & adjust compression, and I think even change image type (by which I mean .jpg to .png or .gif anything you like). I worked in seo and page optimization was a huge part of my Job I generally tried to make the images the size the needed to be no smaller or bigger. compress JS & CSS and that's really all most people need to worry about.
It looks like you could use the PageSpeed Insights API to get the compressions scores: https://developers.google.com/speed/docs/insights/v1/reference, though I'm guessing you'd want to run a quality check/compression locally, rather than sending everything through this API.
It also looks like they've got a standalone image optimizer available at http://code.google.com/p/page-speed/wiki/DownloadPageSpeed?tm=2, though this appears to be a Windows binary. They've got an SDK there as well, though I haven't looked into what it entails, or how easy it would be to integrate into an existing site.
I am currently working on a PHP application which is ran from the command line to optimize a folder of Images.
The PHP application is more of a wrapper for other Image Optimizer's and it simply iterates the directory and grabs all the images, it then runs the Image through the appropriate program to get the best result.
Below are the Programs that I will be using and what each will be used for...
imagemagick to determine file type and convert non-animated gif's to png
gifsicle to optimize Animated Gif images
jpegtran to optimize jpg images
pngcrush to optimize png images
pngquant to optimize png images to png8 format
pngout to optimize png images to png8 format
My problem: With 1-10 images, everything runs smooth and fairly fast however, once I run on a larger folder with 10 or more images, it becomes really slow. I do not really see a good solution around this but one thing that would help is to avoid re-processing images that have already been Optimized. So if I have a folder with 100 images and I optimize that folder and then add 5 new images, re-run the optimizer. It then has to optimize 105 images, my goal is to have it only optimize the 5 newer images since the previous 100 would have already been optimized. This alone would greatly improve performance when new images are added to the image folder.
I realize the simple solution would be to simply copy or move the images to a new folder after processing them, my problem with that simple solution is that these images are used for the web and websites, so the images are generally hard-linked into a websites source code and changing the path to the images would complicate that and possibly break it sometimes.
Some ideas I have had are: Write some kind of text file database to the image folders that will list all the images that have already been processed, so when the application is ran, it will only run on images that are not in that file already. Another idea was to cheange the file name to have some kind of identification in the name to show it has been optimized, a third idea is to move each optimized file to a final destination folder once it is optimized. Idea 2 and 3 are not good though because they will break all image path links in the websites source code.
So please if you can think of a decent/good solution to this problem, please share?
Meta data
You could put a flag in the meta info of each image after it is optimized. First check for that flag and only proceed if it's not there. You can use exif_read_data() to read the data. Writing it maybe like this.
The above is for JPGs. Metdata for PNGs is also possible take a look at this question, and this one.
I'm not sure about GIFs, but you could definitely convert them to PNGs and then add metadata... although I'm pretty sure they have their own meta info, since meta data extraction tools allow GIFs.
Database Support
Another solution would be to store information about the images in a MySQL database. This way, as you tweak your optimizations you could keep track of when and which optimization was tried on which image. You could pick which images to optimize according to any parameters of your choosing. You could build an admin panel for this. This method would allow easy experimentation.
You could also combine the above two methods.
Maximum File Size
Since this is for saving space, you could have the program only work on images that are larger than a certain file size. Ideally, after running the compressor once, all the images would be below this file size, and after that only newly added images that are too big would be touched. I don't know how practical this is in terms of implementation, since it would require that the compressor gets any image below some arbitrary files size. You could make the maximum file size dependent on image size.....
The easiest way would most likely be to look at the time of the last change for each image. If an image was changed after the last run of your script, you have to run it on this particular image.
The timestamp when the script was ran could be saved easily in a short text file.
A thought that comes to my head is to mix the simple solution with a more complicated one. When you optimize the image, move it to a separate folder. When an access is made into the original image folder, have your .htaccess file capture those links and route them to an area of which can see if that same image exists within the optimized folder section, if not, optimize, move, then proceed.
I know i said simple solution, this is a sightly complicated solution, but the nice part is that the solution will provide a scalable approach to your issue.
Edit: One more thing
I like the idea of a MySQL database because you can add a level security (not all images can be viewed by everyone) If thats a need of course. But it also makes your links problem (the hard coded one) not so much a problem. Since all links are a single file of which retrieves the images from the db and the only thing that changes are get variables which are generated. This way your project becomes significantly more scalable and easier to do a design change.
Sorry this is late, but since there is a way to address this issue without creating any files, storing any data of any kind or keeping track of anything. I thought I'd share my solution of how I address things like this.
Goal
Setup an idempotent solution that efficiently optimizes images without dependencies that require keeping track of its current status.
Why
This allows for a truly portable solution that can work in a new environment, an environment that somehow lost its tracker, or an environment that is sensitive as to what files you can actually save in there.
Diagnose
Although metadata might be the first source you'd think to check for this information, it's true that in some cases it will not be available and the nature of metadata itself is arbitrary, like comments, they can come and go and not affect the image in any way. We want something more concrete, something that is a definite descriptor of the asset at hand. Ideally you would want to "identify" if one has been optimized or not, and the way to do that is to review the image to see if it has been based on its characteristics.
Strategy
When you optimize an image, you are providing different options of all sorts in order to reach the final state of optimization. These are the very traits you will also check to come to the conclusion of whether or not it had been in fact optimized.
Example
Lets say we have a function in our script called optimize(path = ''), and let's assume that part of our optimization does the following:
$ convert /path/to/image.jpg -bit-depth=8 -quality=87% -colors=255 -colorspace sRGB ...
Note that these options are ones that you choose to specify, they will be applied to the image and are properties that can be reviewed later...
$ identify -verbose /path/to/image.jpg
Image: /path/to/image.jpg
Format: JPEG (Joint Photographic Experts Group JFIF format)
Mime type: image/jpeg
Geometry: 1250x703+0+0
Colorspace: sRGB <<<<<<
Depth: 8-bit <<<<<<
Channel depth:
Red: 8-bit
Green: 8-bit
Blue: 8-bit
Channel statistics:
Pixels: 878750
Red:
...
Green:
...
Blue:
...
Image statistics:
Overall:
...
Rendering intent: Perceptual
Gamma: 0.454545
Transparent color: none
Interlace: JPEG
Compose: Over
Page geometry: 1250x703+0+0
Dispose: Undefined
Iterations: 0
Compression: JPEG
Quality: 87 <<<<<<
Properties:
...
Artifacts:
...
Number pixels: 878750
As you can see here, the output quite literally has everything I would want to know to determine whether or not I should optimize this image or not, and it costs nothing in terms of a performance hit.
Conclusion
When you are iterating through a list of files in a folder, you can do so as many times as you like without worrying about over optimizing the images or keeping track of anything. You would simply filter out all the extensions you don't want to optimize (eg .bmp, .jpg, .png) then check their stats to see if they possess the attributes your function will apply to the image in the first place. If it has the same values, skip, if not, optimize.
Advanced
If you want to get extremely efficient, you would check each attribute of the image that you plan on optimizing and in your optimization execution you would only apply the options that have not been applied to the command.
Note
This technique is obviously meant to show an example of how you can accurately determine whether or not an image needs to be optimized. The actual options I have listed above are not the complete scope of elements that can be chosen. The are a variety of available options to choose from, and you can apply and check for as many as you want.
Currently i am using gd to converting the image uploaded in jpg with
imagjpeg($data,NULL,85)
and storing it in a blob mysql field. Problem is the quality. Yes the quality is just awful. A screenshot of a window with some text inside becomes pretty hugly. (While with a game screenshot the quality is passable)
Should I use other lib than GD?
Should I use other gd format? (imagepng?)
ps. please note: this is not a duplicate.. i am not asking how to save an img to db
ps2. note that if i should change my currently format/method I have to convert something like 5000-6000 images already stored as jpg in my db
ps3. i use 85 for imagejpeg, can't use a higher value because the image size would grow more than the original version
ps4. I need to store image in db, please stay in topic
Thanks all
================================
REFORMULATING THE QUESTION
I save image (mostly software screenshots, sometime games screenshots) in BLOB field after doing an imagejpeg($resourceFromImageCreateFromString,NULL,85);
Problem I am experiencing is the image uploaded looks pretty bad in confront of the original even if it's the same size. (sometime it gets bigger because of 85)
Is the imagejpeg implementations that sucks?
ps. my simple upload script does the follow calls:
upload file
open it with "rb" (binary)
pass the data to imagecreatefromstring
pass the result to imagejpeg
Thanks again
Is there a special reason you want to store your image inside a BLOB?
What quality argument did you pass to imagejpeg()?
You can set the quality of your output with the 3rd parameter to imagejpeg(). The problem is that JPEG is not the best format for images with a lot of text - you have to choose between turning down the quality and getting a lot of artifacts in your image, or turning up the quality and having a large file size. You might consider using PNG instead.
my first recommendation would be to try saving it as a png as the quality is better.
my second recommendation is to try the quality parameter in imagejpeg($image, $filename, $quality).
recommendation #3 - I worked on a site that I had to rewrite the image storage system because they implemented just what you are doing, storing images as BLOBs in the database. Their system only had about 20K images in it and it was crashing weekly because the database could not handle the load.
The conversion process for 5-6K images should not take more than a couple hours no matter how large the images. When I converted my images to disk storage from db storage, I ran 1K images at a time and each run took 5-10 minutes.
Why do you re-code anyway if it already IS a jpeg? To save space? Well, but then you already have your answer, in part - to save space from jpg to jpg you have to save in lower quality.
You cannot specify quality with imagecopyresampled, with imagejpeg you can. Use a high(er) value for the quality parameter.
Please note that JPEG is a format optimized for storing FOTOS. Images with large areas of the same color, straight lines etc. are much better stored in a lossless format like PNG. Of course, if you have a screenshot taken from a screen with fotos as desk background you have elements of both - so choose your format according to what you want. If the foto-part is more important and you don't care as much about fonts and lines, go with jpeg. Of course JPEG can be just as good quality as PNG for screenshots as well, but then you miss out on most of its compression capabilities, you have to use a 90+% quality setting. At 100% JPEG too is lossless, but by then a PNG might actually be of smaller (file)size.
I am working on a project that resizes images using PHP. This project utilizes PEL to copy EXIF data from the source image to the resized image. However, when resizing images to be really small, the embedded EXIF data contains a thumbnail that is actually much larger than the resized image itself. I would love to use PEL to remove this thumbnail from the EXIF data while keeping all of the other goodies. Any ideas?
I'm sure there are other ways but the simplest way I can see from here is to read the file in using GD, output a new image and copy the EXIF across. This is by no means a simple process but if you're already doing resizing, it would make some sense.
PEL is a bit stale. If you're doing this on any real scale, you might want to look at playing around with some simple Python scripting. You have access to much better graphics and EXIF libraries and you can cron the script to run on a set period... Though I admit this might not suit your website's image processing flow.
You would need to use an image processing library, imagemagik (imagik) or gmagik would work.
Basically you would grab all of the info you wanted via exif then use $image -> stripimage() to remove ALL the meta-data, including the thumbnail, then use gmagic again to take the data you pulled out via exif to add it back in. ($image -> commentimage for example).
The only catch is that none of the documentation specifies whether the image will be re-added as exif, or some other meta-data type.