ImageMagick single convert command performance - php

Question

I have a few thousand images to be processed so each millisecond counts. Each image is ~2-3Mb in size.
Source file fed to the converter:
image.jpg
Files to be generated out of the source:
orig_image.jpg // original image
1024x768_image.jpg // large image
250x250_image.jpg // thumbnail 1
174x174_image.jpg // thumbnail 2
While browsing different topics on imagemagick convert performance I got a feeling that a single command should be way faster than individual converts for each image size. Also a memory utilization was mentioned as a performance boost. (ImageMagick batch resizing performance)
Multiple command conversion (each command run via php's exec() in a loop):
convert "image.jpg" \
-coalesce -resize "1024x768>" +repage "1024x768_image.jpg"
convert "1024x768_image.jpg" \
-coalesce \
-resize "250x250>" \
+repage \
-gravity center \
-extent "250x250" "250x250_image.jpg"
convert "1024x768_image.jpg" \
-coalesce \
-resize "174x174>" \
+repage \
-gravity center \
-extent "174x174" "174x174_image.jpg"
mv image.jpg orig_image.jpg
Single command conversion incorporating ImageMagicks mpr:
convert "image.jpg" -quality 85 -colorspace rgb -coalesce \
-resize "1024x768>" \'
-write "1024x768_image.jpg" \
-write mpr:myoriginal +delete \
mpr:myoriginal -coalesce \
-resize "250x250>" \
-gravity center \
-extent "250x250" \
-write "250x250_image.jpg" +delete \
mpr:myoriginal -coalesce \'
-resize "174x174>" \
-gravity center \
-extent "174x174" \
-write "174x174_image.jpg"
After performance testing the results are somewhat unexpected. Single command convert in a loop finishes in 62 seconds while multiple command conversion executes in just 16 seconds?
# convert -version
Version: ImageMagick 7.0.2-1 Q8 i686 2017-02-03 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2016 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Features: Cipher DPC HDRI OpenMP
Delegates (built-in): bzlib freetype jng jpeg lzma png tiff wmf xml zlib
Also installed libjpeg-turbo jpg processing library but I cannot tell (don't know how to check) if ImageMagic is using it or the old libjpeg.
Any ideas how to speed up image converting process?
Edit:
Don't know how to format it properly here on stackoverflow, but I just noticed that single line command had an argument "-colorspace rgb" and multiple line commands did not which actually results in such strange results where multiple commands are processed faster.
Removed the "-colorspace rgb" argument after which the MPR convert version works the best and gave additional boost in performance.
To sum it all up I ended up using this command:
// MPR
convert "orig_image.jpg" -quality 80 -coalesce \
-resize "1024x768>" \
-write 1024x768_image.jpg \
-write mpr:myoriginal +delete \
mpr:myoriginal -resize "250x250>" \
+repage -gravity center -extent "250x250" \
-write "250x250_image.jpg" \
-write mpr:myoriginal +delete \
mpr:myoriginal -coalesce -resize "174x174>" \
+repage -gravity center -extent "174x174" \
-write "174x174_image.jpg"

Answer 1

You're not using jpeg shrink-on-load, that'll give an easy speedup.
The jpeg library has a neat feature where it'll let you decompress at full resolution, at 1/2, 1/4 or 1/8th. 1/8th resolution is especially quick because of the way jpg works internally.
To exploit this in convert you need to hint to the jpeg loader that you need an image of a particular size. To avoid aliasing you should ask for an image at least 200% larger than your target size.
On this machine, I see:
$ vipsheader image.jpg
image.jpg: 5112x3470 uchar, 3 bands, srgb, jpegload
$ time convert image.jpg -resize 1024x768 1024x768_image.jpg
real 0m0.405s
user 0m1.896s
sys 0m0.068s
If I set the shrink-on-load hint, it's about 2x faster:
$ time convert -define jpeg:size=2048x1536 image.jpg -resize 1024x768 1024x768_image.jpg
real 0m0.195s
user 0m0.604s
sys 0m0.016s
You'll see a dramatic speedup for very large jpg files.
You could also consider another thumbnailer. vipsthumbnail, for example, is quite a bit faster again:
$ time vipsthumbnail image.jpg -s 1024x768 -o 1024x768_image.jpg
real 0m0.111s
user 0m0.132s
sys 0m0.024s
Although real time is down by only a factor of 2, user time is down by a factor of 5 or so. This makes it useful to run with gnu parallel. For example:
parallel vipsthumbnail image.jpg -s {} -o {}_image.jpg ::: \
1024x768 250x250 174x174

Answer 2

Eric's and John's suggestions share much wisdom, and can be mixed in with my suggestion - which is to use GNU Parallel. It will REALLY count when you have thousands of images.
I created 100 images (actually using GNU Parallel, but that's not the point) called image-0.jpg through image-99.jpg. I then did a simple resize operation just to show how to do it without getting too hung up on the ImageMagick aspect. First I did it sequentially and it took 48 seconds to resize 100 images, then I did the exact same thing with GNU Parallel and came in under 10 seconds - so there are massive time savings to be made.
#!/bin/bash
# Create a function used by both sequential and parallel versions - it's only fair
doit(){
echo Converting $1 to $2
convert -define jpeg:size=2048x1536 "$1" -resize 1024x768 "$2"
}
export -f doit
# First do them all sequentially - 48 seconds on iMac
time for img in image*.jpg; do
doit $img "seq-$img"
done
# Now do them in parallel - 10 seconds on iMac
time parallel doit {} "par-{}" ::: image*.jpg
Just for kicks - watch the CPU meter (at top right corner of the movie) and the rate the files pop out of GNU Parallel in the last 1/6th of the movie.

Answer 3

It is funny as I did some conversions a while ago and found mpr was slower as well.
Anyway try this:
$cmd = " convert "image.jpg" -colorspace rgb -coalesce \( -clone 0 -resize 1024x768> -write 1024x768_image.jpg +delete \)".
" \( -clone 0 -resize 250x250> -gravity center -extent 250x250 -write 250_wide.jpg +delete \) ".
" -resize 174x174> -gravity center -extent 174x174 null: ";
exec("convert $cmd 174x174_image.jpg ");
I notice you do not have a background colour for you extent.
You can also add a -define to your loop method check it out in this list: https://www.imagemagick.org/script/command-line-options.php#define
jpeg:size=geometry only reads the amount of data you need to create the image without reading the whole image. You could probably also add it into your first line. -quality is for the output and will have no effect where you put it.
$cmd = " convert "image.jpg" jpeg:size=1024x768 -colorspace rgb -coalesce \( -clone 0 -resize 1024x768> -write 1024x768_image.jpg +delete \)"..
I can never remember if it comes before or after the file name though

Answer 4

Try Magick Persistent Cache image file format (.mpc) over Magick Persistent Registry (.mpr).
convert "image.jpg" -quality 85 -colorspace rgb myoriginal.mpc
convert myoriginal.mpc \
-resize "1024x768>" \
-write "1024x768_image.jpg" \
-resize "250x250>" \
-gravity center \
-extent "250x250" \
-write "250x250_image.jpg" \
-resize "174x174>" \
-gravity center \
-extent "174x174" \
"174x174_image.jpg"
Which results in the following times when tested with 1.8mb jpeg.
real 0m0.051s
user 0m0.133s
sys 0m0.013s
It's true that this will take two commands (although could be simplified to one with -write ... +delete), but very little I/O cost after .mpc is loaded back into the image stack.
Or
You can probably skip .mpc all together with ...
convert "image.jpg" -quality 85 -colorspace rgb \
-resize "1024x768>" \
-write "1024x768_image.jpg" \
-resize "250x250>" \
-gravity center \
-extent "250x250" \
-write "250x250_image.jpg" \
-resize "174x174>" \
-gravity center \
-extent "174x174" \
"174x174_image.jpg"
With results...
real 0m0.061s
user 0m0.163s
sys 0m0.012s

Answer 5

ImageMagick has a special resize operator variation named -thumbnail <geometry> for converting very large images to small thumbnails. Internally it uses
-sample to shrink the image down to 5 times the final height which is much faster than -resize if the thumbnail is much smaller than the original image. Because this operator uses a reduced filter set, the -filter operator is ignored!
-strip to remove all profiles from the image which are usually not required for thumbnails. This also further reduces the size of the resulting image file.
-resize to finally create the requested size and ratio
When it comes to creating thumbnails from JPEG images, then the special JPEG shrink-on-load option -define jpeg:size=<size> can be used instead, as stated out by user894763. Be aware that this option has to be specified immediately after convert, e.g.:
convert -define jpeg:size=<size> input-image.jpg ...
Anyway the -thumbnail operator can be specified additionaly then because it removes all profiles from the thumbnail image and thus reduces the file size.
Detailed information can be found in the ImageMagick usage documentation.

Answer 6

I get about half the time using one command line in ImageMagick 6.9.10.0 Q16 Mac OSX starting with a 3 MB input JPG image.
list="image.jpg"
time for img in $list; do
convert "image.jpg" \
-coalesce -resize "1024x768>" +repage "1024x768_image.jpg"
convert "1024x768_image.jpg" \
-coalesce \
-resize "250x250>" \
+repage \
-gravity center \
-extent "250x250" "250x250_image.jpg"
convert "1024x768_image.jpg" \
-coalesce \
-resize "174x174>" \
+repage \
-gravity center \
-extent "174x174" "174x174_image.jpg"
done
Time: 0m0.952s
time convert "image.jpg" \
-resize "1024x768>" \
+write "1024x768_image.jpg" \
-resize "250x250>" \
-gravity center \
-extent "250x250" \
+write "250x250_image.jpg" \
-resize "174x174>" \
-gravity center \
-extent "174x174" \
"174x174_image.jpg"
Time: 0m0.478s
The coalesce in your multiple commands is not needed, since JPG does not support a virtual canvas. So removing that reduces the multiply command line time to 0m0.738s
Multiple commands should be longer since you have to write and read intermediate images. Since your intermediate images are JPG, you will lose more visual quality each time you write and read the intermediate images. So the quality of one long command line should be better, also.