gd-png cannot allocate image data - php

Question

I have an old code base that makes extensive use of GD (no, switching to imagemagick is not an option). It's been running for several years, through several versions. However, when I run it in my current development environment, I'm running into a mysterious gd-png error: cannot allocate image data errors when calling imagecreatefrompng(). The PNG file I'm using is the same as I've been using, so I know it works.
Current setup:
Ansible-provisioned vagrant box
Ubuntu 14.04
PHP 5.5.9
GD 2.1.1
libPNG 1.2.50
PHP's memory limit at the time the script is run is 650M, though it's ultimately the kernel itself that ends up killing the script, and changing PHP's memory limit doesn't seem to have an effect.
The image dimensions are 7200x6600 and are about 500KiB on disk. This hasn't been a problem in my other environments and is only newly-occurring in my development environment. Unfortunately, I don't have access to the other environments anymore to do a comparison, though the setup was similar in the last working one -- Ubuntu 14.04, PHP 5.5, sufficient memory allocations.
What could be happening in this setup that wasn't happening in my previous setups? How can I fix this?

Answer 1

I was browsing a bit through the PHP 5.5.9 source to try and find your specific error string "cannot allocate image data", but the closest I could find is "gd-png error: cannot allocate gdImage struct". The bundled version of GD for PHP 5.5.9 (all the way up to 7.0.0) is version 2.0.35, so it seems you're looking at a custom build(?). Bugging the PHP guys might not help here.
Looking at the GD source (2.1.2, can't seem to find 2.1.1), the only place this error occurs is:
https://github.com/libgd/libgd/blob/master/src/gd_png.c#L435
image_data = (png_bytep) gdMalloc (rowbytes * height);
if (!image_data) {
gd_error("gd-png error: cannot allocate image data\n");
png_destroy_read_struct (&png_ptr, &info_ptr, NULL);
if (im) {
gdImageDestroy(im);
}
if (palette_allocated) {
gdFree (palette);
}
return NULL;
}
Where gdMalloc is:
https://github.com/libgd/libgd/blob/GD-2.1/src/gdhelpers.c#L73
void *
gdMalloc (size_t size)
{
return malloc (size);
}
I'm afraid this is as far as my detective work goes. As far as I can tell the bundled 2.0.35 GD versions in PHP also just use malloc, so at first glance there's no real difference there. I also tried to find some equivalent piece of code in the bundled versions, but so far I haven't found it and it seems to be here:
https://github.com/php/php-src/blob/PHP-5.5.9/ext/gd/libgd/gd_png.c#L323
image_data = (png_bytep) safe_emalloc(rowbytes, height, 0);
I can't seem to find safe_emalloc, but it seems the old PHP bundled versions did use a different memory allocation here than the version your environment uses. Perhaps your best bet is to check with the GD devs?. To avoid a merry goose chase, I think trying another environment is your best bet -after confirming they are indeed using a non-strandard GD version.
After some more PHP source safari, it seems safe_emalloc is used throughout all extensions, so I guess (guess mind you) that this is the preferred way to allocate memory for PHP extensions (looks like it). If your environment is in fact using an unaltered GD 2.1.1, it's likely it is ignoring any PHP memory settings/limits. You might be able to find some other way of specifying the ('stand-alone') GD memory limit?
Edit: Looking at the official libgd faq, near the bottom it states that the PHP extension should indeed respect the memory limit and it specifies that an 8000x8000 pixel image would take about 256MB, so your 650MB limit should really be enough (unless you're working with multiple copies in the same run?) and the fact that it's not working corroborates that something fishy is going on with the memory allocation.

Answer 2

If I was you I would do the following:
Send messages to the php-devel list and see if it is a known issue. You'll also want to search their bug tracker. http://php.net/mailing-lists.php and https://bugs.php.net/
Use GD command line utilities to see if you can process the same image with the same version with cmd line stuff. This is trying to determine if it's a problem with GD and the image or if it is an issue with the PHP-gd lib or some combination.
1.a Create a PHP CLI program to resize the image and see if it works
Figure out exactly what happens when you call the php function in the underlying (probably c) code. This will involve downloading the source to the php-gd module and looking through it.
Code a minimal c application that does the same thing as the underlying PHP-gd library and see if you can reproduce the error
Something in there should tell you exactly what is going on, though it will take a bit of work. You should be able to get all the tools/sources for your environment. The beauty of open source, right?
The other option would be to try different versions of these applications in your environment and see if you find one that works. This is the "shotgun" approach and not the best, IMO.

Answer 3

From my experience:
Try a file with a different row size. I had issues with another image library that did not read images longer than 3000px/row. It was otherwise not restricted in absolute size.
You have quite an image there, if this is in RGBA in memory, you end up with 180M uncompressed image data, 140M on RGB, still 45M as 8Bit. You are sure, your process limits will not be exceeded when this is loaded?

Answer 4

You used the same image with the same libraries, it means libgd and libpng have no restrictions about the memory size (or at least not one for the size you're using), the problem may be php but you are sure the memory limit is 650M (if your php installation is using suhosin, you may want to check suhosin.memory_limit)
You tell the kernel is killing the script (but I can't find why you are telling so, is there a dmesg log about?) but the kernel kill a process only when is out of memory.
One problem could be memory fragmentation / contiguous alloc, nowadays it should be more a kernel space problem than user space, but the size your script is allocating is not an every day size.
Fragmentation could be a problem if your script is running on a server with a long uptime with a lot of process allocating and freeing memory, if is a virtual machine that you just start up and the script fail at the first launch, then the problem isn't fragmentation.
You can do a simple test, command and code below, it use the gcc compiler to create (in the directory where you launch the command) a a.out executable which allocate and set to 0 a memory chunk with size 191 x 1024 x 1024 (about ~190M)
echo '#include <stdlib.h>\n#include <string.h>\n#define SIZ 191*1024*1024\nint main() {void *p=malloc(SIZ);memset(p,0,SIZ);return (p==NULL?1:0);}' \
| gcc -O2 -xc -
run the executable with:
./a.out
If the problem is the system memory the kernel should kill the executable (I'm not sure 100%, the kernel have some policy about which process to kill when there's no memory, to be checked)
You should see the kill message though, with dmesg perhaps.
(I think this shouldn't happen, instead the malloc should just fail)
If the malloc succeed, the executable can allocate enough memory, it should return 0, if it fail should return 1
to verify the return code just use
./a.out
echo $?
(don't execute any other command in the middle)
If the malloc fail you probably just need to add physical or virtual memory to the system.
Keep in mind that ~190M is just the memory for one uncompressed image, depending on how php, libgd and libpng are working the memory for the whole process may be even twice (or more).
I did a simple test and profiled the memory usage, the peak with an image 7600x2200 (1.5M on disk) on my system is about ~342M, here's the test:
<?php
$im = imagecreatefrompng("input.png");
$string = "Sinker sucker socks pants";
$orange = imagecolorallocate($im, 220, 210, 60);
$px = (imagesx($im) - 7.5 * strlen($string)) / 2;
imagestring($im, 3, $px, 9, $string, $orange);
imagepng($im);
imagedestroy($im);
I tried xhprof at first but it returned values too low.
So I tried a simple script memusg
memusg /usr/bin/php -f test.php >output.png
(the test work btw)

Answer 5

I met this problem yesterday and fixed it today.
Yesterday's env:
php-7.0.12
libpng-1.6.26
libgd-2.1.1
This suite will crash when I resize a png image.
After memory check, I thought it might be a bug in the latest php or libpng.
So I changed the env to this:
php-5.6.27
libpng-1.2.56
libgd-2.1.1
I changed php and libpng to mature versions which are used longtime.
Recompile and re-install these - it works well for png and jpeg.