Delay a while loop in PHP to prevent memory exhaustion? - php

Question

I have a pretty simple image resize script that loops through 1000 folders, with different amounts of images in each folder. On my testing server, there aren't a lot of images, so it runs fine...but live with about 100K images...it will run into the memory problem. The php memory is already set to 384M, but i'd rather have the script maybe only process 10 folders at a time, stop for a bit, then start again. How can I do that within the while loop? use the sleep function maybe?
Here it is...
class DevTestHelper {
//This will Go through all full size images and resize images into requested folder
function resizeGalleryImages($destFolder, $width, $height, $isCrop = false) {
$count = 1000;
$source = JPATH_SITE."/images/gallery/full";
$dest = JPATH_SITE."/images/gallery/".$destFolder;
echo 'Processing Images<br>';
//Loop through all 1000 folders 0 to 999 in the /full dir
for($i=0; $i < $count;$i++) {
$iPath = $source.'/'.$i;
if(is_dir($iPath)) {
$h = opendir($iPath);
//Loop through all the files in numbered folder
while ($entry = readdir($h)) {
//only read files
if ($entry != "." && $entry != ".." && !is_dir($entry)) {
$img = new GImage($source.'/'.$i.'/'.$entry);
$tmp = $img->resize($width, $height, true, 3);
if($isCrop) {
$tmpWidth = $tmp->getWidth();
$tmpHeight = $tmp->getHeight();
$xOffset = ($tmpWidth - $width) / 2;
$yOffset = ($tmpHeight - $height) / 2;
$tmp->crop($width, $height, $xOffset, $yOffset, false, 3);
}
$destination = $dest.'/'.$i.'/'.$entry;
if(!$tmp->toFile($destination)) {
echo 'error in creating resized image at: '.$destination.'<br>';
}
flush();
ob_flush();
sleep(10);
echo $entry ;
//echo $entry.'<br>';
}
}
echo 'Processed: '.$i.' Folder<br>';
}
}
Thanks for any suggestions.
/**
* Class to manipulate an image.
*
* #package Gallery.Libraries
* #subpackage Media
* #version 1.0
*/
class GImage extends JObject
{
/**
* The image handle
*
* #access private
* #var resource
*/
var $_handle = null;
/**
* The source image path
*
* #access private
* #var string
*/
var $_path = null;
var $_support = array();
/**
* Constructor
*
* #access public
* #return void
* #since 1.0
*/
function __construct($source=null)
{
// First we test if dependencies are met.
if (!GImageHelper::test()) {
$this->setError('Unmet Dependencies');
return false;
}
// Determine which image types are supported by GD.
$info = gd_info();
if ($info['JPG Support']) {
$this->_support['JPG'] = true;
}
if ($info['GIF Create Support']) {
$this->_support['GIF'] = true;
}
if ($info['PNG Support']) {
$this->_support['PNG'] = true;
}
// If the source input is a resource, set it as the image handle.
if ((is_resource($source) && get_resource_type($source) == 'gd')) {
$this->_handle = &$source;
} elseif (!empty($source) && is_string($source)) {
// If the source input is not empty, assume it is a path and populate the image handle.
//Andy - Big freaking cockroach: if file is wrong type or doesn't even exist the error is not handled.
$this->loadFromFile($source);
}
}
function crop($width, $height, $left, $top, $createNew = true, $scaleMethod = JXIMAGE_SCALE_INSIDE)
{
// Make sure the file handle is valid.
if ((!is_resource($this->_handle) || get_resource_type($this->_handle) != 'gd')) {
$this->setError('Invalid File Handle');
return false;
}
// Sanitize width.
$width = ($width === null) ? $height : $width;
if (preg_match('/^[0-9]+(\.[0-9]+)?\%$/', $width)) {
$width = intval(round($this->getWidth() * floatval(str_replace('%', '', $width)) / 100));
} else {
$width = intval(round(floatval($width)));
}
// Sanitize height.
$height = ($height === null) ? $width : $height;
if (preg_match('/^[0-9]+(\.[0-9]+)?\%$/', $height)) {
$height = intval(round($this->getHeight() * floatval(str_replace('%', '', $height)) / 100));
} else {
$height = intval(round(floatval($height)));
}
// Sanitize left.
$left = intval(round(floatval($left)));
// Sanitize top.
$top = intval(round(floatval($top)));
// Create the new truecolor image handle.
$handle = imagecreatetruecolor($width, $height);
// Allow transparency for the new image handle.
imagealphablending($handle, false);
imagesavealpha($handle, true);
if ($this->isTransparent()) {
// Get the transparent color values for the current image.
$rgba = imageColorsForIndex($this->_handle, imagecolortransparent($this->_handle));
$color = imageColorAllocate($this->_handle, $rgba['red'], $rgba['green'], $rgba['blue']);
// Set the transparent color values for the new image.
imagecolortransparent($handle, $color);
imagefill($handle, 0, 0, $color);
imagecopyresized(
$handle,
$this->_handle,
0, 0,
$left,
$top,
$width,
$height,
$width,
$height
);
} else {
imagecopyresampled(
$handle,
$this->_handle,
0, 0,
$left,
$top,
$width,
$height,
$width,
$height
);
}
// If we are cropping to a new image, create a new GImage object.
if ($createNew)
{
// Create the new GImage object for the new truecolor image handle.
$new = new GImage($handle);
return $new;
} else
{
// Swap out the current handle for the new image handle.
$this->_handle = &$handle;
return true;
}
}
function filter($type)
{
// Initialize variables.
$name = preg_replace('#[^A-Z0-9_]#i', '', $type);
$className = 'GImageFilter_'.ucfirst($name);
if (!class_exists($className))
{
jimport('joomla.filesystem.path');
if ($path = JPath::find(GImageFilter::addIncludePath(), strtolower($name).'.php'))
{
require_once $path;
if (!class_exists($className)) {
$this->setError($className.' not found in file.');
return false;
}
}
else {
$this->setError($className.' not supported. File not found.');
return false;
}
}
$instance = new $className;
if (is_callable(array($instance, 'execute')))
{
// Setup the arguments to call the filter execute method.
$args = func_get_args();
array_shift($args);
array_unshift($args, $this->_handle);
// Call the filter execute method.
$return = call_user_func_array(array($instance, 'execute'), $args);
// If the filter failed, proxy the error and return false.
if (!$return) {
$this->setError($instance->getError());
return false;
}
return true;
}
else {
$this->setError($className.' not valid.');
return false;
}
}
function getHeight()
{
return imagesy($this->_handle);
}
function getWidth()
{
return imagesx($this->_handle);
}
function isTransparent()
{
// Make sure the file handle is valid.
if ((!is_resource($this->_handle) || get_resource_type($this->_handle) != 'gd')) {
$this->setError('Invalid File Handle');
return false;
}
return (imagecolortransparent($this->_handle) >= 0);
}
function loadFromFile($path)
{
// Make sure the file exists.
if (!JFile::exists($path)) {
$this->setError('File Does Not Exist');
return false;
}
// Get the image properties.
$properties = GImageHelper::getProperties($path);
if (!$properties) {
return false;
}
// Attempt to load the image based on the MIME-Type
switch ($properties->get('mime'))
{
case 'image/gif':
// Make sure the image type is supported.
if (empty($this->_support['GIF'])) {
$this->setError('File Type Not Supported');
return false;
}
// Attempt to create the image handle.
$handle = #imagecreatefromgif($path);
if (!is_resource($handle)) {
$this->setError('Unable To Process Image');
return false;
}
$this->_handle = &$handle;
break;
case 'image/jpeg':
// Make sure the image type is supported.
if (empty($this->_support['JPG'])) {
$this->setError('File Type Not Supported');
return false;
}
// Attempt to create the image handle.
$handle = #imagecreatefromjpeg($path);
if (!is_resource($handle)) {
$this->setError('Unable To Process Image');
return false;
}
$this->_handle = &$handle;
break;
case 'image/png':
// Make sure the image type is supported.
if (empty($this->_support['PNG'])) {
$this->setError('File Type Not Supported');
return false;
}
// Attempt to create the image handle.
$handle = #imagecreatefrompng($path);
if (!is_resource($handle)) {
$this->setError('Unable To Process Image');
return false;
}
$this->_handle = &$handle;
break;
default:
$this->setError('File Type Not Supported');
return false;
break;
}
// Set the filesystem path to the source image.
$this->_path = $path;
return true;
}
function resize($width, $height, $createNew = true, $scaleMethod = JXIMAGE_SCALE_INSIDE)
{
// Make sure the file handle is valid.
if ((!is_resource($this->_handle) || get_resource_type($this->_handle) != 'gd')) {
$this->setError('Invalid File Handle');
return false;
}
// Prepare the dimensions for the resize operation.
$dimensions = $this->_prepareDimensions($width, $height, $scaleMethod);
if (empty($dimensions)) {
return false;
}
//var_dump($dimensions);
// Create the new truecolor image handle.
$handle = imagecreatetruecolor($dimensions['width'], $dimensions['height']);
// Allow transparency for the new image handle.
imagealphablending($handle, false);
imagesavealpha($handle, true);
if ($this->isTransparent()) {
// Get the transparent color values for the current image.
$rgba = imageColorsForIndex($this->_handle, imagecolortransparent($this->_handle));
$color = imageColorAllocate($this->_handle, $rgba['red'], $rgba['green'], $rgba['blue']);
// Set the transparent color values for the new image.
imagecolortransparent($handle, $color);
imagefill($handle, 0, 0, $color);
imagecopyresized(
$handle,
$this->_handle,
0, 0, 0, 0,
$dimensions['width'],
$dimensions['height'],
$this->getWidth(),
$this->getHeight()
);
} else {
imagecopyresampled(
$handle,
$this->_handle,
0, 0, 0, 0,
$dimensions['width'],
$dimensions['height'],
$this->getWidth(),
$this->getHeight()
);
}
// If we are resizing to a new image, create a new GImage object.
if ($createNew)
{
// Create the new GImage object for the new truecolor image handle.
$new = new GImage($handle);
return $new;
} else
{
// Swap out the current handle for the new image handle.
$this->_handle = &$handle;
return true;
}
}
function toFile($path, $type = IMAGETYPE_JPEG, $options=array())
{
switch ($type)
{
case IMAGETYPE_GIF:
$ret = imagegif($this->_handle, $path);
break;
case IMAGETYPE_PNG:
$ret = imagepng($this->_handle, $path, (array_key_exists('quality', $options)) ? $options['quality'] : 0);
break;
case IMAGETYPE_JPEG:
default:
$ret = imagejpeg($this->_handle, $path, (array_key_exists('quality', $options)) ? $options['quality'] : 100);
break;
}
return $ret;
}
function display() {
//header('Content-type: image/jpeg');
imagejpeg($this->_handle,'',100);
}
function _prepareDimensions($width, $height, $scaleMethod)
{
// Sanitize width.
$width = ($width === null) ? $height : $width;
if (preg_match('/^[0-9]+(\.[0-9]+)?\%$/', $width)) {
$width = intval(round($this->getWidth() * floatval(str_replace('%', '', $width)) / 100));
} else {
$width = intval(round(floatval($width)));
}
// Sanitize height.
$height = ($height === null) ? $width : $height;
if (preg_match('/^[0-9]+(\.[0-9]+)?\%$/', $height)) {
$height = intval(round($this->getHeight() * floatval(str_replace('%', '', $height)) / 100));
} else {
$height = intval(round(floatval($height)));
}
$dimensions = array();
if ($scaleMethod == JXIMAGE_SCALE_FILL)
{
$dimensions['width'] = $width;
$dimensions['height'] = $height;
}
elseif ($scaleMethod == JXIMAGE_SCALE_INSIDE || $scaleMethod == JXIMAGE_SCALE_OUTSIDE)
{
$rx = $this->getWidth() / $width;
$ry = $this->getHeight() / $height;
if ($scaleMethod == JXIMAGE_SCALE_INSIDE)
$ratio = ($rx > $ry) ? $rx : $ry;
else
$ratio = ($rx < $ry) ? $rx : $ry;
$dimensions['width'] = round($this->getWidth() / $ratio);
$dimensions['height'] = round($this->getHeight() / $ratio);
}
else {
$this->setError('Invalid Fit Option');
return false;
}
return $dimensions;
}
}
class GImageFilter extends JObject
{
/**
* Add a directory where GImage should search for filters. You may
* either pass a string or an array of directories.
*
* #access public
* #param string A path to search.
* #return array An array with directory elements
* #since 1.5
*/
function addIncludePath($path='')
{
static $paths;
if (!isset($paths)) {
$paths = array(dirname(__FILE__).'/image');
}
// force path to array
settype($path, 'array');
// loop through the path directories
foreach ($path as $dir)
{
if (!empty($dir) && !in_array($dir, $paths)) {
array_unshift($paths, JPath::clean( $dir ));
}
}
return $paths;
}
function execute()
{
$this->setError('Method Not Implemented');
return false;
}
}
class GImageHelper
{
function getProperties($path)
{
// Initialize the path variable.
$path = (empty($path)) ? $this->_path : $path;
// Make sure the file exists.
if (!JFile::exists($path)) {
$e = new JException('File Does Not Exist');
return false;
}
// Get the image file information.
$info = #getimagesize($path);
if (!$info) {
$e = new JException('Unable To Get Image Size');
return false;
}
// Build the response object.
$result = new JObject;
$result->set('width', $info[0]);
$result->set('height', $info[1]);
$result->set('type', $info[2]);
$result->set('attributes', $info[3]);
$result->set('bits', #$info['bits']);
$result->set('channels', #$info['channels']);
$result->set('mime', $info['mime']);
return $result;
}
function test()
{
return (function_exists('gd_info') && function_exists('imagecreatetruecolor'));
}
}

Answer 1

It looks to me like you have a memory leak, when creating $img using new GImage it will likely be reading the image into memory which you never release.
You take a copy of the resized image:
$tmp = $img->resize($width, $height, true, 3);
but next time around the loop you create a new GImage again using the $img variable.
Does GIMage have some kind of close or cleanup function? If so, do that at the end of each loop before you call new GImage again.
closeGImage($img);
UPDATE:
In response to your updated question it looks like you can do one of two things:
Change the 3rd parameter of $img->resize() to false as this will stop a new GImage being created on image resize.
Write your own image destroy function as part of the GImage class. Because GImage is based upon the GD library you can use the imagedestroy function, see here.
UPDATE 2:
Ok, so following on from your comment... What you'll want to do to create a destroy image function is inside the GImage class, add the following function:
function closeImage()
{
imagedestroy($this->_handle);
}
Now at the bottom of the for loop in devtest.php add a call to this function:
...
ob_flush();
sleep(10);
echo $entry ;
$img->closeImage();
...

Answer 2

Try using sleep or usleep to avoid CPU overload.
Maybe GImage have a dispose() or destroy() method to avoid memory retention.
You could also do batch processing.