I'm using a php script to create image thumbnails and this error is thrown while creating some thumbs:
Fatal error: Allowed memory size of 31457280 bytes exhausted (tried to
allocate 227 bytes)
this is what top shows:
top - 07:43:49 up 44 days, 22:21, 1 user, load average: 0.00, 0.00, 0.00
Tasks: 171 total, 1 running, 170 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.2%sy, 0.0%ni, 99.7%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 6097648k total, 3459060k used, 2638588k free, 566924k buffers
Swap: 4194296k total, 0k used, 4194296k free, 1991920k cached
I haven't looked at optimizing phpthumb code. But is there any other way to free the already used memory? May be a cron job can be used to free this memory on regular intervals?
Your image is probably larger than ~10-15MB. PHP has a limit on the amount of memory it can take up per script (memory_limit in php.ini)
What happens is that you load an image in memory (And then resize it, creating a second image)...
Change the memory limit if you're allowed, or don't load such large image ...
AFAIK there is no stream image reader ...
If you can't change the memory limit, a workaround might be calling the commandline ImageMagick or GraphicsMagick tools if they're installed ...
This is a typical php.ini problem, if you are running this script on a VPS or a dedicated server, edit the php.ini file and set memory_limit to 99(or more)MB, also look out for max_run_time as that can stop a script after x number of seconds.
Don't forget to reboot Apache after you have done the changes,
If you are running this on a shared server, you might have some problems trying to solve this as you can't edit the settings file, you can try to set the settings in the actual script, however this usually doesn't .
Related
Background
I understand how PHP's memory_limit setting can be used to control how much memory is available for PHP to use.
As well as using this property to raise/lower the memory limit for your script, you can also set it to -1 to disable the memory limit altogether.
However, as we all know, a computer does not have infinite memory, therefore all we are really talking about is removing any self-imposed limits implemented by PHP itself.
An illustration
We can demonstrate that this is true, by using the following script:
<?php
print("Original: ");
print(ini_get('memory_limit'));
ini_set('memory_limit', -1);
print(", New: ");
print(ini_get('memory_limit'));
$x = "123456789ABCDEF";
while (true)
$x .= $x;
?>
When running from the command-line, I get the following output:
Original: 128M, New: -1
PHP Fatal error: Out of memory (allocated 503840768) (tried to allocate 1006632961 bytes) in test.php on line 14
Fatal error: Out of memory (allocated 503840768) (tried to allocate 1006632961 bytes) in test.php on line 14
zend_mm_heap corrupted
And from the web (via Apache) I get something similar:
Original: 128M, New: -1
Fatal error: Out of memory (allocated 503840768) (tried to allocate 1006632961 bytes) in test.php on line 14
In my examples, the values are the same (~480MB) so it doesn't appear that the web server is imposing a limit. Also, this is nowhere near the amount of RAM installed in the system (even ignoring virtual memory) so it is not a hardware limitation.
Note that these tests were run on PHP 5.6 on a Windows machine with 4GB of RAM. However, output is similar on other PHP versions.
Finally, we come to a question!
Given the above:
What actually dictates the memory limit when we set it to -1?
Is there a way of finding out what this limit is from within PHP?
When you set memory_limit to -1 the actual limit is memory available to operating system.
By available memory I mean sum of physical memory(RAM) and virtual memory(SWAP) minus memory currently used by others processes, and operating system itself.
This is hardware limitation, you just have to consider other processes as well.
For example given 4GB of RAM, no SWAP, and other processes(DB, WebBrowser, etc.) consuming 3GB of memory, you have ~1GB available to be used by PHP. When you try to use more operating system rejects memory allocation request, and subsequently PHP exits with "out of memory" error.
There is no method which can be used across different operating systems, PHP don't provide functions to check how much memory is available and used in the environment. If you know your application is running on Linux you can try to read memory info from /proc/meminfo.
<?php
echo file_get_contents("/proc/meminfo");
which will output something like:
MemTotal: 1999084 kB
MemFree: 174344 kB
MemAvailable: 1077264 kB
Buffers: 151328 kB
Cached: 726752 kB
...
Based on that you can try to estimate amount of available memory, be advised memory used by other processes will vary with time.
I'm busy migrating a customer application written in PHP/MySQL from Hetzner to AWS. Everything works fine but a few scripts. These scripts are poorly written and loop through millions of records, creating hundreds of local variables in each run of the loop, writing each row to an excel file, opening another file, writing a status update and closing a file in each run of the loop. The script is spawned as an independent process using shell_exec from the main web application.
When I first tested the script on EC2, it crashed quite quickly as the php memory_limit parameter was set to 128M on my EC2 instance. The error was similar to the one below:
Fatal error: Allowed memory size of X bytes exhausted (tried to allocate Y bytes)
I increased the memory_limit from 128M to 256M, then to 512M, then to 1024M, 4096M and eventually set to -1 to see where the problem lies.
Setting it to -1 did initially seemed to work but it then froze the entire instance (t2.micro). I then realized it was because of lack of swap space that system memory limits are being reached so just for testing purposes, I added a swap space of 4GB and set memory_limit to -1. This did work as expected and the script never crashed but as the script ran, it became progressively slower. For instance, it wrote the first 10% of records to excel file much faster than the 10% after 50%. All this while I watched the memory usage expand to 6 GB.
Below are screenshots from htop while the script was running:
I'm using the default php.ini that comes php installation except for memory_limit which is set to -1.
However, when I run this very same script on another server (Hetzner shared hosting in this case), the server somehow limits the memory usage and the script runs fine on 128M of memory_limit setting. Although this is not entirely true (as confirmed by htop on the server), it seems like it does not crash on the server with much less memory/swap space than on my EC2 instance.
Below are screenshots from htop running on Hetzner:
The third screenshot was taken towards the end of the script - and as you can see the memory didn't change much between when it started and the end. Here's the php.ini settings from the server:
display_errors=1
memory_limit=128M
max_execution_time=90
max_input_vars=3500
upload_max_filesize=64M
post_max_size=64M
allow_url_fopen=0
The database and code base are exact replicas in both instances.The resultant excel file size is about 225MB in both cases.
So, any ideas what might be causing this behavior and how should I go about fixing it on my EC2 instance?
Thank you for your help!
EasyPHP-Devserver-17-depanne-erreur Fatal error: Allowed memory size
of 134217728 bytes exhausted (tried to allocate 471698889 bytes) in
C:\Program Files
(x86)\EasyPHP-Devserver-17\eds-binaries\httpserver\apache2425vc11x86x180705170153\eds-app-dashboard.php
on line 77
SOLUTION
In my case, it was because Easyphp was trying to load a too large error.log file.
Deleting the server log file in eds-binaries\httpserver\apache2418x160331124251\logs
Test ok
I'm trying to increase the allowed memory for certain PHP script. No matter what I do, for instance this:
ini_set('memory_limit', '512M');
... the script always runs out of memory at around 300MB:
Fatal error: Out of memory (allocated 25165824) (tried to allocate 343810589 bytes) in \\Foo\phpQuery\phpQuery.php on line 255
I've verified by several means that memory_limit is actually changed. The issue seems to be that PHP can't physically allocate a total of 300 MB of memory (25165824 bytes + 343810589 bytes = 352 MB).
I've tried both PHP/5.3.0 and PHP/5.3.9 in two different Windows-based computers with the following specs:
Windows XP / Windows Server 2003 (both computers are 32-bit boxes with 1GB or RAM)
Official PHP 32-bit VC9 binaries
Running as Apache 2.2 module (third-party 32-bit VC9 binaries)
I understand that using half of the physical RAM will force swapping and slow things down as hell but I just need to make sure the script actually works so it can be deployed to the live server. I've also tried larger values (which procuded the same error) and smaller values (with either made my script hit the limit or made Apache crash).
What can be the source of this apparently hard-coded memory limit?
Update #1: I've done further testing with the Windows Server 2003 box (which is actually a VMWare virtual machine). I've increased the "physical" RAM to 2 GB and I've verified that the paging file is allowed to grow up to 1152 MB. Task manager shows that current transaction load is 886 MB and there're 1,5 GB of free physical memory. However, I'm getting the same error with exactly the same figures.
Update #2: As I said, the memory_limit directive is fine. It shows up in both ini_get() and phpinfo(). The error message you'd get is slightly different from mine; mine indicates a PHP crash. Please compare:
Out of memory (allocated 25165824) (tried to allocate 343810589 bytes)
Allowed memory size of 25165824 bytes exhausted (tried to allocate 343810589 bytes)
I'll try to compose a script to reproduce the issue and report back.
An OOM exception is different to the memory limit warninigs.
This means PHP can't actually allocate the memory because insufficient resources are available within your operating system.
You'll need to check the system has sufficient memory/paging available to support this.
Try with max_input_time, sometimes when PHP says memory_limit it actually means max_input_time (-1 is infinite for this one).
I had a similar problem with out-of-memory errors popping up at numbers as low as 250MB. If you find your apache config file that controls ThreadsPerChild (for me it was /conf/extra/httpd-mpm.conf) and reduce the ThreadsPerChild from 150 to 50 or so under you should see a noticeable improvement... here's a script to test it out:
echo "Memory limit: ".ini_get("memory_limit")."<br><br>";
$a=array();
if (ob_get_level() == 0) ob_start();
for($i=0;$i<200;$i++)
{
$a[]=str_pad('',1024*1024*32);
echo "Pass ".$i.", memory used: ".number_format((memory_get_usage())/(1024*1024),0)." MB<br>";
ob_flush();
flush();
}
Hello All i have following ini variable set in phpini file.
max_execution_time 50000
memory_limit 40M
post_max_size 8M
When i try to make thumbnail of image its gives me following error :
Fatal error: Out of memory (allocated 30670848) (tried to allocate 14976 bytes)
image size is 700 kb.
can any one help me ?
Your problem is not that single call to imagecreatefromjpeg() but memory that you have allocated earlier. After all, the allocation fails with a memory request for only ~14kB.
Maybe you created images before in the same script without releasing their memory with imagedestroy() or you have another memory problem. In the latter case you could use a debugging tool (e.g. webgrind) to find the memory hog.
What is the resolution of your 700kb image? It doesn't really matter how big the original .JPG is. If it's a very "simple" image, it could literally be 10,000 x 10,000 pixels. When it's loaded/uncompressed by PHP, those pixels will require 10k x 10k x 3 = 286 megabytes (10x squared times 3 for each red/green/blue component value).
And of course, you say you've set memory limit to 40M, but you've run out of memory at just 29.25 meg. Could be that there's an override somewhere in the setup, perhaps in a site-specific web server .conf file, or a .htaccess, which is setting a lower limit than 40M.
At some point in my php script that makes use of curl, the following error shows up: "Fatal error: Allowed memory size of 262144 bytes exhausted (tried to
allocate 77824 bytes) in"... It points out this part of the script: "$s = curl_exec($c);"
What is the problem? And how to settle it down?
In this case, your server is misconfigured.
Allowed memory size of 262144 bytes
200 kilobytes of RAM per script are not enough for most PHP scripts. The standard in my experience is 8 MB minimum; 16 MB is normal. A blog system like WordPress (it is admittedly fat, but still one of the most popular blog systems around) chokes on 8 MB and runs half-way decently with 16.
You should change the memory_limit value in your php.ini. If you're on shared hosting, demand that the provider increase it to at least 8M, better 16M or more. If they deny, get out of there: It's sub-standard hosting.
you are tying to allocate more memory than the heap can handle
set your limit higher, for xample
at the top of the script::
ini_set("memory_limit","10M");
or in your php.ini
memory_limit = 10M
this set your memory_limit to 10M
ini_set() is probably better than setting something in php.ini. If you've got a specific application you know needs more than the standard memory - then it's fine to increase the memory limit for that application. You want to be really careful opening up all your code to having a higher memory limit though.
That said, if you set memory_limit to 0, there is no memory limit & the script will use as much memory as it needs (and the system can give it).
Here are a couple of suggestions:
edit your php.ini file and change the line that says memory_limit = .25M so that it says
memory_limit = 16M
make sure you are calling
curl_close($c);
consistently.
If you want to include a larger code snippet, maybe we can see where you have memory leaks.