I have 2 servers
Both server's have the same php memory_limit of 128M of data.
My Dev Server runs a script just fine, while on my prod server I am receiving a Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 32 bytes) in ...
My question is what are other reasons I would be running out of memory in the prod environment even though my php memory_limits are the same?
Preface
PHP is module that runs top of Apache [HTTPD Server] this involves linking the php interpreter against a library of hooks published by the webserver
Cause
Now it can exhaust due to scripts running allocating memory [RAM] & reach its threshold & get such errors.
Example big loops running & saving lots of data in memory which may over RUN the Memory
Possible Optimization you can do
memory_limit = 32M to your server's main php.ini file (recommended, if you have access)
php_value memory_limit 32M in your .htaccess file in the
These are some work around for pages where you RUN out of Memory
ini_set('memory_limit', '-1'); overrides the default PHP memory limit (On individual php pages wherever you need extra memory)
Also some optimization you can do on HTTP Server (apache.conf or http.conf)
RLimitCPU,RLimitNPROC, RLimitMEM parameters can be adusted
Since you are running out of Memory you can adjust RLimitMEM
Syntax: RLimitMEM soft-bytes [hard-bytes]
Example: RLimitMEM 1048576 2097152
This directive sets the soft and hard limits for maximum memory usage of a process in bytes. It takes one or two parameters. The first parameter sets the soft resource limit for all processes. The second parameter sets the maximum resource limit. Either parameter can be a number, or ``max'', which indicates to the server that the limit should match the maximum allowed by the operating system configuration. Raising the maximum resource limit requires the server to be running as the user ``root'' or in the initial start-up phase.
You can put the lines in .htaccess files too [since in shared hosting you didnt have access to php.ini & http.conf files
Related
I'm busy migrating a customer application written in PHP/MySQL from Hetzner to AWS. Everything works fine but a few scripts. These scripts are poorly written and loop through millions of records, creating hundreds of local variables in each run of the loop, writing each row to an excel file, opening another file, writing a status update and closing a file in each run of the loop. The script is spawned as an independent process using shell_exec from the main web application.
When I first tested the script on EC2, it crashed quite quickly as the php memory_limit parameter was set to 128M on my EC2 instance. The error was similar to the one below:
Fatal error: Allowed memory size of X bytes exhausted (tried to allocate Y bytes)
I increased the memory_limit from 128M to 256M, then to 512M, then to 1024M, 4096M and eventually set to -1 to see where the problem lies.
Setting it to -1 did initially seemed to work but it then froze the entire instance (t2.micro). I then realized it was because of lack of swap space that system memory limits are being reached so just for testing purposes, I added a swap space of 4GB and set memory_limit to -1. This did work as expected and the script never crashed but as the script ran, it became progressively slower. For instance, it wrote the first 10% of records to excel file much faster than the 10% after 50%. All this while I watched the memory usage expand to 6 GB.
Below are screenshots from htop while the script was running:
I'm using the default php.ini that comes php installation except for memory_limit which is set to -1.
However, when I run this very same script on another server (Hetzner shared hosting in this case), the server somehow limits the memory usage and the script runs fine on 128M of memory_limit setting. Although this is not entirely true (as confirmed by htop on the server), it seems like it does not crash on the server with much less memory/swap space than on my EC2 instance.
Below are screenshots from htop running on Hetzner:
The third screenshot was taken towards the end of the script - and as you can see the memory didn't change much between when it started and the end. Here's the php.ini settings from the server:
display_errors=1
memory_limit=128M
max_execution_time=90
max_input_vars=3500
upload_max_filesize=64M
post_max_size=64M
allow_url_fopen=0
The database and code base are exact replicas in both instances.The resultant excel file size is about 225MB in both cases.
So, any ideas what might be causing this behavior and how should I go about fixing it on my EC2 instance?
Thank you for your help!
EasyPHP-Devserver-17-depanne-erreur Fatal error: Allowed memory size
of 134217728 bytes exhausted (tried to allocate 471698889 bytes) in
C:\Program Files
(x86)\EasyPHP-Devserver-17\eds-binaries\httpserver\apache2425vc11x86x180705170153\eds-app-dashboard.php
on line 77
SOLUTION
In my case, it was because Easyphp was trying to load a too large error.log file.
Deleting the server log file in eds-binaries\httpserver\apache2418x160331124251\logs
Test ok
I'm getting the error when I run my PHP script....
Fatal error: Out of memory (allocated 1827405824) (tried to allocate 88800 bytes)
I've added this line to my PHP script..
ini_set("memory_limit","3000M");
This statement does seem to correctly control the memory usage, but I dont seem to be able to get it above about 1.8GB.
Its as if the upper memory limit is being restricted somewhere else.
I've also added to the php.ini...
memory_limit = 3000M
Does anyone know if the memory is restricted elsewhere?
I'm running a local server with Xampp.
I have Windows 7, 64-bit with 4GB RAM.
My script uses PHP's GD image library and I get the error when trying to allocate an image reference with ImageCreateTrueColor().
(I know this is a huge amount of memory - but this is just a one-of script, and its just a lot easier to do it this way.)
Thanks.
Update....
#elusive #Orbling
I expect everybody's bored whith this question, but here is the simplified code which illustrates the problem.
<?php
ini_set("memory_limit","4000000000");
echo "ini_get = " . ini_get('memory_limit') . "<br>\n";
echo "memory_get_usage = " . memory_get_usage(true) . "<br>\n";
$bigImageHandle = imagecreatetruecolor(22200, 24800); //this is line 5
?>
Browser output...
ini_get = 4000000000
memory_get_usage = 524288
Fatal error: Out of memory (allocated 1843396608) (tried to allocate 88800 bytes) in
E:\User\My_Webs\experiments\houseshunting\temp\osMaps\t1.php on line 5
I tested this out with a smaller set of tiles and the memory used by imagecreatetruecolor() and I estimate I need 2.7GB
Using Acquia Dev Desktop, I had many memory limit crashes.
After having increased the memory limit into PHP.ini.
php_value memory_limit 1024M
php_value max_execution_time 3000
This issue was less frequent but still occuring ( Especially with Feature Recreate )
Into my httpd.conf I increased the StackThread to 16M
ThreadStackSize 16*1024*1024
And it solved the memory crash issue.
Hope it can help
You're running on a 64-bit operating system, but Apache and PHP are likely still 32-bit. If you're using mod_php, apache would be the limiting factor here.
32-bit processes are limited about 2GiB of RAM unless you used the /3GB switch and the software is aware of 3GB support.
That still leaves up about 200 MiB that seems unused, but its small enough that it can be used by various libraries that all have to be loaded in memory
As far as I know, the library usage won't show up in the committed memory, but still counts towards the 2GiB limit (much like device memory counts towards the 4GiB limit on 32-bit windows. Where installing 2 GiB graphics card brings you down to under 2GiB of usable RAM).
Most likely solution? Install a 64-bit PHP, and then dispatch it to that (using a system() call, perhaps)
In Ubuntu 18.04
Check version PHP: php -v
In my case I have PHP 7.4
Edit file: nano /etc/php/7.4/apache2/php.ini
Search and change memory_limit = 2048M
Edit file: nano /etc/php/7.4/cli/php.ini
Search and change memory_limit = 2048M
Finally: systemctl restart apache2
Which PHP version are you using?
The memory_limit variable is, or was, contained in a 32-bit integer, so can not go above 2GB.
See: http://bugs.php.net/bug.php?id=39132&edit=1
From the bottom comment on that bug report, it might be the routine that translates the human readable form to a number, try putting it in digits.
Check your Apache config (e.g., httpd.conf). There's likely an RLimitMEM directive limiting the memory allow for children processes handling requests.
So, you can set your PHP limit all you want, if Apache spawns the process with a memory limit, you can't exceed that.
If you're on a hosted service and have a shared server, likely you don't have access to this config and need to work with your provider. As you can see, it's configuration that applies server-wide... you're not likely going to get them to change this. Then again, if you're looking to spawn bigger than 1.5Gig processes, you prolly should be either solving the problem a different way (others have suggested this) or getting a dedicated server of some kind (e.g. EC2).
For example:
/usr/local/apache/conf
#RLimitMEM 85643200 104857600 # Limit to: 80Mb / process, 100Mb total
RLimitMEM 134217728 537395200 # Limit to: 128Mb / Process, 512Mb total
The issue is likely to be caused by running 32-bit apache and php. Try upgrading these to 64-bit binaries and see if that fixes the issue.
I've had exactly the same problem and after searching a lot I discovered it had nothing to do wih memory limit but with a bug in my code: I had a function using an array, located in external file. In this function I had set the array as "global" but missed to include the file with tis array....
Like the spiderplant0's error, this was giving me an error with a very very huge memory allocation.
So check your code and try to see which part create this error.
Try this
set_time_limit(300);
ini_set('memory_limit', '20000M');
Try this:
#php_value memory_limit 300M
#php_value upload_max_filesize 200M
#php_value post_max_size 200M
#php_value max_execution_time 80000
#php_value max_input_time 80000
try ini_set('memory_limit', '-1');
Fatal error: Allowed memory size of 104857600 bytes exhausted (tried
to allocate 32345609 bytes)
It mean I need 137203209 bytes memory limit to run my php script.
I got this error although I have the following code in the script.
ini_set('memory_limit', 268435456); # 256 MB
The script is on my shared hosting server. I don't have access to php.ini.
The memory_limit did work for other scripts in the same hosting server.
When I checked phpinfo() of my server, I see these
suhosin.memory_limit - 128
memory_limit - 100M
My script definitely needs more than 128M because it is a mail sending script with large file attachment.
You're on a shared host, and shared hosts almost never let you using the ini_set('memory_limit',XXX) functions (otherwise everyone would always try to grab the whole server memory and lock up the whole server). Check phpinfo() to see if safe-mode is on... it probably is.
Also, side-note: if you want to set the memory limit to 256MB in any case, you can just use the:
ini_set('memory_limit', '256M');
notation instead of writing out the whole integer.
suhosin.memory_limit - 128 does not allow you to increase memory limit higher that 128mb, set it to 268435456 to be allowed to increase memory_limit up to 256Mb. I doubt you can do it on a shared hosting, though, because you need an access to suhosin ini file (you can not do something like ini_set('suhosin.memory_limit', 268435456);).
More about suhosin.memory_limit
I'm getting the error when I run my PHP script....
Fatal error: Out of memory (allocated 1827405824) (tried to allocate 88800 bytes)
I've added this line to my PHP script..
ini_set("memory_limit","3000M");
This statement does seem to correctly control the memory usage, but I dont seem to be able to get it above about 1.8GB.
Its as if the upper memory limit is being restricted somewhere else.
I've also added to the php.ini...
memory_limit = 3000M
Does anyone know if the memory is restricted elsewhere?
I'm running a local server with Xampp.
I have Windows 7, 64-bit with 4GB RAM.
My script uses PHP's GD image library and I get the error when trying to allocate an image reference with ImageCreateTrueColor().
(I know this is a huge amount of memory - but this is just a one-of script, and its just a lot easier to do it this way.)
Thanks.
Update....
#elusive #Orbling
I expect everybody's bored whith this question, but here is the simplified code which illustrates the problem.
<?php
ini_set("memory_limit","4000000000");
echo "ini_get = " . ini_get('memory_limit') . "<br>\n";
echo "memory_get_usage = " . memory_get_usage(true) . "<br>\n";
$bigImageHandle = imagecreatetruecolor(22200, 24800); //this is line 5
?>
Browser output...
ini_get = 4000000000
memory_get_usage = 524288
Fatal error: Out of memory (allocated 1843396608) (tried to allocate 88800 bytes) in
E:\User\My_Webs\experiments\houseshunting\temp\osMaps\t1.php on line 5
I tested this out with a smaller set of tiles and the memory used by imagecreatetruecolor() and I estimate I need 2.7GB
Using Acquia Dev Desktop, I had many memory limit crashes.
After having increased the memory limit into PHP.ini.
php_value memory_limit 1024M
php_value max_execution_time 3000
This issue was less frequent but still occuring ( Especially with Feature Recreate )
Into my httpd.conf I increased the StackThread to 16M
ThreadStackSize 16*1024*1024
And it solved the memory crash issue.
Hope it can help
You're running on a 64-bit operating system, but Apache and PHP are likely still 32-bit. If you're using mod_php, apache would be the limiting factor here.
32-bit processes are limited about 2GiB of RAM unless you used the /3GB switch and the software is aware of 3GB support.
That still leaves up about 200 MiB that seems unused, but its small enough that it can be used by various libraries that all have to be loaded in memory
As far as I know, the library usage won't show up in the committed memory, but still counts towards the 2GiB limit (much like device memory counts towards the 4GiB limit on 32-bit windows. Where installing 2 GiB graphics card brings you down to under 2GiB of usable RAM).
Most likely solution? Install a 64-bit PHP, and then dispatch it to that (using a system() call, perhaps)
In Ubuntu 18.04
Check version PHP: php -v
In my case I have PHP 7.4
Edit file: nano /etc/php/7.4/apache2/php.ini
Search and change memory_limit = 2048M
Edit file: nano /etc/php/7.4/cli/php.ini
Search and change memory_limit = 2048M
Finally: systemctl restart apache2
Which PHP version are you using?
The memory_limit variable is, or was, contained in a 32-bit integer, so can not go above 2GB.
See: http://bugs.php.net/bug.php?id=39132&edit=1
From the bottom comment on that bug report, it might be the routine that translates the human readable form to a number, try putting it in digits.
Check your Apache config (e.g., httpd.conf). There's likely an RLimitMEM directive limiting the memory allow for children processes handling requests.
So, you can set your PHP limit all you want, if Apache spawns the process with a memory limit, you can't exceed that.
If you're on a hosted service and have a shared server, likely you don't have access to this config and need to work with your provider. As you can see, it's configuration that applies server-wide... you're not likely going to get them to change this. Then again, if you're looking to spawn bigger than 1.5Gig processes, you prolly should be either solving the problem a different way (others have suggested this) or getting a dedicated server of some kind (e.g. EC2).
For example:
/usr/local/apache/conf
#RLimitMEM 85643200 104857600 # Limit to: 80Mb / process, 100Mb total
RLimitMEM 134217728 537395200 # Limit to: 128Mb / Process, 512Mb total
The issue is likely to be caused by running 32-bit apache and php. Try upgrading these to 64-bit binaries and see if that fixes the issue.
I've had exactly the same problem and after searching a lot I discovered it had nothing to do wih memory limit but with a bug in my code: I had a function using an array, located in external file. In this function I had set the array as "global" but missed to include the file with tis array....
Like the spiderplant0's error, this was giving me an error with a very very huge memory allocation.
So check your code and try to see which part create this error.
Try this
set_time_limit(300);
ini_set('memory_limit', '20000M');
Try this:
#php_value memory_limit 300M
#php_value upload_max_filesize 200M
#php_value post_max_size 200M
#php_value max_execution_time 80000
#php_value max_input_time 80000
try ini_set('memory_limit', '-1');
I just scaffolded a new module and when I save using the generated form I get:
Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 19456 bytes) in /Applications/MAMP/htdocs/ats/lib/vendor/symfony/lib/plugins/sfDoctrinePlugin/lib/vendor/doctrine/Doctrine/Hydrator/Graph.php on line 404
Any ideas why?
symfony is quite greedy for memory, so this sometimes happens. You can increase the memory available to PHP/symfony via php.ini as suggested above:
php.ini:
memory_limit = 128M
128M here is just an example, but one which you might need to evaluate.
because that's the configured memory limit of your php. you may want to find what php.ini it uses from phpinfo() and edit it to give symfony more memory.
The memory_limit is the maximum amount of memory a PHP script is allowed to use. Basically, this is a security configuration option, to ensure you don't have a PHP script that does mad and consumes all the memory of the server -- or worse, that you don't have several PHP script that eat more memory than the server has.
This configuration directive can be set in the php.ini file ; it's the file that sets the configuration of PHP.
To find out where the php.ini file is on your server, you can use the phpinfo() function : somewhere near the top of the output, there should be a "Loaded Configuration File" option.
Which value should be used for memory_limit is an interesting question... In the past, when we were only writting small script, 8MB was generally enough.
Now, with Frameworks, bigger applications structured in layers, ORM, and all that, 8MB is generally not enough (as you obviously noticed) -- I generally set memory_limit to 32M on my production servers, which is almost always enough for my applications, without being too much.
So, in my php.ini file, I have :
memory_limit = 32M
Note : it would be tempting to put a very high value for memory_limit, to just get rid of the problem... But remember that memory_limit is here as a security : you should make sure your server has enough memory to answer several requests at the same time !