Need advise here. Currently I am get the memory usage of the script by using this code
$sysMem = escapeshellcmd(system('echo $(free)'));
memory_get_usage();
This is the result i am getting :
Total Mem : 1034708
Used Mem : 1014572
Free Mem : 20136
Shared Mem : 0
Buff Mem : 73456
Cached Mem : 480752
----------------------------------------------------
Mem Usegae : 509464
Total Mem : 1034708
Used Mem : 1014564
Free Mem : 20144
Shared Mem : 0
Buff Mem : 73456
Cached Mem : 480828
----------------------------------------------------
Mem Usegae : 343904
However i found out that the memory usage is kind of inconsistent and at time the memory usage might even exceed the total memory which is impossible.
Is memory_get_usage(); the best option to get the memory usage of the php script?
Or is it i need to use unset() function. However even if i use the memory still about the same
if there are other methods please kindly advise.
Thanks a millions
Guys and Ladies
Maybe this will help using pure PHP.
You can get the exact total memory using:
ini_get('memory_limit');
But this will be something like '128M'.
To convert this form into bytes look at Stack-Overflow-Convert String Mem-Bytes into bytes.
Now you have 2 methods for getting the script memory usage:
memory_get_peak_usage();
and
memory_get_usage();
With these ingedients the puzzle is easy (imho).
This is going to be difficult to state precisely. You might need to specify more carefully exactly what you are trying to do.
For one, calling out to the 'free' command is stepping off into the void of Linux's virtual memory scheme. You need to know how to interpret these meaningfully.
As far as I can tell, get_memory_usage could behave in a number of different ways depending on how PHP is running. It appears to be returning how much memory is allocated to the current PHP interpreter in total. This is unlikely to be indicative of how much memory the current script is using.
Also, free returns memory in kilobytes. PHP's get_memory_usage returns it in bytes. You are off by a factor of 1024. Divide the latter by 1024 to have comparable units.
So what are you trying to do?
Related
Suppose I have a php application that runs with php-fpm. Below are the options for php/php-fpm and opcache. How can I calculate how much memory is used by all of the components?
memory_limit=128M
...
opcache.memory_consumption=128M
...
[www]
pm = dynamic
pm.max_children = 2
pm.start_servers = 1
pm.min_spare_servers = 1
pm.max_spare_servers = 2
Here is my thinking. php-fpm is starting 1 process with 128 MB and opcache 128MB = 256 MB. If php-fpm creates a new server/process we will have 2 processes with 256 MB and opcache 128 MB = 384 MB. Is that correct?
The background is to calculator the correct memory limits for a Kubernetes deployment.
If you're talking about how much memory the app uses, you can use something like
ps aux | grep -P "fpm|COMMAND"
Once you have that, 10MB for e.g., you know how many MB your FPM thread is using. This is your child.
Next you need to know how much memory your server is using. Open htop, sort by mem (shift + m), take all your top mem usages above 1MB, which are not php related, add them up, thats your current server usage (non php).
If you can load your server when you check this that helps, particularly if your running mysql, that uses alot of memory, so helps if you load your server a bit there when you do your check.
If you find for e.g. your 8GB server uses only 512MB mem, you can reserve some extra space for it, say 750MB, or 1GB. that leaves the remaining 7GB to divide by your app mem usage. 10MB/7GB = 700 threads/childs.
PHP Warning: exec(): Unable to fork [rm some_file.txt] in some.php on
line 111
There is a question that has been asked before about this subject PHP Warning: exec() unable to fork I have similar problem but it not the same.
ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 31364
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 31364
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
My limits are shown above and it looks like there is nothing with low limit on server that can affect this error.
I tried to unsetting variables after using them both with unset and by setting them to null to free up memory. But it has no effect.
unset($var);
$var = null;
Unable to fork error occuring because of exhausting of some resources but I can't find the reason. Can you suggest me to which logs I should look?
Any ideas or workaround for this problem?
Any ideas or workaround for this problem?
The problem is likely a flaw in your code - like it was in https://stackoverflow.com/a/20649541/2038383. So the work around is fixing it.
Can you suggest to me in which logs I should look?
There is your PHP logs, then your system / kernel logs.
You already know where to get the PHP log and what is in it by default. Unfortunately your not going to get much more out of PHP. You could catch the error yourself with set_error_handler() but that won't give you any more useful info (it'll give you PHP's "errno" but not UNIX's errno).
As for system logs; I said in comments check your syslog. There might be something in there, and it's always a good starting point. But actually you won't generally see ulimit violations in syslog. Some will get logged (example stack size generates segfault which gets logged) but many won't. This post deals with how to get logs of ulimit violations: https://unix.stackexchange.com/questions/139011/how-do-i-configure-logging-for-ulimits. Surprisingly non trivial.
The way system ulimit violations are supposed to be reported by system call is by setting an errno. For example, if max user processes is hit fork() will return EAGAIN.
So ... you need to get at that UNIX errno to know what is really going on. Unfortunately I don't think there is a way in PHP (there is posix_errno(), but pretty sure that is limited to PHP's posix_XXX function library). Also note, it's PHP generating the "Unable to fork" message. How that maps to actual system call error is not completely transparent.
So best off looking at other ways to debug of which there are plenty. Like system monitoring tools: ps, dstat, strace might be a good start.
When running my PHP CLI script, my OS turns out of memory while memory_get_usage() returns 40MB memory use.
Here is a sample of what this script performs.
<?php
$aInstances = array();
$aInstances = myClass::getInstances(); // returns ~ 1500 objects in array()
while ( count($aInstances) ) {
$instance = array_pop($aInstances);
// do some stuff - memory usage increases
$instance->loadFromDb();
......
// free memory
unset($instance);
gc_collect_cycles(); // force garbage collector cleanup (memory usage grows up to 110MB otherwise)
}
// Display Php Memory Usage
echo number_format(memory_get_usage());
// returns 40,320,200
echo number_format(memory_get_peak_usage());
// returns 41,002,056
// Display Php System Memory Usage
echo memory_get_usage(true);
// returns 41,943,040
echo memory_get_peak_usage(true)
// returns 41,943,040
?>
This script seems to do his job, but when i try to see system memory usage by another way (like a ps aux on another shell), memory usage is more than 90%.
Script was killed by system one time because it was out of memory.
Some more information :
OS : Debian 6, 2GB memory
PHP version : 5.3.3-7
php.ini (cli) memory_limit = 128M
Do you have any ideas to explain and / or resolve my problem ?
Many thanks...
I have no idea why or how this came to be, but for some odd reason PHP scripts on my server, once they utilize ini_set trying to influence the memory_limit setting, cause the script to completely crash. No error messages, no nothing. If i call the script through the browser, all i get is a blank page.
Any hints on this?
Update:
running 'free' returns
total used free shared buffers cached
Mem: 8190820 7922056 268764 0 565124 6598656
-/+ buffers/cache: 758276 7432544
Swap: 2102456 0 2102456
Is something hogging my memory?
running ps aux |grep apache gives me 'ERROR: Unsupported option (BSD syntax)'
Checking manually i found a whole bunch of lines refering to:
/usr/sbin/apache2 -k start
All at about 0.3% memory usage and owned by 'www-data'.
The scary part is that none of the processes listed by 'ps aux' uses more than 0.8% of the memory. And if i add up all the percentages listed, i never arrive at where i should according to what 'free' is telling me.
I seem to remember there being a problem with requesting anything over 2GB. I think 2GB is the magic cut-off in at least some versions of PHP.
try with this code:
ini_set('memory_limit', '-1');
I need to run a PHP CLI script and give it a LOT of memory (it's an automatic documentation generator that needs to cover a large code base). I have a powerful machine and have allocated 5GB to php, both in php.ini and in an inline ini_set('memory_limit','5120M') declaration in the script file.
If I add these lines to top of the script:
phpinfo();
exit();
... it claims that it has a memory limit of 5120M, as I specified.
But the script still errors out, saying
Fatal error: Allowed memory size of 1073741824 bytes exhausted
... which is 1GB, not 5GB as I specified.
Is there any other place that the script might be looking to determine its memory limit? This is running in Fedora Linux.
(Yes, the ultimate solution may be to rewrite the script to be more efficient, but I didn't write it in the first place, so before I resort to that, I want to just throw resources at it and see if that works.)
The heap limit property is a size_t, which is 32 bits on a 32-bit machine. If this is in bytes, this would limit the memory limit to 4 GB. You may try running it on a 64 bit machine, with 64-bit PHP.
Edit: confirmed, heap->limit is a size_t (unsigned int) and is in bytes. A memory_limit of -1 sets the heap->limit to 4GB, and does not disable it as the documentation implies. Setting it to 5GB makes it wrap around to 1GB.