I have done some investigations and I have found this to calculate and adjust pm.max_children value
https://myshell.co.uk/blog/2012/07/adjusting-child-processes-for-php-fpm-nginx/
but for example :
I have 8Gb on my server
I'm hosting 30 websites
php-fpm average process size around 40mb
php-fpm max process size around 80mb
I want to allocate 5Gb max of my memory to php-fpm processes
If I apply this :
pm.max_children = Total RAM dedicated to the web server / Max child process size
So in my case :
pm.max_children = 5120 / 80 = 64
But If I add pm.max_children = 64 on each php-fpm website conf files, this means every website can use 64 children process X size of 1 process (ex 40mb) = 2560Mb
And If we imagine, at the same time, all the 30 website have reach the pm.max_children value, we will have : 2560Mb (max per website) x 30 websites = 76 800 Mb
Am I correct?
Si yes, this means when many websites are hosted on the same server, we have to divide the result of the calcul pm.max_children = 5120 / 80 = 64 by the number of websites hosted (here 30).
So 64 / 30 = 2,1 and pm.max_children = 2 per website
Is it correct or not?
Thanks
Related
I am setting up a website. I have PHP code that will run for long periods of time. When the code runs, it takes 1-2 hours to finish the process.
This script works great up to 6, and interestingly the 7th link is "pending" until the previous one is finished.
CPU & Ram and Disk stable.
I tried the following;
Change PHP limits:
pm = dynamic
pm.max_children = 200
pm.start_servers = 1
pm.min_spare_servers = 1
pm.max_spare_servers = 200
Change Apache Limits (I wrote these numbers very high on purpose, but it didn't work.):
FcgidConnectTimeout 5
FcgidMaxProcesses 1000000
FcgidMaxProcessesPerClass 1000000
FcgidMaxRequestInMem 65536000000
FcgidMaxRequestLen 13107200000
PHP Version: 7.2
In chrome console this php showing "pending", apache error log and journal error log is empty, just white screen and wait until the other is finished. What else should I try?
I moved me shop to new server and I have problem with loading time. I have 500 products in one page (no pagination), first loading takes about 8 seconds (in old server was 2 sec maximum). Then website works very fast because cache plugin is working.
I have Prestashop 1.7.5.2 with very good cache plugin and two powerful servers:
Only with database: Apache, phpMyAdmin (RAM 60 GB, Processor 16 vCores, SSD)
Nginx, php-fpm 7.2 (RAM 60 GB, Processor 16 vCores, SSD)
Only page with products have this problem. I know, 500 products without pagination is not perfect idea but it have to be like that.
Maybe can be wrong php-fpm config?
Currently I have this:
pm = ondemand
pm.max_children = 16
pm.max_requests = 4000
pm.process_idle_timeout = 30s
I will be very grateful for your help.
I have a strange problem. On an amazon m4.large with nginx and php-fpm , that sits behind a loadbalancer and with a memcached and s3 bucket when the visitors reach certain number, the CPU of the instance hits almost 97% for a short period of time. I have enabled the slow logging functionality of php-fpm and found multiple occurances of this:
script_filename = /var/www/xxx/web/app.php
[0x00007f231c23c6c0] curl_multi_select() /var/www/xxx/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php:101
[0x00007f231c23c560] tick() /var/www/xxx/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php:125
[0x00007f231c23c448] execute() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:246
[0x00007f231c23c338] invokeWaitFn() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:223
[0x00007f231c23c208] waitIfPending() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:267
[0x00007f231c23c0d8] invokeWaitList() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:225
[0x00007f231c23bfa8] waitIfPending() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:267
[0x00007f231c23be78] invokeWaitList() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:225
[0x00007f231c23bd48] waitIfPending() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:62
[0x00007f231c23bc08] wait() /var/www/xxx/vendor/aws/aws-sdk-php/src/AwsClientTrait.php:58
[0x00007f231c23bac0] execute() /var/www/xxx/vendor/league/flysystem-aws-s3-v3/src/AwsS3Adapter.php:319
[0x00007f231c23b998] getMetadata() /var/www/xxx/vendor/league/flysystem-aws-s3-v3/src/AwsS3Adapter.php:342
[0x00007f231c23b8a0] getSize() /var/www/xxx/vendor/league/flysystem/src/Filesystem.php:329
[0x00007f231c23b720] getSize() /var/www/xxx/vendor/league/glide-symfony/src/Responses/SymfonyResponseFactory.php:38
What i am trying to do is, to fetch an image from a S3 bucket. Both the instances from where i am executing this code and the bucket sit in the same aws zone. So the latency shall be pretty low. I have configured the aws-sdk-php with the following params:
timeout = 5
connect_timeout = 1
delay = 0
PHP and PHP-FPM are in version 5.6.36
this is an outtake from the php-fpm configuration:
pm = dynamic
pm.max_children = 505
pm.start_servers = 15
pm.min_spare_servers = 15
pm.max_spare_servers = 135
pm.max_requests = 5000
The latency time of the loadbalancer peaks, when the visitors rise and the CPU load reaches ~97%. The RAM usage is pretty low though.
Does someone had a similar problem with curl_multi_selection() taking so long?
Suppose I have a php application that runs with php-fpm. Below are the options for php/php-fpm and opcache. How can I calculate how much memory is used by all of the components?
memory_limit=128M
...
opcache.memory_consumption=128M
...
[www]
pm = dynamic
pm.max_children = 2
pm.start_servers = 1
pm.min_spare_servers = 1
pm.max_spare_servers = 2
Here is my thinking. php-fpm is starting 1 process with 128 MB and opcache 128MB = 256 MB. If php-fpm creates a new server/process we will have 2 processes with 256 MB and opcache 128 MB = 384 MB. Is that correct?
The background is to calculator the correct memory limits for a Kubernetes deployment.
If you're talking about how much memory the app uses, you can use something like
ps aux | grep -P "fpm|COMMAND"
Once you have that, 10MB for e.g., you know how many MB your FPM thread is using. This is your child.
Next you need to know how much memory your server is using. Open htop, sort by mem (shift + m), take all your top mem usages above 1MB, which are not php related, add them up, thats your current server usage (non php).
If you can load your server when you check this that helps, particularly if your running mysql, that uses alot of memory, so helps if you load your server a bit there when you do your check.
If you find for e.g. your 8GB server uses only 512MB mem, you can reserve some extra space for it, say 750MB, or 1GB. that leaves the remaining 7GB to divide by your app mem usage. 10MB/7GB = 700 threads/childs.
I am running a drupal 7 website on a Linux server (with 4 cores and 12GB RAM) with LEMP (nginx+php5-fpm+mysql).
The drupal installation has a large number of modules enabled(all of which are needed).
I also use apc + boost + memcache + authcache for caching. The caching seems to be working (i see pages being served from cache)
and the site has a reasonable response time.
I have run stress tests with the website running in a url like www-1.example.com. (www-1.example.com points to the ip of my webserver, let's say x.x.x.x)
and the results are fine (for up to 100 concurrent users)
The problem starts when I change the dns so that www.example.com also points to x.x.x.x. Then the cpu of my webserver (all 4 cores) reach 100% at short time.
I have been expereimenting with the following parameters on the www.conf file with no luck:
Configuration 1:
pm.max_children = 100
pm.start_servers = 20
pm.min_spare_servers = 10
pm.max_spare_servers = 20
pm.max_requests = 200
result: 100% cpu usage, low memory usage
Configuration final:
pm.max_children = 300
pm.start_servers = 20
pm.min_spare_servers = 10
pm.max_spare_servers = 20
pm.max_requests = 200
result: low cpu usage, 100% memory usage
Can anyone guide me to find the optimal comfiguration or has any idea on what can cause the 100% cpu usage?
How do i culculate the maximum number of concurrent usages that can run without problem based on server parameters?