I have a strange problem. On an amazon m4.large with nginx and php-fpm , that sits behind a loadbalancer and with a memcached and s3 bucket when the visitors reach certain number, the CPU of the instance hits almost 97% for a short period of time. I have enabled the slow logging functionality of php-fpm and found multiple occurances of this:
script_filename = /var/www/xxx/web/app.php
[0x00007f231c23c6c0] curl_multi_select() /var/www/xxx/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php:101
[0x00007f231c23c560] tick() /var/www/xxx/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php:125
[0x00007f231c23c448] execute() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:246
[0x00007f231c23c338] invokeWaitFn() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:223
[0x00007f231c23c208] waitIfPending() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:267
[0x00007f231c23c0d8] invokeWaitList() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:225
[0x00007f231c23bfa8] waitIfPending() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:267
[0x00007f231c23be78] invokeWaitList() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:225
[0x00007f231c23bd48] waitIfPending() /var/www/xxx/vendor/guzzlehttp/promises/src/Promise.php:62
[0x00007f231c23bc08] wait() /var/www/xxx/vendor/aws/aws-sdk-php/src/AwsClientTrait.php:58
[0x00007f231c23bac0] execute() /var/www/xxx/vendor/league/flysystem-aws-s3-v3/src/AwsS3Adapter.php:319
[0x00007f231c23b998] getMetadata() /var/www/xxx/vendor/league/flysystem-aws-s3-v3/src/AwsS3Adapter.php:342
[0x00007f231c23b8a0] getSize() /var/www/xxx/vendor/league/flysystem/src/Filesystem.php:329
[0x00007f231c23b720] getSize() /var/www/xxx/vendor/league/glide-symfony/src/Responses/SymfonyResponseFactory.php:38
What i am trying to do is, to fetch an image from a S3 bucket. Both the instances from where i am executing this code and the bucket sit in the same aws zone. So the latency shall be pretty low. I have configured the aws-sdk-php with the following params:
timeout = 5
connect_timeout = 1
delay = 0
PHP and PHP-FPM are in version 5.6.36
this is an outtake from the php-fpm configuration:
pm = dynamic
pm.max_children = 505
pm.start_servers = 15
pm.min_spare_servers = 15
pm.max_spare_servers = 135
pm.max_requests = 5000
The latency time of the loadbalancer peaks, when the visitors rise and the CPU load reaches ~97%. The RAM usage is pretty low though.
Does someone had a similar problem with curl_multi_selection() taking so long?
Related
I am running magento2 on an nginx server that uses php-fpm. On refreshing the webpage multiple times on each reload being within milliseconds the memory increases until the system runs out of memory. A single user could go to the website and refresh the page multiple times and crash the website how can I prevent this from happening. What modifications can I make to NGINX or PHP-FPM to prevent this from happening.
Current PHP-FPM Configuraton:
pm = dynamic
pm.max_children = 50
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 35
I moved me shop to new server and I have problem with loading time. I have 500 products in one page (no pagination), first loading takes about 8 seconds (in old server was 2 sec maximum). Then website works very fast because cache plugin is working.
I have Prestashop 1.7.5.2 with very good cache plugin and two powerful servers:
Only with database: Apache, phpMyAdmin (RAM 60 GB, Processor 16 vCores, SSD)
Nginx, php-fpm 7.2 (RAM 60 GB, Processor 16 vCores, SSD)
Only page with products have this problem. I know, 500 products without pagination is not perfect idea but it have to be like that.
Maybe can be wrong php-fpm config?
Currently I have this:
pm = ondemand
pm.max_children = 16
pm.max_requests = 4000
pm.process_idle_timeout = 30s
I will be very grateful for your help.
Simple question: I'm getting Maximum execution time of 0 seconds exceeded, but I don't know why: there are a lot of things written about it, but in script I've no set_timeout or anything like that, just in PHP-FPM config I've php_value[max_execution_time] = 0 and even with this I'm gettting timeout from PHP-FPM's side (nginx is OK, no 504 Gateway Timeout or what's throwing). The thing is long running http request being killed in aprox. 3 mins. Setup is standard nginx + php-fpm (7.2) running in docker.
Thanks for any points!
I have done some investigations and I have found this to calculate and adjust pm.max_children value
https://myshell.co.uk/blog/2012/07/adjusting-child-processes-for-php-fpm-nginx/
but for example :
I have 8Gb on my server
I'm hosting 30 websites
php-fpm average process size around 40mb
php-fpm max process size around 80mb
I want to allocate 5Gb max of my memory to php-fpm processes
If I apply this :
pm.max_children = Total RAM dedicated to the web server / Max child process size
So in my case :
pm.max_children = 5120 / 80 = 64
But If I add pm.max_children = 64 on each php-fpm website conf files, this means every website can use 64 children process X size of 1 process (ex 40mb) = 2560Mb
And If we imagine, at the same time, all the 30 website have reach the pm.max_children value, we will have : 2560Mb (max per website) x 30 websites = 76 800 Mb
Am I correct?
Si yes, this means when many websites are hosted on the same server, we have to divide the result of the calcul pm.max_children = 5120 / 80 = 64 by the number of websites hosted (here 30).
So 64 / 30 = 2,1 and pm.max_children = 2 per website
Is it correct or not?
Thanks
I am running a drupal 7 website on a Linux server (with 4 cores and 12GB RAM) with LEMP (nginx+php5-fpm+mysql).
The drupal installation has a large number of modules enabled(all of which are needed).
I also use apc + boost + memcache + authcache for caching. The caching seems to be working (i see pages being served from cache)
and the site has a reasonable response time.
I have run stress tests with the website running in a url like www-1.example.com. (www-1.example.com points to the ip of my webserver, let's say x.x.x.x)
and the results are fine (for up to 100 concurrent users)
The problem starts when I change the dns so that www.example.com also points to x.x.x.x. Then the cpu of my webserver (all 4 cores) reach 100% at short time.
I have been expereimenting with the following parameters on the www.conf file with no luck:
Configuration 1:
pm.max_children = 100
pm.start_servers = 20
pm.min_spare_servers = 10
pm.max_spare_servers = 20
pm.max_requests = 200
result: 100% cpu usage, low memory usage
Configuration final:
pm.max_children = 300
pm.start_servers = 20
pm.min_spare_servers = 10
pm.max_spare_servers = 20
pm.max_requests = 200
result: low cpu usage, 100% memory usage
Can anyone guide me to find the optimal comfiguration or has any idea on what can cause the 100% cpu usage?
How do i culculate the maximum number of concurrent usages that can run without problem based on server parameters?