I'm getting 502 erros when I try to place an order in Magento, I also tried to disable Downloadable products and RSS feeds.
I receive the orde confirmation email, the stock is update and at the admin panel I can see the order correctly but I don't get the success message at checkout/onepage/success/ instead I get an 502 error caused by a timeout.
The only error message that I'm getting is os nginx log:
2013/12/21 18:49:47 [error] 18449#0: *191 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 177.103.69.141, server: dev.canalagricola.com.br, request: "GET /checkout/onepage/success/ HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "dev.canalagricola.com.br", referrer: "http://dev.canalagricola.com.br/checkout/onepage/"
I'm using Amazon Linux (64 bits) on a small instance (PHP5.5, PHPFPM-5.5, Nginx 1.4.3, Mysql 5.5, Magento 1.8.1)
magento.conf (nginx)
server {
listen 80;
server_name dev.canalagricola.com.br;
access_log off;
root /var/www/canalagricola;
index index.php;
gzip on;
gzip_min_length 10240;
gzip_comp_level 9;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml application/xml+rss;
gzip_disable "MSIE [1-6]\.";
sendfile on;
tcp_nodelay on;
tcp_nopush on;
keepalive_timeout 30;
keepalive_requests 100000;
reset_timedout_connection on;
client_body_timeout 10;
send_timeout 2;
location / {
try_files $uri $uri/ #handler;
expires 30d;
}
location ~* \.(jpg|jpeg|gif|png|css|js|ico|xml)$ {
log_not_found off;
expires 1y;
}
location ^~ /(app|includes|lib|media/downloadable|pkginfo|report/config.xml|var)/ { internal; }
location /var/export/ { internal; }
location /. { return 404; }
location #handler { rewrite / /index.php; }
location ~* .php/ { rewrite ^(.*.php)/ $1 last; }
location ~* .php$ {
if (!-e $request_filename) { rewrite / /index.php last; }
expires off;
fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param MAGE_RUN_CODE default;
fastcgi_param MAGE_RUN_TYPE store;
fastcgi_read_timeout 300;
include fastcgi_params;
}
rewrite ^/minify/([0-9]+)(/.*.(js|css))$ /lib/minify/m.php?f=$2&d=$1 last;
rewrite ^/skin/m/([0-9]+)(/.*.(js|css))$ /lib/minify/m.php?f=$2&d=$1 last;
location /lib/minify/ {
allow all;
}
}
www.conf (php-fpm)
[www]
listen = /var/run/php-fpm/php-fpm.sock
listen.allowed_clients = 127.0.0.1
listen.owner = nginx
listen.group = nginx
listen.mode = 0664
user = nginx
group = nginx
pm = dynamic
pm.max_children = 4
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
pm.max_requests = 200
request_terminate_timeout = 300s
rlimit_files = 131072
rlimit_core = unlimited
catch_workers_output = yes
php_admin_value[memory_limit] = 1024M
php_admin_value[max_execution_time] = 300000
php_admin_value[error_log] = /var/log/php-fpm/5.5/www-error.log
php_admin_flag[log_errors] = on
php_value[session.save_handler] = files
php_value[session.save_path] = /var/lib/php/5.5/session
apc needs to be disabled few hours, then you can plug it in. and read logs again.
there is eaccelerator as well, works sometimes even better.
disable full page cache and apc and run
ab -c 1 -n 1 http://www.campsaver.com/
then restart with apc
see speed, read logs and enable full page cache if good.
Okay,
The above nginx error indicate that issue is related to the fastcgi(php-fpm).
After enabling the "catch_workers_output" on /etc/php5/fpm/pool.d/www.conf Got the relevant error on php5-fpm.log. Now understand that issue is related to AMQP module.
[21-Aug-2013 23:32:57] WARNING: [pool www] child 16091 said into stderr: "php-fpm: pool www: symbol lookup error: /usr/lib/php5/20090626/amqp.so: undefined symbol: amqp_open_socket"
Issue has been fixed afterrecompiled and install amqp module
I notice you are using PHP 5.5
According to the requirements the recommended PHP version to use is 5.2.13 - 5.3.24
http://magento.com/resources/system-requirements
Install the recommended PHP version to see if you have any issue.
Restart your PHP-FPM:
/etc/init.d/php55-php-fpm restart
If this issue persists, you can increase buffer and timeouts inside HTTP block:
http {
...
fastcgi_buffers 8 16k;
fastcgi_buffer_size 32k;
fastcgi_connect_timeout 300;
fastcgi_send_timeout 300;
fastcgi_read_timeout 300;
...
}
Related
I've installed nginx on a fresh Ubuntu 18.04 server. Its for an osTicket installation if that matters at all. I put all the files in the /var/www/html directory and setup my sites-available/enabled. But when I go to the site I get an Access Denied message and the error log shows:
2019/04/07 13:50:24 [error] 17708#17708: *1 FastCGI sent in stderr:
"PHP message: PHP Warning: Unknown: failed to open stream: Permission
denied in Unknown on line 0
Unable to open primary script: /var/www/html/upload/index.php
(Permission denied)" while reading response header from upstream,
client: [my IP], server: server.domain.local, request: "GET / HTTP/1.1",
upstream: "fastcgi://unix:/run/php/php7.2-fpm.sock:", host: "server.domain.local"
Already made sure all directories from /var/www/html down have www-data:www-data and +x.
/etc/nginx/sites-available/osticket.conf
server {
listen 80;
server_name server.domain.local;
root /var/www/html/upload/;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
index index.php;
client_max_body_size 2000M;
client_body_buffer_size 100M;
client_header_buffer_size 10M;
large_client_header_buffers 2 10M;
client_body_timeout 12;
client_header_timeout 12;
keepalive_timeout 15;
send_timeout 10;
gzip on;
gzip_comp_level 2;
gzip_min_length 1000;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain application/x-javascript text/xml text/css application/xml;
set $path_info "";
location ~ /include {
deny all;
return 403;
}
if ($request_uri ~ "^/api(/[^\?]+)") {
set $path_info $1;
}
location ~ ^/api/(?:tickets|tasks).*$ {
try_files $uri $uri/ /api/http.php?$query_string;
}
if ($request_uri ~ "^/scp/.*\.php(/[^\?]+)") {
set $path_info $1;
}
location ~ ^/scp/ajax.php/.*$ {
try_files $uri $uri/ /scp/ajax.php?$query_string;
}
location / {
try_files $uri $uri/ index.php;
}
location ~ \.php$ {
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
include snippets/fastcgi-php.conf;
fastcgi_pass unix:/run/php/php7.2-fpm.sock;
fastcgi_param PATH_INFO $path_info;
}
}
User portion of /etc/php/7.2/fpm/pool.d/www.conf
; Unix user/group of processes
user = www-data
group = www-data
Well, I jumped straight to configs and other ideas. Forgot to KISS. For whatever reason, the files from the osTicket download came with 755 permissions on the folders, but absolutely no permissions on the files. Did a chmod 755 on everything and now its good.
I'm running a nginx 1.12 and a php-fpm 7.1 as seperate docker containers on a synology nas and i get a 504 Gateway error if the php-script runs longer than 60s. I've tried already several nginx configuration parameters but the error still exists.
Here is my actual nginx config:
#user www-data;
#group http
worker_processes 1;
error_log /opt/data/logs/nginx_error.log notice;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#keepalive_timeout 30s;
sendfile on;
#tcp_nopush off;
tcp_nodelay on;
#gzip off;
send_timeout 300
server {
listen 80;
server_name "";
root /opt/php;
index index.php;
location /data/ {
sendfile on;
root /opt;
}
location ~ \.php$ {
include fastcgi_params;
fastcgi_split_path_info ^(.+?\.php)(/.*)$;
if (!-f $document_root$fastcgi_script_name) {
return 404;
}
# Mitigate https://httpoxy.org/ vulnerabilities
fastcgi_param HTTP_PROXY "";
fastcgi_pass php:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_read_timeout 300;
#fastcgi_buffering off;
#fastcgi_keep_conn on;
#fastcgi_intercept_errors on;
#fastcgi_cache off;
#fastcgi_ignore_client_abort on;
}
location ~ ^/(status|ping)$ {
access_log off;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_pass php:9000;
}
}
}
The php-testscript:
<?php
sleep(65);
echo "done!";
file_put_contents("/opt/data/timetest.txt", "\nEnd", FILE_APPEND);
After 60s the browser shows up the 504 Gateway Time-out. The php-script is still running and is also writing the text to the file.
Nginx errorlog:
2017/07/22 08:16:32 [error] 8#8: *10 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.17.0.1, server: , request: "GET /timetest.php HTTP/1.1", upstream: "fastcgi://172.17.0.3:9000", host: "192.168.0.100:8081"
Has anyone an idea?
The question is probably why does your backend take so long to respond? Not sure about your usecase but normally it's not user-friendly to wait to long for a response.
To answer your question:
I found this link: https://easyengine.io/tutorials/php/increase-script-execution-time/
Add in /etc/php5/fpm/php.ini
max_execution_time = 300
Set in /etc/php5/fpm/pool.d/www.conf
request_terminate_timeout = 300
Set in /etc/nginx/nginx.conf
http {
#...
fastcgi_read_timeout 300;
#...
}
And in your config:
location ~ \.php$ {
include /etc/nginx/fastcgi_params;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_read_timeout 300;
}
And reload services
service php5-fpm reload
service nginx reload
I'm running a nginx 1.12 and a php-fpm 7.1 as seperate docker containers on a synology nas and i get a 504 Gateway error if the php-script runs longer than 60s. I've tried already several nginx configuration parameters but the error still exists.
Here is my actual nginx config:
#user www-data;
#group http
worker_processes 1;
error_log /opt/data/logs/nginx_error.log notice;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#keepalive_timeout 30s;
sendfile on;
#tcp_nopush off;
tcp_nodelay on;
#gzip off;
send_timeout 300
server {
listen 80;
server_name "";
root /opt/php;
index index.php;
location /data/ {
sendfile on;
root /opt;
}
location ~ \.php$ {
include fastcgi_params;
fastcgi_split_path_info ^(.+?\.php)(/.*)$;
if (!-f $document_root$fastcgi_script_name) {
return 404;
}
# Mitigate https://httpoxy.org/ vulnerabilities
fastcgi_param HTTP_PROXY "";
fastcgi_pass php:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_read_timeout 300;
#fastcgi_buffering off;
#fastcgi_keep_conn on;
#fastcgi_intercept_errors on;
#fastcgi_cache off;
#fastcgi_ignore_client_abort on;
}
location ~ ^/(status|ping)$ {
access_log off;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_pass php:9000;
}
}
}
The php-testscript:
<?php
sleep(65);
echo "done!";
file_put_contents("/opt/data/timetest.txt", "\nEnd", FILE_APPEND);
After 60s the browser shows up the 504 Gateway Time-out. The php-script is still running and is also writing the text to the file.
Nginx errorlog:
2017/07/22 08:16:32 [error] 8#8: *10 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.17.0.1, server: , request: "GET /timetest.php HTTP/1.1", upstream: "fastcgi://172.17.0.3:9000", host: "192.168.0.100:8081"
Has anyone an idea?
The question is probably why does your backend take so long to respond? Not sure about your usecase but normally it's not user-friendly to wait to long for a response.
To answer your question:
I found this link: https://easyengine.io/tutorials/php/increase-script-execution-time/
Add in /etc/php5/fpm/php.ini
max_execution_time = 300
Set in /etc/php5/fpm/pool.d/www.conf
request_terminate_timeout = 300
Set in /etc/nginx/nginx.conf
http {
#...
fastcgi_read_timeout 300;
#...
}
And in your config:
location ~ \.php$ {
include /etc/nginx/fastcgi_params;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_read_timeout 300;
}
And reload services
service php5-fpm reload
service nginx reload
Running shopware 5 on a Debian Jessie machine with nginx and php5-fpm, we get very often a 502 Bad Gateway. This happens mostly in backend when longer operations are working like thumbnail creation, even if this is done within small chunks of single ajax requests.
The used server with 64 GB RAM and 16 Cores is sleeping at all, because there is no real traffic on it. We use it like a staging system currently unless we have fixed all errors like this one.
Error log:
In the nginx-error log the following lines can be found then:
[error] 20524#0: *175 connect() failed (111: Connection refused) while connecting to upstream, client: xx.xx.xx.xx, server: domain.com, request: "POST /backend/MediaManager/createThumbnails HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "www.domain.com", referrer: "http://www.domain.com/backend/"
[error] 20524#0: *175 no live upstreams while connecting to upstream, client: xx.xx.xx.xx, server: domain.com, request: "POST /backend/Log/createLog HTTP/1.1", upstream: "fastcgi://php-fpm", host: "www.domain.com", referrer: "http://www.domain.com/backend/"
[error] 20524#0: *175 connect() failed (111: Connection refused) while connecting to upstream, client: xx.xx.xx.xx, server: domain.com, request: "GET /backend/login/getLoginStatus?_dc=1457014588680 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "www.domain.com", referrer: "http://www.domain.com/backend/"
[error] 20522#0: *209 connect() failed (111: Connection refused) while connecting to upstream, client: xx.xx.xx.xx, server: domain.com, request: "GET /backend/login/getLoginStatus?_dc=1457014618682 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "www.domain.com", referrer: "http://www.domain.com/backend/"
Maybe it is notable, that at first lot of "*175 connect" errors occure and then finally a "*209 connect".
Config files:
I'll try to post only significant lines related to this topic and will leave out all those lines which are commented out.
php-fpm:
/etc/php5-fpm/pool.d/www.conf:
[www]
user = www-data
group = www-data
listen = /var/run/php5-fpm.sock
listen.owner = www-data
listen.group = www-data
pm.max_children = 5
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
nginx:
/etc/nginx/nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
events {
worker_connections 768;
multi_accept on;
}
http {
## MIME types.
include /etc/nginx/mime.types;
default_type application/octet-stream;
## Default log and error files.
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
## Use sendfile() syscall to speed up I/O operations and speed up
## static file serving.
sendfile on;
## Handling of IPs in proxied and load balancing situations.
# set_real_ip_from 192.168.1.0/24; # set to your proxies ip or range
# real_ip_header X-Forwarded-For;
## Timeouts.
client_body_timeout 60;
client_header_timeout 60;
keepalive_timeout 10 10;
send_timeout 60;
## Reset lingering timed out connections. Deflect DDoS.
reset_timedout_connection on;
## Body size.
client_max_body_size 10m;
## TCP options.
tcp_nodelay on;
## Optimization of socket handling when using sendfile.
tcp_nopush on;
## Compression.
gzip on;
gzip_buffers 16 8k;
gzip_comp_level 1;
gzip_http_version 1.1;
gzip_min_length 10;
gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript image/x-icon application/vnd.ms-fontobject font/opentype application/x-font-ttf;
gzip_vary on;
gzip_proxied any; # Compression for all requests.
gzip_disable "msie6";
## Hide the Nginx version number.
server_tokens off;
## Upstream to abstract backend connection(s) for PHP.
upstream php-fpm {
server unix:/var/run/php5-fpm.sock;
# server 127.0.0.1:9000;
## Create a backend connection cache.
keepalive 32;
}
## Include additional configs
include /etc/nginx/conf.d/*.conf;
## Include all vhosts.
include /etc/nginx/sites-enabled/*;
}
/etc/nginx/sites-available/site.conf:
server {
listen 80;
listen 443 ssl;
server_name xxxxxxxx.com;
root /var/www/shopware;
## Access and error logs.
access_log /var/log/nginx/xxxxxxxx.com.access.log;
error_log /var/log/nginx/xxxxxxxx.com.error.log;
## leaving out lots of shopware/mediafiles-related settings
## ....
## continue:
location ~ \.php$ {
try_files $uri $uri/ =404;
## NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
fastcgi_split_path_info ^(.+\.php)(/.+)$;
## required for upstream keepalive
# disabled due to failed connections
#fastcgi_keep_conn on;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param SHOPWARE_ENV $shopware_env if_not_empty;
fastcgi_param ENV $shopware_env if_not_empty; # BC for older SW versions
fastcgi_buffers 8 16k;
fastcgi_buffer_size 32k;
client_max_body_size 24M;
client_body_buffer_size 128k;
## upstream "php-fpm" must be configured in http context
fastcgi_pass php-fpm;
}
}
What to do now? Please let me now if i should provide further information to this question.
Update
After applying nginx- and fpm-settings from #peixotorms, the errors in nginx-logs changed to:
30 upstream timed out (110: Connection timed out) while reading response header from upstream
But the issue itself isn't solved. It has just another face...
It might sound strange to you, but your problem is most probably due to the fact that you're running PHP on a socket instead of a tcp port. You will start seeing 502 errors (and others) when you have around 300 concurrent requests (sometimes less) to php on a socket configuration.
Also your pm.max_children is way too low, unless you want to limit your server to around 5 simultaneous php requests maximum: http://php.net/manual/en/install.fpm.configuration.php
Configure it this way, and those errors should go away:
For your nginx.conf change the following values:
worker_processes 4;
worker_rlimit_nofile 750000;
# handles connection stuff
events {
worker_connections 50000;
multi_accept on;
use epoll;
}
upstream php-fpm {
keepalive 30;
server 127.0.0.1:9001;
}
Your /etc/php5-fpm/pool.d/www.conf
(Use these settings because you have plenty or RAM and CPU)
[www]
user = www-data
group = www-data
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
listen = 127.0.0.1:9001
listen.allowed_clients = 127.0.0.1
listen.backlog = 65000
pm = dynamic
pm.max_children = 1024
pm.start_servers = 8
pm.min_spare_servers = 4
pm.max_spare_servers = 16
pm.max_requests = 10000
Also add this on your location ~ \.php$ { block:
location ~ \.php$ {
try_files $uri $uri/ =404;
## NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
fastcgi_split_path_info ^(.+\.php)(/.+)$;
## required for upstream keepalive
# disabled due to failed connections
#fastcgi_keep_conn on;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param SHOPWARE_ENV $shopware_env if_not_empty;
fastcgi_param ENV $shopware_env if_not_empty; # BC for older SW versions
fastcgi_keep_conn on;
fastcgi_connect_timeout 20s;
fastcgi_send_timeout 60s;
fastcgi_read_timeout 60s;
fastcgi_pass php-fpm;
}
EDIT:
Change the values below on your /etc/php5/fpm/php.ini file to this and restart:
safe_mode = Off
output_buffering = Off
zlib.output_compression = Off
max_execution_time = 900
max_input_time = 900
memory_limit = 2048M
post_max_size = 120M
file_uploads = On
upload_max_filesize = 120M
Try binding to 0.0.0.0:9000:
listen = 0.0.0.0:9000
I've moved my server from apache2+fcgi to nginx+fpm because I wanted a lighter environment, and apache's memory footprint was high. The server is a dual core (I know, not very much) with 8G of ram. It runs also a rather busy FreeRadius server and related MySQL. CPU load average is ~1, with some obvious peaks.
One of those peaks happens every 30 minutes when I get web pings from some controlled devices. With Apache the server load was spiking up a lot, slowing down everything. Now with nginx the process is much faster (I also did some optimization in the code), tough now I miss some of these connections. I configured both nginx and fpm to what I believe should be enough, but I must be missing something because in these moments php isn't (apparently) able to reply to nginx. This is a recap of the config:
nginx/1.8.1
user www-data;
worker_processes auto;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
# multi_accept on;
}
client_body_buffer_size 10K;
client_header_buffer_size 1k;
client_max_body_size 20m;
large_client_header_buffers 2 1k;
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(.*)$;
set $fsn /$yii_bootstrap;
if (-f $document_root$fastcgi_script_name){
set $fsn $fastcgi_script_name;
}
fastcgi_pass 127.0.0.1:9011;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fsn;
fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_param PATH_TRANSLATED $document_root$fsn;
fastcgi_read_timeout 150s;
}
php5-fpm 5.4.45-1~dotdeb+6.1
[pool01]
listen = 127.0.0.1:9011
listen.allowed_clients = 127.0.0.1
pm = dynamic
pm.max_children = 150
pm.start_servers = 2
pm.min_spare_servers = 2
pm.max_spare_servers = 8
pm.max_requests = 2000
pm.process_idle_timeout = 10s
When the peak arrives I start seeing this in fpm logs:
[18-Feb-2016 11:30:04] WARNING: [pool pool01] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 c
hildren, there are 0 idle, and 13 total children
[18-Feb-2016 11:30:05] WARNING: [pool pool01] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16
children, there are 0 idle, and 15 total children
[18-Feb-2016 11:30:06] WARNING: [pool pool01] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32
children, there are 0 idle, and 17 total children
[18-Feb-2016 11:30:07] WARNING: [pool pool01] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32
children, there are 0 idle, and 19 total children
and worse in nginx's error.log
2016/02/18 11:30:22 [error] 23400#23400: *209920 connect() failed (110: Connection timed out) while connecting to upstream, client: 79.1.1.9,
server: host.domain.com, request: "GET /ping/?whoami=abc02 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9011", host: "host.domain.com"
2016/02/18 11:30:22 [error] 23400#23400: *209923 connect() failed (110: Connection timed out) while connecting to upstream, client: 1.1.9.71,
server: host.domain.com, request: "GET /utilz/pingme.php?whoami=abc01 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9011", host: "host.domain.com"
2016/02/18 11:30:22 [error] 23400#23400: *209925 connect() failed (110: Connection timed out) while connecting to upstream, client: 3.7.0.4,
server: host.domain.com, request: "GET /ping/?whoami=abc03 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9011", host: "host.domain.com"
2016/02/18 11:30:22 [error] 23400#23400: *209926 connect() failed (110: Connection timed out) while connecting to upstream, client: 1.7.2.1
, server: host.domain.com, request: "GET /ping/?whoami=abc04 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9011", host: "host.domain.com"
Those connections are lost!
First question, why nginx returns timeout within 22s (the pings are made at 00 and 30 minutes of every hour) if fastcgi_read_timeout is set to 150s?
Second question: why do I get so many fpm warnings? The total children displayed never reaches pm.max_children. I know warnings are not errors, but I get warned... Is there a relation between those messages and nginx's timeouts?
Given that the server handles perfectly fine the regular traffic, and it has no problem with ram and swap neither during these peak times (it always has ~1.5G or more free), is there a better tuning to handle those ping connections (which doesn't involve changing the schedule)? Should I raise pm.start_servers and/or pm.min_spare_servers?
You need some changes + I would recommend upgrading your php to 5.6.
Nginx tunning: /etc/nginx/nginx.conf
user www-data;
pid /var/run/nginx.pid;
error_log /var/log/nginx/error.log crit;
# NOTE: Max simultaneous requests = worker_processes*worker_connections/(keepalive_timeout*2)
worker_processes 1;
worker_rlimit_nofile 750000;
# handles connection stuff
events {
worker_connections 50000;
multi_accept on;
use epoll;
}
# http request stuff
http {
access_log off;
log_format main '$remote_addr $host $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $ssl_cipher $request_time';
types_hash_max_size 2048;
server_tokens off;
fastcgi_read_timeout 180;
keepalive_timeout 20;
keepalive_requests 1000;
reset_timedout_connection on;
client_body_timeout 20;
client_header_timeout 10;
send_timeout 10;
tcp_nodelay on;
tcp_nopush on;
sendfile on;
directio 100m;
client_max_body_size 100m;
server_names_hash_bucket_size 100;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# index default files
index index.html index.htm index.php;
# use hhvm with php-fpm as backup
upstream php {
keepalive 30;
server 127.0.0.1:9001; # php5-fpm (check your port)
}
# Virtual Host Configs
include /etc/nginx/sites-available/*;
}
For a default server, create and add to /etc/nginx/sites-available/default.conf
# default virtual host
server {
listen 80;
server_name localhost;
root /path/to/your/files;
access_log off;
log_not_found off;
# handle staic files first
location / {
index index.html index.htm index.php ;
}
# serve static content directly by nginx without logs
location ~* \.(jpg|jpeg|gif|png|bmp|css|js|ico|txt|pdf|swf|flv|mp4|mp3)$ {
access_log off;
log_not_found off;
expires 7d;
# Enable gzip for some static content only
gzip on;
gzip_comp_level 6;
gzip_vary on;
gzip_types text/plain text/css application/json application/x-javascript application/javascript text/javascript image/svg+xml application/vnd.ms-fontobject application/x-font-ttf font/opentype;
}
# no cache for xml files
location ~* \.(xml)$ {
access_log off;
log_not_found off;
expires 0s;
add_header Pragma no-cache;
add_header Cache-Control "no-cache, no-store, must-revalidate, post-check=0, pre-check=0";
gzip on;
gzip_comp_level 6;
gzip_vary on;
gzip_types text/plain text/xml application/xml application/rss+xml;
}
# run php only when needed
location ~ .php$ {
# basic php params
fastcgi_pass php;
fastcgi_index index.php;
fastcgi_keep_conn on;
fastcgi_connect_timeout 20s;
fastcgi_send_timeout 30s;
fastcgi_read_timeout 30s;
# fast cgi params
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
}
}
Idealy, you want php5-fpm to autorestart if it starts failing, therefore you can this to /etc/php5/fpm/php-fpm.conf
emergency_restart_threshold = 60
emergency_restart_interval = 1m
process_control_timeout = 10s
Change /etc/php5/fpm/pool.d/www.conf
[www]
user = www-data
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
listen = 127.0.0.1:9001
listen.allowed_clients = 127.0.0.1
listen.backlog = 65000
pm = dynamic
pm.start_servers = 8
pm.min_spare_servers = 4
pm.max_spare_servers = 16
; maxnumber of simultaneous requests that will be served (if each php page needs 32 Mb, then 128x32 = 4G RAM)
pm.max_children = 128
; We want to keep it hight (10k to 50k) to prevent server respawn, however if there are memory leak on PHP code we will have a problem.
pm.max_requests = 10000