I have a Vagrant guest I'm using to run a Symfony 2 application locally for development. In general this is working fine, however, I am regularly finding the processes lock in the 'D+' state (waiting for I/O).
eg. I try to run my unit tests:
./bin/phpunit -c app
The task launches, but then never exits. In the process list I see:
vagrant 3279 0.5 4.9 378440 101132 pts/0 D+ 02:43 0:03 php ./bin/phpunit -c app
The task is unkillable. I need to power cycle the Vagrant guest to get it back again. This seems to happen mostly with PHP command line apps (but it's also the main command line tasks I do, so it might not be relevant).
The syslog reports a hung task:
Aug 20 03:04:40 precise64 kernel: [ 6240.210396] INFO: task php:3279 blocked for more than 120 seconds.
Aug 20 03:04:40 precise64 kernel: [ 6240.211920] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 20 03:04:40 precise64 kernel: [ 6240.212843] php D 0000000000000000 0 3279 3091 0x00000004
Aug 20 03:04:40 precise64 kernel: [ 6240.212846] ffff88007aa13c98 0000000000000082 ffff88007aa13c38 ffffffff810830df
Aug 20 03:04:40 precise64 kernel: [ 6240.212849] ffff88007aa13fd8 ffff88007aa13fd8 ffff88007aa13fd8 0000000000013780
Aug 20 03:04:40 precise64 kernel: [ 6240.212851] ffff88007aa9c4d0 ffff880079e596f0 ffff88007aa13c78 ffff88007fc14040
Aug 20 03:04:40 precise64 kernel: [ 6240.212853] Call Trace:
Aug 20 03:04:40 precise64 kernel: [ 6240.212859] [<ffffffff810830df>] ? queue_work+0x1f/0x30
Aug 20 03:04:40 precise64 kernel: [ 6240.212863] [<ffffffff811170e0>] ? __lock_page+0x70/0x70
Aug 20 03:04:40 precise64 kernel: [ 6240.212866] [<ffffffff8165a55f>] schedule+0x3f/0x60
Aug 20 03:04:40 precise64 kernel: [ 6240.212867] [<ffffffff8165a60f>] io_schedule+0x8f/0xd0
Aug 20 03:04:40 precise64 kernel: [ 6240.212869] [<ffffffff811170ee>] sleep_on_page+0xe/0x20
Aug 20 03:04:40 precise64 kernel: [ 6240.212871] [<ffffffff8165ae2f>] __wait_on_bit+0x5f/0x90
Aug 20 03:04:40 precise64 kernel: [ 6240.212873] [<ffffffff81117258>] wait_on_page_bit+0x78/0x80
Aug 20 03:04:40 precise64 kernel: [ 6240.212875] [<ffffffff8108af00>] ? autoremove_wake_function+0x40/0x40
Aug 20 03:04:40 precise64 kernel: [ 6240.212877] [<ffffffff8111736c>] filemap_fdatawait_range+0x10c/0x1a0
Aug 20 03:04:40 precise64 kernel: [ 6240.212882] [<ffffffff81122a01>] ? do_writepages+0x21/0x40
Aug 20 03:04:40 precise64 kernel: [ 6240.212884] [<ffffffff81118da8>] filemap_write_and_wait_range+0x68/0x80
Aug 20 03:04:40 precise64 kernel: [ 6240.212892] [<ffffffffa01269fe>] nfs_file_fsync+0x5e/0x130 [nfs]
Aug 20 03:04:40 precise64 kernel: [ 6240.212896] [<ffffffff811a632b>] vfs_fsync+0x2b/0x40
Aug 20 03:04:40 precise64 kernel: [ 6240.212900] [<ffffffffa01272c3>] nfs_file_flush+0x53/0x80 [nfs]
Aug 20 03:04:40 precise64 kernel: [ 6240.212903] [<ffffffff81175d6f>] filp_close+0x3f/0x90
Aug 20 03:04:40 precise64 kernel: [ 6240.212905] [<ffffffff81175e72>] sys_close+0xb2/0x120
Aug 20 03:04:40 precise64 kernel: [ 6240.212907] [<ffffffff81664a82>] system_call_fastpath+0x16/0x1b`
To provision the box, I'm sharing a local folder using:
config.vm.synced_folder "/my/local/path.dev", "/var/www", :nfs => true
Vagrant creates the following /etc/exports file on the OSX host:
# VAGRANT-BEGIN: c7d0c56a-a126-46f5-a293-605bf554bc9a
"/Users/djdrey-local/Sites/oddswop.dev" 192.168.33.101 -mapall=501:20
# VAGRANT-END: c7d0c56a-a126-46f5-a293-605bf554bc9a
Output of nfsstat on the vagrant guest
Server rpc stats:
calls badcalls badclnt badauth xdrcall
0 0 0 0 0
Client rpc stats:
calls retrans authrefrsh
87751 0 87751
Client nfs v3:
null getattr setattr lookup access readlink
0 0% 35018 39% 1110 1% 8756 9% 19086 21% 0 0%
read write create mkdir symlink mknod
5100 5% 7059 8% 4603 5% 192 0% 0 0% 0 0%
remove rmdir rename link readdir readdirplus
4962 5% 262 0% 313 0% 0 0% 0 0% 1056 1%
fsstat fsinfo pathconf commit
1 0% 2 0% 1 0% 229 0%
I've ensured the Guest Additions are up to date on the guest using the plugin: vagrant-vbguest
I'm not sure how to go about debugging this. It's pretty clear to me this is a NFS issue between the guest and the Mac OSX host. If I try and up the debug logging for NFS on OSX using NFS Manager, I get a kernel panic in OSX.
Has anyone else had a similar issue? Any suggestions on a way forward would be appreciated - as power cycling the guest several times per day is unworkable.
Environment
OSX 10.8.4
Vagrant 1.2.7
Virtualbox 4.2.16
Vagrant guest O/S: Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-23-generic x86_64) [precise64.box]
I had a similar problem when running npm install within a shared nfs folder and subsequently found that disabling nfs_udp fixed the hanging issues :
config.vm.synced_folder ".", "/vagrant", type: "nfs", nfs_udp: false
You don't give enough detail on the specific configuration (e.g., the exports file, the fstab file, firewall config, etc.) for a specific answer. Here are some ideas though:
In the fstab try adding the "hard,intr" flags to the mount options -- this makes it possible to kill processes waiting for I/O on a dead mount.
Also make sure your firewall is open for rpc calls and the rpc-statd service is running.
Also figure out what version of nfs you're running and that you have the correct TCP/UDP ports open. If NFS v4 isn't working out, maybe try NFS v3.
Finally, are you connecting via IP address or hostname? Hostname is great, but make sure it always resolves correctly -- maybe in your /etc/hosts file. Alternatively, hard-code the IP addresses so there is no chance of name resolution failing...
Related
I don't know what I am missing. I have a docker image, and a docker compose file.
My goal is to execute PHP Unit tests in Jenkins
I guess this
volumes:
- .:/var/www/html
is the source of the problems, but shouldn't the container last image have already the vendor directory?
Dockerfile
FROM quay/php-base:7.2.
COPY . /var/www/html
RUN curl -sS https://getcomposer.org/installer | php
RUN php composer.phar install
RUN rm composer.phar
EXPOSE 9000
CMD ["php-fpm"]
build.testing.yml
version: "3"
services:
phpfpm:
build:
context: .
dockerfile: Dockerfile
image: xxx-phpfpm:${GIT_COMMIT}
environment:
- APP_ENV=testing
volumes:
- .:/var/www/html
Jenkins build shell command
printenv
echo $GIT_BRANCH
echo $GIT_COMMIT
docker-compose -f build.testing.yml up -d --build --remove-orphans
docker-compose -f build.testing.yml exec -T phpfpm bash build/phpunit.sh
phpunit.sh
#!/usr/bin/env bash
cd /var/www/html;
ls -la;
ls -la vendor;
ls -la vendor/bin;
php vendor/bin/phpunit;
exit $?
Build log (last lines)
phpunit/php-code-coverage suggests installing ext-xdebug (^2.6.0)
phpunit/phpunit suggests installing ext-soap (*)
phpunit/phpunit suggests installing ext-xdebug (*)
phpunit/phpunit suggests installing phpunit/php-invoker (^2.0)
Generating optimized autoload files
Removing intermediate container 56245736b26b
---> 818e587ac18a
Step 6/8 : RUN rm composer.phar
---> Running in 5ccbffc37e05
Removing intermediate container 5ccbffc37e05
---> 086e88f369fc
Step 7/8 : EXPOSE 9000
---> Running in 1c10e32ba134
Removing intermediate container 1c10e32ba134
---> 84eb9cbc3477
Step 8/8 : CMD ["php-fpm"]
---> Running in b6381d9b0f9a
Removing intermediate container b6381d9b0f9a
---> 40b32fa81d5f
Successfully built 40b32fa81d5f
Successfully tagged xxx-phpfpm:aa01355d1e8ab6d0f3daf87fb8f1f3b1be3e45be
Recreating xxxmicroservicetesting_phpfpm_1 ...
Recreating xxxmicroservicetesting_phpfpm_1 ... done
+ docker-compose -f build.testing.yml exec -T phpfpm bash build/phpunit.sh
total 224
drwxrwxr-x 13 1002 1002 4096 Jul 29 19:35 .
drwxr-xr-x 1 root root 18 May 5 2018 ..
-rw-rw-r-- 1 1002 1002 213 Jul 26 15:38 .editorconfig
-rw-rw-r-- 1 1002 1002 388 Jul 26 15:38 .env.example
drwxrwxr-x 8 1002 1002 162 Jul 29 19:41 .git
-rw-rw-r-- 1 1002 1002 55 Jul 26 15:38 .gitignore
-rw-rw-r-- 1 1002 1002 71 Jul 26 15:38 .styleci.yml
-rw-rw-r-- 1 1002 1002 232 Jul 26 15:38 Dockerfile
-rw-rw-r-- 1 1002 1002 895 Jul 29 18:25 Jenkinsfile
drwxrwxr-x 16 1002 1002 240 Jul 26 15:38 app
-rw-rw-r-- 1 1002 1002 1094 Jul 26 15:38 artisan
drwxrwxr-x 2 1002 1002 21 Jul 26 15:38 bootstrap
drwxrwxr-x 2 1002 1002 24 Jul 29 19:41 build
-rw-rw-r-- 1 1002 1002 250 Jul 29 19:35 build.testing.yml
-rw-rw-r-- 1 1002 1002 1256 Jul 26 15:38 composer.json
-rw-rw-r-- 1 1002 1002 179335 Jul 26 15:38 composer.lock
drwxrwxr-x 2 1002 1002 45 Jul 26 15:38 config
drwxrwxr-x 5 1002 1002 54 Jul 26 15:38 database
-rw-rw-r-- 1 1002 1002 193 Jul 26 15:38 phpcs.xml
-rw-rw-r-- 1 1002 1002 924 Jul 26 15:38 phpunit.xml
drwxrwxr-x 2 1002 1002 40 Jul 26 15:38 public
drwxrwxr-x 3 1002 1002 19 Jul 26 15:38 resources
drwxrwxr-x 2 1002 1002 40 Jul 26 15:38 routes
drwxrwxr-x 5 1002 1002 46 Jul 26 15:38 storage
drwxrwxr-x 4 1002 1002 113 Jul 26 15:38 tests
ls: vendor: No such file or directory
ls: vendor/bin: No such file or directory
Could not open input file: vendor/bin/phpunit
Build step 'Execute shell' marked build as failure
Finished: FAILURE
When you have a volumes: directive in a docker-compose.yml like you show, it tells Docker to completely ignore the contents of the Docker image and use whatever's in the local filesystem. You should remove that.
If you're running this in Jenkins or another continuous integration (CI) system, this behavior would mean that the image that comes out of the build system hasn't actually been tested. You've run tests on whatever local source tree Jenkins has checked out, glued over the top of the built image. If the host system and the image have any sort of incompatibility around library versions or a vendored library tree, you can easily wind up in a situation where you're shipping a completely broken image, that happens to pass this artificial test environment.
In this context I'd suggest a sequence like this:
Check out your source tree.
Run any build steps and unit-test steps. Jenkins has a path to run these inside a Docker container, which can be more convenient than maintaining a Jenkins image with the language runtime you need, but this isn't "building and running the image". You can do this totally without Docker too.
Actually build the Docker image. I'd do this with the Jenkins Docker workflow plugin, or a manual sh "docker build" step.
Run integration tests against the built image. This could use a docker-compose.yml file like you show above, but without the build: or volumes: blocks. The test itself wouldn't be built into the image, but it would make external calls into the image's published API.
Push the built image to a registry. Trigger an automated deployment sequence, if that's appropriate.
I have long running worker that iterates over 5M records using batch processing. I use standard Laravel's function chunkById for this.
As long as i can see, i have not reached 200M of memory usage, which i can see in output of docker stats:
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
fd05760e5d96 case-place-partners_case-place-partners_app_1 21.71% 140.5MiB / 7.666GiB 1.79% 919MB / 103MB 113MB / 21.4MB 19
Additionally, i have memory_get_usage() and memory_get_usage(true) everywhere and i dont see numbers higher than 52428800.
Output of journalctl -k | grep -i -e memory -e oom:
Aug 19 09:28:41 mirokko-i3 kernel: Memory: 8023772K/8259584K available (12291K kernel code, 1319K rwdata, 3900K rodata, 1612K init, 3616K bss, 235812K reserved, 0K cma-reserved)
Aug 19 09:28:41 mirokko-i3 kernel: Freeing SMP alternatives memory: 32K
Aug 19 09:28:41 mirokko-i3 kernel: x86/mm: Memory block size: 128MB
Aug 19 09:28:41 mirokko-i3 kernel: Freeing initrd memory: 9024K
Aug 19 09:28:41 mirokko-i3 kernel: check: Scanning for low memory corruption every 60 seconds
Aug 19 09:28:41 mirokko-i3 kernel: Freeing unused decrypted memory: 2040K
Aug 19 09:28:41 mirokko-i3 kernel: Freeing unused kernel image memory: 1612K
Aug 19 09:28:41 mirokko-i3 kernel: Freeing unused kernel image memory: 2012K
Aug 19 09:28:41 mirokko-i3 kernel: Freeing unused kernel image memory: 196K
Aug 19 09:28:57 mirokko-i3 kernel: [TTM] Zone kernel: Available graphics memory: 4019344 KiB
Aug 19 09:28:57 mirokko-i3 kernel: [TTM] Zone dma32: Available graphics memory: 2097152 KiB
Output of docker inspect container_id located here
Seems like Laravel has job timeout. Just run worker with --timeout="0" to disable this feature or set your own value.
I am using docker images for running web-app on PHP and MongoDB. Docker container is running perfectly on my local machine (Windows10) but it is not updating on AWS Elastic Beanstalk. I have made a zipped file consist of run.sh, Dockerfile, and Dockerrun.aws.json which can be found from here.
On AWS Elastic Beanstalk I am using single docker container and instance type is Linux Docker Version 1.11.2 . after deploying I am getting following error:
It looks like your run.sh needs to be executable, based on this inspection of the files:
ls
total 16K
drwxr-xr-x 7 adam 238 Jan 11 11:04 .
drwxr-xr-x+ 139 adam 4.7K Jan 11 11:04 ..
drwxr-xr-x 12 adam 408 Jan 11 11:04 .git
-rw-r--r-- 1 adam 1.9K Jan 11 11:04 Dockerfile
-rw-r--r-- 1 adam 248 Jan 11 11:04 Dockerrun.aws.json
-rw-r--r-- 1 adam 544 Jan 11 11:04 index.php
-rw-r--r-- 1 adam 72 Jan 11 11:04 run.sh
It's possible that you have other issues, but running chmod +x run.sh before you recreate the zip file should fix that specific problem.
I had a problem on my main php server, wherein the main php5-fpm process would be killed by an HUP signal. After the main process would be killed it would fail to respawn. Since each child process is allowed only to server a certain number of requests, they would eventually die without spawning any other child process. This would cause the server to die and my users would receive a 502 response from the server. I was initially able to solve this issue by have a cron that would check the thread count of PHP processes and then restart if its less than 5.
Sep 14 11:41:41 ubuntu kernel: [ 3699.092724] init: php5-fpm main process (3592) killed by HUP signal
Sep 14 11:41:41 ubuntu kernel: [ 3699.092740] init: php5-fpm main process ended, respawning
Sep 14 11:41:42 ubuntu kernel: [ 3699.160940] init: php5-fpm main process (3611) terminated with status 78
Sep 14 11:41:42 ubuntu kernel: [ 3699.160954] init: php5-fpm main process ended, respawning
Sep 14 11:41:42 ubuntu kernel: [ 3699.216950] init: php5-fpm main process (3619) terminated with status 78
Sep 14 11:41:42 ubuntu kernel: [ 3699.216966] init: php5-fpm main process ended, respawning
Sep 14 11:41:42 ubuntu kernel: [ 3699.283573] init: php5-fpm main process (3627) terminated with status 78
Sep 14 11:41:42 ubuntu kernel: [ 3699.283590] init: php5-fpm main process ended, respawning
Sep 14 11:41:42 ubuntu kernel: [ 3699.337563] init: php5-fpm main process (3635) terminated with status 78
Sep 14 11:41:42 ubuntu kernel: [ 3699.337579] init: php5-fpm main process ended, respawning
Sep 14 11:41:42 ubuntu kernel: [ 3699.385293] init: php5-fpm main process (3643) terminated with status 78
Sep 14 11:41:42 ubuntu kernel: [ 3699.385305] init: php5-fpm main process ended, respawning
Sep 14 11:41:42 ubuntu kernel: [ 3699.430903] init: php5-fpm main process (3651) terminated with status 78
Sep 14 11:41:42 ubuntu kernel: [ 3699.430913] init: php5-fpm main process ended, respawning
Sep 14 11:41:42 ubuntu kernel: [ 3699.482790] init: php5-fpm main process (3659) terminated with status 78
Sep 14 11:41:42 ubuntu kernel: [ 3699.482800] init: php5-fpm main process ended, respawning
Sep 14 11:41:42 ubuntu kernel: [ 3699.532239] init: php5-fpm main process (3667) terminated with status 78
Sep 14 11:41:42 ubuntu kernel: [ 3699.532249] init: php5-fpm main process ended, respawning
Sep 14 11:41:42 ubuntu kernel: [ 3699.595810] init: php5-fpm main process (3675) terminated with status 78
Sep 14 11:41:42 ubuntu kernel: [ 3699.595825] init: php5-fpm main process ended, respawning
Sep 14 11:41:42 ubuntu kernel: [ 3699.648253] init: php5-fpm main process (3683) terminated with status 78
Sep 14 11:41:42 ubuntu0 kernel: [ 3699.648265] init: php5-fpm respawning too fast, stopped
My upstart script config
# php5-fpm - The PHP FastCGI Process Manager
description "The PHP FastCGI Process Manager"
author "Ondřej Surý <ondrej#debian.org>"
start on runlevel [2345]
stop on runlevel [016]
# Precise upstart does not support reload signal, and thus rejects the
# job. We'd rather start the daemon, instead of forcing users to
# reboot https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1272788
#
#reload signal USR2
pre-start exec /usr/lib/php5/php5-fpm-checkconf
respawn
exec /usr/sbin/php5-fpm --nodaemonize --fpm-config /etc/php5/fpm/php-fpm.conf
After searching the internet was finally able to get a solution to this by modifying the upstart script of php5-fpm in /etc/init/php5-fpm.conf
# php5-fpm - The PHP FastCGI Process Manager
description "The PHP FastCGI Process Manager"
author "Ondřej Surý <ondrej#debian.org>"
start on runlevel [2345]
stop on runlevel [016]
# Precise upstart does not support reload signal, and thus rejects the
# job. We'd rather start the daemon, instead of forcing users to
# reboot https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1272788
#
#reload signal USR2
pre-start exec /usr/lib/php5/php5-fpm-checkconf
pre-start exec /bin/bash /etc/init/php5-fpm.sh
post-start exec /bin/bash /etc/init/php5-fpm-onstart.sh
respawn
exec /usr/sbin/php5-fpm --nodaemonize --fpm-config /etc/php5/fpm/php-fpm.conf
So added additional scripts pre-start and post-start in the php5-fpm.conf. The pre-start script is
#!/bin/bash
rm /var/run/php5-fpm.pid
rm /var/run/php5-fpm.sock
CHILD_PIDS_FILE="/var/run/php5-fpm-child.pid"
CHILD_PIDS=`ps -ef | grep 'php' | grep -v grep |awk '{print $2}'`
echo "$CHILD_PIDS" > "$CHILD_PIDS_FILE"
The script basically deletes the main process pid and the sock file. Then writes the pids of the child processes to the file so than they can be killed once a new php5-fpm process is created.
The post-start script is
#!/bin/bash
CHILD_PIDS_FILE="/var/run/php5-fpm-child.pid"
while read PID; do
kill -9 $PID
done < $CHILD_PIDS_FILE
>$CHILD_PIDS_FILE
The post-start script deletes all the child-pids that were running before php5-fpm restarted.
I'm trying to install php-fpm for the first time on Ubunutu. I have installed it via
sudo apt-get install php5-fpm
When the install finishes, I run ps -waux | grep php5 to see if anything is running. I get the following:
root 9625 0.5 0.6 133612 12836 ? Ss 22:49 0:00 php-fpm: master process (/etc/php5/fpm/php-fpm.conf)
It looks like it is running, so I tried to stop it by executing:
sudo /etc/init.d/php5-fpm stop
This appears to do nothing. No error message is given on the command line and when I run ps again, the same process exists. I am able to get it to stop if I run:
sudo service php5-fpm stop
The first time I ran this command I got an error about not being able to find a directory, but the process did stop. When I start and stop the service again, it seems to work fine and I don't get an error message about a missing directory.
Can someone explain to me why the /etc/init.d/php5-fpm method does not work? All of the tutorials I've seen use this method to start and stop php5-fpm.
Quick answer...you don't use the init.d scripts to control (stop / start /status etc) of php-fpm because it's controlled by the newer 'upstart'. Use the following service command to stop and restart etc. I think it's a bit of a 'bug' that it quietly exits without stating that it's now controlled by upstart.
More detailed look at this problem, that leads to above explanation.
I see exactly the same problem on Ubuntu 14.04, you can stop / start / get status of php5-fpm using the service commands: I.E these work fine :-
# sudo service php5-fpm status
php5-fpm start/running, process 18793
# sudo service php5-fpm stop
php5-fpm stop/waiting
# sudo service php5-fpm status
php5-fpm stop/waiting
# sudo service php5-fpm start
php5-fpm start/running, process 18949
# sudo service php5-fpm status
php5-fpm start/running, process 18949
BUT the /etc/init.d/php5-fpm command has no effect :-
# /etc/init.d/php5-fpm stop
# ps -ef | grep php
root 18949 1 0 12:15 ? 00:00:00 php-fpm: master process (/etc/php5/fpm/php-fpm.conf)
www-data 18952 18949 0 12:15 ? 00:00:00 php-fpm: pool www
www-data 18953 18949 0 12:15 ? 00:00:00 php-fpm: pool www
root 18970 10051 0 12:17 pts/2 00:00:00 grep --color=auto php
Note nothing is returned, and the process is not stopped. Similarly for status and start versions, nothing is returned and the process is unaffected.
I found the line that it exits the init.d script on.... init_is_upstart:
# Don't run if we are running upstart
if init_is_upstart; then
exit 1
fi
Surely it should state an error message here...like 'Use service commands to stop/start php-fpm'.
I had similar problem that relates to php5-fpm on Ubuntu 14.10.
I have configured php5-fpm for one of the virtual hosts and when trying to browse the site i got the 503 Service Unavailable.
Every time I tried
root#testupgrade:~# service php5-fpm restart
stop: Unknown instance:
php5-fpm start/running, process 2775
I got in the syslog the following:
Dec 7 14:08:53 testupgrade kernel: [ 230.711612] init: php5-fpm main process (2775) terminated with status 78
Dec 7 14:08:53 testupgrade kernel: [ 230.711639] init: php5-fpm main process ended, respawning
Dec 7 14:08:53 testupgrade kernel: [ 230.866617] init: php5-fpm main process (2783) terminated with status 78
Dec 7 14:08:53 testupgrade kernel: [ 230.866643] init: php5-fpm main process ended, respawning
Dec 7 14:08:54 testupgrade kernel: [ 231.027522] init: php5-fpm main process (2791) terminated with status 78
Dec 7 14:08:54 testupgrade kernel: [ 231.027548] init: php5-fpm main process ended, respawning
Dec 7 14:08:54 testupgrade kernel: [ 231.137792] init: php5-fpm main process (2799) terminated with status 78
Dec 7 14:08:54 testupgrade kernel: [ 231.137807] init: php5-fpm main process ended, respawning
Dec 7 14:08:54 testupgrade kernel: [ 231.221146] init: php5-fpm main process (2807) terminated with status 78
Dec 7 14:08:54 testupgrade kernel: [ 231.221161] init: php5-fpm main process ended, respawning
Dec 7 14:08:54 testupgrade kernel: [ 231.301859] init: php5-fpm main process (2815) terminated with status 78
Dec 7 14:08:54 testupgrade kernel: [ 231.301874] init: php5-fpm main process ended, respawning
Dec 7 14:08:54 testupgrade kernel: [ 231.381635] init: php5-fpm main process (2823) terminated with status 78
Dec 7 14:08:54 testupgrade kernel: [ 231.381649] init: php5-fpm main process ended, respawning
Dec 7 14:08:54 testupgrade kernel: [ 231.469211] init: php5-fpm main process (2831) terminated with status 78
Dec 7 14:08:54 testupgrade kernel: [ 231.469225] init: php5-fpm main process ended, respawning
Dec 7 14:08:54 testupgrade kernel: [ 231.548950] init: php5-fpm main process (2839) terminated with status 78
Dec 7 14:08:54 testupgrade kernel: [ 231.548964] init: php5-fpm main process ended, respawning
Dec 7 14:08:54 testupgrade kernel: [ 231.628781] init: php5-fpm main process (2847) terminated with status 78
Dec 7 14:08:54 testupgrade kernel: [ 231.628795] init: php5-fpm main process ended, respawning
Dec 7 14:08:54 testupgrade kernel: [ 231.711933] init: php5-fpm main process (2855) terminated with status 78
Dec 7 14:08:54 testupgrade kernel: [ 231.711947] init: php5-fpm respawning too fast, stopped
What I tried then is to check if my php5-fpm configuration for this virtual host pool is actually working.
It was configured to listed on a TCP port with the following configuration in /etc/php5/fpm/pool.d/www.conf
I have correctly instructed the pool to listed as follows:
listen = 127.0.0.1:9000
I tried restarting few more times with the service command and since it did not work I decided checking the startup scripts.
I edited the /etc/init/php5-fpm.conf and I noticed that I already have the latest bug-fixed version of the line "reload signal USR2".
Then I noticed the following line:
pre-start exec /usr/lib/php5/php5-fpm-checkconf
I checked the /usr/lib/php5/php5-fpm-checkconf script and noticed the syntax check that is appended to the $errors variable:
/usr/sbin/php5-fpm --fpm-config /etc/php5/fpm/php-fpm.conf
I executed it in the shell and noticed the following:
root#testupgrade:~# /usr/sbin/php5-fpm --fpm-config /etc/php5/fpm/php-fpm.conf
[07-Dec-2014 13:46:14] ERROR: [pool www] 'slowlog' must be specified for use with 'request_slowlog_timeout'
[07-Dec-2014 13:46:14] ERROR: failed to post process the configuration
[07-Dec-2014 13:46:14] ERROR: FPM initialization failed
This means that I have touched my php-fpm.conf and messed it up however the error was nowhere to be found and I had no indication that this is the reason for the php5-fpm service to fail starting.
I fixed the syntax in the /etc/php5/fpm/pool.d/www.conf file and then tried restarting again with the service command.
The service started and opened a tcp listener on the required port 9000.
The vhost also started working.
So always check the php-fpm syntax and configuration with the above command to avoid issues with the service not starting.
Typically an init script will not always spit out the problem causing the situation to happen. Your best option is to cat /etc/init.d/php5-fpm and find the command it is trying to run and then use that command to start the process manually.
Speaking from experience with FPM start-up scripts, FPM will normally spit out the error you cannot see or are missing from random log files and then you'll be able to find and fix that error. Once resolved, your init script should be good to go afterwards.