pcntl_fork() results in defunct parent process - php

So, I have this PHP daemon worker that listens to IPC messages.
Weird thing is that the parent process (result from pcntl_fork) leaves a [php] < defunct> process untill the child process is done but ONLY when the script is started form a cronjob, not directly from command line.
I know < defunct> processes aren't evil, but I can't figure out why it's happening only when running from a cronjob.
Command
/path/to/php/binary/php /path/to/php/file/IpcServer.php
Forking code:
$iParent = posix_getpid();
$iChild = pcntl_fork();
if ($iChild == -1)
throw new Exception("Unable to fork into child process.");
elseif ($iChild)
{
echo "Forking into background [{$iChild}].\n";
Log::d('Killing parent process (' . $iParent. ').');
exit;
}
Output
Forking into background [20835].
Killing parent process (20834).
ps aux | grep php
root 20834 0.0 0.0 0 0 ? Zs 14:28 0:00 [php] <defunct>
root 20835 0.0 0.2 275620 8064 ? Ss 15:35 0:00 /path/to/php/binary/php /path/to/php/file/IpcServer.php
I've found out that it's a known apache bug, but then why do I get this bug when running from cronjob?

In PHP the child will become a zombie process unless the parent waits for it to return with a pcntl_wait() or pcntl_waitpid(). The zombies will be destroyed once all processes have ended or are handled. It looks like the parent will become a zombie too if children aren't handled and a child runs longer than the parent.
Example from pcntl_fork page:
$pid = pcntl_fork();
if ($pid == -1) {
die('could not fork');
} else if ($pid) {
// we are the parent
pcntl_wait($status); //Protect against Zombie children
} else {
// we are the child
}
Or use signal handling like so to prevent waiting on the main thread:
$processes = array(); // List of running processes
$signal_queue = array(); // List of signals for main thread read
// register signal handler
pcntl_signal(SIGCHLD, 'childSignalHandler');
// fork. Can loop around this for lots of children too.
switch ($pid = pcntl_fork()) {
case -1: // FAILED
break;
case 0: // CHILD
break;
default: // PARENT
// ID the process. Auto Increment or whatever unique thing you want
$processes[$pid] = $someID;
if(isset($signal_queue[$pid])){
childSignalHandler(SIGCHLD, $pid, $signal_queue[$pid]);
unset($signal_queue[$pid]);
}
break;
}
function childSignalHandler($signo, $pid=null, $status=null){
global $processes, $signal_queue;
// If no pid is provided, Let's wait to figure out which child process ended
if(!$pid){
$pid = pcntl_waitpid(-1, $status, WNOHANG);
}
// Get all exited children
while($pid > 0){
if($pid && isset($processes[$pid])){
// I don't care about exit status right now.
// $exitCode = pcntl_wexitstatus($status);
// if($exitCode != 0){
// echo "$pid exited with status ".$exitCode."\n";
// }
// Process is finished, so remove it from the list.
unset($processes[$pid]);
}
else if($pid){
// Job finished before the parent process could record it as launched.
// Store it to handle when the parent process is ready
$signal_queue[$pid] = $status;
}
$pid = pcntl_waitpid(-1, $status, WNOHANG);
}
return true;
}

Related

Connection errors in forked PHP processes

I have a PHP script that takes N documents from MongoDB, forks the process into K child PHP processes, each process does some things with each document and tries to update document's info (see the code below).
On my local environment (Docker) everything is cool, but on the server (no Docker there) sometimes during the loop strange things happen...
Randomly all forked processes can not connect to MongoDB. The updateOne command returns an error :
"Failed to send "update" command with database "databasename": Invalid reply from server. in /vendor/mongodb/mongodb/src/Operation/Update.php on line 158".
This happens to all processes at the same time only for one (or several) random loop iterations. When each process goes to another iteration (takes the next document) -- everything is ok again. I make 5 tries to write to MongoDB.
Each try is with delay +1 sec to the previous, so the first try makes immediately, if any exception is caught -- wait a second and try again, the next try will be in 2 seconds and so on. But this does not help, all these 5 tries are broken.
This is not mongoDB problem, it's log is empty and it even don't receive anything from PHP, when error happens.
Also I have admitted, the more simultaneous processes I run -- the more frequent errors occur.
Also it is not server resource problem, when error occurs, half of RAM (4 gig) is free and CPU is working for the half of it's power.
Maybe PHP has some configuration for this? Some memory limits or something...
I use PHP v 7.1.30
MongoDB v 3.2.16
PHP Package mongodb/mongodb v 1.1.2
<?php
$processesAmount = 5;
$documents = $this->mongoResource->getDocuments();
for ($processNumber = 0; $processNumber < $processesAmount; $processNumber++) {
// create child process
$pid = pcntl_fork();
// do not create new processes in child processes
if ($pid === 0) {
break;
}
if ($pid === -1) {
// some errors catching staff here...
}
else if ($pid === 0) {
// create new MongoDB connection
}
else {
// Protect against Zombie children
// main process waits before all child processes end
for ($i = 0; $i < $processesAmount; $i++) {
pcntl_wait($status);
}
return null;
}
// spread documents to each process without duplicates
for ($i = $processNumber; $i < count($documents); $i += $processesAmount) {
$newDocumentData = $this->doSomeStaffWithDocument($documents[$i]);
$this->mongoResource->updateDocument($documents[$i], $newDocumentData);
}
}
There could be many issues here, one being that all processes are sharing 1 DB connection and the first to connect is then disconnecting and killing the connection for them all. Check the second example in the docs here: https://www.php.net/manual/en/ref.pcntl.php
If that doesn't help, the way I read your code the "spreading" part is happening in every process, when it should be happening once. Shouldn't you be putting the "work" in the child section like below?
$processesAmount = 5;
$documents = $this->mongoResource->getDocuments();
$numDocs = count($documents);
$i = 0;
$children = [];
for ($processNumber = 0; $processNumber < $processesAmount; $processNumber++) {
// create child
$pid = pcntl_fork();
if ($pid === -1) {
// some errors catching staff here...
} else if ($pid) {
//parent
$children[] = $pid;
} else {
//child
while (!empty($documents) && $i <= $numDocs) {
$i += $processNumber;
$doc = $documents[$i] ?? null;
unset($documents[$i]);
$newDocumentData = $this->doSomeStaffWithDocument($doc);
$this->mongoResource->updateDocument($doc, $newDocumentData);
}
}
}
//protect against zombies and wait for parent
//children is always empty unless in parent
while (!empty($children)) {
foreach ($children as $key => $pid) {
$status = null;
$res = pcntl_waitpid($pid, $status, WNOHANG);
if ($res == -1 || $res > 0) { //if the process has already exited
unset($children[$key]);
}
}
}
}

How is the main process running after return in PHP and Shell

I am new to daemons and shell script.
I have followed a tutorial to run a PHP file as a service.
Here is the code. I did chmod a+x on both the /etc/init.d/daemon and the /home/user/Work/Daemon.php files.
Here is the code of the bash file daemon. The problem that I am facing is that, when I do a sudo service daemon start, it just prints Starting Program Name: and does not close it(i.e. I have to do a ctrl+c to close it). When I check the log that the PHP file it printing, it does show that the PHP file was running when the command was given.
#!/bin/bash
#
# /etc/init.d/Daemon
#
# Starts the at daemon
#
# chkconfig: 345 95 5
# description: Runs the demonstration daemon.
# processname: Daemon
# Source function library.
. /etc/init.d/functions
#startup values
log=/var/log/Daemon.log
#verify that the executable exists
test -x /home/user/Work/Daemon.php || exit 0RETVAL=0
#
# Set prog, proc and bin variables.
#
prog="Program Name"
proc=/var/lock/subsys/Daemon
bin=/home/user/Work/Daemon.php
start() {
# Check if Daemon is already running
if [ ! -f $proc ]; then
echo -n $"Starting $prog: "
daemon $bin --log=$log
RETVAL=$?
[ $RETVAL -eq 0 ] && touch $proc
echo
fi
return $RETVAL
}
stop() {
echo -n $"Stopping $prog: "
killproc $bin
RETVAL=$?
[ $RETVAL -eq 0 ] && rm -f $proc
echo
return $RETVAL
}
restart() {
stop
start
}
reload() {
restart
}
status_at() {
status $bin
}
case "$1" in
start)
start
;;
stop)
stop
;;
reload|restart)
restart
;;
condrestart)
if [ -f $proc ]; then
restart
fi
;;
status)
status_at
;;
*)
echo $"Usage: $0 {start|stop|restart|condrestart|status}"
exit 1
esac
exit $?
exit $RETVAL
And here is the code of Daemon.php
#!/usr/bin/php
<?php
while(true){
file_put_contents('/var/log/Daemon.log', 'Running...', FILE_APPEND);
sleep(1);
}//end while
?>
UPDATE
I changed the PHP file to the following code, and it stared working. The daemon was expecting a return. But I don't understand why it goes to the main process block in PHP after the return. Could someone please explain it?
New Code.
#!/usr/bin/php
<?php
$log = '/var/log/vb_q.log';
//fork the process to work in a daemonized environment
file_put_contents($log, "Status: starting up.n", FILE_APPEND);
$pid = pcntl_fork();
if($pid == -1){
file_put_contents($log, "Error: could not daemonize process.n", FILE_APPEND);
return 1; //error
}
else if($pid){
return 0; //success
}
else{
//the main process
while(true){
file_put_contents($log, 'Running...', FILE_APPEND);
sleep(1);
}//end while
}//end if
?>
But I don't understand why it goes to the main process block in PHP
after the return. Could someone please explain it?
It does not go to the main process block after the return. The $pid = pcntl_fork() creates a child process; now there are two processes executing this same code:
the parent, where $pid is the ID of the child process, executes the block
else if($pid){
return 0; //success
}
and after the return the Daemon.php program terminates (which causes your shell script's start() function to resume)
the child, where $pid is 0, executes the block you call main process
else{
//the main process
while(true){
file_put_contents($log, 'Running...', FILE_APPEND);
sleep(1);
}//end while
}//end if
where it did not go after a return, but just thru the if … else.
It sounds like your bash script is running the php process alright, but it is running in the foreground and the bash script is waiting for it to end before it continues. Since the php script runs forever, try adding an ampersand after the command like this: daemon $bin --log=$log & to make your php process run in the background.
Try adding a running user to your daemon so that it runs with appropriate privileges instead of root.
Additionally move your touch $proc above the daemon start. To ensure it is able to create the lock file prior to starting the daemon.
The stop command looks identical to mine.
eg:
start(){
if [ ! -f $proc ]; then
echo -n "Starting $prog: "
touch $proc
daemon --user=apache --log=$log $bin
RETVAL=$?
[ $RETVAL -ne 0 ] && rm $proc
echo
else
echo -n "$prog is already running"
echo
fi
return $RETVAL
}
Also you can test your lock file to ensure you are able to access it.
. /etc/init.d/functions
#startup values
log=/var/log/Daemon.log
prog="Program Name"
proc=/var/lock/subsys/Daemon
bin=/home/user/Work/Daemon.php
if [ ! -w /var/lock/subsys ]; then
echo -n "You do not have permissions to run this service"
echo
exit 0
fi
if [ ! -x $bin ]; then
echo $bin is not executable!
echo
exit 0
fi
RETVAL=0
Update
PHP scripts are not meant to run as a normal service.
In order to control it you have to fork it for the service to continue running and allow the parent PHP process to close the script, rather than in the current thread that called it. See: http://man7.org/linux/man-pages/man3/daemon.3.html#RETURN_VALUE
For example launch a service - service never returned that it started or errored, since it is in a while loop.
You should additionally check for your child process and ensure it is assigned as a main process (session leader), after it has been forked as well. Otherwise the calling process remains the leader.
if ($pid == -1) {
return 1; //return error to service
} elseif ($pid) {
//parent process - break out of it
return 0;
} else {
/* child Process - run by calling pcntl_fork */
try {
if (posix_setsid() < 0) {
return; //error assigning a session id
}
$sid = posix_getpid(); //retrieve current process name
//... Your Code Here
} catch(\Exception $e) {
return 1;
}
}
See: http://linux.die.net/man/2/setsid

Ensure that there is only one running PHP process on Linux

I have a php script running each minute from cron. but sometimes it works longer than 1 minute.
My question is: what is the best way to ensure that only one process is running right now?
I use this code:
$output = shell_exec('ps aux | grep some_script.php | grep -v grep'); //get all processes containing "some_script.php" and exclude current grep process
$trimmed = rtrim($output, PHP_EOL); //trim newline symbol in the end
$processes = explode(PHP_EOL, $trimmed); //get the array of lines (i.e. processes)
$procCnt = count($processes); //get number of lines
if ($procCnt > 2) {
echo "busy\n";
exit(); //exit if number of processes more than 2 (see explaination below)
}
If there is one some_script process running from cron, shell_exec returns something like that:
apache 13593 0.0 0.0 9228 1068 ? Ss 18:20 0:00 /bin/bash -c php -f /srv/www/robot/some_script.php 2>&1
apache 13602 0.0 0.0 290640 10544 ? S 18:20 0:00 php -f /srv/www/robot/some_script.php
so if I have more than 2 lines in output, I call exit()
I want to ask: am I on a right way? Or is there a better way?
Any help would be appreciated
Checking that way creates a race condition where two processes can retrieve the list at the same time, then both decide to exit. Depending on what you're trying to do, this may or may not be a problem.
A possible better alternative is to create a lock of some sort. A simple one I've used is a directory that only exists while the process is running - mkdir is atomic, it will either succeed (no other process is running) or fail (another process has already created it). Just make sure to remove it when complete:
if (!mkdir("lock_dir")) {
echo "busy\n";
exit();
}
register_shutdown_function(function() {
rmdir("lock_dir");
});
Or better, it looks like flock was made for a similar purpose. This is the example from the manual:
<?php
$fp = fopen("/tmp/lock.txt", "r+");
if (flock($fp, LOCK_EX)) { // acquire an exclusive lock
ftruncate($fp, 0); // truncate file
fwrite($fp, "Write something here\n");
fflush($fp); // flush output before releasing the lock
flock($fp, LOCK_UN); // release the lock
} else {
echo "Couldn't get the lock!";
}
fclose($fp);
?>
Just hold the lock for the script's runtime, similar to my first example.
Simplest method is to create file while process start, and delete it at very end.
And check if file exist, then proceed and create it or die.
Another one is limit MaxClients for apache to one (if it is valid option in your case).
Just for OOP style, I am using separated Helper class.
flock — Portable advisory file locking
register_shutdown_function — Register a function for execution on shutdown
<?php
class ProcessHelper
{
const PIDFILE = 'yourProcessName.pid';
public static function isLocked()
{
$fp = fopen(self::PIDFILE, 'w');
if (!flock($fp, LOCK_EX | LOCK_NB, $wouldBlock)) {
if ($wouldBlock) {
// if this file locked by other process
var_dump('DO NOTHING');
return true;
}
} else {
var_dump('Do something and remove PID file');
register_shutdown_function(function () {
unlink(self::PIDFILE);
});
sleep(5); //for test, uncomment
}
return false;
}
}
class FooService
{
public function init()
{
if (!ProcessHelper::isLocked()) {
var_dump('count 1000\n');
for ($i = 0; $i < 10000; $i++) {
echo $i;
}
}
}
}
$foo = new FooService();
$foo->init();

PHP: how to start a detached process?

Currently my solution is:
exec('php file.php >/dev/null 2>&1 &');
and in file.php
if (posix_getpid() != posix_getsid(getmypid()))
posix_setsid();
is there any way I can do this just with exec?
No, detaching can't be done with exec() directly (nor shell_exec() or system(), at least not without changing the command, and using third-party tool which does detach).
If you have the pcntl extension installed, it would be:
function detached_exec($cmd) {
$pid = pcntl_fork();
switch($pid) {
// fork errror
case -1 : return false;
// this code runs in child process
case 0 :
// obtain a new process group
posix_setsid();
// exec the command
exec($cmd);
break;
// return the child pid in father
default:
return $pid;
}
}
Call it like this:
$pid = detached_exec($cmd);
if($pid === FALSE) {
echo 'exec failed';
}
// ... do some work ...
// Optionally, kill child process (if no longer required).
posix_kill($pid, SIGINT);
waitpid($pid, $status);
echo 'Child exited with ' . $status;
Provided your current user has sufficient permissions to do so this should be possible with exec and alike:
/*
/ Start your child (otherscript.php)
*/
function startMyScript() {
exec('nohup php otherscript.php > nohup.out & > /dev/null');
}
/*
/ Kill the script (otherscript.php)
/ NB: only kills one process at the time, otherwise simply expand to
/ loop over all complete exec() output rows
*/
function stopMyScript() {
exec('ps a | grep otherscript.php | grep -v grep', $otherProcessInfo);
$otherProcessInfo = array_filter(explode(' ', $otherProcessInfo[0]));
$otherProcessId = $otherProcessInfo[0];
exec("kill $otherProcessId");
}
// ensure child is killed when parent php script / process exits
register_shutdown_function('stopMyScript');
startMyScript();

How to run a PHP script asynchronously?

I am creating a PHP script that will be run via the command line. As part of this script, there are times where I might need to spawn/fork a different script that could take a long time to complete. I don't want to block the original script from completing. If I were doing this with JavaScript, I could run AJAX requests in the background. That is essentially what I am trying to do here. I don't need to know when the forks complete, just that they start and complete themselves.
How can I run these PHP scripts asynchronously?
foreach ($lotsOfItems as $item) {
if ($item->needsExtraHelp) {
//start some asynchronous process here, and pass it $item
}
}
$pids = array();
foreach ($lotsOfItems as $item) {
if ($item->needsExtraHelp) {
$pid = pcntl_fork();
if ($pid == 0) {
// you're in the child
var_dump($item);
exit(0); // don't forget this one!!
} else if ($pid == -1) {
// failed to fork process
} else {
// you're in the parent
$pids[] = $pid;
}
}
usleep(100); // prevent CPU from peaking
foreach ($pids as $pid) {
pcntl_waitpid($pid, $exitcode, WNOHANG); // prevents zombie processes
}
}
Looking the user contributed notes on exec, it looks like you could use it, check out:
http://de3.php.net/manual/en/function.exec.php#86329
<?php
function execInBackground($cmd) {
if (substr(php_uname(), 0, 7) == "Windows"){
pclose(popen("start /B ". $cmd, "r"));
}
else {
exec($cmd . " > /dev/null &");
}
}
?>
This will execute $cmd in the
background (no cmd window) without PHP
waiting for it to finish, on both
Windows and Unix.
int pcntl_fork ( void )
The pcntl_fork() function creates a child process that differs from the parent process only in its PID and PPID. Please see your system's fork(2) man page for specific details as to how fork works on your system.
details : http://php.net/manual/en/function.pcntl-fork.php
related question : PHP: What does pcntl_fork() really do?
Process control should not be enabled within a web server environment and unexpected results may happen if any Process Control functions are used within a web server environment.
details: http://www.php.net/manual/en/intro.pcntl.php

Categories