Well the child process may exit with error but pcntl_wifexited always return true
<?php
//Sample code :
$child_pid = pcntl_fork();
if ($child_pid === 0)
{
//This is child process
$my_var = rand(1,5);
if($my_var == 2)
{
//2 is not allowed
exit(1); //exit with error
}
if($my_var == 4)
{
//4 is unknown
i_crash_now(); //crash
}
echo 'end of child' . "\n";
exit(0); //exit standard
}
else
{
sleep(1); //waiting for child with ninja code (don't do that at home, this was made by professional ninjas)
pcntl_waitpid($child_pid, $status, WNOHANG);
var_dump(pcntl_wstopsig($status)); //in case 2 > 1, in case 4 > 255, else 0
var_dump(pcntl_wifexited($status)); //always true;
}
exit(0);
I may use the signal to find errors, but I don't get what's wrong with pcntl_wifexited().
Is this related to WNOHANG option?
I imagine the pcntl_waitpid behaves similarly to the normal waitpid call.
WNOHANG forces waitpid to return 0 if no process is terminated [it's akin to a zero timeout].
Hence, exit code will appear to be normal.
Related
I have php7 CLI daemon which serially parses json with filesize over 50M. I'm trying to save every 1000 entries of parsed data using a separate process with pcntl_fork() to mysql, and for ~200k rows it works fine.
Then I get pcntl_fork(): Error 35.
I assume this is happening because mysql insertion becomes slower than parsing, which causes more and more forks to be generated until CentOS 6.3 can't handle it any more.
Is there a way to catch this error to resort to single-process parsing and saving? Or is there a way to check child process count?
Here is the solution that I did based on #Sander Visser comment. Key part is checking existing processes and resorting to same process if there are too many of them
class serialJsonReader{
const MAX_CHILD_PROCESSES = 50;
private $child_processes=[]; //will store alive child PIDs
private function flushCachedDataToStore() {
//resort to single process
if (count($this->child_processes) > self::MAX_CHILD_PROCESSES) {
$this->checkChildProcesses();
$this->storeCollectedData() //main work here
}
//use as much as possible
else {
$pid = pcntl_fork();
if (!$pid) {
$this->storeCollectedData(); //main work here
exit();
}
elseif ($pid == -1) {
die('could not fork');
}
else {
$this->child_processes[] = $pid;
$this->checkChildProcesses();
}
}
}
private function checkChildProcesses() {
if (count($this->child_processes) > self::MAX_CHILD_PROCESSES) {
foreach ($this->child_processes as $key => $pid) {
$res = pcntl_waitpid($pid, $status, WNOHANG);
// If the process has already exited
if ($res == -1 || $res > 0) {
unset($this->child_processes[$key]);
}
}
}
}
}
I'm using PCNTL to multiprocess a big script in PHP on an ubuntu server.
Here is the code ( simplified and commented )
function signalHandler($signo = null) {
$pid = posix_getpid();
switch ($signo) {
case SIGTERM:
case SIGINT:
case SIGKILL:
// a process is asked to stop (from user or father)
exit(3);
break;
case SIGCHLD:
case SIGHUP:
// ignore signals
break;
case 10: // signal user 1
// a process finished its work
exit(0);
break;
case 12: // signal user 2
// a process got an error.
exit(3);
break;
default:
// nothing
}
}
public static function run($nbProcess, $nbTasks, $taskFunc, $args) {
$pid = 0;
// there will be $nbTasks tasks to do, and no more than $nbProcess children must work at the same time
$MAX_PROCESS = $nbProcess;
$pidFather = posix_getpid();
$data = array();
pcntl_signal(SIGTERM, "signalHandler");
pcntl_signal(SIGINT, "signalHandler");
// pcntl_signal(SIGKILL, "signalHandler"); // SIGKILL can't be overloaded
pcntl_signal(SIGCHLD, "signalHandler");
pcntl_signal(SIGHUP, "signalHandler");
pcntl_signal(10, "signalHandler"); // user signal 1
pcntl_signal(12, "signalHandler"); // user signal 2
for ($indexTask = 0; $indexTask < $nbTasks ; $indexTask++) {
$pid = pcntl_fork();
// Father and new child both read code from here
if ($pid == -1) {
// log error
return false;
} elseif ($pid > 0) {
// We are in father process
// storing child id in an array
$arrayPid[$pid] = $indexTask;
} else {
// We are in child, nothing to do now
}
if ($pid == 0) {
// We are in child process
$pidChild = posix_getpid();
try {
//$taskFunc is an array containing an object, and the method to call from that object
$ret = (array) call_user_func($taskFunc, $indexTask, $args);// similar to $ret = (array) $taskFunc($indexTask, $args);
$returnArray = array(
"tasknb" => $indexTask,
"time" => $timer,
"data" => $ret,
);
} catch(Exception $e) {
// some stuff to exit child
}
$pdata = array();
array_push($pdata, $returnArray);
$data_str = serialize($pdata);
$shm_id = shmop_open($pidChild, "c", 0644, strlen($data_str));
if (!$shm_id) {
// log error
} else {
if(shmop_write($shm_id, $data_str, 0) != strlen($data_str)) {
// log error
}
}
// We are in a child and job is done. Let's exit !
posix_kill($pidChild, 10); // sending user signal 1 (OK)
pcntl_signal_dispatch();
} else {
// we are in father process,
// we check number of running children
while (count($arrayPid) >= $MAX_PROCESS) {
// There are more children than allowed
// waiting for any child to send signal
$pid = pcntl_wait($status);
// A child sent a signal !
if ($pid == -1) {
// log error
}
if (pcntl_wifexited($status)) {
$statusChild = pcntl_wexitstatus($status);
} else
$statusChild = $status;
// father ($pidFather) saw a child ($pid) exiting with status $statusChild (or $status ?)
// ^^^^ ^^^^^^
// (=3) (= random number ?)
if(isset($arrayPid[$pid])) {
// father knows this child
unset($arrayPid[$pid]);
if ($statusChild == 0 || $statusChild == 10 || $statusChild == 255) {
// Child did not report any error
$shm_id = shmop_open($pid, "a", 0, 0);
if ($shm_id === false)
// log error
else {
$shm_data = unserialize(shmop_read($shm_id, 0, shmop_size($shm_id)));
shmop_delete($shm_id);
shmop_close($shm_id);
$data = array_merge($data, $shm_data);
}
// kill (again) child
posix_kill($pid, 10);
pcntl_signal_dispatch();;
}
else {
// Child reported an error
}
}
}
}
}
}
The problem I'm facing is about the value returned by wexitstatus.
To make it simple, there is a father-process, that must create 200 threads.
He makes process one at a time, and wait for a process to finish if there are more than 8 threads actually running.
I added many logs, so I see a child finished its work.
I see it calling the line posix_kill($pidChild, 10);.
I see the signal handler is called with signal user 1 (which results in an exit(0)).
I see the father awakening, but when he gets the returned code from wexitstatus, he sees a code 3, and so thinks the child got an error, whereas it has exited with code 0 !!.
The pid is the good child's pid.
Maybe I misunderstand how signals work... Any clue ?
I found the problem.
In my application, register_shutdown_function(myFrameworkShutdownFunction) was used to shut down the script "smoothly".
So an exit(0) didn't immediately stop the child process. It first went into myFrameworkShutdownFunction, and converted the return code 0 to a code 3 (because of a misconfigured variable).
Is there something in php that can halt or let the script proceed if things are ok?
In my current scripts I do like this (for this example):
$fetch = false;
if($i == 10){
echo 'I is clear'
$fetch = true;
}
if($fetch){
//Do database work here
}
echo 'This should be seen no matter what the above';
Instead of the $fetch in there, can I do something else? I don't want to stop the entire script after that like what die() or exit does.
Here's an example that should help you:
<?php
function doLoop() {
for($i=0;$i<100;$i++) {
if($i != 50) {
continue; //It's not 50, skip it
}
//Otherwise
printf("Loop: $i");
}
}
function doBreak() {
for($i=0;$i<100;$i++) {
if($i != 49) {
continue; //It's not 49 yet, continue
} //Otherwise, break
printf("Loop: $i");
break;
}
}
doLoop();
doBreak();
?>
break; can be used to end a loop when a condition is met, while continue; can also be used to skip a certain value if a condition is not met. Using die(); would stop your whole script from executing, preventing it to call anything that comes after the die(); statement because that's how the execution of the scripts pretty much go, from the top to the end.
* SOLUTION IN THE ANSWER BELOW *
I got a problem with limiting childs in the multifork php script... Seems like last child never ends... I'm really tired and can't find the error, could You please help? It does not end most of times...
<?php
declare(ticks = 1);
$max=5;
$child=0;
function sig_handler($signo) {
global $child;
switch ($signo) {
case SIGCHLD:
$child -= 1;
echo "[-]";
}
}
pcntl_signal(SIGCHLD, "sig_handler");
$found = array(1,2,3,4,5,6,7,8,9,10,11,12);
echo "LETS GO!\n";
foreach($found as $item){
while ($child >= $max) {
sleep(1);
}
$child++;
echo "[+]";
$pid=pcntl_fork();
if($pid){
}else{ // CHILD
sleep(rand(1,5));
echo "[~]";
exit(0);
}
}
while($child != 0){
echo "($child)";
sleep(1);
}
echo "THE END.\n"
?>
Result most times is:
[+][+][+][+][+][~][-][+][~][-][+][~][-][+][~][-][+][~][-][+][~][-][+][~][~][~][-][+] (5)[-](4)(4)[~][-](3)[~][-](2)(2)[~](2)[-](1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)... etc etc ...
Seems like last child does not end at all or at least it doesnt trigger sig handler...
[+] <- just before forking - count: 12
[~] <- just before child exit - count: 12
[-] <- sig handler after child exit - count: 11
Help?
PS. The weird thing is that it sometimes does end.
Ok found the solution:
function sig_handler($signo){
global $child;
switch ($signo) {
case SIGCLD:
while( ( $pid = pcntl_wait ( $signo, WNOHANG ) ) > 0 ){
$signal = pcntl_wexitstatus ($signo);
$child -= 1;
echo "[-]";
}
break;
}
}
The problem was in the way of handling signals on UNIX systems. All sigs sent in similar time are being grouped to one. The function above fixes that problem. And not only that one... when child exits immediately, it doesn't segfault ("zend_mm_heap corrupted" error in some PHP configurations/versions) anymore. And it did in previous case.
PS. I should delete this question, but I will leave the solution for everyone experiencing similar problems as multithreading in PHP scripts is very useful for some tasks.
The following PHP code is working as expected. I need to echo "copy error" message when the code is unable to execute the command in 4 attempts.
for($num_tries = 0 ; $num_tries < 4 ; $num_tries++)
{
$cloudcmd = "cp abc xyz ";
system($cloudcmd,$status);
if($status != 0)
{
sleep(3) ;
continue ;
}
break ;
}
I tried to add echo command after break; but it does not seem to work.
Strictly, you don't need to introduce additional variables and checks to do this.
Keeping it simple, you can inspect the value of the for loop variable $num_tries once outside the for loop:
if($num_tries==4)....
You also have the $status variable available after the for loop.
if($status!=0)....
$success = false;
for($num_tries = 0 ; $num_tries < 4 ; $num_tries++)
{
$cloudcmd = "cp abc xyz ";
system($cloudcmd,$status);
if($status != 0)
{
sleep(3) ;
continue ;
}
$success=true;
break ;
}
if($success)
{
//do your thing
}
Try doing this:
for ($num_tries=0;$num_tries < 4;$num_tries++)
{
system('cp abc xyz',$status);
if ($status === 0)
{
break;
}
sleep(3);
}
if ($status !== 0)
{
echo 'error';
}
This loop will break as soon as $status is 0, meaning the system() call was a success, after the loop, the $status value will be accessible, still (PHP doesn't have block scope), so check it's value and echo if needed. You can easily replace the echo statement using an exit(), to stop the rest of the script.
<?php
for($num_tries = 1 ; $num_tries <=4 ; $num_tries++){
$cloudcmd = "cp abc xyz ";
system($cloudcmd,$status);
if($status != 0) {
if($num_tries!=4){
sleep(3);
continue;
} else {
echo "error";
}
}
break;
}
?>
A break statement causes the loop to break, and any code that occurs after the break, within the loop is skipped over.
Perhaps, you could do something like this:
$success = false;
for($num_tries = 0 ; $num_tries < 4 ; $num_tries++) {
if (success()) {
// do something
$success = true;
}
}
echo ($success) ? 'Success' : 'Failure';