I'm having a weird problem in PHP with symfony 1.4
I have a task that launches multiple workers, and, sometimes, I need to stop all the workers (for example, after a deployment).
I launch the task using start-stop-daemon, and I want to stop it by sending the signal SIGINT to it.
So, here is my code:
protected function execute($arguments = array(), $options = array())
{
$pid_arr = array();
$thread = $this->forkChildren($arguments, $options, $options['nb_children']);
if ($this->iAmParent())
{
declare(ticks = 1);
pcntl_signal(SIGINT, array($this, 'signalHandler'));
// Retrieve list of children PIDs
$pid_arr = $this->getChildrenPids();
// While there are still children processes
while(count($pid_arr) > 0)
{
$myId = pcntl_waitpid(-1, $status);
foreach($pid_arr as $key => $pid)
{
// If the stopped process is indeed a children of the parent process
if ($myId == $pid)
{
$this->removeChildrenPid($key);
// Recreate a child
$this->createNewChildren($arguments, $options, 1, $pid_arr);
}
}
usleep(1000000);
$pid_arr = $this->getChildrenPids();
}
}
else
$thread->run();
}
public function signalHandler($signal)
{
echo "HANDLED SIGNAL $signal\n";
foreach ($this->getChildrenPids() as $childrenPid)
{
echo "KILLING $childrenPid\n";
posix_kill($childrenPid, $signal);
}
exit();
}
What I do is quite straightforward: I fork, create N children processes, and in the parent, I add a pcntl_signal to catch the SIGINT signal. The signalHanlder function retrieves the list of children pids and sends them the same signal it just received (so SIGINT).
The problem is that the signalHandler function is never called when I send a INT signal (via kill) to the parent process. And I don't understand why!
The weird thing is that when I launch my task in cli and uses Ctrl-C, the signalHandler function is called and all the children are stopped.
So, do you understand why is this happening? Am I doing something wrong?
OK, forget about it, I found the problem just after asking the question:
I just replaced
$myId = pcntl_waitpid(-1, $status);
by
$myId = pcntl_waitpid(-1, $status, WNOHANG);
because of course, the process was hung waiting for one of the children to die.
Related
Problem
The title really says it all. I had a Timeout class that handled timing out using pcntl_alarm(), and during ssh2_* calls, the signal handler simply does not fire when it should.
In its most basic form,
print date('H:i:s'); // A - start waiting, ideally 5 seconds at the most
pcntl_alarm(5);
// SSH attempt to download remote file /dev/random to ensure it blocks *hard*
// (omitted)
print date('H:i:s'); // B - get here once it realizes it's a tripeless-cat scenario
with a signal handler that also outputs the date, I would expect this (rows B and C might be inverted, that does not matter)
"A, at 00:00:00"
"C, at 00:00:05" <-- from the signal handler
"B, at 00:00:05"
but the above will obtain this disappointing result instead:
"A, at 00:00:00"
"C, at 00:01:29" <-- from the signal handler
"B, at 00:01:29"
In other words, the signal handler does fire, but it does so once another, longer, timeout has expired. I can only guess this timeout is inside ssh2_*. Quickly browsing through the source code yielded nothing obvious. What's worse, this ~90 seconds timeout is what I can reproduce by halting a download. In other cases when the firewall dropped the wrong SSH2 packet, I got a stuck process with an effectively infinite timeout.
As the title might reveal, I have already checked other questions and this is definitely not a mis-quoting of SIGALRM (also, the Timeout class worked beautifully). It seems like I need a louder alarm when libssh2 is involved.
Workaround
This modification of the Timeout yields a slightly less precise, but always working system, yet it does so in an awkward, clunky way. I'd really prefer for pcntl_alarm to work in all cases as it's supposed to.
public static function install() {
self::$enabled = function_exists('pcntl_async_signals') && function_exists('posix_kill') && function_exists('pcntl_wait');
if (self::$enabled) {
// Just to be on the safe side
pcntl_async_signals(true);
pcntl_signal(SIGALRM, static function(int $signal, $siginfo) {
// Child just died.
$status = null;
pcntl_wait($status);
static::$expired = true;
});
}
return self::$enabled;
}
public static function wait(int $time) {
if (self::$enabled) {
static::$expired = false;
// arrange for a SIGALRM to be delivered in $time seconds
// pcntl_alarm($time);
$ppid = posix_getpid();
$pid = pcntl_fork();
if ($pid == -1) {
throw new RuntimeException('Could not fork alarming child');
}
if ($pid) {
// save the child's PID
self::$thread = $pid;
return;
}
// we are the child. Send SIGALRM to the parent after requested timeout
sleep($time);
posix_kill($ppid, SIGALRM);
die();
}
}
/**
* Cancel the timeout and verify whether it expired.
**/
public static function expired(): bool {
if (self::$enabled) {
// Have we spawned an alarm?
if (self::$thread) {
// Yes, so kill it.
posix_kill(self::$thread, SIGTERM);
self::$thread = 0;
}
// Maybe.
$status = null;
pcntl_wait($status);
// pcntl_alarm(0);
}
return static::$expired;
}
Question
Can pcntl_alarm() be made to work as expected?
How should I multithread some php-cli code that needs a timeout?
I'm using PHP 5.6 on Centos 6.6 from the command line.
I'm not very familiar with multithreading terminology or code. I'll simplify the code here but it is 100% representative of what I want to do.
The non-threaded code currently looks something like this:
$datasets = MyLibrary::getAllRawDataFromDBasArrays();
foreach ($datasets as $dataset) {
MyLibrary::processRawDataAndStoreResultInDB($dataset);
}
exit; // just for clarity
I need to prefetch all my datasets, and each processRawDataAndStoreResultInDB() cannot fetch it's own dataset. Sometimes processRawDataAndStoreResultInDB() takes too long to process a dataset, so I want to limit the amount of time it has to process it.
So you can see that making it multithreaded would
Speed it up by allowing multiple processRawDataAndStoreResultInDB() to execute at the same time
Use set_time_limit() to limit the amount of time each one has to process each dataset
Notice that I don't need to come back to my main program. Since this is a simplification, you can trust that I don't want to collect all the processed datasets and do a single save into the DB after they are all done.
I'd like to do something like:
class MyWorkerThread extends SomeThreadType {
public function __construct($timeout, $dataset) {
$this->timeout = $timeout;
$this->dataset = $dataset;
}
public function run() {
set_time_limit($this->timeout);
MyLibrary::processRawDataAndStoreResultInDB($this->dataset);
}
}
$numberOfThreads = 4;
$pool = somePoolClass($numberOfThreads);
$pool->start();
$datasets = MyLibrary::getAllRawDataFromDBasArrays();
$timeoutForEachThread = 5; // seconds
foreach ($datasets as $dataset) {
$thread = new MyWorkerThread($timeoutForEachThread, $dataset);
$thread->addCallbackOnTerminated(function() {
if ($this->isTimeout()) {
MyLibrary::saveBadDatasetToDb($dataset);
}
}
$pool->addToQueue($thread);
}
$pool->waitUntilAllWorkersAreFinished();
exit; // for clarity
From my research online I've found the PHP extension pthreads which I can use with my thread-safe php CLI, or I could use the PCNTL extension or a wrapper library around it (say, Arara/Process)
https://github.com/krakjoe/pthreads (and the example directory)
https://github.com/Arara/Process (pcntl wrapper)
When I look at them and their examples though (especially the pthreads pool example) I get confused quickly by the terminology and which classes I should use to achieve the kind of multithreading I'm looking for.
I even wouldn't mind creating the pool class myself, if I had a isRunning(), isTerminated(), getTerminationStatus() and execute() function on a thread class, as it would be a simple queue.
Can someone with more experience please direct me to which library, classes and functions I should be using to map to my example above? Am I taking the wrong approach completely?
Thanks in advance.
Here comes an example using worker processes. I'm using the pcntl extension.
/**
* Spawns a worker process and returns it pid or -1
* if something goes wrong.
*
* #param callback function, closure or method to call
* #return integer
*/
function worker($callback) {
$pid = pcntl_fork();
if($pid === 0) {
// Child process
exit($callback());
} else {
// Main process or an error
return $pid;
}
}
$datasets = array(
array('test', '123'),
array('foo', 'bar')
);
$maxWorkers = 1;
$numWorkers = 0;
foreach($datasets as $dataset) {
$pid = worker(function () use ($dataset) {
// Do DB stuff here
var_dump($dataset);
return 0;
});
if($pid !== -1) {
$numWorkers++;
} else {
// Handle fork errors here
echo 'Failed to spawn worker';
}
// If $maxWorkers is reached we need to wait
// for at least one child to return
if($numWorkers === $maxWorkers) {
// $status is passed by reference
$pid = pcntl_wait($status);
echo "child process $pid returned $status\n";
$numWorkers--;
}
}
// (Non blocking) wait for the remaining childs
while(true) {
// $status is passed by reference
$pid = pcntl_wait($status, WNOHANG);
if(is_null($pid) || $pid === -1) {
break;
}
if($pid === 0) {
// Be patient ...
usleep(50000);
continue;
}
echo "child process $pid returned $status\n";
}
I have a function that needs to go over around 20K rows from an array, and apply an external script to each. This is a slow process, as PHP is waiting for the script to be executed before continuing with the next row.
In order to make this process faster I was thinking on running the function in different parts, at the same time. So, for example, rows 0 to 2000 as one function, 2001 to 4000 on another one, and so on. How can I do this in a neat way? I could make different cron jobs, one for each function with different params: myFunction(0, 2000), then another cron job with myFunction(2001, 4000), etc. but that doesn't seem too clean. What's a good way of doing this?
If you'd like to execute parallel tasks in PHP, I would consider using Gearman. Another approach would be to use pcntl_fork(), but I'd prefer actual workers when it's task based.
The only waiting time you suffer is between getting the data and processing the data. Processing the data is actually completely blocking anyway (you just simply have to wait for it). You will not likely gain any benefits past increasing the number of processes to the number of cores that you have. Basically I think this means the number of processes is small so scheduling the execution of 2-8 processes doesn't sound that hideous. If you are worried about not being able to process data while retrieving data, you could in theory get your data from the database in small blocks, and then distribute the processing load between a few processes, one for each core.
I think I align more with the forking child processes approach for actually running the processing threads. There is a brilliant demonstration in the comments on the pcntl_fork doc page showing an implementation of a job daemon class
http://php.net/manual/en/function.pcntl-fork.php
<?php
declare(ticks=1);
//A very basic job daemon that you can extend to your needs.
class JobDaemon{
public $maxProcesses = 25;
protected $jobsStarted = 0;
protected $currentJobs = array();
protected $signalQueue=array();
protected $parentPID;
public function __construct(){
echo "constructed \n";
$this->parentPID = getmypid();
pcntl_signal(SIGCHLD, array($this, "childSignalHandler"));
}
/**
* Run the Daemon
*/
public function run(){
echo "Running \n";
for($i=0; $i<10000; $i++){
$jobID = rand(0,10000000000000);
while(count($this->currentJobs) >= $this->maxProcesses){
echo "Maximum children allowed, waiting...\n";
sleep(1);
}
$launched = $this->launchJob($jobID);
}
//Wait for child processes to finish before exiting here
while(count($this->currentJobs)){
echo "Waiting for current jobs to finish... \n";
sleep(1);
}
}
/**
* Launch a job from the job queue
*/
protected function launchJob($jobID){
$pid = pcntl_fork();
if($pid == -1){
//Problem launching the job
error_log('Could not launch new job, exiting');
return false;
}
else if ($pid){
// Parent process
// Sometimes you can receive a signal to the childSignalHandler function before this code executes if
// the child script executes quickly enough!
//
$this->currentJobs[$pid] = $jobID;
// In the event that a signal for this pid was caught before we get here, it will be in our signalQueue array
// So let's go ahead and process it now as if we'd just received the signal
if(isset($this->signalQueue[$pid])){
echo "found $pid in the signal queue, processing it now \n";
$this->childSignalHandler(SIGCHLD, $pid, $this->signalQueue[$pid]);
unset($this->signalQueue[$pid]);
}
}
else{
//Forked child, do your deeds....
$exitStatus = 0; //Error code if you need to or whatever
echo "Doing something fun in pid ".getmypid()."\n";
exit($exitStatus);
}
return true;
}
public function childSignalHandler($signo, $pid=null, $status=null){
//If no pid is provided, that means we're getting the signal from the system. Let's figure out
//which child process ended
if(!$pid){
$pid = pcntl_waitpid(-1, $status, WNOHANG);
}
//Make sure we get all of the exited children
while($pid > 0){
if($pid && isset($this->currentJobs[$pid])){
$exitCode = pcntl_wexitstatus($status);
if($exitCode != 0){
echo "$pid exited with status ".$exitCode."\n";
}
unset($this->currentJobs[$pid]);
}
else if($pid){
//Oh no, our job has finished before this parent process could even note that it had been launched!
//Let's make note of it and handle it when the parent process is ready for it
echo "..... Adding $pid to the signal queue ..... \n";
$this->signalQueue[$pid] = $status;
}
$pid = pcntl_waitpid(-1, $status, WNOHANG);
}
return true;
}
}
you can use "PTHREADS"
very easy to install and works great on windows
download from here -> http://windows.php.net/downloads/pecl/releases/pthreads/2.0.4/
Extract the zip file and then
move the file 'php_pthreads.dll' to php\ext\ directory.
move the file 'pthreadVC2.dll' to php\ directory.
then add this line in your 'php.ini' file:
extension=php_pthreads.dll
save the file.
you just done :-)
now lets see example of how to use it:
class ChildThread extends Thread {
public $data;
public function run() {
/* Do some expensive work */
$this->data = 'result of expensive work';
}
}
$thread = new ChildThread();
if ($thread->start()) {
/*
* Do some expensive work, while already doing other
* work in the child thread.
*/
// wait until thread is finished
$thread->join();
// we can now even access $thread->data
}
for more information about PTHREADS read php docs here:
PHP DOCS PTHREADS
if you'r using WAMP like me, then you should add 'pthreadVC2.dll' into
\wamp\bin\apache\ApacheX.X.X\bin
and also edit the 'php.ini' file (same path) and add the same line as before
extension=php_pthreads.dll
GOOD LUCK!
What you are looking for is parallel which is a succinct concurrency API for PHP 7.2+
$runtime = new \parallel\Runtime();
$future = $runtime->run(function() {
for ($i = 0; $i < 500; $i++) {
echo "*";
}
return "easy";
});
for ($i = 0; $i < 500; $i++) {
echo ".";
}
printf("\nUsing \\parallel\\Runtime is %s\n", $future->value());
Output:
.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*..*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*
Using \parallel\Runtime is easy
Have a look at pcntl_fork. This allows you to spawn child processes which can then do the separate work that you need.
Not sure if a solution for your situation but you can redirect the output of system calls to a file, thus PHP will not wait until the program is finished. Although this may result in overloading your server.
http://www.php.net/manual/en/function.exec.php - If a program is started with this function, in order for it to continue running in the background, the output of the program must be redirected to a file or another output stream. Failing to do so will cause PHP to hang until the execution of the program ends.
There's Guzzle with its concurrent requests
use GuzzleHttp\Client;
use GuzzleHttp\Promise;
$client = new Client(['base_uri' => 'http://httpbin.org/']);
$promises = [
'image' => $client->getAsync('/image'),
'png' => $client->getAsync('/image/png'),
'jpeg' => $client->getAsync('/image/jpeg'),
'webp' => $client->getAsync('/image/webp')
];
$responses = Promise\Utils::unwrap($promises);
There's the overhead of promises; but more importantly Guzzle only works with HTTP requests and it works with version 7+ and frameworks like Laravel.
Is there a way you can abort a block of code if it's taking too long in PHP? Perhaps something like:
//Set the max time to 2 seconds
$time = new TimeOut(2);
$time->startTime();
sleep(3)
$time->endTime();
if ($time->timeExpired()){
echo 'This function took too long to execute and was aborted.';
}
It doesn't have to be exactly like above, but are there any native PHP functions or classes that do something like this?
Edit: Ben Lee's answer with pcnt_fork would be the perfect solution except that it's not available for Windows. Is there any other way to accomplish this with PHP that works for Windows and Linux, but doesn't require an external library?
Edit 2: XzKto's solution works in some cases, but not consistently and I can't seem to catch the exception, no matter what I try. The use case is detecting a timeout for a unit test. If the test times out, I want to terminate it and then move on to the next test.
You can do this by forking the process, and then using the parent process to monitor the child process. pcntl_fork is a method that forks the process, so you have two nearly identical programs in memory running in parallel. The only difference is that in one process, the parent, pcntl_fork returns a positive integer which corresponds to the process id of the child process. And in the other process, the child, pcntl_fork returns 0.
Here's an example:
$pid = pcntl_fork();
if ($pid == 0) {
// this is the child process
} else {
// this is the parent process, and we know the child process id is in $pid
}
That's the basic structure. Next step is to add a process expiration. Your stuff will run in the child process, and the parent process will be responsible only for monitoring and timing the child process. But in order for one process (the parent) to kill another (the child), there needs to be a signal. Signals are how processes communicate, and the signal that means "you should end immediately" is SIGKILL. You can send this signal using posix_kill. So the parent should just wait 2 seconds then kill the child, like so:
$pid = pcntl_fork();
if ($pid == 0) {
// this is the child process
// run your potentially time-consuming method
} else {
// this is the parent process, and we know the child process id is in $pid
sleep(2); // wait 2 seconds
posix_kill($pid, SIGKILL); // then kill the child
}
You can't really do that if you script pauses on one command (for example sleep()) besides forking, but there are a lot of work arounds for special cases: like asynchronous queries if you programm pauses on DB query, proc_open if you programm pauses at some external execution etc. Unfortunately they are all different so there is no general solution.
If you script waits for a long loop/many lines of code you can do a dirty trick like this:
declare(ticks=1);
class Timouter {
private static $start_time = false,
$timeout;
public static function start($timeout) {
self::$start_time = microtime(true);
self::$timeout = (float) $timeout;
register_tick_function(array('Timouter', 'tick'));
}
public static function end() {
unregister_tick_function(array('Timouter', 'tick'));
}
public static function tick() {
if ((microtime(true) - self::$start_time) > self::$timeout)
throw new Exception;
}
}
//Main code
try {
//Start timeout
Timouter::start(3);
//Some long code to execute that you want to set timeout for.
while (1);
} catch (Exception $e) {
Timouter::end();
echo "Timeouted!";
}
but I don't think it is very good. If you specify the exact case I think we can help you better.
This is an old question, and has probably been solved many times by now, but for people looking for an easy way to solve this problem, there is a library now: PHP Invoker.
You can use declare function if the execution time exceeds the limits. http://www.php.net/manual/en/control-structures.declare.php
Here a code example of how to use
define("MAX_EXECUTION_TIME", 2); # seconds
$timeline = time() + MAX_EXECUTION_TIME;
function check_timeout()
{
if( time() < $GLOBALS['timeline'] ) return;
# timeout reached:
print "Timeout!".PHP_EOL;
exit;
}
register_tick_function("check_timeout");
$data = "";
declare( ticks=1 ){
# here the process that might require long execution time
sleep(5); // Comment this line to see this data text
$data = "Long process result".PHP_EOL;
}
# Ok, process completed, output the result:
print $data;
With this code you will see the timeout message.
If you want to get the Long process result inside the declare block you can just remove the sleep(5) line or increase the Max Execution Time declared at the start of the script
What about set-time-limit if you are not in the safe mode.
Cooked this up in about two minutes, I forgot to call $time->startTime(); so I don't really know exactly how long it took ;)
class TimeOut{
public function __construct($time=0)
{
$this->limit = $time;
}
public function startTime()
{
$this->old = microtime(true);
}
public function checkTime()
{
$this->new = microtime(true);
}
public function timeExpired()
{
$this->checkTime();
return ($this->new - $this->old > $this->limit);
}
}
And the demo.
I don't really get what your endTime() call does, so I made checkTime() instead, which also serves no real purpose but to update the internal values. timeExpired() calls it automatically because it would sure stink if you forgot to call checkTime() and it was using the old times.
You can also use a 2nd script that has the pause code in it that is executed via a curl call with a timeout set. The other obvious solution is to fix the cause of the pause.
Here is my way to do that. Thanks to others answers:
<?php
class Timeouter
{
private static $start_time = FALSE, $timeout;
/**
* #param integer $seconds Time in seconds
* #param null $error_msg
*/
public static function limit($seconds, $error_msg = NULL)
: void
{
self::$start_time = microtime(TRUE);
self::$timeout = (float) $seconds;
register_tick_function([ self::class, 'tick' ], $error_msg);
}
public static function end()
: void
{
unregister_tick_function([ self::class, 'tick' ]);
}
public static function tick($error)
: void
{
if ((microtime(TRUE) - self::$start_time) > self::$timeout) {
throw new \RuntimeException($error ?? 'You code took too much time.');
}
}
public static function step()
: void
{
usleep(1);
}
}
Then you can try like this:
<?php
try {
//Start timeout
Timeouter::limit(2, 'You code is heavy. Sorry.');
//Some long code to execute that you want to set timeout for.
declare(ticks=1) {
foreach (range(1, 100000) as $x) {
Timeouter::step(); // Not always necessary
echo $x . "-";
}
}
Timeouter::end();
} catch (Exception $e) {
Timeouter::end();
echo $e->getMessage(); // 'You code is heavy. Sorry.'
}
I made a script in php using pcntl_fork and lockfile to control the execution of external calls doing the kill after the timeout.
#!/usr/bin/env php
<?php
if(count($argv)<4){
print "\n\n\n";
print "./fork.php PATH \"COMMAND\" TIMEOUT\n"; // TIMEOUT IN SECS
print "Example:\n";
print "./fork.php /root/ \"php run.php\" 20";
print "\n\n\n";
die;
}
$PATH = $argv[1];
$LOCKFILE = $argv[1].$argv[2].".lock";
$TIMEOUT = (int)$argv[3];
$RUN = $argv[2];
chdir($PATH);
$fp = fopen($LOCKFILE,"w");
if (!flock($fp, LOCK_EX | LOCK_NB)) {
print "Already Running\n";
exit();
}
$tasks = [
"kill",
"run",
];
function killChilds($pid,$signal) {
exec("ps -ef| awk '\$3 == '$pid' { print \$2 }'", $output, $ret);
if($ret) return 'you need ps, grep, and awk';
while(list(,$t) = each($output)) {
if ( $t != $pid && $t != posix_getpid()) {
posix_kill($t, $signal);
}
}
}
$pidmaster = getmypid();
print "Add PID: ".(string)$pidmaster." MASTER\n";
foreach ($tasks as $task) {
$pid = pcntl_fork();
$pidslave = posix_getpid();
if($pidslave != $pidmaster){
print "Add PID: ".(string)$pidslave." ".strtoupper($task)."\n";
}
if ($pid == -1) {
exit("Error forking...\n");
}
else if ($pid == 0) {
execute_task($task);
exit();
}
}
while(pcntl_waitpid(0, $status) != -1);
echo "Do stuff after all parallel execution is complete.\n";
unlink($LOCKFILE);
function execute_task($task_id) {
global $pidmaster;
global $TIMEOUT;
global $RUN;
if($task_id=='kill'){
print("SET TIMEOUT = ". (string)$TIMEOUT."\n");
sleep($TIMEOUT);
print("FINISHED BY TIMEOUT: ". (string)$TIMEOUT."\n");
killChilds($pidmaster,SIGTERM);
die;
}elseif($task_id=='run'){
###############################################
### START EXECUTION CODE OR EXTERNAL SCRIPT ###
###############################################
system($RUN);
################################
### END ###
################################
killChilds($pidmaster,SIGTERM);
die;
}
}
Test Script run.php
<?php
$i=0;
while($i<25){
print "test... $i\n";
$i++;
sleep(1);
}
I have a queue (Amazon SQS) of data that needs to be processed, and I would like to do it with multiple processes (in PHP).
I want the child workers to do something like this (pseduoish code):
while(true) {
$array = $queue->fetchNItems(10); // get 10 items
if(!count($array))
killProcess();
foreach($array as $item) {
... // process the item
$queue->remove($item);
}
sleep(2);
}
I always need 1 child process to be running, but in times of need I want to (fork?) a child process so that it can help process the queue faster.
Can someone help me with a rough PHP skeleton of what I need, or point me in the right direction?
I think I need to take a look at http://php.net/manual/en/function.pcntl-fork.php, but I'm not sure how I can use this to manage multiple processes.
When you fork a process. you make a duplicate of that process. In other words the copy (fork) contains everything the original process had (including file handles)
So how do you know if you are the parent or the forked process?
The example from the linked page shows this pretty clearly
<?php
$pid = pcntl_fork();
if ($pid == -1) {
die('could not fork');
} else if ($pid) {
// we are the parent
pcntl_wait($status); //Protect against Zombie children
} else {
// we are the child
}
?>
To extend this to what you want
<?php
$pid = pcntl_fork();
if ($pid == -1) {
die('could not fork');
} else if ($pid) {
// we are the parent
pcntl_wait($status); //Protect against Zombie children
} else {
// we are the child
while(true) {
$array = $queue->fetchNItems(10); // get 10 items
if(!count($array)) {
exit();
}
foreach($array as $item) {
... // process the item
$queue->remove($item);
}
sleep(2);
}
}
?>
This will create on forked process ( a waste in this instance ) use a loop to create multiple processes. when the child has finished exit will kill the child process. and pcntl_wait() will return allowing the parent to continue. I am not sure about php but if the parent process dies or exits it will kill the child process even if the child is not finished. hence the pcntl_wait. a more elaborate system is required if you spawn multiple children.
perhaps rather than forking you should look at the range of exec functions?
A caveat.
forking process can be wrought with problems, database handles being closed when a child exits etc. You can also kill a server with to many processes if something goes wrong. spend a lot of time playing and testing and reading.
DC
I know this is an old thread, but looked like it could use a more complete answer. This is how I generally spawn multiple processes in PHP.
A word of caution: PHP was meant to die. Meaning, the language was mean to execute for a few seconds then exit. Though, garbage cleanup in PHP has come a long way, be careful. Monitor your processes for unexpected memory consumption, or other oddities. Watch everything like a hawk for a while before you set it and forget it, and even then, still check the processes once in a while or have them automatically notify if something becomes amiss.
As I was typing this up, seemed like a good idea to slap it on github too.
When ready to run the program, I recommend, doing a tail -f on the log to see the output.
<?php
/*
* date: 27-sep-2015
* auth: robert smith
* info: run a php daemon process
* lic : MIT License (see LICENSE.txt for details)
*/
$pwd = realpath("");
$daemon = array(
"log" => $pwd."/service.log",
"errorLog" => $pwd."/service.error.log",
"pid_file" => $pwd."/",
"pid" => "",
"stdout" => NULL,
"stderr" => NULL,
"callback" => array("myProcessA", "myProcessB")
);
/*
* main (spawn new process)
*/
foreach ($daemon["callback"] as $k => &$v)
{
$pid = pcntl_fork();
if ($pid < 0)
exit("fork failed: unable to fork\n");
if ($pid == 0)
spawnChores($daemon, $v);
}
exit("fork succeeded, spawning process\n");
/*
* end main
*/
/*
* functions
*/
function spawnChores(&$daemon, &$callback)
{
// become own session
$sid = posix_setsid();
if ($sid < 0)
exit("fork failed: unable to become a session leader\n");
// set working directory as root (so files & dirs are not locked because of process)
chdir("/");
// close open parent file descriptors system STDIN, STDOUT, STDERR
fclose(STDIN);
fclose(STDOUT);
fclose(STDERR);
// setup custom file descriptors
$daemon["stdout"] = fopen($daemon["log"], "ab");
$daemon["stderr"] = fopen($daemon["errorLog"], "ab");
// publish pid
$daemon["pid"] = sprintf("%d", getmypid());
file_put_contents($daemon["pid_file"].$callback.".pid", $daemon["pid"]."\n");
// publish start message to log
fprintf($daemon["stdout"], "%s daemon %s started with pid %s\n", date("Y-M-d H:i:s"), $callback, $daemon["pid"]);
call_user_func($callback, $daemon);
// publish finish message to log
fprintf($daemon["stdout"], "%s daemon %s terminated with pid %s\n", date("Y-M-d H:i:s"), $callback, $daemon["pid"]);
exit(0);
}
function myProcessA(&$daemon)
{
$run_for_seconds = 30;
for($i=0; $i<$run_for_seconds; $i++)
{
fprintf($daemon["stdout"], "Just being a process, %s, for %d more seconds\n", __FUNCTION__, $run_for_seconds - $i);
sleep(1);
}
}
function myProcessB(&$daemon)
{
$run_for_seconds = 30;
for($i=0; $i<$run_for_seconds; $i++)
{
fprintf($daemon["stdout"], "Just being a process, %s, for %d / %d seconds\n", __FUNCTION__, $i, $run_for_seconds);
sleep(1);
}
}
?>