waiting for all pids to exit in php - php

Question

My issue is this. I am forking a process so that I can speed up access time to files on disk. I store any data from these files in a tmp file on local desk. ideally, after all processes have finished, I need to access that tmp file and get that data into an array. I then unlink the tmp file as it is no longer needed. My problem is that it would seem that pcntl_wait() does not acutally wait until all child processes are done before moving on to the final set of operations. So I end up unlinking that file before some random process can finish up.
I can't seem to find a solid way to wait for all processes to exit cleanly and then access my data.
$numChild = 0;
$maxChild = 20; // max number of forked processes.
// get a list of "availableCabs"
foreach ($availableCabs as $cab) {
// fork the process
$pids[$numChild] = pcntl_fork();
if (!$pids[$numChild]) {
// do some work
exit(0);
} else {
$numChild++;
if ($numChild == $maxChild) {
pcntl_wait($status);
$numChild--;
}
} // end fork
}
// Below is where things fall apart. I need to be able to print the complete serialized data. but several child processes don't actually exit before i unlink the file.
$dataFile = fopen($pid, 'r');
while(($values = fgetcsv($dataFile,',')) !== FALSE) {
$fvalues[] = $values;
}
print serialize($fvalues);
fclose($dataFile);
unlink($file);
please note that i'm leaving a lot of code out regarding what i'm actually doing, if we need that posted thats not issue.

Answer 1

Try restructuring you code so that you have two loops - one that spawns processes and one that waits for them to finish. You should also use pcntl_waitpid() to check for specific process IDs, rather than the simple child counting approach you are currently using.
Something like this:
<?php
$maxChildren = 20; // Max number of forked processes
$pids = array(); // Child process tracking array
// Get a list of "availableCabs"
foreach ($availableCabs as $cab) {
// Limit the number of child processes
// If $maxChildren or more processes exist, wait until one exits
if (count($pids) >= $maxChildren) {
$pid = pcntl_waitpid(-1, $status);
unset($pids[$pid]); // Remove PID that exited from the list
}
// Fork the process
$pid = pcntl_fork();
if ($pid) { // Parent
if ($pid < 0) {
// Unable to fork process, handle error here
continue;
} else {
// Add child PID to tracker array
// Use PID as key for easy use of unset()
$pids[$pid] = $pid;
}
} else { // Child
// If you aren't doing this already, consider using include() here - it
// will keep the code in the parent script more readable and separate
// the logic for the parent and children
exit(0);
}
}
// Now wait for the child processes to exit. This approach may seem overly
// simple, but because of the way it works it will have the effect of
// waiting until the last process exits and pretty much no longer
foreach ($pids as $pid) {
pcntl_waitpid($pid, $status);
unset($pids[$pid]);
}
// Now the parent process can do it's cleanup of the results