I would like to start using thread with PHP.
Could someone give me an example about how to start two simultaneous processes?
If you want to spawn multiple PHP processes that run in the background (or not), you should have a look at the manual entry on Program Execution. It lists all the methods that allow you to spawn a new process to handle background tasks
http://www.php.net/manual/en/ref.exec.php
You can fork processes with PCNTL extension. http://php.net/manual/tr/book.pcntl.php
$pid = pcntl_fork();
if ($pid) {
// main
pcntl_wait($status);
} else {
// child
}
Related
I have a php script that divides a task into multiple parts and runs each part in a separate child process. The code looks like this:
foreach($users as $k => $arr) {
if(($pid = pcntl_fork()) === -1) continue;
if($pid) {
pcntl_wait($status,WNOHANG);
continue;
}
ob_start();
posix_setsid();
dbConnect();
// do stuff to data
exit();
}
I'm running this script using crontab on a Debian server, but the problem is some processes keep running and reserve memory. After a while the server's memory is flooded.
I need to find a way to make sure all processes finish correctly.
I think the issue is the use of WNOHANG in the pcntl_wait call. This means the pcntl_wait function exist before the child process - which you want, in order to be able to fork the other child processes concurrently. But it has the side-effect that the main parent finishes before some of the children. This link http://www.devshed.com/c/a/PHP/Managing-Standalone-Scripts-in-PHP/2/ describes how to loop using pcntl_wait with WNOHANG until all children complete.
The stuff you do to the data takes to long or forever. You need to debug the operations you execute.
As all of you know when you fork the child gets a copy of everything, including file and network descriptors - man fork.
In PHP, when you use pcntl_fork all of your connections created with mysql_connect are copied and this is somewhat of a problem - php docs and SO question. Common sense in this situation says close the parent connection, create new and let the child use the old one. But what if said parent needs create many children ever few seconds? In that case you end up creating loads of new connections - one for every bunch of forks.
What does that mean in code:
while (42) {
$db = mysql_connect($host, $user, $pass);
// do some stuff with $db
// ...
foreach ($jobs as $job) {
if (($pid = pcntl_fork()) == -1) {
continue;
} else if ($pid) {
continue;
}
fork_for_job($job);
}
mysql_close($db);
wait_children();
sleep(5);
}
function fork_for_job($job) {
// do something.
// does not use the global $db
// ...
exit(0);
}
Well, I do not want to do that - thats way too many connections to the database. Ideally I would want to be able to achieve behaviour similar to this one:
$db = mysql_connect($host, $user, $pass);
while (42) {
// do some stuff with $db
// ...
foreach ($jobs as $job) {
if (($pid = pcntl_fork()) == -1) {
continue;
} else if ($pid) {
continue;
}
fork_for_job($job);
}
wait_children();
sleep(5);
}
function fork_for_job($job) {
// do something
// does not use the global $db
// ...
exit(0);
}
Do you think it is possible?
Some other things:
This is php-cli script
I've tried using mysql_pconnect in the first example but as far as I can tell there is no difference - the mysql server receives as much new connections. Maybe that's because it is cli and pconnect does not work as it was in mod_php. As Marc has noticed - pconnect in php-cli does not make sense.
The only thing you could try, is to let your children wait until each other child has finished its job. This way you could use the same database connection (provided there aren't any synchronization issues). But of course you'll have a lot of processes, which is not very good too (in my experience PHP has quite a big memory usage). If having multiple processes accessing the same database connection is not a problem, you could try to make "groups" of processes which share a connection. So you don't have to wait until each job finished (you can clean up when the whole group finished) and you don't have a lot of connections either..
You should ask yourself whether you really need a database connection for your worker processes. Why not let the parent fetch the data and write your results to a file?
If you do need the connection, you should consider using another language for the job. PHPs cli itself is not a "typical" use case (it was added in 4.3) and multiprocessing is more of a hack than a supported feature.
If the child calls exec() or _exit() fairly quickly, you're alright. The problem is if the child sticks around and holds on to copies of your file descriptors.
You could also use posix_spawn if PHP has an API for that. That might work well.
My advice (from personal experience on the same issue) is to close the connection before pcntl_fork() then open new connections in parent and/or the child process as needed.
If you open a new connection in the parent process then you have to block the SIGCHLD signal (using pcntl_sigprocmask(SIG_BLOCK, array(SIGCHLD)). No special care is needed in the children processes (except when they also launch their own children, becoming parents this way.)
SIGCHLD is a signal that is received by the parent process when one of its children completes.
During the communication with the server, the MySQL client library uses nanosleep() to suspend the execution of the program for some amounts of time. The sleep() functions return when the time passes but they also return before the time passes if the process receives a signal while it is suspended.
When nanosleep() returns because of a signal (i.e. before enough time has passed), the MySQL library gets confused and reports the error "MySQL server has gone away" and the connection cannot be used any more. It is a false alarm, the MySQL server is still there waiting for queries but the client code is fooled by the signal arrived at the wrong moment.
If you are interested in receiving the SIGCHLD signal then you can block it before running a MySQL query then unblock it again (to avoid it being received during the communication with MySQL server.
Also read this answer and this answer I wrote on similar questions (it's the same information, but with more details and explanation.)
Can PHP call a function and don't wait for it to return? So something like this:
function callback($pause, $arg) {
sleep($pause);
echo $arg, "\n";
}
header('Content-Type: text/plain');
fast_call_user_func_array('callback', array(3, 'three'));
fast_call_user_func_array('callback', array(2, 'two'));
fast_call_user_func_array('callback', array(1, 'one'));
would output
one (after 1 second)
two (after 2 seconds)
three (after 3 seconds)
rather than
three (after 3 seconds)
two (after 3 + 2 = 5 seconds)
one (after 3 + 2 + 1 = 6 seconds)
Main script is intended to be run as a permanent process (TCP server). callback() function would receive data from client, execute external PHP script and then do something based on other arguments that are passed to callback(). The problem is that main script must not wait for external PHP script to finish. Result of external script is important, so exec('php -f file.php &') is not an option.
Edit:
Many have recommended to take a look at PCNTL, so it seems that such functionality can be achieved. PCNTL is not available in Windows, and I don't have an access to a Linux machine right now, so I can't test it, but if so many people have advised it, then it should do the trick :)
Thanks, everyone!
On Unix platforms you can enable the PCNTL functions, and use pcntl_fork to fork the process and run your jobs in child processes.
Something like:
function fast_call_user_func_array($func, $args) {
if (pcntl_fork() == 0) {
call_user_func_array($func, $args);
}
}
Once you call pcntl_fork, two processes will execute your code from the same position. The parent process will get a PID returned from pcntl_fork, while the child process will get 0. (If there's an error the parent process will return -1, which is worth checking for in production code).
You can check out PHP Process Control:
http://us.php.net/manual/en/intro.pcntl.php
Note: This is not threading, but the handling of separate processes. There is more overhead attached.
Wouldn't it solve your problem to fork, keeping the parent process free for other connections & actions? See http://www.php.net/pcntl_fork. If you need an answer back you could possibly listen to a socket in the parent, and write with the child. A simple while(true) loop with a read could possibly do, and probably you already have that basic functionality if you run a permanent TCP server. Another option would be to keep track of your childprocess-ids, keep a accessable store somewhere (file/database/memcached etc), with a pcnt_wait in the main process with a WNOHANG to check which process has exited, and retrieve the data from the store.
You can do some threading in PHP if you use the method pcntl_fork.
http://ca.php.net/manual/en/function.pcntl-fork.php
I have never use this myself, but the are some good example of how to use it on php.net.
PHP doesn't have this functionality as far as I know
You can emulate the function using a different technique, like this one:
Parallel functions in PHP
PHP does not support multi-threading, so there's no other option than taking advantage of the OS or the web server multi processing capabilities. Note that actually you can fetch both the result and output of exec:
string exec ( string $command [,
array &$output [, int &$return_var
]] )
You can, at least, prevent the parent process from hanging until the child process is done by ignoring the child signals using pcntl_signal(SIGCHLD, SIG_IGN).
So, let's say you want to fork a process and execute another PHP function that takes a while without making the parent wait for it to finish (since you want the main process to finish in a timely manner):
pcntl_signal(SIGCHLD, SIG_IGN);
$pid = pcntl_fork();
if ($pid < 0) {
exit(0);
} elseif (!$pid) {
my_slow_function();
exit(0);
}
// Parent keeps executing and finishes before the child does
If you want to execute a slow external script as the child process, pcntl_exec is handy:
$script = array('/path/to/my/script'); // E.g. /home/my_user/my_script.php
pcntl_exec('/path/to/program/executable',$script); // E.g. /usr/bin/php
I am converting a PDF with PDF2SWF and Indexing with XPDF.. with exec.. only this requires the execution time to be really high.
Is it possible to run it as background process and then launch a script when it is done converting?
in general, php does not implement threads.
But there is an ZF-class which may be suitable for you:
http://framework.zend.com/manual/en/zendx.console.process.unix.overview.html
ZendX_Console_Process_Unix allows
developers to spawn an object as a new
process, and so do multiple tasks in
parallel on console environments.
Through its specific nature, it is
only working on nix based systems
like Linux, Solaris, Mac/OSx and such.
Additionally, the shmop_, pcntl_* and
posix_* modules are required for this
component to run. If one of the
requirements is not met, it will throw
an exception after instantiating the
component.
suitable example:
class MyProcess extends ZendX_Console_Process_Unix
{
protected function _run()
{
// doing pdf and flash stuff
}
}
$process1 = new MyProcess();
$process1->start();
while ($process1->isRunning()) {
sleep(1);
}
echo 'Process completed';
.
Try using popen() instead of exec().
This hack will work on any standard PHP installation, even on Windows, no additional libraries required. Yo can't really control all aspects of the processes you spawn this way, but sometimes this is enough:
$p1 = popen("/bin/bash ./some_shell_script.sh argument_1","r");
$p2 = popen("/bin/bash ./some_other_shell_script.sh argument_2","r");
$p2 = popen("/bin/bash ./yet_other_shell_script.sh argument_3","r");
The three spawned shell scripts will run simultaneously, and as long as you don't do a pclose($p1) (or $p2 or $p3) or try to read from any of these pipes, they will not block your PHP execution.
When you're done with your other stuff (the one that you are doing with your PHP script) you can call pclose() on the pipes, and that will pause your script execution until the process you are pclosing finishes. Then your script can do something else.
Note that your PHP will not conclude or die() until those scripts have finished. Reaching the end of the script or calling die() will make it wait.
If you are running it from the command line, you can fork a php process using pcntl_fork
There are also daemon classes that would do the same trick:
http://pear.php.net/package/System_Daemon
$pid = pcntl_fork();
if ($pid == -1) {
die('could not fork');
} else if ($pid) {
//We are the parent, exit
exit();
} else {
// We are the child, do something interesting then call the script at the end.
}
Hey there, I have a simple script that which is suppose to load 2 separate pages at the same time and grab some text from them, however it loads either the parent process or the child process depending on what finishes first, what am i doing wrong ? I want the 2 processes to work simultaneously, here is the example code:
<?php
$pid = pcntl_fork();
if ($pid == -1) {
die("could not fork");
}
else if($pid) {
$url = "http://www.englishpage.com/verbpage/simplepresent.html";
$readurl = file_get_contents($url);
$pattern = '#Examples(.*?)Forms#s';
preg_match($pattern, $readurl, $match);
echo "Test1:".$match[1];
}
else {
$url = "http://www.englishpage.com/verbpage/simplepresent.html";
$readurl = file_get_contents($url);
$pattern = '#Examples(.*?)Forms#s';
preg_match($pattern, $readurl, $match);
echo "Test2:".$match[1];
}
echo "<br>Finished<br>";
?>
any help would be appreciated!
I am not quite sure that I really understand what you are willing to get, but if you want your "Finished" message to be displayed :
only once
only when the two processes have done their work
You should :
Use pcntl_wait in the parent process, so it waits for its child to die
Echo "finished" from the parent process, after it has finished waiting.
For instance, something like this should do :
$pid = pcntl_fork();
if ($pid == -1) {
die("could not fork");
}
else if($pid) { // Father
sleep(mt_rand(0, 5));
echo "Father done\n";
pcntl_wait($status); // Wait for the children to finish / die
echo "All Finished\n\n";
}
else { // Child
sleep(mt_rand(0, 5));
echo "Child done\n";
}
With this, each process will do its work, and only when both have finished, the parent will display that everything is done :
if the parent is done first, it'll wait for the child
if the child ends first, the parent will not wait... But still finish after it.
As a sidenote : you are using two separate processes ; once forked, you cannot "easily" share data between them -- so it's not easy to pass data from the child to the father, nor is it the other way arround.
If you need to do that, you can take a look at Shared Memory Functions -- or just use plain files ^^
Hope this helps -- and that I understood the question correctly ^^
From the Process Control Extension Introduction
Process Control support in PHP implements the Unix style of process creation, program execution, signal handling and process termination. Process Control should not be enabled within a web server environment and unexpected results may happen if any Process Control functions are used within a web server environment.
So basically, you shouldn't use any of the pcntl functions when you are running a PHP script through the apache module.
If you just want to fetch the data from those 2 pages simultaneously then you should be able to use stream_select to achieve this. You can find an example at http://www.ibm.com/developerworks/web/library/os-php-multitask/.
BTW Apparently curl supports this too, using curl_multi_select, an example on how to use that can be found at http://www.somacon.com/p537.php.