that's a very simple question
I have a crawler php script.
When I run a sh script:
php crawler.php
php crawler.php
It is synchronous: it wait until first php is ended to start another one.
When trying:
php crawler.php &
php crawler.php &
That looks ok, Porblem is: they do remain in processes:
ps - a | grep php
8689 pts/3 00:00:00 php
8747 pts/3 00:00:00 php
and the goal would be to have a cron that lucnhes 10 php scripts in background.
so once a php script has finished, it is not in processes any more, so another can be lunched.
Any clue ?
regards
use php crawler.php > /dev/null 2>/dev/null &
Related
My issue seems to be asked before but hold on, this one is a bit different.
I have 2 php files, I run the following commands:
nohup php file_1.php >/dev/null 2>&1 &
nohup php file_2.php >/dev/null 2>&1 &
This will create basically 2 php processes.
My problem is that sometimes one of the files or even both of them are killed by the server for unknown reason and I have to re-enter the commands over again. I tried 'forever' but doesn't help.
If the server is rebooted I will have to enter those 2 commands, I thought about Cronjob but I'm not sure if it would launch it twice which would create more confusion.
My question is how to start automatically the files if one or both of them got killed? What is the best way to achieve this which would check exactly that file_1.php or that file_2.php is indeed running?
There are a couple of ways you can do this. As #Chris Haas mentioned in the comments, supervisord can do this for you, or you can run a watchdog shell script from cron at regular intervals that checks to see if your script is running, and if not, start it. Here's one I use.
#!/bin/bash
FIND_PROC=`ps -ef |grep "php queue_agent.php --env prod" | awk '{if ($8 !~ /grep/) print $2}'`
# if FIND_PROC is empty, the process has died; restart it
if [ -z "${FIND_PROC}" ]; then
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
echo queue_agent.php failed at `date`
cd ${DIR}
nohup nice -n 10 php queue_agent.php --env prod -v > ../../sandbox/logs/queue_agent.log &
fi
exit 0
I think u can try to figure out why these two php scripts shut down as the first step. To solve this problem, u can use this php function:
void register_shutdown_function(callback $callback[.mixed $parameter]);
which Registers a callback to be executed after script execution finishes or exit() is called. So u can log some info when php files get shut down like this:
function myLogFunction() {
//log some info here
}
register_shutdown_function(myLogFunction);
Instead of putting the standard output and error output into /dev/null, u can put them into a log file(Since maybe we can get some helpful info from the output). So instead of using:
nohup php file_1.php >/dev/null 2>&1 &
nohup php file_2.php >/dev/null 2>&1 &
try:
nohup php file_1.php 2>yourLog.log &
nohup php file_2.php 2>yourLog.log &
If u want to autorun these two php files when u boot the server, try edit /etc/rc.local(which is autorun when the os start up). Add your php cli command lines in this file.
If u can't figure out why php threads get shut down, try supervisor as #Chris Haas mensioned.
There is a commonly discussed method of starting a background process in PHP using the exec or shell_exec functionality.
I have had success with this in the past with batch email sending, and sending data to APIs in the background.
In a PHP page that you would call by ajax, you do something like this:
echo 'process running';
shell_exec('/usr/bin/php -q path_to_background_script.php > /dev/null &' );
exit;
The background process normally runs as if called by the owner of the php user like a terminal process.
Recently however, under a FASTCGI system (ea-php56) I have found this method has stopped working.
Instead of one process beginning from one web-request to the calling page, I am getting the background script continually terminating and being re-spawned with a new process id. Interestingly, the only way to stop this continual re-spawning is to disable the line in the calling script that starts the process. The re-spawning stops immediately when you save the calling file without the call to the background script.
This tells me that it is actually the calling script (requested by the browser) which is actually being re-spawned.
This is what the re-spawning looks like from the root terminal. Notice the PID changes everytime I look:
[root#*** public_html]# ps -ef | grep php
*user* 725 1 7 23:53 ? 00:00:00 /opt/cpanel/ea-php56/root/usr/bin/php-cgi /home/*user*/public_html/background-script_exec.php
root 727 32411 0 23:53 pts/1 00:00:00 grep php
[root#dev public_html]# ps -ef | grep php
*user* 757 1 5 23:53 ? 00:00:00 /opt/cpanel/ea-php56/root/usr/bin/php-cgi /home/*user*/public_html/background-script_exec.php
root 759 32411 0 23:53 pts/1 00:00:00 grep php
[root#dev public_html]# ps -ef | grep php
*user* 781 1 12 23:54 ? 00:00:00 /opt/cpanel/ea-php56/root/usr/bin/php-cgi /home/*user*/public_html/background-script_exec.php
I have tried disabling "PHP-FPM service for cPanel Daemons". I have tried 'ignore_user_abort()'. fastcgi_finish_request() function is not available so could not try that. I have tried creating a shell script instead to call the background PHP script, which I call from the calling script - but this also does exactly the same thing.
Apart from disabling the ability to trigger background scripts from a PHP web-page, this new PHP FastCGI behaviour is creating an erratic re-spawning process that does not stop without intervention mentioned above. It has made shell_exec / exec functions unstable!
Problem seems similar to that reported here:
php exec/shell_exec/system/popen/proc_open runs calling script itself infinite number of times on linux
but, suggestion reported here does not help in this case.
This seems to solve the problem. Using the 'php5-cli' instead of 'php'
shell_exec('/usr/bin/php5-cli path_to_background_script.php > /dev/null &' );
I had early tried 'php-cli' and found it did not exist - I did not think to check to see that it was named differently!
If having similar problems, have a look for the php binaries:
>>ls /usr/bin/php*
>>/usr/bin/php /usr/bin/php5 /usr/bin/php5-cli /usr/bin/php-config /usr/bin/phpize
use the one which is for command line, and it should then run properly.
-note that this issue was specifically on a linux cpanel easy-apache fastcgi-php system running on Centos.
I have this simple sample script:
#!/usr/bin/php
<?php
while(true) {
error_log("hello " . time() . "\n", 3, "logs.log");
sleep(3);
}
Which I execute with the command:
me:~/Desktop$ php worker.php &
As expected it returns me 6683: the id of the process. Fine.
Now to make sure everything is fine I do a ps and get:
PID TTY TIME CMD
5561 pts/1 00:00:00 bash
6683 pts/1 00:00:00 php
6705 pts/1 00:00:00 ps
[1]+ Stopped php worker.php
Why is it stopped? Is it not supposed to run continuously untill someone kill -9 it?
If I do a second ps I get:
PID TTY TIME CMD
5561 pts/1 00:00:00 bash
6683 pts/1 00:00:00 php
7395 pts/1 00:00:00 ps
The process is there but it is not logging (dormant?)
Anyone?
Ta
The script must have a place where to dump stdout or else it shows the above behaviour (gets stopped). This can be done like this:
php worker.php > /dev/null &
Like this it works and the timed infinte loop does its job. The only problem is: if you now stop the parent process (the shell's) then the background php process stops too.
To get around it run:
disown -h php-process-id
Youi can now close all the shells, log out and go to the pub.
Well done me! ;)
Not sure if my reasoning is correct, but I've run into this problem, too, and solved it by using nohup.
My reasoning was this:
A PHP process has a handle on stdin. By running a process on the background, you can't read from the stdin, perhaps at some point a hangup signal might be sent.
By using nohup, you can ignore that signal.
Here's a few extra words of explanation on the matter
I have a pdf file that makes the pdf2swf tool to run forever. when running the following command manually :
pdf2swf -Q 10 test.pdf
The script aborts after 10 seconds becouse of the -Q 10 flag, but when running the same command using php the script runs forever. I've tried using shell_exec() ,exec() and passthru() and all of them ignored the -Q flag.
Has anyone encontered anything like this with pdf2swf tool or with any other PHP exec ?
EDIT
when I run it manually
php -r "exec('pdf2swf -Q 10 test.pdf');"
It aborts after 10 seconds, but when running as a deamon, again, it won't abort
I tried same command in my test project and it worked fine with no issues, please find my code as -
$out = exec("pdf2swf font_example.pdf -o varun.swf", $rest);
I am trying to run a php CLI script in the background and it just won't run - it has a status of Stopped SIGTOU (Trying to write output) - Here are the details
Mac OS X Lion 10.7.2
PHP 5.3.6 with Suhosin-Patch (cli) (built: Sep 8 2011 19:34:00)
I created a basic script test.php
<?php echo 'Hello world'.PHP_EOL; ?>
Here are the results of various tests:-
php -f test.php (Hello world gets displayed)
php -f test.php >test.log 2>&1 (Hello world gets put into test.log)
php -f test.php >test.log 2>&1 & --- I get [1]+ Stopped(SIGTTOU) php -f test.php > test.log 2>&1 -- and the job just sits there doing nothing nothing gets logged however lsof shows the log file is open
It is something to do with PHP? A similar shell script gets executed no problems in the background.
If readline is enabled in your build of php, simply pass /dev/null as the input.
In your example above, it would be:
php -f test.php </dev/null >test.log 2>&1
This is resolved now -- thanks to all who responded. The problem was that Apple provide PHP pre-built with the OS - the CLI version was built with readline included - http://www.php.net/manual/en/intro.readline.php ... this prevents any background running of scripts because readline automatically starts IO with the TTY ...
My problem was that I couldn't build my own version of PHP because of this -> http://forums.macrumors.com/showthread.php?t=1284479 - once I got that resolved my background script issue was gone :)
Well, a PHP script stops when its done execution, ie a simple echo "Hello World" is done execution as soon as it outputted the string, i would guess it have something to do with it ;-)