I want to start Hadoop daemons with PHP instead of terminal.
When i do start-dfs.sh, it works well.
The PHP code I'm using is
<?php
echo shell_exec('/usr/local/hadoop3/sbin/stop-dfs.sh');
?>
the output of the above code is
Starting namenodes on [localhost] starting datanodes. Starting
secondary namenodes [chbpc-VirtualBox] .
but when i type the url http://localhost:9870/dfshealth.html#tab-overview in the browser (to check the status of my Hadoop), it actually did not start despite having the output above.
Answer
<?php
echo shell_exec('/usr/local/hadoop3/sbin/stop-dfs.sh > /dev/null &');
?>
Description
When you run the daemon process with shell_exec, you should detach from this process for finished your shell_exec command, but you should stay daemon process in progress state.
& - provide doing this.
Also instead /dev/null you may be using your custom log file and all info from stop-dfs.sh will redirect to this file.
My test for this solution
Copy this snippet to your bash console:
cat <<EOT > daemon.sh
while true; do date; sleep 1; done
EOT
chmod +x daemon.sh
cat <<EOT > daemon-runner.php
<?php
echo shell_exec(__DIR__ . '/daemon.sh > daemon-log &');
EOT
touch daemon-log
php daemon-runner.php
tail -f daemon-log
PS
You using stop-dfs.sh script name for start your hadoop, also maybe you mixed up script name with start.sh?
Related
My issue seems to be asked before but hold on, this one is a bit different.
I have 2 php files, I run the following commands:
nohup php file_1.php >/dev/null 2>&1 &
nohup php file_2.php >/dev/null 2>&1 &
This will create basically 2 php processes.
My problem is that sometimes one of the files or even both of them are killed by the server for unknown reason and I have to re-enter the commands over again. I tried 'forever' but doesn't help.
If the server is rebooted I will have to enter those 2 commands, I thought about Cronjob but I'm not sure if it would launch it twice which would create more confusion.
My question is how to start automatically the files if one or both of them got killed? What is the best way to achieve this which would check exactly that file_1.php or that file_2.php is indeed running?
There are a couple of ways you can do this. As #Chris Haas mentioned in the comments, supervisord can do this for you, or you can run a watchdog shell script from cron at regular intervals that checks to see if your script is running, and if not, start it. Here's one I use.
#!/bin/bash
FIND_PROC=`ps -ef |grep "php queue_agent.php --env prod" | awk '{if ($8 !~ /grep/) print $2}'`
# if FIND_PROC is empty, the process has died; restart it
if [ -z "${FIND_PROC}" ]; then
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
echo queue_agent.php failed at `date`
cd ${DIR}
nohup nice -n 10 php queue_agent.php --env prod -v > ../../sandbox/logs/queue_agent.log &
fi
exit 0
I think u can try to figure out why these two php scripts shut down as the first step. To solve this problem, u can use this php function:
void register_shutdown_function(callback $callback[.mixed $parameter]);
which Registers a callback to be executed after script execution finishes or exit() is called. So u can log some info when php files get shut down like this:
function myLogFunction() {
//log some info here
}
register_shutdown_function(myLogFunction);
Instead of putting the standard output and error output into /dev/null, u can put them into a log file(Since maybe we can get some helpful info from the output). So instead of using:
nohup php file_1.php >/dev/null 2>&1 &
nohup php file_2.php >/dev/null 2>&1 &
try:
nohup php file_1.php 2>yourLog.log &
nohup php file_2.php 2>yourLog.log &
If u want to autorun these two php files when u boot the server, try edit /etc/rc.local(which is autorun when the os start up). Add your php cli command lines in this file.
If u can't figure out why php threads get shut down, try supervisor as #Chris Haas mensioned.
I have a Beanstalk MQ where I put task to create APK, and a consumer named AppBuilder.php, which reads messages from Beanstalk MQ, and then exec the command which creates the App (android app).
The AppBuilder.php is run from the crontab. the process is
Crontab runs a health-check.sh shell script
health-check.sh runs AppBuilder.php in background
AppBuilder.php calls exec to create the process
Following is the relevant code snippet(s) from each file:
Root crontab is like so:
* * * * * /opt/cron/health-check/health-check.sh
health-check.sh is like this:
#!/bin/bash
PATH=$PATH:/sbin/
#HEALTH CHECK FOR AppBuilder Daemon
if (( $(ps -ef | grep "[A]ppBuilder" | wc -l) > 0 ))
then
echo "AppBuilder is Running"
else
echo "AppBuilder is Stopped, attempting to restart"
$PHP_CMD /opt/appbuilder/AppBuilder.php &
if pgrep "AppBuilder" > /dev/null
then
echo "AppBuilder is now Running"
else
echo "AppBuilder is still not Running"
fi
fi
AppBuilder.php has following exec command:
exec('sudo sh /var/www/cgi-bin/appbuilder/oneClickApk.sh &', $output, $resultCode);
If I run the AppBuilder.php directly, things work fine. However, from cron, it does not.
I've followed this SO Post, and modified the exec command to the following:
exec('/usr/bin/nohup /usr/bin/sudo /usr/bin/env TERM=xterm /bin/sh /var/www/cgi-bin/appbuilder/oneClickApk.sh &', $output, $resultCode);
However, still things fail. Any clues where this may be going wrong? I've spend a lot of time digging the forums, none helping. Please help!
EDIT 1:
The crontab runs, AppBuilder.php gets initialized, but after exec command, I could not see the oneClickApk.sh in process list
EDIT 2:
I changed the crontab from root to ec2-user, as suggested in comments: Still the process does not run.
Just following your first approach like below. But to fix the issue you have to do the following.
Check your cron-tab entry whether properly set or not.
In cron-tab provide full path to your executable
Check whether cron-tab user account has proper setting to execute sudo
Check whether cron-tab account has proper permission to execute your PHP application as well as your shell script. If not add the account into the proper group so that it can run both the PHP and shell script.
exec('sudo sh /var/www/cgi-bin/appbuilder/oneClickApk.sh &', $output, $resultCode);
Use above command its seems OK.
In you shell script put a wait command and check whether it is working fine or not. Just add only below two lines without any if condition first. If working then you need to check your if condition why they are not satisfying as per your requirement.
$PHP_CMD /opt/appbuilder/AppBuilder.php &
while (( $(ps -ef | grep "[h]ealth-check.sh" | wc -l) <= 0 ))
do
sleep 5
done
sleep 30
I'd need you precious help on a matter I am spending hours on.
Scope: Apache2 and PHP running on a raspberry pi;
Premise: my little knowledge of Linux environment!
The objective: launching a long-run php script, that opens sockets, from another php script running as webpage. In other terms, the application is a chat and I need to start the php server script form a web page, for my convenience.
The issue: if I run it from the the console, logged as "pi", with the following command
php -q /var/www/chatSocket.php > /var/www/tmp/socketProcessOutput.txt 2>&1 & echo $!
it works like a charm, but if it try to do so from a script, with the following (don't mind the concatenated strings and assignment of output to variables - it made no difference removing them):
$result .= "Result of pkill (killed process): " .shell_exec('sudo pkill -f SongWebSocket.php') ."\n";
$result .= "Launching new process: id returned:". shell_exec('php -q /var/www/chatSocket.php > /var/www/tmp/socketProcessOutput.txt 2>&1 & echo $!') ."\n";
$result .= "Checking running SongWebSocket.php process:" ."\n";
$result .= shell_exec('ps -A aux| grep -e SongWebSocket.php -e USER') ."\n";
.. it does not work (it seems like it launch the script but the sockets ar not open).
Any clue why this happens?
Also, and this can be for my little knowledge of Linux, why i get a dioffrent aoutput from the command
ps aux| grep -e SongWebSocket.php -e USER
if I launch it from the shell, as user pi, or from the sript, as www-data user.
I Look forward for your help. Thanks in advance!
Marco.
www-data user doesn't have the permisson I guess. why not add "sudo" for every shell_exec line? (sudo starts the programm with root permission). it's not pretty and not secure but it might work on you local home-network. sudo php ... sudo ps -A aux etc. In addition you should make sure that the php safe_mode is off. you can see that by adding phpinfo(); to you php code
I am struggling with getting a php file to run in the background with PHP's exec(). As a first test, I tried :
exec("ls -l > logfile.txt 2> errfile.txt &");
That works fine. logfile.txt gets filled with a directory listing.
Per instructions in the php documentation, since the exec kicks off a process that runs in the background, standard out (and standard error) are redirected to a file.
Now, I try
exec("/usr/bin/php -f /path/to/my.php > logfile.txt 2> errorfile.txt &");
It appears nothing happens.
Here are test files that I'm trying:
alpha.php
<?php
$version="a";
// Go do something we do not need to wait for.
exec("/usr/bin/php -f /path/to/beta.php > logfile.txt 2> errorfile.txt &");
?>
<html>
<head><title>Test</title></head>
<body>
<p>This is Alpha version <?php echo $version; ?></p>
</body>
</html>
beta.php
<?php
if (!($fp = fopen('/home/johnst12/public_html/workshops/admin/betadata.txt', 'w'))) { exit;}
fprintf($fp, "Proof that Beta executed.");
fclose($fp);
?>
If I run beta.php directly, it works fine. Betadata.txt gets the message.
If I run alpha.php to launch beta.php, betadata.txt is not created. logfile.txt and errorfile.txt remain empty (expected).
I am sure that the path to php, and the path to my php file are correct.
Googling for clarification has not been fruitful. A couple of common themes seem to be (a) running out of resources? (b) lack of permission on the target php file? Out of resources seems unlikely. The permission on the script is global read 644 (rw-r--r--). I tried adding execute (755) just in case it would help. It made no difference.
PHP version 5.3.21
Linux/Apache system.
safe_mode Off
What am I missing? Thanks.
First of all : Have you verified that /usr/bin/php is the correct path to PHP?
Php doesn't like running like that. Something to do with stdin. Try with nohup:
exec("nohup /usr/bin/php -f /path/to/beta.php > logfile.txt 2> errorfile.txt &");
With -f anything else that looks like a flag will go to PHP, so if you wanted to pass a "-x" option to your script then you'd have to
/usr/bin/php -f /path/to/beta.php -- -x
Without, options before the filename go to PHP and after go to the script.
/usr/bin/php /path/to/beta.php -x
I assume you've already looked at the two files in case they have output or errors?
A few other things to check:
Delete the two files. Are they recreated each time this code runs?
exec("nohup /usr/bin/php -v > logfile.txt &");
should output version information to that log file.
exec("/usr/bin/php -f /path/to/beta.php > logfile.txt 2> errorfile.txt");
should run the script properly (but not in the background).
I have a PHP script that listens on a queue. Theoretically, it's never supposed to die. Is there something to check if it's still running? Something like Ruby's God ( http://god.rubyforge.org/ ) for PHP?
God is language agnostic but it would be nice to have a solution that works on windows as well.
I had the same issue - wanting to check if a script is running. So I came up with this and I run it as a cron job. It grabs the running processes as an array and cycles though each line and checks for the file name. Seems to work fine. Replace #user# with your script user.
exec("ps -U #user# -u #user# u", $output, $result);
foreach ($output AS $line) if(strpos($line, "test.php")) echo "found";
In linux run ps as follows:
ps -C php -f
You could then do in a php script:
$output = shell_exec('ps -C php -f');
if (strpos($output, "php my_script.php")===false) {
shell_exec('php my_script.php > /dev/null 2>&1 &');
}
The above code lists all php processes running in full, then checks to see if "my_script.php" is in the list of running processes, if not it runs the process and does not wait for the process to terminate to carry on doing what it was doing.
Just append a second command after the script. When/if it stops, the second command is invoked. Eg.:
php daemon.php 2>&1 | mail -s "Daemon stopped" you#example.org
Edit:
Technically, this invokes the mailer right away, but only completes the command when the php script ends. Doing this captures the output of the php-script and includes in the mail body, which can be useful for debugging what caused the script to halt.
Simple bash script
#!/bin/bash
while [true]; do
if ! pidof -x script.php;
then
php script.php &
fi
done
Not for windows, but...
I've got a couple of long-running PHP scripts, that have a shell script wrapping it. You can optionally return a value from the script that will be checked in the shell-script to exit, restart immediately, or sleep for a few seconds -and then restart.
Here's a simple one that just keeps running the PHP script till it's manually stopped.
#!/bin/bash
clear
date
php -f cli-SCRIPT.php
echo "wait a little while ..."; sleep 10
exec $0
The "exec $0" restarts the script, without creating a sub-process that will have to unravel later (and take up resources in the meantime). This bash script wraps a mail-sender, so it's not a problem if it exits and pauses for a moment.
Here is what I did to combat a similar issue. This helps in the event anyone else has a parameterized php script that you want cron to execute frequently, but only want one execution to run at any time. Add this to the top of your php script, or create a common method.
$runningScripts = shell_exec('ps -ef |grep '.strtolower($parameter).' |grep '.dirname(__FILE__).' |grep '.basename(__FILE__).' |grep -v grep |wc -l');
if($runningScripts > 1){
die();
}
You can write in your crontab something like this:
0 3 * * * /usr/bin/php -f /home/test/test.php my_special_cron
Your test.php file should look like this:
<?php
php_sapi_name() == 'cli' || exit;
if($argv[1]) {
substr_count(shell_exec('ps -ax'), $argv[1]) < 3 || exit;
}
// your code here
That way you will have only one active instace of the cron job with my-special-cron as process key. So you can add more jobs within the same php file.
test.php system_send_emails sendEmails
test.php system_create_orders orderExport
Inspired from Justin Levene's answer and improved it as ps -C doesn't work in Mac, which I need in my case. So you can use this in a php script (maybe just before you need daemon alive), tested in both Mac OS X 10.11.4 & Ubuntu 14.04:
$daemonPath = "FULL_PATH_TO_DAEMON";
$runningPhpProcessesOfDaemon = (int) shell_exec("ps aux | grep -c '[p]hp ".$daemonPath."'");
if ($runningPhpProcessesOfDaemon === 0) {
shell_exec('php ' . $daemonPath . ' > /dev/null 2>&1 &');
}
Small but useful detail: Why grep -c '[p]hp ...' instead of grep -c 'php ...'?
Because while counting processes grep -c 'php ...' will be counted as a process that fits in our pattern. So using a regex for first letter of php makes our command different from pattern we search.
One possible solution is to have it listen on a port using the socket functions. You can check that the socket is still listening with a simple script. Even a monitoring service like pingdom could monitor its status. If it dies, the socket is no longer listening.
Plenty of solutions.. Good luck.
If you have your hands on the script, you can just ask him to set a time value every X times in db, and then let a cron job check if that value is up to date.
troelskn wrote:
Just append a second command after the script. When/if it stops, the second command is invoked. Eg.:
php daemon.php | mail -s "Daemon stopped" you#example.org
This will call mail each time a line is printed in daemon.php (which should be never, but still.)
Instead, use the double ampersand operator to separate the commands, i.e.
php daemon.php & mail -s "Daemon stopped" you#example.org
If you're having trouble checking for the PHP script directly, you can make a trivial wrapper and check for that. I'm not sufficiently familiar with Windows scripting to put how it's done here, but in Bash, it'd look like...
wrapper_for_test_php.sh
#!/bin/bash
php test.php
Then you'd just check for the wrapper like you'd check for any other bash script: pidof -x wrapper_for_test_php.sh
I have used cmder for windows and based on this script I came up with this one that I managed to deploy on linux later.
#!/bin/bash
clear
date
while true
do
php -f processEmails.php
echo "wait a little while for 5 secobds...";
sleep 5
done