Printing out one line to screen from popen in PHP? - php

I'm making a batch megadownload site in PHP. I'm parsing the input, validating it, then passing it to a bash script. I'm piping the output to the web page, but the contents of the webpage show like this.
8lX0JsBi.part01.rar: 0.00% - 0 bytes of 1000.0?MiB
8lX0JsBi.part01.rar: 0.09% - 917.7?KiB (939752 bytes) of 1000.0?MiB (916.7?KiB/s)
8lX0JsBi.part01.rar: 0.63% - 6.3?MiB (6566240 bytes) of 1000.0?MiB (5.3?MiB/s)
8lX0JsBi.part01.rar: 1.38% - 13.8?MiB (14430560 bytes) of 1000.0?MiB (7.0?MiB/s)
8lX0JsBi.part01.rar: 2.30% - 23.0?MiB (24129888 bytes) of 1000.0?MiB (9.0?MiB/s)
I'd like it to only show one line at a time that updates a single line.
8lX0JsBi.part01.rar: 2.30% - 23.0?MiB (24129888 bytes) of 1000.0?MiB (9.0?MiB/s)
Here is the function I'm using to pipe stdout to the webpage.
<?php
/**
* Execute the given command by displaying console output live to the user.
* #param string cmd : command to be executed
* #return array exit_status : exit status of the executed command
* output : console output of the executed command
*/
function liveExecuteCommand($cmd)
{
while (# ob_end_flush()); // end all output buffers if any
$proc = popen("$cmd 2>&1 ; echo Exit status : $?", 'r');
$live_output = "";
$complete_output = "";
while (!feof($proc))
{
$live_output = fread($proc, 4096);
$complete_output = $complete_output . $live_output;
echo "$live_output";
# flush();
}
pclose($proc);
// get exit status
preg_match('/[0-9]+$/', $complete_output, $matches);
// return exit status and intended output
return array (
'exit_status' => intval($matches[0]),
'output' => str_replace("Exit status : " . $matches[0], '', $complete_output)
);
}
?>

You have a few options:
Using clear, though this will clear the entire console.
system('clear');
echo '5%';
sleep(5);
system('clear');
echo '10%';
Or using /r, This will replace the last printed out line.
echo "5%";
sleep(5);
echo "\r10%";
You can also user cursor manipulation and character counting or the output buffer. But I think that will be a bit much for this.

Related

PHP not executing shell script and overloading apache

I want to execute my book generator script with php.
I tried to run my script on my linux terminal and all worked out fine.
sudo /home/repoadmin/apple/arduino.raamatu.generaator.sh
I used some execute function from stackoverflow to see what it echos during execute(also tried regular exec() and shell_exec()):
function liveExecuteCommand($cmd)
{
while (# ob_end_flush()); // end all output buffers if any
$proc = popen("$cmd 2>&1 ; echo Exit status : $?", 'r');
$live_output = "";
$complete_output = "";
while (!feof($proc))
{
$live_output = fread($proc, 4096);
$complete_output = $complete_output . $live_output;
echo "$live_output";
# flush();
}
pclose($proc);
// get exit status
preg_match('/[0-9]+$/', $complete_output, $matches);
// return exit status and intended output
return array (
'exit_status' => intval($matches[0]),
'output' => str_replace("Exit status : " . $matches[0], '', $complete_output)
);
}
define('__FILEDIR__','/home/repoadmin/apple');
chdir(__FILEDIR__);
readdir(__FILEDIR__);
liveExecuteCommand("sudo /home/repoadmin/apple/arduino.raamatu.generaator.sh");
closedir(__FILEDIR__);
I added www-data ALL=NOPASSWD:/home/repoadmin/apple/arduino.raamatu.generaator.sh to /etc/sudoer for executing with sudo.
My apache2 server freezes when I'm executing this php. When I restart my apache2, the output what I get is this:
/home/repoadmin/apple/arduino/arduino_raamat.pdf Could not open input file: /home/www/xxx/xxx/bookcreator.php?lang=et&toc=arduinotoc&book=book --2017-07-14 15:34:11-- http://xxx.xxx.xx/bookcreator.php Resolving xxx.xxx.xx (xxx.xxx.xx)... 195.xxx.xxx.xxx Connecting to xxx.xxx.xx (xxx.xxx.xx)|195.xxx.xxx.xxx|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: '/home/repoadmin/apple/bookcreator_output.html' 0K .
What could be the problem and how can I fix it?

Trying in PHP to mysqldump -v to popen and not getting output

I'm writing a wrapper for mysqldump and want to show the output nicely in a PHP CLI application.
I'm attempting to run mysqldump -v using popen so that I can get the verbose output and display progress indicators to the user. However no output is returned (by default it gets logged to the screen via stdErr).
I tried adding 2>&1 to the command to push the verbose output from stdErr to stdOut, but fread still doesn't get any output even though the output nolonger goes to the screen via stdErr.
$cmd = "mysqldump -uroot -proot -v dbname 2>&1 | mysql -uroot -proot dbname2";
$handle = popen($cmd, "r");
$buffer = "";
while ($handle && !feof($handle)){
$output = fread($handle, 100);
$buffer .= $output;
echo sprintf("Buffer: %s\n", $buffer);
}
pclose($handle);
Should I use something else instead of popen? Or am I simply incorrectly redirecting the output?
You seem to actually pipe the mysqldump data into mysql, in which case it might be a bad idea to include error messages into the pipe.
Of course, in this scenario, you cannot capture the mysqldump's output.
You should use the tee command:
mysqldump -uroot -proot -v dbname | tee /tmp/output | mysql -uroot -proot dbname2
This way, you can have the output both in pipe for mysql and /tmp/output.
In such a way you can then fopen /tmp/output for results.
Please note that you might not have the possible errors in the output, as mysql would not be happy seeing them later down the pipe.
I figured it out, without having to use a file as a stream buffer.
/**
* PROCESS
*
* Process the command
*
* #param int $buffer The size of the buffer
* #param int $pipe The pipe to subscribe to (0=stdIn, 1=stdOut, 2=stdErr)
*
* #return bool Success or not
*/
public function process($buffer=10, $pipe=1) {
$handle = proc_open(
$this->command,
[
["pipe","r"],
["pipe","w"],
["pipe","w"]
],
$pipes
);
if (!is_resource($handle)) {
return false;
}
$output = "";
$buffer = "";
$hasLineCallbacks = count($this->onNewLine);
while ($buffer = fread($pipes[$pipe], 10)) {
$output .= $buffer;
if (preg_match("/[\r\n]/", $output)) {
$segments = preg_split("/[\r\n]+/", $output);
while (count($segments) > 1) {
$line = array_shift($segments);
if (!$hasLineCallbacks) { continue; }
foreach ($this->onNewLine as $callback) {
if (!$callback["pattern"] || preg_match($callback["pattern"], $line)) {
call_user_func($callback["callback"], $line);
}
}
}
$output = array_shift($segments);
}
}
proc_close($handle);
return true;
}
I'm basically making Background class to run a terminal command and pipe the output to callback functions. It obviously still has a long way to go though.
Thanks for your help, #Victor

Run shell command asynchronously using PHP and node

I need to execute a shell program that will run a rather long process and I dont want to wait until that process has ended for my PHP script to carry on execution. So far i tried:
1:Pure PHP
exec("longCommand &");
2:Node and php
exec("/usr/local/bin/node nodeLauncher.js &");
Node:
var spawn = require('child_process').spawn,
proc = spawn('longCommand', ['&']);
console.log('return');
In both cases the script carry on execution only after the "longCommand" has returned. Am I doing something wrong?
From PHP's page on exec():
If a program is started with this function, in order for it to
continue running in the background, the output of the program must be
redirected to a file or another output stream. Failing to do so will
cause PHP to hang until the execution of the program ends.
That means, unless you direct the output to a file, exec() is blocking and will pause execution of your PHP script until the command you issued exits.
You can redirect the output to a file, or if you don't care about the output, redirect it to /dev/null.
Finally, yet another alternate could be to fork a new PHP process and exec the command from there. You can fork a new PHP process using pcntl_fork.
for node try passing detached option
var spawn = require('child_process').spawn,
proc = spawn('longCommand', ['&'], { detached: true } );
Node documentation on spawn
Although my filenames used here seems weird, why dont try to look at my working prototype of the raw code below... i can't post the other parts dude as I have attached to it my private DB passwords..eheheh
LINK: http://affiliateproductpromotions.net/sml1r.php
<?php
if(isset($_GET['y']))
$y =false;
else $y =true;
if(isset($_GET['count']))
{
echo getCount($_GET['f'],$y);
exit;
}
if(isset($_GET['stop']) && $_GET['stop']=='true')
{
$fr=fopen("huhu.txt","w");
fwrite($fr,"<script>document.getElementById('send').disabled=false;document.getElementById('stop').disabled=true;document.getElementById('process').innerHTML='<b style=color:GREY>Current Status: Stopped!</b>';document.getElementById('stop').style='width:90px;color:LIGHTYELLOW;background-color:GREY';document.getElementById('send').style='width:90px;color:LIGHTYELLOW;background-color:BLUE';</script>");
fclose($fr);
include('../semail/killexec.php');
sleep(2);
//exit;
}
else
{
header("Connection: close");
ignore_user_abort(); // optional
ob_start();
echo ('Text the user will see');
$size = ob_get_length();
header("Content-Length: $size");
function run_in_background($Command, $Priority = 0)
{
if($Priority)
$PID = shell_exec("nohup nice -n $Priority $Command > /dev/null 2>&1 & echo $!");
else
$PID = shell_exec("nohup $Command > /dev/null 2>&1 & echo $!");
return($PID);
}
function is_process_running($PID)
{
exec("ps $PID", $ProcessState);
return(count($ProcessState) >= 2);
}
//ob_end_clean();
echo("Running hmmsearch. . .");
$ps = run_in_background("hmmsearch $hmmfile $fastafile > $outfile");
$fpf = fopen("pid.txt","w");
fwrite($fpf,exec('ps '.$ps));
fclose($fpf);
while($i<=getCount())
{
$fp2 = fopen("sent1email.txt","w");
fwrite($fp2,getEmailSent($i));
fclose($fp2);
$fp = fopen("haha.txt","w");
fwrite($fp,"$i\n");
// echo("<br> [ ".$i++." ] ");
// ob_flush(); flush();
$i++;
sleep(2);
if($i==getCount())
{
$fr=fopen("huhu.txt","w");
fwrite($fr,"<script>document.getElementById('send').disabled=false;document.getElementById('stop').disabled=true;document.getElementById('process').innerHTML='<b style=color:GREY>Current Status: Finished Sending!</b>';document.getElementById('stop').style='width:90px;color:LIGHTYELLOW;background-color:GREY';document.getElementById('send').style='width:90px;color:LIGHTYELLOW;background-color:BLUE';</script>");
fclose($fr);
sleep(1);
include('../semail/killexec.php');
}
if($i<getCount())
{
$fr=fopen("huhu.txt","w");
fwrite($fr,"<script>document.getElementById('send').disabled=true;document.getElementById('stop').disabled=false;document.getElementById('process').innerHTML='<b style=color:GREY>Current Status: Sending...</b>';document.getElementById('send').style='width:90px;color:LIGHTYELLOW;background-color:GREY';document.getElementById('stop').style='width:90px;color:LIGHTYELLOW;background-color:RED';</script>");
fclose($fr);
sleep(2);
}
}
fclose($fp);
//sleep(1);
ob_end_flush(); // <-- this trash will not work
flush(); // <--- if this garbage dont exist
sleep(5);// <-- but dont worry, a collector is here...
}
?>

How to prevent PHP script running more than once?

Currently, I tried to prevent an onlytask.php script from running more than once:
$fp = fopen("/tmp/"."onlyme.lock", "a+");
if (flock($fp, LOCK_EX | LOCK_NB)) {
echo "task started\n";
//
while (true) {
// do something lengthy
sleep(10);
}
//
flock($fp, LOCK_UN);
} else {
echo "task already running\n";
}
fclose($fp);
and there is a cron job to execute the above script every minute:
* * * * * php /usr/local/src/onlytask.php
It works for a while. After a few day, when I do:
ps auxwww | grep onlytask
I found that there are two instances running! Not three or more, not one. I killed one of the instances. After a few days, there are two instances again.
What's wrong in the code? Are there other alternatives to limit only one instance of the onlytask.php is running?
p.s. my /tmp/ folder is not cleaned up. ls -al /tmp/*.lock show the lock file was created in day one:
-rw-r--r-- 1 root root 0 Dec 4 04:03 onlyme.lock
You should use x flag when opening the lock file:
<?php
$lock = '/tmp/myscript.lock';
$f = fopen($lock, 'x');
if ($f === false) {
die("\nCan't acquire lock\n");
} else {
// Do processing
while (true) {
echo "Working\n";
sleep(2);
}
fclose($f);
unlink($lock);
}
Note from the PHP manual
'x' - Create and open for writing only; place the file pointer at the
beginning of the file. If the file already exists, the fopen() call
will fail by returning FALSE and generating an error of level
E_WARNING. If the file does not exist, attempt to create it. This is
equivalent to specifying O_EXCL|O_CREAT flags for the underlying
open(2) system call.
And here is O_EXCL explanation from man page:
O_EXCL - If O_CREAT and O_EXCL are set, open() shall fail if the file
exists. The check for the existence of the file and the creation of
the file if it does not exist shall be atomic with respect to other
threads executing open() naming the same filename in the same
directory with O_EXCL and O_CREAT set. If O_EXCL and O_CREAT are set,
and path names a symbolic link, open() shall fail and set errno to
[EEXIST], regardless of the contents of the symbolic link. If O_EXCL
is set and O_CREAT is not set, the result is undefined.
UPDATE:
More reliable approach - run main script, which acquires lock, runs worker script and releases the lock.
<?php
// File: main.php
$lock = '/tmp/myscript.lock';
$f = fopen($lock, 'x');
if ($f === false) {
die("\nCan't acquire lock\n");
} else {
// Spawn worker which does processing (redirect stderr to stdout)
$worker = './worker 2>&1';
$output = array();
$retval = 0;
exec($worker, $output, $retval);
echo "Worker exited with code: $retval\n";
echo "Output:\n";
echo implode("\n", $output) . "\n";
// Cleanup the lock
fclose($f);
unlink($lock);
}
Here goes the worker. Let's raise a fake fatal error in it:
#!/usr/bin/env php
<?php
// File: worker (must be executable +x)
for ($i = 0; $i < 3; $i++) {
echo "Processing $i\n";
if ($i == 2) {
// Fake fatal error
trigger_error("Oh, fatal error!", E_USER_ERROR);
}
sleep(1);
}
Here is the output I got:
galymzhan#atom:~$ php main.php
Worker exited with code: 255
Output:
Processing 0
Processing 1
Processing 2
PHP Fatal error: Oh, fatal error! in /home/galymzhan/worker on line 8
PHP Stack trace:
PHP 1. {main}() /home/galymzhan/worker:0
PHP 2. trigger_error() /home/galymzhan/worker:8
The main point is that the lock file is cleaned up properly so you can run main.php again without problems.
Now I check whether the process is running by ps and warp the php script by a bash script:
#!/bin/bash
PIDS=`ps aux | grep onlytask.php | grep -v grep`
if [ -z "$PIDS" ]; then
echo "Starting onlytask.php ..."
php /usr/local/src/onlytask.php >> /var/log/onlytask.log &
else
echo "onlytask.php already running."
fi
and run the bash script by cron every minute.
<?php
$sLock = '/tmp/yourScript.lock';
if( file_exist($sLock) ) {
die( 'There is a lock file' );
}
file_put_content( $sLock, 1 );
// A lot of code
unlink( $sLock );
You can add an extra check by writing the pid and then check it within file_exist-statement.
To secure it even more you can fetch all running applications by "ps fax" end check if this file is in the list.
try using the presence of the file and not its flock flag :
$lockFile = "/tmp/"."onlyme.lock";
if (!file_exists($lockFile)) {
touch($lockFile);
echo "task started\n";
//
// do something lengthy
//
unlink($lockFile);
} else {
echo "task already running\n";
}
You can use lock files, as some have suggested, but what you are really looking for is the PHP Semaphore functions. These are kind of like file locks, but designed specifically for what you are doing, restricting access to shared resources.
Never use unlink for lock files or other functions like rename. It's break your LOCK_EX on Linux. For example, after unlink or rename lock file, any other script always get true from flock().
Best way to detect previous valid end - write to lock file few bytes on the end lock, before LOCK_UN to handle. And after LOCK_EX read few bytes from lock files and ftruncate handle.
Important note: All tested on PHP 5.4.17 on Linux and 5.4.22 on Windows 7.
Example code:
set semaphore:
$handle = fopen($lockFile, 'c+');
if (!is_resource($handle) || !flock($handle, LOCK_EX | LOCK_NB)) {
if (is_resource($handle)) {
fclose($handle);
}
$handle = false;
echo SEMAPHORE_DENY;
exit;
} else {
$data = fread($handle, 2);
if ($data !== 'OK') {
$timePreviousEnter = fileatime($lockFile);
echo SEMAPHORE_ALLOW_AFTER_FAIL;
} else {
echo SEMAPHORE_ALLOW;
}
fseek($handle, 0);
ftruncate($handle, 0);
}
leave semaphore (better call in shutdown handler):
if (is_resource($handle)) {
fwrite($handle, 'OK');
flock($handle, LOCK_UN);
fclose($handle);
$handle = false;
}
Added a check for old stale locks to galimzhan's answer (not enough *s to comment), so that if the process dies, old lock files would be cleared after three minutes and let cron start the process again. That's what I use:
<?php
$lock = '/tmp/myscript.lock';
if(time()-filemtime($lock) > 180){
// remove stale locks older than 180 seconds
unlink($lock);
}
$f = fopen($lock, 'x');
if ($f === false) {
die("\nCan't acquire lock\n");
} else {
// Do processing
while (true) {
echo "Working\n";
sleep(2);
}
fclose($f);
unlink($lock);
}
You can also add a timeout to the cron job so that the php process will be killed after, let's say 60 seconds, with something like:
* * * * * user timeout -s 9 60 php /dir/process.php >/dev/null

allow php invoked by exec() to write to console

I invoked a php file by exec() doing this :
<?php
$cmd = "php -q nah.php";
exec($cmd);
echo "lalal";
?>
and the nah.php has this :
<?php
echo "yaya";
sleep(3);
?>
It does sleep for 3 seconds but the file can echo out to the command.
How can I do echo the output from nah.php
If you want to capture the output of another process, then you can use backticks
$output=`command line`;
or capture it with exec()
exec('command line', $output);
However, both of these techniques only give the output when the external process has run to completion. If you want to grab the output as it happens, you can use popen (or proc_open for more control), e.g. something like this
$handle = popen('command line 2>&1', 'r');
while (!feof($handle))
{
$read = fread($handle, 2096);
echo $read;
}
pclose($handle);
The 2>&1 at the end of the command line is a handy idiom if running within a shell like bash. It redirects stderr to stdout, so any errors the command generates will be returned to PHP. There are more advanced ways to capture stderr, this just makes it easy.
Change your first script to:
<?php
$cmd = "php -q nah.php";
echo `$cmd`;
echo "lalal";
exec() creates a new console and executes the php file there, it returns the output in an array (One element per line return) that is passed in as a second argument. So if you change your code to:
<?php
$output = array();
$cmd = "php -q nah.php";
exec($cmd, $output);
$lines = count($output);
for($i = 0; $i < $lines; $i++)
echo $output[$i];
echo "lalal";
?>

Categories