get real time Result shell using PHP [duplicate] - php

This question already has answers here:
PHP reading shell_exec live output
(3 answers)
Closed 6 years ago.
I have to execute a batch file (I.e. .bat ) and get the result on an HTML page.
I used this PHP code to get a result:
<?php
echo "<br>";
$result = shell_exec('start file.bat');
iconv("CP850","UTF-8",$result);
echo "<pre>$result </pre>";
?>
Now the problem is that I get a result only when the batch file execution finishes, and I want to have the result in real time, like running via command line.

Found in the comments of the shell_exec documentation's page of PHP
If you're trying to run a command such as "gunzip -t" in shell_exec and getting an empty result, you might need to add 2>&1 to the end of the command, eg:
Won't always work:
echo shell_exec("gunzip -c -t $path_to_backup_file");
Should work:
echo shell_exec("gunzip -c -t $path_to_backup_file 2>&1");
In the above example, a line break at the beginning of the gunzip output seemed to prevent shell_exec printing anything else. Hope this saves someone else an hour or two.
I believe it is what you really have to do.
Reference: http://php.net/manual/en/function.shell-exec.php#106250

As already mentioned in the comments to your question you need to fork a process in such way that you can communicate with it. That is not possible with imple functions like exec() and the like.
Instead take a look at this simple example:
File test.sh:
#!/bin/bash
echo start counting ...
for counter in 1 2 3
do
echo * counter is at value $counter *
sleep 3
done
echo ... finished counting.
File test.php:
<?php
$descriptorspec = array(
0 => array('pipe', 'r'),
1 => array('pipe', 'w'),
);
echo "forking process ...\n";
$process = proc_open('./test.sh', $descriptorspec, $pipes);
if (is_resource($process)) {
echo "... successfully forked process ...\n";
while (!feof($pipes[1])) {
echo fread($pipes[1], 1024);
flush();
}
echo "... process has finished ...\n";
fclose($pipes[0]);
fclose($pipes[1]);
echo "... pipes closed ...\n";
proc_close($process);
echo "... process closed.\n";
} else {
echo "... failed to fork process!\n";
}
The obvious output of a test run of that php script is:
forking process ...
... successfully forked process ...
start counting ...
* counter is at value 1 *
* counter is at value 2 *
* counter is at value 3 *
... finished counting.
... process has finished ...
... pipes closed ...
... process closed.
But the interesting part here is that this output is not sent in one go once that forked process has finished, but in what you referred to as "a live manner". So the first four lines appear immediately, the next two with a 3 second delay each, then the rest of the output.
Please note that the above example is meant as a demonstration for a Linux CLI environment. So it does not care about html markup but outputs plain text and relies on bash as a shell environment for the forked process. You will have to adapt that simple demonstration to your needs, obviously.

Related

PHP exec() function only runs extremely short Python scripts

I'm having some trouble using the PHP exec() function. Whenever the script I'm attempting to run is short, exec() works just fine. But if the script takes any more than a second or so, it fails. Note, I've attempted run the long script manually on the command line, and it works just fine. It seems as though the PHP interpreter is killing my external script if it takes any longer than a second or so to run. Any thoughts or suggestions? Here is my code:
<?php
$fileName = "foobar.docx";
$argVar = $fileName;
exec("python3 /var/www/html/jan8/alexandrina.py /var/www/html/jan8/$argVar");
echo "$output";
?>
And here is my script:
#!/usr/bin/env python3
import docx
import sys
docxFile = "".join(sys.argv[1:])
# The Three Lines Below Create a New Variable "htmlFile"
# "htmlFile" is the same as "docxFile", except ".docx" is cut off
# and replaced with ".html"
myNumber = len(docxFile) - 5
htmlFile = docxFile[0:myNumber]
htmlFile = htmlFile + '.html'
def generateHTML(filename):
doc = docx.Document(filename)
fullText = []
for para in doc.paragraphs:
fullText.append('<p>')
fullText.append(para.text)
fullText.append('</p>')
fullText.append('\n')
return '\n'.join(fullText)
file = open(htmlFile, "w")
file.write(generateHTML(docxFile))
file.close()
print("end python script")
Additional notes: I've increased the max execution time limits in php.ini, but I don't think that should matter, as the "long script" should only take a few seconds to run. Also, when I refer to the "short script", and the "long script", I'm actually referring to the same script. The difference between the "long script" and the "short script" is just the time to execute as it may vary depending on the size of the file I'm asking the script to process. Anyway... any suggestions would really be appreciated!
Ordinarily, php exec function should block until the command you run has completed. I.e., the PHP script will halt, waiting for the command to finish until continuing with the rest of your script. I was half thinking that your server was experiencing a max_execution_time timeout, but you've clearly stated that even just a couple of seconds is too long and even these fairly short scripts are having trouble.
A couple of solutions occur to me. The simplest one is to alter the python command so that a) any output is routed to a file or output stream and b) the process is run in the background. According to the docs on exec:
If a program is started with this function, in order for it to continue running in the background, the output of the program must be redirected to a file or another output stream. Failing to do so will cause PHP to hang until the execution of the program ends.
I also would like you to make use of the two additional optional parameters for the exec function.
$fileName = "foobar.docx";
$argVar = $fileName;
$cmd = "python3 /var/www/html/jan8/alexandrina.py /var/www/html/jan8/$argVar";
// modify your command to toss output, background the process, and output the process id
$cmd_modified = $cmd . " >/dev/null & echo \$!";
$cmd_output = NULL; // this will be an array of output
$cmd_return_value = NULL; // this will be the return value of the script
exec($cmd_modified, $cmd_output, $cmd_return_value);
echo "exec has completed</br>";
echo "output:<br>" . print_r($cmd_output, TRUE) . "<br>";
echo "return value: " . print_r($cmd_return_value, TRUE);
This may help or may not. If it does not, we still might be able to solve the problem using posix commands.
EDIT: according to crispytx, the long scripts are resulting in a $cmd_return_val of 1 which means an error is happening. Try changing this one line:
$cmd_modified = $cmd . " >/dev/null & echo \$!";
to this
$cmd_modified = $cmd . " & echo \$!";
And let us know what the output of $cmd_output is -- it should at the very least have the process id of the newly spawned process.
Thanks for all the help S. Imp. I had a little trouble debugging using your suggestions because I happened to be using AJAX to call the script. However, I wrote simpler script using your suggestions to try and debug the problem and this is what I found:
Array ( [0] => Traceback (most recent call last): [1] => File "/var/www/html/jan8/alexandrina.py", line 28, in [2] => file.write(generateHTML(docxFile)) [3] => UnicodeEncodeError: 'ascii' codec can't encode character '\u2026' in position 25: ordinal not in range(128) )
So it looks like the problem has to do with ascii encoding! Even though the larger file was just a docx file with the same text as the shorter docx file repeated over and over again for 300 pages. It seems that if a docx file exceeds 1 pages, ascii characters are inserted that aren't present in single page docx files. I have no idea if this post will ever end up helping anyone, but who knows!
[SOLVED]

Running Stockfish chess engine under php script

I'm thinking about making a php script which opens stockfish chess engine CLI, send fews commands and get back the output.
I think I can achieve this by using proc_open with pipes array but I can't figure out how to wait the whole output... If there's another better solution it's appreciated!
Here's my code:
// ok, define the pipes
$descr = array(
0 => array("pipe", "r"),
1 => array("pipe", "w"),
2 => array("pipe", "w")
);
$pipes = array();
// open the process with those pipes
$process = proc_open("./stockfish", $descr, $pipes);
// check if it's running
if (is_resource($process)) {
// send first universal chess interface command
fwrite($pipes[0], "uci");
// send analysis (5 seconds) command
fwrite($pipes[0], "go movetime 5000");
// close read pipe or STDOUTPUT can't be read
fclose($pipes[0]);
// read and print all output comes from the pipe
while (!feof($pipes[1])) {
echo fgets($pipes[1]);
}
// close the last opened pipe
fclose($pipes[1]);
// at the end, close the process
proc_close($process);
}
The process seems to start, but the second STDINPUT I send to process isn't able to wait until it finishes because the second command produces analysis lines for about 5 seconds and the result it prints it's immediate.
How can I get on it?
CLI link, CLI documentation link
Please, ask me for more information about this engine if you need.
Thank you!
fwrite($pipes[0], "uci/n");
fwrite($pipes[0], "go movetime 5000/n");
Without /n Stockfish see this as one command (ucigo movetime 5000) and don't recognise it.
Actually, your code works. $pipes[1] contained all the output from stockfish...
You might need to change the line
position startpos moves 5000
to a different number, as the 5000 means 5000 ms = 5 seconds, ie. the time when the engine stops. Try 10000 and the engine stops after 10 seconds etc.
You need to remove: fclose($pipes[0]); and make check for bestmove in while cycle, if bestmove is found - break the cycle, and after that only put fclose($pipes[0]);. That worked for me. And add \n separator at the end of commands.
Thanks for the code!

Running at from PHP gives no output

I've been wrestling with exec(), trying to capture the output from it when I add a task using the at Unix system command. My problem is that it is giving no output when run from my script, however running it from the terminal and PHP in interactive mode prints out a couple of lines.
The command I want to execute is this:
echo exec("echo 'php -f /path/to/file.php foo=1' | at now + 1 minutes", $result);
var_dump() gives string(0) "", and print_r() spits out Array (). I've tried using shell_exec(), which outputs NULL, however the following outputs hi when run in a web page context:
echo exec("echo 'hi'");
This also outputs stuff:
echo exec("atq");
However, as soon as I use at, nothing is output. How can I get the output of:
exec("echo 'php -f /path/to/file.php foo=1' | at now + 1 minutes", $result);
Because at present it outputs nothing when run as "normal" by PHP through Apache, however running the command in the terminal as well as in PHP's interactive console gives me the expected result of something like:
php > echo exec("echo 'php -f /path/to/file.php foo=1' | at now + 1 minutes", $result);
warning: commands will be executed using /bin/sh
job 1219 at Sun Jun 10 12:43:00 2012
safe_mode is off, and I cannot work out why I don't get any output from at with a piped-in echo statement, when executing atq or any other commend with exec() gives me output. I've searched and read this question, all to no avail.
How can I get exec() to return the output from at to either a string, or an array if using a second argument with exec()?
Working, one line solution
I didn't realise it could be this simple. All that is required is to reroute stderr to stdout by putting 2>&1 at the end of the command to execute. Now any output from at is printed to stdout, therefore captured by exec():
echo exec("echo 'php -f /path/to/file.php foo=1' | at now + 1 minutes 2>&1", $result);
My old solution:
I was trying to keep to a one/two line solution, however the only thing that worked in the end was using proc_open() because at logs to stderr, which exec() doesn't read! I'd like to thank #Tourniquet for pointing this out, however he has deleted his answer. To quote:
As far as i can see, at outputs to stderr, which isn't captured by
exec. I'm not really confident in it, but consider using
http://php.net/manual/en/function.proc-open.php, which allows you to
direct stderr to its own pipe.
This is actually the correct way of doing things. My solution (because I only want stderr) was to do this:
// Open process to run `at` command
$process = proc_open("echo 'php -f /path/to/file.php foo=1' | at now + 1 minutes", array(2 => array("pipe", "w")), $pipes);
// Get stuff from stderr, because `at` prints out there for some odd reason
if(is_resource($process)) {
$output = stream_get_contents($pipes[2], 100);
fclose($pipes[2]);
$return_value = proc_close($process);
}
$output now contains whatever at printed to stderr (which should really go to stdout because it's not an error), and $return_value contains 0 on success.
Here a more complex solution with proc_open. I'm writing this answer because, in my case, the '2>&1' workaround doesn't work.
function runCommand($bin, $command = '', $force = true)
{
$stream = null;
$bin .= $force ? ' 2>&1' : '';
$descriptorSpec = array
(
0 => array('pipe', 'r'),
1 => array('pipe', 'w')
);
$process = proc_open($bin, $descriptorSpec, $pipes);
if (is_resource($process))
{
fwrite($pipes[0], $command);
fclose($pipes[0]);
$stream = stream_get_contents($pipes[1]);
fclose($pipes[1]);
proc_close($process);
}
return $stream;
}
Usage examples:
// getting the mime type of a supplied file
echo runCommand('file -bi ' . escapeshellarg($file));
Another example using the command parameter:
// executing php code on the fly
echo runCommand('/usr/bin/php', '<?php echo "hello world!"; ?>');
Another example using the force parameter (this can be useful for commands that will change the output during the execution process):
// converting an mp3 to a wav file
echo runCommand('lame --decode ' . escapeshellarg($source) . ' ' . escapeshellarg($dest), '', true);
I hope this helps :-)
Try to create a minimum (non-)working example. Break everything down, and test only one thing at a time.
Here is one error in your bash:
hpek#melda:~/temp$ echo 'php -f /path/to/file.php foo=1' | at now + 1 minutes
at: pluralization is wrong
job 22 at Sun Jun 10 14:48:00 2012
hpek#melda:~/temp$
Write minute instead og minutes.
My output from at is send to me by mail!!

always blocked on reading from pipe opened through php's proc_open when used with stream_select

I'm talking to a process requiring user interaction using the following (PHP 5.3/ Ubuntu 12.04),
$pdes = array(
0 => array('pipe', 'r'), //child's stdin
1 => array('pipe', 'w'), //child's stdout
);
$process = proc_open($cmd, $pdes, $pipes);
sleep(1);
if(is_resource($process)){
while($iter-->0){
$r=array($pipes[1]);
$w=array($pipes[0]);
$e=array();
if(0<($streams=stream_select($r,$w,$e,2))){
if($streams){
if($r){
echo "reading\n";
$rbuf.=fread($pipes[1],$rlen); //reading rlen bytes from pipe
}else{
echo "writing\n";
fwrite($pipes[0],$wbuf."\n"); //writing to pipe
fflush($pipes[0]);
}}}}
fclose($pipes[0]);
fclose($pipes[1]);
echo "exitcode: ".proc_close($process)."\n";
}
And this is my test program in C,
#include <stdio.h>
int main(){
char buf[512];
printf("before input\n");
scanf("%s",buf);
printf("after input\n");
return 0;
}
Now, the problem is $r is always empty after stream_select even if $pipes[1] is set to non-blocking where as write to $pipes[0] never blocks. However, things work fine without stream_select i.e. if I match reads and writes to the test program,
echo fread($pipes[1],$rlen); //matching printf before input
fwrite($pipes[0],$wbuf."\n"); //matching scanf
fflush($pipes[0]);
echo fread($pipes[1],$rlen); //matching printf after input
I couldn't figure out what's happening here. I'm trying to achieve something sort of web based terminal emulator here. Any suggestions on how to do this are welcome :)
Sorry guys for wasting your time. I figured out the problem a while later (sorry for the late update). Read was blocking due to a race condition.
I write to the process and immediately check the streams for availability. Write never blocks and data is not ready for reading yet (somehow even for 1 byte of data to be available it took 600ms). So, I was able to fix the problem by adding sleep(1) at the end of write block.

Shell output not being fully retrieved by PHP!

I have a PHP script which executes a shell command:
$handle = popen('python last', 'r');
$read = fread($handle, 4096);
print_r($read);
pclose($handle);
I echo the output of the shell output. When I run this in the command I get something like this:
[root#localhost tester]# python last
[last] ZVZX-W3vo9I: Downloading video webpage
[last] ZVZX-W3vo9I: Extracting video information
[last] ZVZX-W3vo9I: URL: x
[download] Destination: here.flv
[download] 0.0% of 10.09M at ---b/s ETA --:--
[download] 0.0% of 10.09M at 22.24k/s ETA 07:44
[download] 0.0% of 10.09M at 66.52k/s ETA 02:35
[download] 0.1% of 10.09M at 154.49k/s ETA 01:06
[download] 0.1% of 10.09M at 162.45k/s ETA 01:03
However, when I run that same command from PHP I get this output:
[last] ZVZX-W3vo9I: Downloading video webpage
[last] ZVZX-W3vo9I: Extracting video information
[last] ZVZX-W3vo9I: URL: x
[download] Destination: here.flv
As you can see the bottom bit is missing which is the bit I need!! The problem before was that the percentages were being updated on the same line but now I have changed my Python script so that it creates a new line. But this made difference! :(
This question is related to this one.
Thank you for any help.
Update
Needed to redirect output "2>&1". Arul got lucky :P since I missed the deadline to pick the one true answer which belonged to Pax!
You read only the first 4,096 bytes from the pipe, you'll need to place the fread/print_r in a loop and check for the end-of-file using the feof function.
$handle = popen('python last', 'r');
while(!feof($handle))
{
print_r(fread($handle, 4096));
}
pclose($handle);
The first step is to see where the output is going. The first thing I would do is choose a slightly smaller file so that you're not waiting around for seven minutes for each test.
Step 1/ See where things are being written in the shell. Execute the command python last >/tmp/stdout 2>/tmp/stderr then look at those two files. Ideally, everything will be written to stdout but that may not be the case. This gives you the baseline behavior of the script.
Step 2/ Do the same thing when run from PHP by using $handle = popen('python last >/tmp/stdout 2>/tmp/stderr', 'r');. Your PHP script probably won't get anything returned in this case but the files should still be populated. This will catch any changed behavior when running in a non-terminal environment.
If some of the output goes to stderr, then the solution should be as simple as $handle = popen('python last 2>&1', 'r');
Additionally, the doco for PHP states (my bolding):
Returns a file pointer identical to that returned by fopen(), except that it is unidirectional (may only be used for reading or writing) and must be closed with pclose(). This pointer may be used with fgets(), fgetss(), and fwrite().
So I'm not sure you should even be using fread(), although it's shown in one of the examples. Still, I think line-based input maps more to what you're trying to achieve.
Irrespective of all this, you should read the output in a loop to ensure you can get the output when it's more than 4K, something like:
$handle = popen ('python last 2>&1', 'r');
if ($handle) {
while(! feof ($handle)) {
$read = fgets ($handle);
echo $read;
}
pclose ($handle);
}
Another thing to look out for, if you're output is going to a browser and it takes too long, the browser itself may time out since it thinks the server-side connection has disappeared. If you find a small file working and your 10M/1minute file not working, this may be the case. You can try flush() but not all browsers will honor this.
It is much much easier to do this:
$output = `python last`;
var_dump($output);
The 'ticks' (`) will execute the line and capture the output. Here is a test example:
File test.php:
<?php
echo "PHP Output Test 1\n";
echo "PHP Output Test 2\n";
echo "PHP Output Test 3\n";
echo "PHP Output Test 4\n";
echo "PHP Output Test 5\n";
?>
File capture.php:
<?php
$output = `php test.php`;
var_dump($output);
?>
Output from php capture.php:
string(80) "Test PHP Script
Test PHP Script
Test PHP Script
Test PHP Script
Test PHP Script
"
You can then split the output into an array based on line breaks:
$outputArray = explode('\n', $output);
OR use proc_open(), which gives you much more control than popen as you can specify where you want stdin, stdout and stderr to be handled.
OR you can fopen STDIN or php://stdin and then pipe to php:
? python last | php script.php
I would go with option 1 and using the backticks, easiest way.
I would not be surprised to find that the progress report is omitted when the output is not going to a tty. That is, the PHP is capturing everything that is sent, but the progress report is not being sent.
There is ample precedent for commands behaving differently depending on where the output goes - starting with the good old ls command.
Of course, if you wrote the Python script that you're running, this is much less likely to be the cause of the trouble.
How can you verify whether this hypothesis is valid? Well, you could start by running the script at the command line with the standard output going to one file and the standard error going to another. If you see the progress information in one of those files, you know a whole lot more about what is going on. If you see the progress information on the screen still, then the script is probably echoing the progress information to /dev/tty, but when PHP runs it, there is no /dev/tty for the process. If you don't see the progress information at all (on screen or in a file), then my original hypothesis is probably verified.
Try running stream_get_contents() on $handle. It's a better way to work with resources where you don't know the exact size of what you're trying to retrieve.
Could you read in a while loop instead of using fread()?
while( !feof($handle) )
echo fgets($handle);
You may have to flush() also.
What do you see with
python last | less
?
Maybe the output you want is emitted on STDERR. Then you have to start it this way:
$handle = popen('python last 2>&1', 'r');
The 2>&1 directs STDERR into STDOUT, which you are capturing with popen.
If you're just trying to show the progress of the python command inside a terminal window, then I would recommend the following instead:
<?php
system("python last > `tty`");
You won't need to capture the output then, and the user can even Ctrl+C the download without aborting the PHP script.
You are missing a flush call. (In your python app, and possibly your php app aswell)
That is because when you use standard stream stdin/stdout interactively (from cmdline) they are in a line-buffered mode (in short system flushes on each new line), but when you call it from within your program streams are in a fully buffered mode(doesn't output till system buffer is full).
More info on this here buffering in standard streams

Categories