I have a PHP script that calls a Python script which loops through a text file outputting specific lines to the webpage. The PHP script calls the Python script and echoes its output like this...
$output = shell_exec($command);
echo ($output);
The relevant Python code is this...
for myStr in myList:
myObj=re.match(r'(\s|\S)+\[(?P<specificVal>\d+)(\s|\S)+',myStr)
specificVal = int(myObj.group('specificVal'))
if specificVal in range(min,max):
print ('<p>%s</p>' % (myStr))
This works for about a hundred lines then it stops outputting, when there might be as many as a few thousand. Is there a better way to do this, or do I just need to edit some php.ini options? Thank you!
This could have to do with shell_exec() not returning all of the output for some reason. You could try using the more-robust proc_open():
<?php
$fileDescriptors = array(
0 => array("pipe", "r"), // STDIN
1 => array("pipe", "w"), // STDOUT
2 => array("pipe", "w") // STDERR
);
$pipes = array();
flush();
$process = proc_open($command, $fileDescriptors, $pipes, realpath('.'), array());
if (!is_resource($process))
{
error_log("Failed to execute '$command'");
return;
}
while ($data = fgets($pipes[1]))
{
print $data;
flush();
}
You might gain some performance benefit from not calling flush(); on every loop iteration as well.
If this still isn't giving you all of the output, I would be suspicious of a problem in the Python script, and you should post more of it to make it easier to debug.
Related
I am running a command using shell_exec()
Let's say I have 400 directories and I can not wait for the command to run completely.
Is there a way, I can get the output Asynchronous?
$output = shell_exec('ls');
echo "<pre>$output</pre>";
$cmd = $command;
$descriptorspec = array(
0 => array("pipe", "r"), // stdin is a pipe that the child will read from
1 => array("pipe", "w"), // stdout is a pipe that the child will write to
2 => array("pipe", "w") // stderr is a pipe that the child will write to
);
flush();
$process = proc_open($cmd, $descriptorspec, $pipes, realpath('./'), array());
echo "<pre>";
if (is_resource($process)) {
while ($s = fgets($pipes[1])) {
print $s;
flush();
}
}
echo "</pre>";
The Code above Worked perfectly for me, This is copied from another answer I can no longer find. If you put a ping 127.0.0.1 in the $command it works exactly like it does in a terminal.
Check this: Is there a way to use shell_exec without waiting for the command to complete?
And instead of redirecting to /dev/null you could redirect to a tmp file that you read later.
I'm trying to execute a bash script from php and getting its output in real time.
I am applying the answers found here:
However they are not working for me.
When I invoke the .sh script on this way, it works fine:
<?php
$output = shell_exec("./test.sh");
echo "<pre>$output</pre>";
?>
However, when doing:
<?php
echo '<pre>';
passthru(./test.sh);
echo '</pre>';
?>
or:
<?php
while (# ob_end_flush()); // end all output buffers if any
$proc = popen(./test.sh, 'r');
echo '<pre>';
while (!feof($proc))
{
echo fread($proc, 4096);
# flush();
}
echo '</pre>';
?>
I have no output in my browser.
I also tried to call the variable instead of the script in both cases, I mean:
<?php
$output = shell_exec("./test.sh");
echo '<pre>';
passthru($output);
echo '</pre>';
?>
This is my test.sh script:
#!/bin/bash
whoami
sleep 3
dmesg
Use the following:
<?php
ob_implicit_flush(true);
ob_end_flush();
$cmd = "bash /path/to/test.sh";
$descriptorspec = array(
0 => array("pipe", "r"), // stdin is a pipe that the child will read from
1 => array("pipe", "w"), // stdout is a pipe that the child will write to
2 => array("pipe", "w") // stderr is a pipe that the child will write to
);
$process = proc_open($cmd, $descriptorspec, $pipes, realpath('./'), array());
if (is_resource($process)) {
while ($s = fgets($pipes[1])) {
print $s;
}
}
?>
Change test.sh to:
#!/bin/bash
whoami
sleep 3
ls /
Explanation:
dmesg requires permissions. You need to grant webserver's user permissions for that. In my case apache2 is being run via www-data user.
ob_implicit_flush(true): Turns implicit flushing on. Implicit flushing will result in a flush operation after every output call, so that explicit calls to flush() will no longer be needed.
ob_end_flush(): Turns off output buffering, so we see results immediately.
I have a "long" script I want to execute in a PHP page, and I want its output to be 'refreshed' as soon as the script outputs something.
I've read plenty of solutions like questions 4706525, 9182094, 8882383, PHP Flush Manual but it's not working as expected in my case!
My test script:
#!/bin/bash
echo "This is a test script"
echo "Sleeping"
sleep 30
echo "Done"
Executable permission is set for www-data.
My PHP page:
<?php
#apache_setenv('no-gzip', 1);
#ini_set('zlib.output_compression', 0);
#ini_set('implicit_flush', 1);
#ini_set('output_buffering', 0);
#apache_setenv('output_buffering', 0);
echo "Here<br>";
flush();
$cmd = "../test.sh";
$pipes = array();
$descriptors = array(
0 => array("pipe", "r"),
1 => array("pipe", "w"),
2 => array("pipe", "w"),
);
echo "Starting process<br>";
flush();
$process = proc_open($cmd, $descriptors, $pipes, realpath('./'), array());
echo "<pre>";
if (is_resource($process)) {
while ($s = fgets($pipes[1])) {
print $s;
flush();
}
} else {
print "Cannot create process\n";
}
echo "</pre>";
fclose($pipes[0]);
fclose($pipes[1]);
fclose($pipes[2]);
proc_close($process);
?>
NB. My test script, test.sh, is in a directory above the PHP page, thus ../test.sh. Not that that changes anything. But it's not a typo.
My php.ini has those (although, I wasn't too keen at changing that server wide, but I wanted to test if that was the issue):
zlib.output_compression = Off
output_buffering = Off
I use LAMPP.
If I run the PHP page in a terminal,
$ php test.php
It works fine: I immediately get "This is a test script" and "Sleeping", and after a while, "Done".
If I load the page in my browser, it does not work: it waits until test.sh has completed before outputting anything.
Edited: If I add echo str_pad('',4096)."\n" in the loop, then it works. However, this fix suggests that for a reason I do not understand, output buffering is still set to its default value (4096) and not off as I tried to configure.
while ($s = fgets($pipes[1])) {
print $s;
echo str_pad('',4096)."\n";
flush();
}
Furthermore, this solution is not perfect because , in reality, it adds spaces to the output.
I am looking for a solution that
refreshes output of the PHP page
does not modify php.ini
does not modify the output
Thanks!
I am trying to run processes using proc_open() function. As specified on the page - I supplied the custom environment variables and tried to print out. It shows all of my supplied variables + always 3 variables : 'SHLVL', 'PWD', '_='. I would like to print/use only my supplied environment variables. Are these 3 always present with this function? Is there any way to have only provided variables? This is all under Linux and PHP5.
//Here is the code to clarify :
$descriptorspec = array(
0 => array("pipe", "r"), // stdin is a pipe that the child will read from
1 => array("pipe", "w"), // stdout is a pipe that the child will write to
2 => array("file", "/tmp/error-output.txt", "a") // stderr is a file to write to
);
$env = array('MY_VAR' => 'FOO');
$process = proc_open('./run.php', $descriptorspec, $pipes, $cwd, $env);
if (is_resource($process)) {
fwrite($pipes[0], escapeshellcmd($args));
fclose($pipes[0]);
$output = "";
while (!feof($pipes[1])) {
$output .= fgets($pipes[1]);
}
print "Output is $output \n";
fclose($pipes[1]);
$return_value = proc_close($process);
}
Thanks.
You could namespace your environment variables, e.g. PHP_MYVAR instead of MYVAR. This way you can filter based on the common prefix PHP_.
Those three variables are created by the shell. If you don't open a shell, they won't be created.
It was just related to Linux. It works as it supposed to under Solaris. I added regex filter to remove those extra variables.
I've this shell script running from a PHP site.
In the shell script (Audit shell script),
I have 3 options:
1) Process script
2) Display results
3) Exit
Tried the codes below and doesn't seem to work, the PHP site displayed blanks.
<?php
session_start();
exec('/Desktop/test.sh');
exec('1');
$output = exec('2');
echo "<pre>$output</pre>";
?>
Any help will be greatly appreciated.
<?php
session_start();
// This line executes '/Desktop/test.sh' as if it had been called from the
// command line
// exec('/Desktop/test.sh');
// This line attempts to execute a file called '1', which would have to be
// in the same directory as this script
// exec('1');
// This line attempts to execute a file called '2', which would have to be
// in the same directory as this script, and capture the first line of the
// output in $output
// $output = exec('2');
// I think you want to be doing something more like this - this executes the
// shell script, passing "1" and "2" as arguments, and captures the whole
// output as an array in $output
exec('/Desktop/test.sh "1" "2"', $output);
// Loop the output array and echo it to the browser
echo "<pre>";
foreach ($output as $lineno => $line) echo "Line $lineno: $line\n";
echo "</pre>";
?>
It seems to me that you could do with reading the manual page for exec() properly...
Try using proc_open instead of exec; it gives you more control of process input/output. Something like:
<?php
$descriptorspec = array(
0 => array("pipe", "r"), // stdin is a pipe that the child will read from
1 => array("pipe", "w"), // stdout is a pipe that the child will write to
2 => array("file", "/dev/null", "a") // stderr is a file to write to
);
$cwd = '/Desktop';
$env = array();
$process = proc_open('/Desktop/test.sh', $descriptorspec, $pipes, $cwd, $env);
if (is_resource($process)) {
// $pipes now looks like this:
// 0 => writeable handle connected to child stdin
// 1 => readable handle connected to child stdout
// Any error output will be sent to /dev/null (ie, discarded)
fwrite($pipes[0], "1\n");
fwrite($pipes[0], "2\n");
fclose($pipes[0]);
$output = stream_get_contents($pipes[1]);
fclose($pipes[1]);
// It is important that you close any pipes before calling
// proc_close in order to avoid a deadlock
$return_value = proc_close($process);
echo $output;
}
?>
Note: I've lifted this code from the PHP Manual's proc_open page