As a simple proof of concept, I tried to share a string between forked processes from node to node or from node to php.
Take this simple php code that should log the output of stdin according to the php docs:
echo 'test' | php -r 'echo trim(fgets(STDIN));'
Working fine, but when I'm spawning the process from nodejs:
spawner.js
var fs = require('fs'); var spawn = require('child_process').spawn;
//dummy stdin file
var stdin = fs.openSync('stdin_file', 'w+');
//write the string
fs.writeSync(stdin, 'test');
spawn('php', ['stdin_test.php'], {
cwd: __dirname,
detached: true,
//to fully detach the process nothing should be piped from or to the parent process
stdio: [stdin, fs.openSync('out.log', 'a'), fs.openSync('err.log', 'a')]
})
stdin_test.php
<?php
error_log('php://stdin');
//this should log 'test' but outputs a newline
error_log(trim(fgets(STDIN)));
$t = fopen('/dev/stdin', 'r');
error_log('/dev/stdin:');
//this is working as expected
error_log(trim(fgets($t)));
Why is php://stdin empty? Is it safe to use /dev/stdin? What is the difference between /dev/stdin and php://stdin anyway?
Note that I have this behavior between 2 node processes too: process.stdin is empty but /dev/stdin has the expected result.
Gist available here
stdin man reference
I tested with the following script ( stdin_test.php ) using:
> echo test | php stdin_test.php
stdin_test.php
<?
echo 'STDIN :' ;
echo trim(fgets(STDIN)) ;
echo PHP_EOL;
$stdin_stream = fopen('php://stdin', 'r');
echo 'php://stdin :';
echo trim(fgets($stdin_stream));
echo PHP_EOL;
fclose($stdin_stream);
$stdin_file = fopen('/dev/stdin', 'r');
echo '/dev/stdin :';
echo trim(fgets($stdin_file));
echo PHP_EOL;
fclose($stdin_file);
I get back :
STDIN :test
php://stdin :
/dev/stdin :
If I then comment out the line:
//echo trim(fgets(STDIN));
I get back:
STDIN :
php://stdin :test
/dev/stdin :
If I comment out both of the first stdin echoes (and the file handler pointers), I get:
STDIN :
php://stdin :
/dev/stdin : test
Looking at documentation on php://input and how it is one-time usable unless (after 5.6) "the request body is saved" which is typical for POST requests but not PUT requests (apparently). This has me thinking that they are called "streams" because you get to walk in them once.
Rewind your stdin stream in JS before spawning PHP, else the file pointer will sit at the end of what you just wrote.
Related
Background:
I'm in a position where I'm placing data into the command line and I need a php loop (what will become a server of sorts) to read STDIN and just echo what it reads to the shell its running in.
The following terrible code works when the process is running in the same shell as the content echoed:
<?php
echo getmypid();
$string = "/proc/" . getmypid() . "/fd/0";
while (true) {
fwrite(STDOUT, fgets(fopen($string, 'r'), 4096) . " worked\n");
}
?>
I've tried many variants:
<?php
echo getmypid();
$string = "/proc/" . getmypid() . "/fd/0";
while (true) {
$fo = fread(STDIN, 1024);
fwrite(STDOUT, $fo);
}
?>
The problem is that whenever I write to this loop from a separate terminal, the output appears in the other terminal but is not processed by the loop.
When I enter text in the same terminal, the text is echoed right back.
I need a way to get command line data into this loop from any source.
I used PHP to call python script successfully and got the result . But I have to wait for the end of script running without anything output. It looks not friendly to my customer.
How can I return the script results to the PHP web in realtime ?
For instance ,for code below , I want to the PHP web will show output message in realtime instead of show them together at the end . How can I change my code?
Thank you .
PHP Code:
<?php
$k = $_REQUEST['k'];
if (!empty($k))
{
$k = trim($k);
$a = array();
exec('python ./some.py '.$k, $a);
echo $a[0];
}
?>
Python Code:
#!/usr/bin/env python
#-*- coding:utf-8 -*-
import sys
def do_some(a):
print 'test1'
time.sleep(30)
print 'test2'
if __name__ == '__main__':
print 'Now the python scritp running'
time.sleep(20)
a = sys.argv[1]
if a:
print 'Now print something'
T = do_some(a)
By specification, exec stop the calling program until the end of the callee. After that, you get back the output in a variable.
If you want to send data as soon as they are produced, you should use popen. It will fork a new process, but will not block the caller. So you can perform other tasks, like looping to read the sub-process output line by line to send it to your client. Something like that:
$handle = popen("python ./some.py ", 'r');
while(!feof($handle)) {
$buffer = fgets($handle);
echo "$buffer<br/>\n";
ob_flush();
}
pclose($handle)
I have a problem running another file from php. I want my php params to be the output of running a python file that calls another file itself.
Here is my php file:
<?php
if (isset($_POST['submit'])) {
$params = solve();
}
function solve() {
exec("python array.py", $output);
return $output;
}
?>
If array.py is simply:
if __name__ == "__main__":
print 1
print 2
print 3
print 4
I will get 1,2,3,4 for my output, but I as soon as I change array.py to the following file that calls os.system, I don't get anything. So the new array.py is:
import os
def main():
os.system("python test.py") #test.py creates tmp.txt with 4 lines w/ values 1,2,3,4
def output():
f = open("tmp.txt", "r")
myReturn = []
currentline = f.readline()
while currentline:
val = currentline[:-1] #Getting rid of '\n'
val = int(val)
myReturn = myReturn + [val]
currentline = f.readline()
f.close()
return myReturn
if __name__ == "__main__":
main()
o = output()
print o[0]
print o[1]
print o[2]
print o[3]
Also if I just run test.py, the output is the file tmp.txt:
1
2
3
4
So now, when I run my php file, the output tmp.txt is not even created in the directory and as a result I don't get any output from my php either.
I am not sure why this is happening because when I just run array.py myself, I get the desired output, and the tmp file is created.
EDIT:
I forgot to include: import os above.
Change exec to:
exec("python array.py 2>&1", $output)
Or check the web server or php error log. This will return the error output from the python script to your php script (not normally what you want in production).
Background
I am writing a simple online judge (a code grading system) using PHP and MySQL. It takes submitted codes in C++ and Java, compiles them, and tests them.
This is Apache running PHP 5.2 on an old version of Ubuntu.
What I am currently doing
I have a php program that loops infinitely, calling another php program by
//for(infinity)
exec("php -f grade.php");
//...
every tenth of a second. Let's call the first one looper.php and the second one grade.php. (Checkpoint: grade.php should completely finish running before the "for" loop continues, correct?)
grade.php pulls the earliest submitted code that needs to be graded from the MySQL database, puts that code in a file (test.[cpp/java]), and calls 2 other php programs in succession, named compile.php and test.php, like so:
//...
exec("php -f compile.php");
//...
//for([all tests])
exec("php -f test.php");
//...
(Checkpoint: compile.php should completely finish running before the "for" loop calling test.php even starts, correct?)
compile.php then compiles the program in test.[cpp/java] as a background process. For now, let's assume that it's compiling a Java program and that test.java is located in a subdirectory. I now have
//...
//$dir = "./sub/" or some other subdirectory; this may be an absolute path
$start_time = microtime(true); //to get elapsed compilation time later
exec("javac ".$dir."test.java -d ".$dir." 2> ".$dir
."compileError.txt 1> ".$dir."compileText.txt & echo $!", $out);
//...
in compile.php. It's redirecting the output from javac, so javac should be running as a background process... and it seems like it works. The $out should be grabbing the process id of javac in $out[0].
The real problem
I want to stop compiling if for some reason compiling takes more than 10 seconds, and I want to end compile.php if the program stops compiling before 10 seconds. Since the exec("javac... I called above is a background process (or is it?), I have no way of knowing when it has completed without looking at the process id, which should have been stored in $out earlier. Right after, in compile.php, I do this with a 10 second loop calling exec("ps ax | grep [pid].*javac"); and seeing if the pid still exists:
//...
$pid = (int)$out[0];
$done_compile = false;
while((microtime(true) - $start_time < 10) && !$done_compile) {
usleep(20000); // only sleep 0.02 seconds between checks
unset($grep);
exec("ps ax | grep ".$pid.".*javac", $grep);
$found_process = false;
//loop through the results from grep
while(!$found_process && list(, $proc) = each($grep)) {
$boom = explode(" ", $proc);
$npid = (int)$boom[0];
if($npid == $pid)
$found_process = true;
}
$done_compile = !$found_process;
}
if(!done_compile)
exec("kill -9 ".$pid);
//...
... which doesn't seem to be working. At least some of the time. Often, what happens is test.php starts running before the javac even stops, resulting in test.php not being able to find the main class when it tries to run the java program. I think that the loop is bypassed for some reason, though this may not be the case. At other times, the entire grading system works as intended.
Meanwhile, test.php also uses the same strategy (with the X-second loop and the grep) in running a program in a certain time limit, and it has a similar bug.
I think the bug lies in the grep not finding javac's pid even when javac is still running, resulting in the 10 second loop breaking early. Can you spot an obvious bug? A more discreet bug? Is there a problem with my usage of exec? Is there a problem with $out? Or is something entirely different happening?
Thank you for reading my long question. All help is appreciated.
I just came up with this code that will run a process, and terminate it if it runs longer than $timeout seconds. If it terminates before the timeout, it will have the program output in $output and the exit status in $return_value.
I have tested it and it seems to work well. Hopefully you can adapt it to your needs.
<?php
$command = 'echo Hello; sleep 30'; // the command to execute
$timeout = 5; // terminate process if it goes longer than this time in seconds
$cwd = '/tmp'; // working directory of executing process
$env = null; // environment variables to set, null to use same as PHP
$descriptorspec = array(
0 => array("pipe", "r"), // stdin is a pipe that the child will read from
1 => array("pipe", "w"), // stdout is a pipe that the child will write to
2 => array("file", "/tmp/error-output.txt", "a") // stderr is a file to write to
);
// start the process
$process = proc_open($command, $descriptorspec, $pipes, $cwd, $env);
$startTime = time();
$terminated = false;
$output = '';
if (is_resource($process)) {
// process was started
// $pipes now looks like this:
// 0 => writeable handle connected to child stdin
// 1 => readable handle connected to child stdout
// Any error output will be appended to /tmp/error-output.txt
// loop infinitely until timeout, or process finishes
for(;;) {
usleep(100000); // dont consume too many resources
$stat = proc_get_status($process); // get info on process
if ($stat['running']) { // still running
if (time() - $startTime > $timeout) { // check for timeout
// close descriptors
fclose($pipes[1]);
fclose($pipes[0]);
proc_terminate($process); // terminate process
$return_value = proc_close($process); // get return value
$terminated = true;
break;
}
} else {
// process finished before timeout
$output = stream_get_contents($pipes[1]); // get output of command
// close descriptors
fclose($pipes[1]);
fclose($pipes[0]);
proc_close($process); // close process
$return_value = $stat['exitcode']; // set exit code
break;
}
}
if (!$terminated) {
echo $output;
}
echo "command returned $return_value\n";
if ($terminated) echo "Process was terminated due to long execution\n";
} else {
echo "Failed to start process!\n";
}
References: proc_open(), proc_close(), proc_get_status(), proc_terminate()
If the php://temp (or php://memory) file is opened more than once, will the handles point to the same file? Or will each handle be unique?
I couldn't find an answer in the php docs, so I'm going to write up a test script to find out. I figured it's worth asking here so someone else can find the answer easily.
Each handle points to an independent stream. Example:
$a = fopen('php://memory', 'w+');
$b = fopen('php://memory', 'w+');
fwrite($a, 'foo');
fwrite($b, 'bar');
rewind($a);
rewind($b);
$a_text = stream_get_contents($a); //=> "foo"
$b_text = stream_get_contents($b); //=> "bar"
fclose($a);
fclose($b);
This is not explicitly documented anywhere, but it is implicit in the documentation for streams and wrappers.
From the official php documentation on streams in general, it is clear that for the standard case of streams, each file handle is associated with it's own independent stream.
And in the documentation on IO stream wrappers, it lists the possible wrappers noting exceptions as they occur. There is an exception listed for the first three (stdin, stdout, stderr):
php://stdin, php://stdout and php://stderr allow direct access to the
corresponding input or output stream of the PHP process. The stream
references a duplicate file descriptor, so if you open php://stdin and
later close it, you close only your copy of the descriptor-the actual
stream referenced by STDIN is unaffected.
But no such exception is listed for php://temp or php://memory. Hence it follows that these would work like normal independent streams.
Also, there are some comments on these pages that further imply the Independence of these streams.
My test code:
$f1 = fopen('php://temp', 'rw');
$f2 = fopen('php://temp', 'rw');
fputs($f1, "File One");
fputs($f2, "File Two");
rewind($f1);
echo "First line from F1: ";
echo fgets($f1) . PHP_EOL;
echo "Second line from F1: ";
echo fgets($f1) . PHP_EOL;
fclose($f1);
rewind($f2);
echo "First line from F2: ";
echo fgets($f2) . PHP_EOL;
echo "Second line from F2: ";
echo fgets($f2) . PHP_EOL;
fclose($f2);
And results:
First line from F1: File One
Second line from F1:
First line from F2: File Two
Second line from F2: