We have a command line php application that maintains special permissions and want to use it to relay piped data into a shell script.
I know that we can read STDIN with:
while(!feof(STDIN)){
$line = fgets(STDIN);
}
But how can I redirect that STDIN into a shell script?
The STDIN is far too large to load into memory, so I can't do something like:
shell_exec("echo ".STDIN." | script.sh");
Using xenon's answer with popen seems to do the trick.
// Open the process handle
$ph = popen("./script.sh","w");
// This puts it into the file line by line.
while(($line = fgets(STDIN)) !== false){
// Put in line from STDIN. (Note that you may have to use `$line . '\n'`. I don't know
fputs($ph,$line);
}
pclose($ph);
As #Devon said, popen/pclose are very useful here.
$scriptHandle = popen("./script.sh","w");
while(($line = fgets(STDIN)) !== false){
fputs($scriptHandle,$line);
}
pclose($scriptHandle);
Alternatively, something along the lines of fputs($scriptHandle, file_get_contents("php://stdin")); might work in the place of a line-by-line approach for a smaller file.
Related
I'm using the PHP function file_put_contents() to put some content into a txt file. The example in the docs doesn't finish using fclose(), should I close the file or it's not necessary?
I'm doing this:
$root = $_SERVER['DOCUMENT_ROOT'];
$log = $root.'/logs/logsContenido.txt';
$agregadoLog = "texto a agregar";
file_put_contents($log, $agregadoLog, FILE_APPEND | LOCK_EX);
And just that. I don't close anything.
Should I rather do something like:
$root = $_SERVER['DOCUMENT_ROOT'];
$log = $root.'/logs/logsContenido.txt';
$agregadoLog = "texto a agregar";
$file = file_put_contents($log, $agregadoLog, FILE_APPEND | LOCK_EX);
fclose($file);
No, you should not/cannot. file_put_contents takes care of opening the file, writing the contents, and closing the file. In fact it does not expose any handle to you which you could close even if you wanted to.
In file_put_contents, PHP handles it for you. So there is no need to fclose it.
But if you are using an handle like fopen, then you need to close it since you'll leave the file open during the entire exection of the script. Oh and also it is a good practice to close the handle once you are done with it
Consider the code snippet here:
$handle = popen("some command that generates an infinite stream of output to stdout", "r");
while ($line = fgets($handle)) {
echo $line;
sleep(3);
}
My question is: what is actually happening during that sleep(3) and the command passed to popen() is still spewing output? Is that getting buffered to PHP's memory?
Is there a chance the output is trashed?
It's OS-dependent. The data may be buffered, the other program's output calls may block, or some combination thereof.
I'm trying to use popen to run a php script in the background. However, I need to pass a (fairly large) serialized object.
$cmd = "php background_test.php >log/output.log &";
$fh = popen($cmd, 'w');
fwrite($fh, $data);
fclose($fh);
//pclose($fh);
Without the ampersand this code executes fine but the parent script will wait until the child is finished running. With the ampersand STDIN gets no data.
Any ideas?
You can try forking letting child process to write data and main script continue as normal.
Something like this
// Fork a child process
$pid = pcntl_fork();
// Unable to fork
if ($pid == -1) {
die('error');
}
// We are the parent
elseif ($pid) {
// do nothing
}
// We are the child
else {
$cmd = "php background_test.php >log/output.log";
$fh = popen($cmd, 'w');
fwrite($fh, $data);
fclose($fh);
exit();
}
// parent will continue here
// child will exit above
Read more about it here: https://sites.google.com/a/van-steenbeek.net/archive/php_pcntl_fork
Also check function pcntl_waitpid() (zombies be gone) in php documentation.
As far as I know there is no way in php to send a process in background and continue to feed its STDIN (but maybe I'm wrong). You have two other choices here:
Refactor your background_test.php to get its input from command line and transform your command line in php background_test.php arg1 arg2 ... >log/output.log &
If your input is pretty long, write it to a temporary file and then feed the background_test.php script with that file as in the following code
Example for point 2:
<?
$tmp_file = tempnam();
file_put_content($tmp_file, $data);
$cmd = "php background_test.php < $tmp_name > log/output.log &";
exec($cmd);
Make a background processes listen to a socket file. Then open socket file from PHP and send your serialized data there. When your background daemon receives connection through the socket, make it fork, read data then process.
You would need to do some reading, but I think that's the best way to achieve this. By socket i mean unix socket file, but you can also use this over the network.
http://gearman.org/ is also a good alternative as mentioned by #Joshua
I got in every php project (around 25!), some sh scripts that help me with routine tasks such as deployment, repo syncronizing, databases exporting/export, etc.
The sh scripts are the same for all the projects I manage, so there must be a configuration file to store diferent parameters that depend on the project:
# example conf, the sintaxys only needs to be able to have comments and be easy to edit.
host=www.host.com
administrator_email=guill#company.com
password=xxx
I just need to find a clean way in which this configuration file can be read (parsed) from a sh script, and at the same time, be able to read those same parameters from my PHP scripts. Without having to use XML.
Do you know a good solution for this?
Guillermo
Simply source the script conf file as another sh file!.
Example:
conf-file.sh:
# A comment
host=www.host.com
administrator_email=guill#company.com
password=xxx
Your actual script:
#!/bin/sh
. ./conf-file.sh
echo $host $administrator_email $passwword
And the same conf-file can be parsed in PHP: http://php.net/manual/en/function.parse-ini-file.php
If you don't want to source the file as pavanlimo showed, another option is to pull in the variables using a loop:
while read propline ; do
# ignore comment lines
echo "$propline" | grep "^#" >/dev/null 2>&1 && continue
# if not empty, set the property using declare
[ ! -z "$propline" ] && declare $propline
done < /path/to/config/file
In PHP, the same basic concepts apply:
// it's been a long time, but this is probably close to what you need
function isDeclaration($line) {
return $line[0] != '#' && strpos($line, "=");
}
$filename = "/path/to/config/file";
$handle = fopen($filename, "r");
$contents = fread($handle, filesize($filename));
$lines = explode("\n", $contents); // assuming unix style
// since we're only interested in declarations, filter accordingly.
$decls = array_filter($lines, "isDeclaration");
// Now you can iterator over $decls exploding on "=" to see param/value
fclose($handle);
For a Bash INI file parser also see:
http://ajdiaz.wordpress.com/2008/02/09/bash-ini-parser/
to parse the ini file from sh/bash
#!/bin/bash
#bash 4
shopt -s extglob
while IFS="=" read -r key value
do
case "$key" in
!(#*) )
echo "key: $key, value: $value"
array["$key"]="$value"
;;
esac
done <"file"
echo php -r myscript.php ${array["host"]}
then from PHP, use argv
INI! http://php.net/manual/en/function.parse-ini-file.php
The code below almost works, but it's not what I really meant:
ob_start();
echo 'xxx';
$contents = ob_get_contents();
ob_end_clean();
file_put_contents($file,$contents);
Is there a more natural way?
It is possible to write STDOUT directly to a file in PHP, which is much easier and more straightforward than using output bufferering.
Do this in the very beginning of your script:
fclose(STDIN);
fclose(STDOUT);
fclose(STDERR);
$STDIN = fopen('/dev/null', 'r');
$STDOUT = fopen('application.log', 'wb');
$STDERR = fopen('error.log', 'wb');
Why at the very beginning you may ask? No file descriptors should be opened yet, because when you close the standard input, output and error file descriptors, the first three new descriptors will become the NEW standard input, output and error file descriptors.
In my example here I redirected standard input to /dev/null and the output and error file descriptors to log files. This is common practice when making a daemon script in PHP.
To write to the application.log file, this would suffice:
echo "Hello world\n";
To write to the error.log, one would have to do:
fwrite($STDERR, "Something went wrong\n");
Please note that when you change the input, output and error descriptors, the build-in PHP constants STDIN, STDOUT and STDERR will be rendered unusable. PHP will not update these constants to the new descriptors and it is not allowed to redefine these constants (they are called constants for a reason after all).
here's a way to divert OUTPUT which appears to be the original problem
$ob_file = fopen('test.txt','w');
function ob_file_callback($buffer)
{
global $ob_file;
fwrite($ob_file,$buffer);
}
ob_start('ob_file_callback');
more info here:
http://my.opera.com/zomg/blog/2007/10/03/how-to-easily-redirect-php-output-to-a-file
None of the answers worked for my particular case where I needed a cross platform way of redirecting the output as soon as it was echo'd out so that I could follow the logs with tail -f log.txt or another log viewing app.
I came up with the following solution:
$logFp = fopen('log.txt', 'w');
ob_start(function($buffer) use($logFp){
fwrite($logFp, $buffer);
}, 1); //notice the use of chunk_size == 1
echo "first output\n";
sleep(10)
echo "second output\n";
ob_end_clean();
I haven't noticed any performance issues but if you do, you can change chunk_size to greater values.
Now just tail -f the log file:
tail -f log.txt
No, output buffering is as good as it gets. Though it's slightly nicer to just do
ob_start();
echo 'xxx';
$contents = ob_get_flush();
file_put_contents($file,$contents);
Using eio pecl module eio is very easy, also you can capture PHP internal errors, var_dump, echo, etc. In this code, you can found some examples of different situations.
$fdout = fopen('/tmp/stdout.log', 'wb');
$fderr = fopen('/tmp/stderr.log', 'wb');
eio_dup2($fdout, STDOUT);
eio_dup2($fderr, STDERR);
eio_event_loop();
fclose($fdout);
fclose($fderr);
// output examples
echo "message to stdout\n";
$v2dump = array(10, "graphinux");
var_dump($v2dump);
// php internal error/warning
$div0 = 10/0;
// user errors messages
fwrite(STDERR, "user controlled error\n");
Call to eio_event_loop is used to be sure that previous eio requests have been processed. If you need append on log, on fopen call, use mode 'ab' instead of 'wb'.
Install eio module is very easy (http://php.net/manual/es/eio.installation.php). I tested this example with version 1.2.6 of eio module.
You can install Eio extension
pecl install eio
and duplicate a file descriptor
$temp=fopen('/tmp/my_stdout','a');
$my_data='my something';
$foo=eio_dup2($temp,STDOUT,EIO_PRI_MAX,function($data,$esult,$request){
var_dump($data,$esult,$request);
var_dump(eio_get_last_error($request));
},$my_data);
eio_event_loop();
echo "something to stdout\n";
fclose($temp);
this creates new file descriptor and rewrites target stream of STDOUT
this can be done with STDERR as well
and constants STD[OUT|ERR] are still usable
I understand that this question is ancient, but people trying to do what this question asks will likely end up here... Both of you.
If you are running under a particular environment...
Running under Linux (probably most other Unix like operating systems, untested)
Running via CLI (Untested on web servers)
You can actually close all of your file descriptors (yes all, which means it's probably best to do this at the very beginning of execution... for example just after a pcntl_fork() call to background the process in a daemon (which seems like the most common need for something like this)
fclose( STDIN ); // fd 3
fclose( STDERR); // fd 2
fclose( STDOUT ); // fd 1
And then re-open the file descriptors, assigning them to a variable that will not fall out of scope and thus be garbage collected. Because Linux will predictably open them in the proper order.
$kept_in_scope_variable_fd1 = fopen(...); // fd 1
$kept_in_scope_variable_fd2 = fopen(...); // fd 2
$kept_in_scope_variable_fd3 = fopen( '/dev/null', ... ); // fd 3
You can use whatever files or devices you want for this. I gave /dev/null as the example for STDIN (fd3) because that's probably the most common case for this kind of code.
Once this is done you should be able to do normal things like echo, print_r, var_dump, etc without specifically needing to write to a file with a function. Which is useful when you're trying to background code that you do not want to, or aren't able to, rewrite to be file-pointer-output-friendly.
YMMV for other environments and things like having other FD's open, etc. My advice is to start with a small test script to prove that it works, or doesn't, in your environment and then move on to integration from there.
Good luck.
Here is an ugly solution that was useful for a problem I had (need to debug).
if(file_get_contents("out.txt") != "in progress")
{
file_put_contents("out.txt","in progress");
$content = file_get_contents('http://'.$_SERVER['HTTP_HOST'].$_SERVER['REQUEST_URI']);
file_put_contents("out.txt",$content);
}
The main drawback of that is that you'd better not to use the $_POST variables.
But you dont have to put it in the very beggining.