PHP: Pass file into script as stdin - php

I am attempting to write some tests for an email parser I am building and having trouble getting started.
For normal operation an email will be piped to the script, but for the tests I want to simulate the piping action : )
My test is starting out like this:
#!/opt/php70/bin/php
<?php
define('INC_ROOT', dirname(__DIR__));
$script = INC_ROOT . '/app/email_parser.php';
//$email = file_get_contents(INC_ROOT . '/tests/test_emails/test.email');
$email = INC_ROOT . '/tests/test_emails/test.email';
passthru("{$script}<<<{$email}");
With the script as is, the only thing passed to stdin is the path to the test email. When using file_get_contents I get:
sh: -c: line 0: syntax error near unexpected token '('
sh: -c: line 0: /myscriptpath/app/email_parser.php<<<TestEmailContents
Where TestEmailContents is the contents of the raw email file. I feel like I have executed scripts in this manner in the past using the heredoc operator to pass data into stdin. But for the last few days I have been unable to find any information to get me past this stumbling block. Any advice will be mucho appreciado!

The syntax error experienced was exactly that. To get the file contents and pass it in as a here string I needed to single quote the string:
$email = file_get_contents(INC_ROOT . '/tests/test_emails/test.email');
passthru("{$script} <<< '{$email}'");
But, in my case passing in a raw email did not require the use of a here string. The line endings are preserved either way. Redirecting the file to the script yielded the same results.
$email = INC_ROOT . '/tests/test_emails/test.email';
passthru("{$script} < {$email}");

To read stdin in PHP you can use php://stdin filename: $content = file_get_contents('php://stdin'); or $f = fopen('php://stdin', 'r');.
To pass a string to an invoked process you have two options: popen or proc_open. The popen function is easier to use, but it has limited use. The proc_open is a bit more complicated, but gives you much finer control of stdio redirection.
Both function give you file handle(s) on which you can use fwrite and fread. In your case the popen should be good enough (simplified):
$f = popen('./script.php', 'w');
fwrite($f, file_get_contents('test.email'));
pclose($f);

Related

PHP exec() function only runs extremely short Python scripts

I'm having some trouble using the PHP exec() function. Whenever the script I'm attempting to run is short, exec() works just fine. But if the script takes any more than a second or so, it fails. Note, I've attempted run the long script manually on the command line, and it works just fine. It seems as though the PHP interpreter is killing my external script if it takes any longer than a second or so to run. Any thoughts or suggestions? Here is my code:
<?php
$fileName = "foobar.docx";
$argVar = $fileName;
exec("python3 /var/www/html/jan8/alexandrina.py /var/www/html/jan8/$argVar");
echo "$output";
?>
And here is my script:
#!/usr/bin/env python3
import docx
import sys
docxFile = "".join(sys.argv[1:])
# The Three Lines Below Create a New Variable "htmlFile"
# "htmlFile" is the same as "docxFile", except ".docx" is cut off
# and replaced with ".html"
myNumber = len(docxFile) - 5
htmlFile = docxFile[0:myNumber]
htmlFile = htmlFile + '.html'
def generateHTML(filename):
doc = docx.Document(filename)
fullText = []
for para in doc.paragraphs:
fullText.append('<p>')
fullText.append(para.text)
fullText.append('</p>')
fullText.append('\n')
return '\n'.join(fullText)
file = open(htmlFile, "w")
file.write(generateHTML(docxFile))
file.close()
print("end python script")
Additional notes: I've increased the max execution time limits in php.ini, but I don't think that should matter, as the "long script" should only take a few seconds to run. Also, when I refer to the "short script", and the "long script", I'm actually referring to the same script. The difference between the "long script" and the "short script" is just the time to execute as it may vary depending on the size of the file I'm asking the script to process. Anyway... any suggestions would really be appreciated!
Ordinarily, php exec function should block until the command you run has completed. I.e., the PHP script will halt, waiting for the command to finish until continuing with the rest of your script. I was half thinking that your server was experiencing a max_execution_time timeout, but you've clearly stated that even just a couple of seconds is too long and even these fairly short scripts are having trouble.
A couple of solutions occur to me. The simplest one is to alter the python command so that a) any output is routed to a file or output stream and b) the process is run in the background. According to the docs on exec:
If a program is started with this function, in order for it to continue running in the background, the output of the program must be redirected to a file or another output stream. Failing to do so will cause PHP to hang until the execution of the program ends.
I also would like you to make use of the two additional optional parameters for the exec function.
$fileName = "foobar.docx";
$argVar = $fileName;
$cmd = "python3 /var/www/html/jan8/alexandrina.py /var/www/html/jan8/$argVar";
// modify your command to toss output, background the process, and output the process id
$cmd_modified = $cmd . " >/dev/null & echo \$!";
$cmd_output = NULL; // this will be an array of output
$cmd_return_value = NULL; // this will be the return value of the script
exec($cmd_modified, $cmd_output, $cmd_return_value);
echo "exec has completed</br>";
echo "output:<br>" . print_r($cmd_output, TRUE) . "<br>";
echo "return value: " . print_r($cmd_return_value, TRUE);
This may help or may not. If it does not, we still might be able to solve the problem using posix commands.
EDIT: according to crispytx, the long scripts are resulting in a $cmd_return_val of 1 which means an error is happening. Try changing this one line:
$cmd_modified = $cmd . " >/dev/null & echo \$!";
to this
$cmd_modified = $cmd . " & echo \$!";
And let us know what the output of $cmd_output is -- it should at the very least have the process id of the newly spawned process.
Thanks for all the help S. Imp. I had a little trouble debugging using your suggestions because I happened to be using AJAX to call the script. However, I wrote simpler script using your suggestions to try and debug the problem and this is what I found:
Array ( [0] => Traceback (most recent call last): [1] => File "/var/www/html/jan8/alexandrina.py", line 28, in [2] => file.write(generateHTML(docxFile)) [3] => UnicodeEncodeError: 'ascii' codec can't encode character '\u2026' in position 25: ordinal not in range(128) )
So it looks like the problem has to do with ascii encoding! Even though the larger file was just a docx file with the same text as the shorter docx file repeated over and over again for 300 pages. It seems that if a docx file exceeds 1 pages, ascii characters are inserted that aren't present in single page docx files. I have no idea if this post will ever end up helping anyone, but who knows!
[SOLVED]

Script reading from STDIN

I have a php script that reads text files. I use fgetc() to get every character one by one. I open file to read from with fopen(),and then I use file descriptor returned from fopen() as a first argument to fgetc(). I tried to do the same thing with reading from STDIN. I wanted to run the script in a terminal, give it the whole text (that was in a text file before) and press enter. I thought that the script would read it and will run as if it read from a text file, but it doesn't work. It only works when a type every single character alone and press enter after it. Why is that? Is there a possibility to make the script behave the way I wanted? That I can give it the whole text to the terminal at once? Should I use different functions or something?
$inputFile = fopen($path, "w");
while(($char = fgetc($inputFile)) !== false){
dosomething();
}
What I'm trying to do is to replace $inputFile in fgetc()with STDIN.
See http://php.net/manual/en/features.commandline.io-streams.php, second comment
Note, without the stream_set_blocking() call, fgetcsv() hangs on STDIN, awaiting input from the user, which isn't useful as we're looking for a piped file. If it isn't here already, it isn't going to be.
<?php
stream_set_blocking(STDIN, 0);
$csv_ar = fgetcsv(STDIN);
I think it's the same for fgetc. After all it
string fgetc ( resource $handle ) Gets a character from the given file pointer.
Emphasis mine.
See http://php.net/manual/en/function.fgetc.php
...

PHP writing lots of text to STDERR

When writing a large chunk to STDOUT in PHP you can do this:
echo <<<END_OF_STUFF
lots and lots of text
over multiple lines
etc.etc
END_OF_STUFF;
(i.e. heredoc)
I have the need to do a similar thing but to STDERR. Is there another command like echo but uses STDERR instead?
For a simple solution - try this
file_put_contents('php://stderr', 'This text goes to STDERR',FILE_APPEND);
The FILE_APPEND parameter will append data and not overwrite it.
You could also write directly to the error stream using the fopen and fwrite functions.
More info can be found at - http://php.net/manual/en/features.commandline.io-streams.php
Yes, using php:// stream wrapper: http://php.net/manual/en/wrappers.php.php
$stuff = <<<END_OF_STUFF
lots and lots of text
over multiple lines
etc.etc
END_OF_STUFF;
$fh = fopen('php://stderr','a'); //both (a)ppending, and (w)riting will work
fwrite($fh,$stuff);
fclose($fh);
In the CLI SAPI, it can be as simple as passing a Heredoc string as an argument to fwrite() with the STDERR constant.
fwrite(STDERR, <<< EOD
Example of string
spanning multiple lines
using heredoc syntax.
EOD
);

Pass multiline html as argument via php system()

Due to reasons that are have nothing to do with the actual question, I need to call and use an external script via PHP to perform a string replacement on a complete html document. The replacement strings and the source code need to be passded to this script via php exec(). For this example, I have used a simple python script to take over the replacement.
PHP script looks like this:
$source = file_get_contents("somehtmlfile.html");
$replaceString = "Some text in the HTML doc";
$replaceTo = "Some other text";
$parsedString = system("python replace.py $replaceString $replaceTo $source", $retval);
print ("Done:" .$mystring);
Then the Python script will do the following:
import sys
import string
dataFrom = sys.argv[1];
dataTo = sys.argv[2];
dataSourceCode = sys.argv[3];
rep = dataSourceCode.replace(dataFrom, dataTo);
print rep;
The problem is that I can't pass the complete html source as an argument to the shell, at least not in the way shown above. From what I understood, while the html code gets passed to the shell it interpretes some sections as commands (multiline could be an issue here I suppose).
The output I receive from the script :
sh: cannot open !DOCTYPE: No such file
sh: cannot open html: No such file
sh: cannot open head: No such file
sh: cannot open title: No such file
... (this goes on)
Any suggestions?
It is not working because there are spaces and quotes in the html text that you pass as an argument, so it is seen as multiple arguments. To solve this, you have to put quotes around the arguments.
The correct code is $parsedString = system("python replace.py '$replaceString' '$replaceTo' '$source'", $retval);

How to pass a file as an argument to php exec?

I would like to know how I can pass the content of a file (csv in my case) as an argument for a command line executable (in C or Objective C) to be called by exec in php.
Here is what I have done: the user loads the content of its file from an URL like this:
http://www.myserver.com/model.php?fileName=test.csv
Then the following code allows php to parse and load the csv file:
<?php
$f = $_GET['fileName'];
$handle = fopen("$f", "r");
$data = array();
while (($line = fgetcsv($handle)) !== FALSE) {
$data[] = $line;
}
?>
where I'm stuck is how to pass the content of this csv file as an argument to exec. Even if I can assume the csv is known to have only two columns, how many rows it has is user-specific, so I cannot pass all the values one by one as parameters, e.g.
exec("/path_to_executable/model -a $data[0][0] -b $data[0][1] .....");
The only alternative solution I guess would be to write something like that:
exec("/path_to_executable/model -fileName test.csv");
and have the command line executable do the csv parsing, but in that case, I think I need to have the csv file physically written on the server side. I'm wondering what happens if several people are accessing the webpage at the same time with their own different csv file, are they over-writing each others?
I guess there must be a much proper way to do this and I have not figured it out. Any idea? Thanks!
I would recommend having that data on disk, and loading it within the command line utility - it is much less messing about. But if you can't do that, just pass it in 1 (unparsed) line at a time:
$command = "/path_to_executable/model";
foreach ($fileData as $line) {
$command .= ' "'.escapeshellarg($line).'"';
}
exec($command);
Then you can just fetch the data into your utility by looping argv, where argv[0] is the first line, argv[1] is the second line, and so on.
you could use popen() to get a handle on the process to write to. If you need to go both ways (read/write) and might requre some more power, have a look a proc_open().
You could also just write your data to some random file (to avoid multiple users kicking each other's race-conditioned butts). Something along the lines of
<?php
$csv = file_get_contents('http://www.myserver.com/model.php?fileName=test.csv
');
$filename = '/tmp/' . uniqid(sha1($csv)) . '.csv';
file_put_contents($filename, $csv);
exec('/your/thing < '. escapeshellarg($filename));
unlink($filename);
And since you're also in charge of the executable, you might figure out how to get the number of arguments passed (hint: argc) and read them in (hint: argv). Passing them through line-based like so:
<?php
$csvRow = fgetcsv($fh);
if ($csvRow) {
$escaped = array_map('escapeshellarg', $csvRow);
exec('/your/thing '. join(' ', $escaped));
}

Categories