How can I force the php exec() to interpret the linux brace expansion?
I am encountering a strange behavior, and did not find a way to fix it the way I want.
I want to execute a linux command containing brace expression to select a batch of files,
from php
I am using php to generate a "random" number of files, and want then to execute a shell script which will make something with the files.
Here is my bash version:
"$ echo $BASH_VERSION"
4.1.5(1)-release
To give a simple example, let's assume I create the following files:
touch /tmp/file_{1..12}.xml
shell.sh
#!/bin/sh
FILES=$*
echo "\n\nFILES: $FILES"
for f in $FILES; do
echo Posting file $f
done
test.php
<?php
$cmd = "./shell.sh /tmp/file_{1..12}.xml";
echo"\n\nCOMMAND:\n".$cmd."\n\n";
var_dump(shell_exec($cmd));
The output of "php test.php" is:
COMMAND:
./shell.sh /tmp/file_{1..12}.xml
string(66) "
FILES: /tmp/file_{1..12}.xml
Posting file /tmp/file_{1..12}.xml
"
I expect to have the same as if I run "./shell.sh /tmp/file_{1..12}.xml" from linux terminal:
$ ./shell.sh /tmp/file_{1..12}.xml
FILES: /tmp/file_1.xml /tmp/file_2.xml /tmp/file_3.xml /tmp/file_4.xml /tmp/file_5.xml /tmp/file_6.xml /tmp/file_7.xml /tmp/file_8.xml /tmp/file_9.xml /tmp/file_10.xml /tmp/file_11.xml /tmp/file_12.xml
Posting file /tmp/file_1.xml
Posting file /tmp/file_2.xml
Posting file /tmp/file_3.xml
Posting file /tmp/file_4.xml
Posting file /tmp/file_5.xml
Posting file /tmp/file_6.xml
Posting file /tmp/file_7.xml
Posting file /tmp/file_8.xml
Posting file /tmp/file_9.xml
Posting file /tmp/file_10.xml
Posting file /tmp/file_11.xml
Posting file /tmp/file_12.xml
But I also tried with or without escapeshellcmd()
with exec($cmd) AND other functions like system() or eval()...
None of them did the job...
I know that I could do the foreach loop in php, but I am sure there is a way to have this command interpreted as if it was launched from command line.
As #Josh Trii Johnston has pointed out, the 'outer' shell you are implicitly using to call your shell script using shell_exec() is probably not Bash in your case. This way, brace expansion never takes place because there is no shell capable of expanding the expression before calling your program (as it would be in an interactive Bash session).
You could
try to change the shell invoked by PHP's shell_exec(), but this may not be possible
call /bin/bash with your program and the brace expression instead of only the brace expression: $cmd = "/bin/bash -c './shell.sh /tmp/file_{1..12}.xml'";
use eval on the argument inside your script to expand the brace expression.
From the bash(1) man page:
If bash is invoked with the name sh, it tries to mimic the startup behavior of historical versions of sh as closely as possible, while conforming to the POSIX standard as well. [...] When invoked as sh, bash enters posix mode after the startup files are read.
Try changing
#!/bin/sh
to
#!/bin/bash
if you expect Bash behavior (there is no brace expansion in POSIX).
If all of the above does not help, you should make sure that brace expansion is activated by executing set -o (while calling your program from the PHP script). If it is off, you can turn it on using:
set -o braceexpand
I used your exact example on my OS X machine and it works as expected. What user are you executing php as? Is that user's shell (/bin/sh) set to a non-bash shell?
$ php test.php
COMMAND:
./shell.sh /tmp/file_{1..12}.xml
string(555) "\n\nFILES: /tmp/file_1.xml /tmp/file_2.xml /tmp/file_3.xml /tmp/file_4.xml /tmp/file_5.xml /tmp/file_6.xml /tmp/file_7.xml /tmp/file_8.xml /tmp/file_9.xml /tmp/file_10.xml /tmp/file_11.xml /tmp/file_12.xml\nPosting file /tmp/file_1.xml\nPosting file /tmp/file_2.xml\nPosting file /tmp/file_3.xml\nPosting file /tmp/file_4.xml\nPosting file /tmp/file_5.xml\nPosting file /tmp/file_6.xml\nPosting file /tmp/file_7.xml\nPosting file /tmp/file_8.xml\nPosting file /tmp/file_9.xml\nPosting file /tmp/file_10.xml\nPost"...
Related
I'm trying to run a Python script from PHP using the following command:
exec('/usr/bin/python2.7 /srv/http/assets/py/switch.py arg1 arg2');
However, PHP simply doesn't produce any output. Error reporting is set to E_ALL and display_errors is on.
Here's what I've tried:
I used python2, /usr/bin/python2 and python2.7 instead of /usr/bin/python2.7
I also used a relative path instead of an absolute path which didn't change anything either.
I tried using the commands exec, shell_exec, system.
However, if I run
if (exec('echo TEST') == 'TEST')
{
echo 'exec works!';
}
it works perfectly fine while shutdown now doesn't do anything.
PHP has the permissions to access and execute the file.
EDIT: Thanks to Alejandro, I was able to fix the problem. If you have the same problem, don't forget that your webserver probably/hopefully doesn't run as root. Try logging in as your webserver's user or a user with similar permissions and try to run the commands yourself.
Tested on Ubuntu Server 10.04. I hope it helps you also on Arch Linux.
In PHP use shell_exec function:
Execute command via shell and return the complete output as a string.
It returns the output from the executed command or NULL if an error
occurred or the command produces no output.
<?php
$command = escapeshellcmd('/usr/custom/test.py');
$output = shell_exec($command);
echo $output;
?>
Into Python file test.py, verify this text in first line: (see shebang explain):
#!/usr/bin/env python
If you have several versions of Python installed, /usr/bin/env will
ensure the interpreter used is the first one on your environment's
$PATH. The alternative would be to hardcode something like
#!/usr/bin/python; that's ok, but less flexible.
In Unix, an executable file that's meant to be interpreted can indicate
what interpreter to use by having a #! at the start of the first line,
followed by the interpreter (and any flags it may need).
If you're talking about other platforms, of course, this rule does not
apply (but that "shebang line" does no harm, and will help if you ever
copy that script to a platform with a Unix base, such as Linux,
Mac, etc).
This applies when you run it in Unix by making it executable
(chmod +x myscript.py) and then running it directly: ./myscript.py,
rather than just python myscript.py
To make executable a file on unix-type platforms:
chmod +x myscript.py
Also Python file must have correct privileges (execution for user www-data / apache if PHP script runs in browser or curl)
and/or must be "executable". Also all commands into .py file must have correct privileges.
Taken from php manual:
Just a quick reminder for those trying to use shell_exec on a
unix-type platform and can't seem to get it to work. PHP executes as
the web user on the system (generally www for Apache), so you need to
make sure that the web user has rights to whatever files or
directories that you are trying to use in the shell_exec command.
Other wise, it won't appear to be doing anything.
I recommend using passthru and handling the output buffer directly:
ob_start();
passthru('/usr/bin/python2.7 /srv/http/assets/py/switch.py arg1 arg2');
$output = ob_get_clean();
If you want to know the return status of the command and get the entire stdout output you can actually use exec:
$command = 'ls';
exec($command, $out, $status);
$out is an array of all lines. $status is the return status. Very useful for debugging.
If you also want to see the stderr output you can either play with proc_open or simply add 2>&1 to your $command. The latter is often sufficient to get things working and way faster to "implement".
To clarify which command to use based on the situation
exec() - Execute an external program
system() - Execute an external program and display the output
passthru() - Execute an external program and display raw output
Source: http://php.net/manual/en/function.exec.php
Alejandro nailed it, adding clarification to the exception (Ubuntu or Debian) - I don't have the rep to add to the answer itself:
sudoers file:
sudo visudo
exception added:
www-data ALL=(ALL) NOPASSWD: ALL
In my case I needed to create a new folder in the www directory called scripts. Within scripts I added a new file called test.py.
I then used sudo chown www-data:root scripts and sudo chown www-data:root test.py.
Then I went to the new scripts directory and used sudo chmod +x test.py.
My test.py file it looks like this. Note the different Python version:
#!/usr/bin/env python3.5
print("Hello World!")
From php I now do this:
$message = exec("/var/www/scripts/test.py 2>&1");
print_r($message);
And you should see: Hello World!
The above methods seem to be complex. Use my method as a reference.
I have these two files:
run.php
mkdir.py
Here, I've created an HTML page which contains a GO button. Whenever you press this button a new folder will be created in directory whose path you have mentioned.
run.php
<html>
<body>
<head>
<title>
run
</title>
</head>
<form method="post">
<input type="submit" value="GO" name="GO">
</form>
</body>
</html>
<?php
if(isset($_POST['GO']))
{
shell_exec("python /var/www/html/lab/mkdir.py");
echo"success";
}
?>
mkdir.py
#!/usr/bin/env python
import os
os.makedirs("thisfolder");
This is so trivial, but just wanted to help anyone who already followed along Alejandro's suggestion but encountered this error:
sh: blabla.py: command not found
If anyone encountered that error, then a little change needs to be made to the php file by Alejandro:
$command = escapeshellcmd('python blabla.py');
All the options above create new system process. Which is a performance nightmare.
For this purpose I stitched together PHP module with "transparent" calls to Python.
https://github.com/kirmorozov/runpy
It may be tricky to compile, but will save system processes and will let you keep Python runtime between PHP calls.
Inspired by Alejandro Quiroz:
<?php
$command = escapeshellcmd('python test.py');
$output = shell_exec($command);
echo $output;
?>
Need to add Python, and don't need the path.
I'm trying to run a Python script from PHP using the following command:
exec('/usr/bin/python2.7 /srv/http/assets/py/switch.py arg1 arg2');
However, PHP simply doesn't produce any output. Error reporting is set to E_ALL and display_errors is on.
Here's what I've tried:
I used python2, /usr/bin/python2 and python2.7 instead of /usr/bin/python2.7
I also used a relative path instead of an absolute path which didn't change anything either.
I tried using the commands exec, shell_exec, system.
However, if I run
if (exec('echo TEST') == 'TEST')
{
echo 'exec works!';
}
it works perfectly fine while shutdown now doesn't do anything.
PHP has the permissions to access and execute the file.
EDIT: Thanks to Alejandro, I was able to fix the problem. If you have the same problem, don't forget that your webserver probably/hopefully doesn't run as root. Try logging in as your webserver's user or a user with similar permissions and try to run the commands yourself.
Tested on Ubuntu Server 10.04. I hope it helps you also on Arch Linux.
In PHP use shell_exec function:
Execute command via shell and return the complete output as a string.
It returns the output from the executed command or NULL if an error
occurred or the command produces no output.
<?php
$command = escapeshellcmd('/usr/custom/test.py');
$output = shell_exec($command);
echo $output;
?>
Into Python file test.py, verify this text in first line: (see shebang explain):
#!/usr/bin/env python
If you have several versions of Python installed, /usr/bin/env will
ensure the interpreter used is the first one on your environment's
$PATH. The alternative would be to hardcode something like
#!/usr/bin/python; that's ok, but less flexible.
In Unix, an executable file that's meant to be interpreted can indicate
what interpreter to use by having a #! at the start of the first line,
followed by the interpreter (and any flags it may need).
If you're talking about other platforms, of course, this rule does not
apply (but that "shebang line" does no harm, and will help if you ever
copy that script to a platform with a Unix base, such as Linux,
Mac, etc).
This applies when you run it in Unix by making it executable
(chmod +x myscript.py) and then running it directly: ./myscript.py,
rather than just python myscript.py
To make executable a file on unix-type platforms:
chmod +x myscript.py
Also Python file must have correct privileges (execution for user www-data / apache if PHP script runs in browser or curl)
and/or must be "executable". Also all commands into .py file must have correct privileges.
Taken from php manual:
Just a quick reminder for those trying to use shell_exec on a
unix-type platform and can't seem to get it to work. PHP executes as
the web user on the system (generally www for Apache), so you need to
make sure that the web user has rights to whatever files or
directories that you are trying to use in the shell_exec command.
Other wise, it won't appear to be doing anything.
I recommend using passthru and handling the output buffer directly:
ob_start();
passthru('/usr/bin/python2.7 /srv/http/assets/py/switch.py arg1 arg2');
$output = ob_get_clean();
If you want to know the return status of the command and get the entire stdout output you can actually use exec:
$command = 'ls';
exec($command, $out, $status);
$out is an array of all lines. $status is the return status. Very useful for debugging.
If you also want to see the stderr output you can either play with proc_open or simply add 2>&1 to your $command. The latter is often sufficient to get things working and way faster to "implement".
To clarify which command to use based on the situation
exec() - Execute an external program
system() - Execute an external program and display the output
passthru() - Execute an external program and display raw output
Source: http://php.net/manual/en/function.exec.php
Alejandro nailed it, adding clarification to the exception (Ubuntu or Debian) - I don't have the rep to add to the answer itself:
sudoers file:
sudo visudo
exception added:
www-data ALL=(ALL) NOPASSWD: ALL
In my case I needed to create a new folder in the www directory called scripts. Within scripts I added a new file called test.py.
I then used sudo chown www-data:root scripts and sudo chown www-data:root test.py.
Then I went to the new scripts directory and used sudo chmod +x test.py.
My test.py file it looks like this. Note the different Python version:
#!/usr/bin/env python3.5
print("Hello World!")
From php I now do this:
$message = exec("/var/www/scripts/test.py 2>&1");
print_r($message);
And you should see: Hello World!
The above methods seem to be complex. Use my method as a reference.
I have these two files:
run.php
mkdir.py
Here, I've created an HTML page which contains a GO button. Whenever you press this button a new folder will be created in directory whose path you have mentioned.
run.php
<html>
<body>
<head>
<title>
run
</title>
</head>
<form method="post">
<input type="submit" value="GO" name="GO">
</form>
</body>
</html>
<?php
if(isset($_POST['GO']))
{
shell_exec("python /var/www/html/lab/mkdir.py");
echo"success";
}
?>
mkdir.py
#!/usr/bin/env python
import os
os.makedirs("thisfolder");
This is so trivial, but just wanted to help anyone who already followed along Alejandro's suggestion but encountered this error:
sh: blabla.py: command not found
If anyone encountered that error, then a little change needs to be made to the php file by Alejandro:
$command = escapeshellcmd('python blabla.py');
All the options above create new system process. Which is a performance nightmare.
For this purpose I stitched together PHP module with "transparent" calls to Python.
https://github.com/kirmorozov/runpy
It may be tricky to compile, but will save system processes and will let you keep Python runtime between PHP calls.
Inspired by Alejandro Quiroz:
<?php
$command = escapeshellcmd('python test.py');
$output = shell_exec($command);
echo $output;
?>
Need to add Python, and don't need the path.
I'm trying to run a Python script from PHP using the following command:
exec('/usr/bin/python2.7 /srv/http/assets/py/switch.py arg1 arg2');
However, PHP simply doesn't produce any output. Error reporting is set to E_ALL and display_errors is on.
Here's what I've tried:
I used python2, /usr/bin/python2 and python2.7 instead of /usr/bin/python2.7
I also used a relative path instead of an absolute path which didn't change anything either.
I tried using the commands exec, shell_exec, system.
However, if I run
if (exec('echo TEST') == 'TEST')
{
echo 'exec works!';
}
it works perfectly fine while shutdown now doesn't do anything.
PHP has the permissions to access and execute the file.
EDIT: Thanks to Alejandro, I was able to fix the problem. If you have the same problem, don't forget that your webserver probably/hopefully doesn't run as root. Try logging in as your webserver's user or a user with similar permissions and try to run the commands yourself.
Tested on Ubuntu Server 10.04. I hope it helps you also on Arch Linux.
In PHP use shell_exec function:
Execute command via shell and return the complete output as a string.
It returns the output from the executed command or NULL if an error
occurred or the command produces no output.
<?php
$command = escapeshellcmd('/usr/custom/test.py');
$output = shell_exec($command);
echo $output;
?>
Into Python file test.py, verify this text in first line: (see shebang explain):
#!/usr/bin/env python
If you have several versions of Python installed, /usr/bin/env will
ensure the interpreter used is the first one on your environment's
$PATH. The alternative would be to hardcode something like
#!/usr/bin/python; that's ok, but less flexible.
In Unix, an executable file that's meant to be interpreted can indicate
what interpreter to use by having a #! at the start of the first line,
followed by the interpreter (and any flags it may need).
If you're talking about other platforms, of course, this rule does not
apply (but that "shebang line" does no harm, and will help if you ever
copy that script to a platform with a Unix base, such as Linux,
Mac, etc).
This applies when you run it in Unix by making it executable
(chmod +x myscript.py) and then running it directly: ./myscript.py,
rather than just python myscript.py
To make executable a file on unix-type platforms:
chmod +x myscript.py
Also Python file must have correct privileges (execution for user www-data / apache if PHP script runs in browser or curl)
and/or must be "executable". Also all commands into .py file must have correct privileges.
Taken from php manual:
Just a quick reminder for those trying to use shell_exec on a
unix-type platform and can't seem to get it to work. PHP executes as
the web user on the system (generally www for Apache), so you need to
make sure that the web user has rights to whatever files or
directories that you are trying to use in the shell_exec command.
Other wise, it won't appear to be doing anything.
I recommend using passthru and handling the output buffer directly:
ob_start();
passthru('/usr/bin/python2.7 /srv/http/assets/py/switch.py arg1 arg2');
$output = ob_get_clean();
If you want to know the return status of the command and get the entire stdout output you can actually use exec:
$command = 'ls';
exec($command, $out, $status);
$out is an array of all lines. $status is the return status. Very useful for debugging.
If you also want to see the stderr output you can either play with proc_open or simply add 2>&1 to your $command. The latter is often sufficient to get things working and way faster to "implement".
To clarify which command to use based on the situation
exec() - Execute an external program
system() - Execute an external program and display the output
passthru() - Execute an external program and display raw output
Source: http://php.net/manual/en/function.exec.php
Alejandro nailed it, adding clarification to the exception (Ubuntu or Debian) - I don't have the rep to add to the answer itself:
sudoers file:
sudo visudo
exception added:
www-data ALL=(ALL) NOPASSWD: ALL
In my case I needed to create a new folder in the www directory called scripts. Within scripts I added a new file called test.py.
I then used sudo chown www-data:root scripts and sudo chown www-data:root test.py.
Then I went to the new scripts directory and used sudo chmod +x test.py.
My test.py file it looks like this. Note the different Python version:
#!/usr/bin/env python3.5
print("Hello World!")
From php I now do this:
$message = exec("/var/www/scripts/test.py 2>&1");
print_r($message);
And you should see: Hello World!
The above methods seem to be complex. Use my method as a reference.
I have these two files:
run.php
mkdir.py
Here, I've created an HTML page which contains a GO button. Whenever you press this button a new folder will be created in directory whose path you have mentioned.
run.php
<html>
<body>
<head>
<title>
run
</title>
</head>
<form method="post">
<input type="submit" value="GO" name="GO">
</form>
</body>
</html>
<?php
if(isset($_POST['GO']))
{
shell_exec("python /var/www/html/lab/mkdir.py");
echo"success";
}
?>
mkdir.py
#!/usr/bin/env python
import os
os.makedirs("thisfolder");
This is so trivial, but just wanted to help anyone who already followed along Alejandro's suggestion but encountered this error:
sh: blabla.py: command not found
If anyone encountered that error, then a little change needs to be made to the php file by Alejandro:
$command = escapeshellcmd('python blabla.py');
All the options above create new system process. Which is a performance nightmare.
For this purpose I stitched together PHP module with "transparent" calls to Python.
https://github.com/kirmorozov/runpy
It may be tricky to compile, but will save system processes and will let you keep Python runtime between PHP calls.
Inspired by Alejandro Quiroz:
<?php
$command = escapeshellcmd('python test.py');
$output = shell_exec($command);
echo $output;
?>
Need to add Python, and don't need the path.
Is there a correct way to use php from the command line...or rather...is one way more correct than another ?
If you create a file, say test.php with the following code:
#!/usr/bin/php
<?php
print "This is a test".PHP_EOL;
print "This is another test!!";
?>
then chmod +x text.php (make it executable on linux).
You can then run in the following ways.....
./test.php
or
php test.php
I prefer just using ./test.php, but often see php test.php in examples.
ALSO
is the following correct syntax for the shebang line
#!/usr/bin/php
or is this more correct
#!/usr/bin/php -q
I've seen both, and see that the -q flag is to quiet the html stuff, but was wondering if
php compiled with cli compatibility really needs the -q flag ???
Thanks for your help :)
#!/usr/bin/php
On the first line of an interpreter script, the "#!", is the name of a program which should be used to interpret the contents of the file. For instance, if the first line contains
"#! /bin/sh"
, then the contents of the file are executed as a shell script.
In your case it means excute the script using the php file in /usr/bin location.
Using php test.php means that you are running your test.php with php so u don't need the first line. where as for ./test.php your file needs to be executable and first line is required.
And about the -q flag look at this
specifically on
rob 23-Mar-2007 11:48
There are better expanations if you see for -q. here's another one
If you are writing little php scripts for your own purposes, #! is fine.
If they are to be on some kind of world visible box (e.g. web server), then I'd say not so fine - since you have made them executable they are now a security risk.
I tend to use #! for perl, sh (which are always little private, non production things) and php somefile.php for PHP (which may or may not end up on a server).
It may help if you are on a Mac and use TextMate, though not entirely necessary.
My php location:
$which php
/opt/local/bin/php
The script:
#!/opt/local/bin/php
<?php
shell_exec("echo -n 'my-string' > out.txt");
?>
The -n to echo suppress the newline that is automatically added to all shell echo commands.
If I run the above php code from the shell:
chmod u+x myfile.php
./myfile.php
I end up with 'out.txt', the contents of which being:
-n my-string
If I run the exact same code within TextMate, the contents of 'out.txt' will be correct:
my-string
I can't figures out what php is up to with putting the literal string '-n' in the output. I really wonder why TextMate does the correct thing. I have checked that both are using the same php, php -i shows mostly the same stuff, of course there are differences as one is run within TextMate, the other in the shell, so one output has pointers to the file whereas the other doesn't. As far as I can tell, $PATH and $ENV are the same.
I have tried a handful of different methods to work around this none of which are working. I actually will not be able to use a workaround, as this has been distilled down to a simple case for posting to SO. My use case for this pipes to pbcopy, which I believed was a Mac OS X only feature, so I used >> redirection here because that is universal.
Ultimately, I want a result on my clipboard that does not have a trailing newline, which is dangerous as pasting that in a shell will execute whatever preceded it.
Thanks
Is it possible that php is calling a different echo than your shell built in echo? Many versions of echo do not support -n and will output it as part of your string.
You could try shell_exec("which echo"); to find out which it is running.
Note that printf will not display the new line unless you explicitly add it. So you can use which printf to figure out where this resides and call it instead.
shell_exec("/usr/local/bin/printf '%s' 'mystring' > out.txt");
PHP just defers the call to popen in Unix platforms. See the manual page for Mac OS X:
The command argument is a pointer to a null-terminated string containing a shell command line. This
command is passed to /bin/sh, using the -c flag; interpretation, if any, is performed by the shell.
So it should be the same as running /bin/sh -c "echo -n 'my-string' > out.txt"