I have a PHP script that executes a shell command to find the common items between two files given. This is the beginning of my PHP script:
$E7Bonded_File = "/opt/IBM/custom/NAC_Dslam/junk/PortParameter_E7_Bonded_cust_stats.csv";
$E7Single_File = "/opt/IBM/custom/NAC_Dslam/junk/PortParameter_E7_Single_cust_stats.csv";
$E7Common_File = "/opt/IBM/custom/NAC_Dslam/junk/Common_tn_SingleBonded_E7_cust_stats.csv";
//only do this once, with old single/bonded filenames. This will be a list to add to the existing Common file.
exec ("comm -12 <(cut -d ',' -f2 $E7Single_File| sort) <(cut -d ',' -f2 $E7Bonded_File| sort)", $outputCommon);
I see this error message when I run the script:
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `comm -12 <(cut -d ',' -f2 /opt/IBM/custom/NAC_Dslam/junk/PortParameter_E7_Single_cust_stats.csv| sort) <(cut -d ',' -f2 /opt/IBM/custom/NAC_Dslam/junk/PortParameter_E7_Bonded_cust_stats.csv| sort)'
I checked, and the parentheses look ok for my exec() line.
When I run the shell command at the command line it returns a listing of numbers like I expect:
comm -12 <(cut -d ',' -f2 junk/PortParameter_E7_Single_cust_stats.csv| sort) <(cut -d ',' -f2 junk/PortParameter_E7_Bonded_cust_stats.csv| sort)
I looked online and I seem to be using exec() correctly. I want the numbers returned to be stored as an array, $outputCommon.
Any ideas about this error message?
*********Update on answer***************
My solution wound up being a combination of both mario and miken32/my co-worker
Adding #!/bin/bash at the top of my php script, and
Adding /bin/bash -c as follows:
exec("/bin/bash -c /opt/IBM/custom/NAC_Dslam/Common_list.sh", $outputShell);
After I moved the comm part to a shell script:
Common_list.sh:
#!/bin/bash
comm -12 <(cut -d ',' -f2 /opt/IBM/custom/NAC_Dslam/junk/PortParameter_E7_Single_cust_stats.csv| sort) <(cut -d ',' -f2 /opt/IBM/custom/NAC_Dslam/junk/PortParameter_E7_Bonded_cust_stats.csv| sort)
This error typically comes up for non-bash shells, which don't support <() expression pipes.
On Ubuntu/Debian servers the default /bin/sh is typically dash.
Check for symlinked binaries:
me#snip:~$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Jul 16 2017 /bin/sh -> dash
Or as #theotherguy mentioned, bash runs as restricted_shell when started as sh.
See $_ENV[SHELL] on what Apache/PHP use as default. Change environment vars.
Either adapt that, or wrap the shell_exec cmdline with /bin/bash -c '…'.
Probably the easiest solution would be to make this into an executable script on the server:
#!/bin/bash
if [[ ! -r "$1" ]] || [[ ! -r "$2" ]]; then
printf "File not found\n" >&2
exit 1
fi
comm -12 <(cut -d ',' -f2 "$1"| sort) <(cut -d ',' -f2 "$2"| sort)
And then call that from PHP:
$E7Bonded_File = escapeshellarg("/opt/IBM/custom/NAC_Dslam/junk/PortParameter_E7_Bonded_cust_stats.csv");
$E7Single_File = escapeshellarg("/opt/IBM/custom/NAC_Dslam/junk/PortParameter_E7_Single_cust_stats.csv");
//only do this once, with old single/bonded filenames. This will be a list to add to the existing Common file.
exec ("/usr/local/bin/your_script.sh $E7Single_File $E7Bonded_File", $outputCommon);
Always escape your shell arguments with escapeshellarg() even if you think they're safe.
when checking the script with a linter, it complains that:
comm -12 <(cut -d ',' -f2 junk/PortParameter_E7_Single_cust_stats.csv| sort) <(cut -d ',' -f2 junk/PortParameter_E7_Bonded_cust_stats.csv| sort)
^-- SC2039: In POSIX sh, process substitution is undefined.
this happens when I define shebang #!/bin/sh, while #!/bin/bash does not complain... therefore the answer might be, to run the script with /usr/bin/bash. escapeshellarg() is indeed useful.
...
try shell_exec(), simply because exec() should not invoke any shell. the one returns the output as string and the other can return an array.
alternatively, you can explicitly invoke bash with exec():
exec('/bin/bash -c " command "', $stdOut);
So in case anyone else still needs to get substitution working in command execution in PHP, there is super dumb simple thing to do:
$a = 'Hello substitution';
var_dump(shell_exec("bash -c 'cat <(echo $a)'"));
Results in:
string(19) "Hello substitution
"
Which is expected/desired.
Related
I have a cron issue with curl:
curl -w "%{time_total}\n" -o /dev/null -s http://myurl.com >> ~/log
works great and add a line in log file with total_time.
But the same line with cron doesn't do anything.
It's not a path problem because curl http://myurl.com >> ~/log works.
% is a special character for crontab. From man 5 crontab:
The "sixth" field (the rest of the line) specifies the command to be
run. The entire command portion of the line, up to a newline or a
"%" character, will be executed by /bin/sh or by the shell specified
in the SHELL variable of the cronfile. A "%" character in the
command, unless escaped with a backslash (\), will be changed into
newline characters, and all data after the first % will be sent to
the command as standard input.
So you need to escape the % character:
curl -w "%{time_total}\n" -o /dev/null -s http://myurl.com >> ~/log
to
curl -w "\%{time_total}\n" -o /dev/null -s http://myurl.com >> ~/log
^
I would want to run strict syntax check:
$ php -d error_reporting=32767 -l test.php
for all php files inside my project in Travis CI.
I tried to use find but it always just returns 0 even if the command for exec flag fails
$ find . -type f -name "*.php" -exec php -d error_reporting=32767 -l {} \;
PHP Parse error: syntax error, unexpected end of file, expecting ',' or ';' in ./test.php on line 3
$ echo $?
0
I solved this by checking if the find + php returns anything into the STDERR.
before_script:
- '! find . -type f -name "*.php" -exec php -d error_reporting=32767 -l {} \; 2>&1 >&- | grep "^"'
How does this work? Adding:
2>&1 >&-
after any command removes STDOUT and redirects STDERR to STDOUT.
Then we can just check if the output contains any lines with grep:
| grep "^"
Then because grep returns 0 if it finds anything and 1 if it doesn't find anything we need to negate the end result by using exclamation mark in the start of this command. Otherwise this command would fail when everything is okay and success when things are failing.
How can I get the CPU % consumed by a Linux process with PHP or by bash? I tried to find any utility for this but couldn't. All I found was always getting the same result for same reason.
I found this amazing answer here: https://unix.stackexchange.com/questions/554/how-to-monitor-cpu-memory-usage-of-a-single-process
To use that information on a script you can do this:
calcPercCpu.sh
#!/bin/bash
nPid=$1;
nTimes=10; # customize it
delay=0.1; # customize it
strCalc=`top -d $delay -b -n $nTimes -p $nPid \
|grep $nPid \
|sed -r -e "s;\s\s*; ;g" -e "s;^ *;;" \
|cut -d' ' -f9 \
|tr '\n' '+' \
|sed -r -e "s;(.*)[+]$;\1;" -e "s/.*/scale=2;(&)\/$nTimes/"`;
nPercCpu=`echo "$strCalc" |bc -l`
echo $nPercCpu
use like: calcPercCpu.sh 1234 where 1234 is the pid
For the specified $nPid, it will measure the average of 10 snapshots of the cpu usage in a whole of 1 second (delay of 0.1s each * nTimes=10); that provides a good and fast accurate result of what is happening in the very moment.
Tweak the variables to your needs.
i try to execute a grep command inside a php shell_exec. And it works fine besides it fails when i have a underscore in the search word. I can not seem to figure out why this fails because of a underscore since, a grep command with a underscore in search word works in shell code below:
$output = shell_exec("grep -l -r '$search_word'");
The content in search_word variable is dynamic from database but the word that gives me trouble is base_64
Try like this:
$output = shell_exec("grep -l -r '$search_word' ./*");
Before PHP spawns a subprocess your command will be $search_word evaluated:
grep -l -r '....'
# So in $search_word is set to `john doe` it will become:
grep -l -r 'john doe'
How PHP behaves I'm not sure, it might be stalling waiting for the process to finish, it might have been closing stdin already.
Your above command will expect input from stdin because no file name is specified, breakdown:
grep [option]... [pattern] [file]...
-l will only print file name of the matched file
-r recursive search.
TLDR: You properly want to specify a file / directory to search in:
$output = shell_exec("grep -l -r '$search_word' .");
// Or maybe
$output = shell_exec("grep -l -r '${search}_word' ."); # will use $search variable as an input from PHP while _word is a string now.
I am extracting information from proftpd logs. I have to call this one-liner from a PHP script but it does not work anymore from there.
This is the original line, which works:
(gunzip -c xferlog*.gz; cat xferlog?(*)!(.gz)) | grep 'host [0-9]\+ file a _ o r ftpuser' | sort -k 5n,5 -k 2M,2 -k 3n,3 -k 4,4 | tail -1 | cut -c 1-24
This is the error I got when executed in PHP:
$cmd = "(gunzip -c $logFile*.gz; cat $logFile?(*)!(.gz)) | grep '$host [0-9]\+ $file a _ o r $ftpUser' | sort -k 5n,5 -k 2M,2 -k 3n,3 -k 4,4 | tail -1 | cut -c 1-24";
exec($cmd);
sh: Syntax error: "(" unexpected (expecting ")")
I tried several bash scripts that would be called by PHP, but it has not been successful. I had errors like:
bash: command substitution: line 9: syntax error near unexpected token `('
bash: command substitution: line 9: `cat ${LOGS}?(*)!(.gz)'
or
bash: ./extract_date_in_xferlog.sh: line 8: syntax error near unexpected token `('
bash: ./extract_date_in_xferlog.sh: line 8: `(gunzip -c ${LOGS}*.gz; cat ${LOGS}?(*)!(.gz)) | grep "$HOST [0-9]\+ $FILE a _ o r $USER" | sort -k 5n,5 -k 2M,2 -k 3n,3 -k 4,4 | tail -1 | cut -c 1-24'
I am a bit confused, thank you for your help!
The weird wildcard uses extended globbing. You need to enable extglob either as part of your script (probably better) or in your Bash setup (probably where it was before, and then it broke when somebody changed it for unrelated reasons).
You're probably not escaping the quotes correctly in the script.
I suggest handling the shell command as a single quoted string assuming you don't want to embed PHP variables in the shell command, and then making sure that all the single quotes in the command are escaped with \' to avoid prematurely terminating the PHP string.
Alternatively you could use a HEREDOC or NOWDOC style string to avoid escaping issues.