I'm using Ant to build my project, generate docs and check coding strandard violations. I use some external tool through exec task. Exec prefixes the output lines with [exec]
This string messes my output up.
If I specify the -e switch in the command line, it remains quiet.
Is there a way to
specify the -e switch on every run in the build.xml file?
or something else equivalent.
Maybe the Java System Properties will help ... see http://ant.apache.org/manual/running.html#sysprops
In particular, I think that setting the property build.compiler.emacs to true may help. Anyway, I doubt that setting (or, re-setting it via ant-extensions) it in the build file helps, as properties are designed to be immutable.
Related
Im trying to call a python file containing a sentence/word tokenizer from my php file like this:
$output = shell_exec('python tokenizer.py $sentence')
I've tried single exec, full paths to python and tokenizer.py, wrapping $sentence in double quotes. But logically, It should not be the problem because calling print(1) at the beginning of python the python code before actually using any nltk packages makes $output equal to '1'. So I came to conclusion that the problem here is the nltk itself, like the path to the modules is not correct or something...
But, calling python from the shell using the same command as above gives me fully tokenized output! To conclude: looks like when calling python from php magically 'turns off' nltk, while it fully works when executed from the shell.
Here's the part of the python code I am using:
import sys
import nltk
from nltk.tokenize import sent_tokenize
sample_text2 = sys.argv[1]
gust = sent_tokenize(sample_text2)
#print(1) here doesn't work, but everywhere above (before calling sent_tokenize) it does.
The server's running on CentOS (Linux), I am accessing it via SSH.
Obvious question: What am I doing wrong here with PHP? Or generally? Any alternatives?
EDIT
As visible in dvhh's answer and its comments, the situation happened because there were two versions installed on the server (2.6 and 2.7), while the www user had access to 2.6 and through console, the default version was 2.7. The solution was to change the default python to 2.7 for both cases and to put the nltk modules to one of the dependency folders. (Or append the dependency directory using sys.path.append)
Your php script is executed by the www user.
You could check if the python script interpreter is correctly called, it is usually in one of the directory in the PATH environment variable (like /usr/bin/python), but the www user don't have a PATH environment variable set.
Solution specify the whole path to your python interpreter in your shell_exec call ( also specify the full path to your script when you're at it )
What about the path the nltk library is installed, you could check if the python interpreter would correctly look for it by looking at the sys.path while running python with the www user.
Diagnostic : use the shell_exec call to run a python script to print the sys.path values
Solution : append the library path to the sys.path in your python script before the import nltk
These would be the most obvious solutions considering the information provided in the question.
Update :
As there is 2 version version of python installed (on that haven't got the library installed ), it is recommended to specify the path to the desired interpreter. The first solution help correct the issue.
In unix like system I would recommend using which python command to determine the path of your default python interpreter.
Option 1
Setup a simple python httpserver listening on localhost. This old answer might help but there are plenty of howtos out there. The advantage is that you don't have the overhead of starting the python interpreter each time the ntlk stuff needs to be executed and you don't have to worry about shell script executions, permissions etc. Disadvantage is a little of extra work and a little overhead.
Option 2
Using a task queue. Whatever said and done it's not safe to execute commands from your web facing PHP scripts. If you are already using RabbitMQ or something similar you can use that here. Or else if you are using redis you can use the lpush, rpop methods to make redis behave like a queue. Disadvantage: the result is not immidiately available.
Option 3
Anbother strategy for your php script to enter the data into a table and setup your python script to run as a cron job to check the table once a minute. Disadvantage: the result is not immidiately available.
Option 4
Your current choice but please make sure that you have escaped the data properly by #lafor if this option is chosen #dvhh 's answer ought to work.
In my case it wasn't problem of python version but problem (as I soon found) of the nltk_data folder. First I thought it was as permission problem, so I changed the permissions to 777, but that didn't work.
I had to copy the folder.
First you have to find where was the nltk folder installed. You will find it by running python3 command from your bash and then put following lines:
import nltk
nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')
It will say something like:
[nltk_data] Downloading package punkt to /root/nltk_data...
[nltk_data] Package punkt is already up-to-date!
Now you know you have it installed eg in /root folder (i had it there).
Then put in your python script which is called by php this:
import nltk
print(nltk.data.path)
It will output something like: (note the folders are another if you run it from php script or if you call it from command line)
['/var/www/nltk_data', '/usr/nltk_data', '/usr/share/nltk_data', '/usr/lib/nltk_data', '/usr/share/nltk_data', '/usr/local/share/nltk_data', '/usr/lib/nltk_data', '/usr/local/lib/nltk_data']
Now just copy your original folder to any of the above folders. I did:
cp /root/nltk_data /var/www/nltk_data -r
Voila. It works now.
I'd use the following code:
$SERVER_PATH = dirname(__FILE__);
shell_exec($PHP_LOCATION.' '.$SERVER_PATH."/script.php?k1=v1&k2=v2 > /dev/null 2>/dev/null &");
Where:
$PHP_LOCATION should contain the path to PHP,
$SERVER_PATH - is current working directory (fortunately the script to run is in the same directory),
> /dev/null 2>/dev/null & added to make this call asynchronous (taken from Asynchronous shell exec in PHP question)
This code has two problems:
As far as I remember ?k1=v1&k2=v2 will work for web-call only, so in this particular case parameters will not be passed to the script.
I don't really know how to init the $PHP_LOCATION variable to be flexible and to work on the most hosts.
I conducted some research regarding both problems:
To solve 1 suggested to use -- 'parameters_string' but it is also recommended to modify the script to parse parameters string which looks a bit clumsy. Is there a better solution?
To solve 2 I found a solution to use PHP_BINARY but this is a PHP 5.4+ case (I'm using 5.3). But the original question was about to run PHP of the same version as the original script version. So for me (as I use PHP 5.3 only) is there probably a solution?
EDIT 0
Let me do some explanation why I stuck to this weird (for PHP) approach:
Those PHP scripts should be separate from each other:
one of those will analyze the data and
the second will generate PNG graphs as a final result.
Those scripts aren't intended to run simultaneously, this means that the second can run at it's own schedule it is only needed that the run should be upon its data will be ready (which done by the first script). So no data should be passed back from second script (child) to the first (parent).
EDIT 1
As seeing from most of the comments the main discussion goes to forking direction. However I'd like to make stress on the point 1 and 2 asked in the original questions. I have some reasons to solve the task in the way I pointed out and I tried to point all that reason. If some of my points looks weird, please post a comment - I will make it more clear or I will change the main question.
Thank you in advance!
How to get executable
Assuming you're using Linux, you can use:
function getBinaryRunner($binary)
{
return trim(shell_exec('which '.$binary));
}
For instance, same may be used for checking if needed stuff is installed:
function checkIfCommandExists($command)
{
$result = shell_exec('which '.$command);
return !empty($result);
}
Some points:
Yes, it will work only for Linux
You should be careful with user input if it is allowed to be passed to shell commands: escapeshellarg() and company
Indeed, normally PHP should not be used for stuff like this as if it is about asynchronous requests, better to either implement forking or run commands from external workers.
How to pass parameters
With doing shell_exec() via file system path you're accessing the file and, obviously, all "GET" parameters are becoming just part of file name, it is no longer "URI" as there is no web-server to process that. So you have two options:
Invoke call via accessing your web-server. So it will be like:
//Yes, you will use wget or, better, curl to make web-request from CLI
shell_exec('wget http://your.web-server.domain/script.php?foo=bar');
Downside here: if you'll access your web-server via public DNS, it will cause network gap and all processing overheads. Benefit - obviously, you will not have to expect anything else in your script and make no distinction between CLI and non-CLI calls
Use $_SERVER array in your script and pass parameters as it should be with CLI:
shell_exec('/usr/bin/php /path/to/script.php foo bar');
//inside your script.php you will see:
//$_SERVER['argv'][0] is "script.php"
//$_SERVER['argv'][1] is "foo"
//$_SERVER['argv'][2] is "bar"
Yes, it will require modification in the script, and, probably, some logic of how to map "regular" web-requests and CLI ones. I would suggest even to think of separating CLI-related stuff to different scripts bundle so not to mess that logic.
More about "asynchronous run"
When you do php script.php & you just run it in background mode. That, however, still keeps parent-child relation for your process. That means - if parent process dies, it's childs will also be removed. To be precise, SIGHUP will be triggered and to avoid this situation you should use nohup command. It will allow to emulate "detaching" of a process and therefore making it's run reliable and independent of circumstances happening to parent process.
I am using PHP on Windows machin. I also use Dev C++. I can perfectly compile .cpp file on CMD using this command:
g++ hello.cpp -O3 -o hello.exe
Now what I am trying to do is running the same command using php system() function, so it looks like this:
system("g++ c:\wamp\www\grader\hello.cpp -O3 -o C:\wamp\www\grader\hello.exe");
but it doesn't compile. I am lost, please tell me what am I missing?
I also looked up at this question and thats exactly what I need, but I couldnt find a usefull solution for my case there:
Php script to compile c++ file and run the executable file with input file
Use the PHP exec command.
echo exec('g++ hello.cpp -O3 -o hello.exe');
should work.
There's a whole family of different exec & system commands in PHP, see here:
http://www.php.net/manual/en/ref.exec.php
If you want the output into a variable, then use :
$variable = exec('g++ hello.cpp -O3 -o hello.exe');
If that doesn't work, then make sure that g++ is available in your path, and that your logged in with sufficient enough privliges to allow it to execute.
You may find also that it's failing beacuse PHP is essentially being executed by your web server (Unless your also running PHP from the cmd prompt) , and the web server user ID may not have write access to the folder where G++ is trying to create the output file.
Temporarily granting write access to 'Everybody' on the output folder will verify if that is the case.
Two things:
You are using double quotes and are not escaping the \ inside the path.
You are not using a full path to g++.
The first one is important as \ followed by something has a special meaning in such a string (you might know \n as new line), the second one is relevant since the PHP environment might have a different search path.
A solution might be
system("c:\\path\\to\\g++ c:\\wamp\\www\\grader\\hello.cpp -O3 -o C:\\wamp\\www\\grader\\hello.exe");
Alternatively you can use single quotes, intead of double quotes, they use diffeent,less strict escaping rules
system('c:\path\to\g++ c:\wamp\www\grader\hello.cpp -O3 -o C:\wamp\www\grader\hello.exe');
or use / instead of \, which is supported by windows, too.
system("c:/path/to/g++ c:/wamp/www/grader/hello.cpp -O3 -o C:/wamp/www/grader/hello.exe");
What you do is your choice, while many might consider the first one as ugly, and the last one as bad style on Windows ;-)
Thanks to everyone. I tried to run the codes given in above posts and it worked like a charm.
I ran the following code using my browser
$var = exec("g++ C:/wamp/www/cpp/hello.cpp -O3 -o C:/wamp/www/cpp/hello.exe");
echo $var;
The exe file is created. I am able to see the result when i run the exe file but the problem is when i run the above code in the browser, the result is not displayed on the webpage. I gave full access permission to all users but still give does not show the result on the webpage.
I really need help on this since i am doing a project on simulated annealing where i want to get the result from compiled c++ program and display it in the webpage with some jquery highcharts.
Thanks again to all, it has helped me alot and i have learnt alot as well.
I have a program that returns a comma-separated string of numbers after doing some background processing. I intended to run this in symfony using shell_exec; however, all I get is NULL (revealed through a var_dump(). I tried the following debugging steps.
I ran the file (it's a PHP class) through a command-line lime unit test in Symfony - it works and gives the correct result there.
Just to check, I tried a simple command ls -l at the same place to see whether I would get anything. Again, I had the same problem - the var_dump in the browser showed NULL, but it worked through the command line.
What could be the problem? Are there restrictions on running shell_exec() in a browser?
EDIT: Just to clarify, shell_exec() commands work when I run them as standalone php scripts on the web server (for example, by putting them in my document root. They don't seem to be working under the symfony framework, for some reason.
I finally solved it, and it turned out to be something quite simple, and quite unrelated.
The shell command I was running was in this format: face_query -D args. I didn't realize that Apache would be executing PHP as user www-data and thus the program face_query wouldn't be in the PATH (the directory is actually ~/bin). Changing the program name to the full path of the program solved it.
I also gather from this that only commands which www-data has permission to execute can be run. In this case, www-data is in the same group as my user, but it might be a problem otherwise.
Have you tried using exec? Or one of the other variants. I am never sure of which one to use and always lump with exec.
http://uk.php.net/manual/en/function.exec.php
Is your web server running php in safe mode?
Note: This function is disabled when PHP is running in safe mode.
From: http://php.net/manual/en/function.shell-exec.php
like func("rm -rf /usr/local/)
Which 'func' will you use to do that?
It depends on what you want to obtain.
If you want to display the ouput of the command that has been executed, passthru does the trick :
This function should be used in place
of exec() or system() when the output
from the Unix command is binary data
which needs to be passed directly back
to the browser. A common use for this
is to execute something like the
pbmplus utilities that can output an
image stream directly.
If you want to get the output of the command in a variable as a whole string, you can use shell_exec (which is the same as using the backtick operator) :
Execute command via shell and return
the complete output as a string
And, if you want to get it line by line, you'll use exec :
If the output argument is present,
then the specified array will be
filled with every line of output from
the command. Trailing whitespace, such
as \n, is not included in this array.
And, finally, if you need a bit more control, you'll use proc_open -- see the documentation, about that one ; it's a bit more tricky, I guess.
Still, when using those :
Don't forget you are calling a program that has to be available on the host machine
This means if you are using a Linux command, your application will most likely not work on windows
Depending on the configuration of PHP, it is possible that only a couple of commands could be available ; see safe_mode_exec_dir, about that
Using this means launching another processus -- probably not that good for performances
So, I'd use those only when it is really necessary.
You would use PHP's exec function. Other options include system, passthru, and the backtick operator.
Personally I think proc_open is the best approach.
http://php.net/proc_open
This gives you far more control over input, output and let's you check it's exit status and STDOUT, which is more than most other commands for system calls. It also doesn't hold up the system while the command exits.