I've been working on a local app over the last few days and I've noticed that one of my 'exec()' functions to call an external program didn't fire correctly. Upon further investigation it was obvious that the program did execute, but it quit prematurely as an important line utilizing 'file_get_contents()' didn't retrieve the contents of the file specified.
The file is a plaintext file without an extension. I'm guessing that 'file_get_contents()' is treating the file as a directory since there is no extension? It's strange because if I manually execute the same program from a web browser, everything works perfectly.
Here's an example line for clarity -
while(file_get_contents('plaintextfile') == "something"){
/// Do This
}
The above works just fine when I visit /program.php from a web browser, but when calling it like this it gives me a file/folder not found error for 'plaintextfile'.
exec('php /program.php', $output);
foreach($output as $output){
print $output . "<br>";
}
Thanks in advance to anyone who can shed some light on this situation. I'm really puzzled by this...
PHP as executed from the browser and executed by the command line (in the exec() call) may use different php.ini configurations, and may have different file search paths. The best course of action is to supply the full path to plaintextfile.
if(!file_get_contents('/path/to/plaintextfile')){
// file couldn't be read
}
Related
Last night I spent 5.5 hours trying make PHP execute and receive the output of Virtualenv’ed Python script. Nothing worked; except for scripts that were not Virtualenv’ed.
What I am trying to do:
I am trying to make PHP call a virtualenv’d install of the Newspaper lib output text when I call it.
What I have now:
PHP: (updated)
<?php
$output = exec('newspaper2/bin/python3 /var/www/html/components/python/test.py 2>&1', $output2);
print_r(error_get_last());
echo $output2;
echo $output;
…this works when using a non-virtualenv script
Python: (updated)
from newspaper import Article
url = 'http://example.com/'
article = Article(url)
article.download()
article.html
article.parse()
article.authors
article.publish_date
string = article.text
print(string)
What the issue is:
I can run the script that PHP is running from the command line and it outputs just fine.
What I have tried:
With PHP, (I have tried all the “exec” calls for PHP) it cannot seem to open the virtual environment and returns nothing.
Before the script I have called “python3” and a few other things to no avail.
Yes, I have chmoded it to be executable…
I feel like this should be so simple.
I have tried suggestions on other posts and all over the web to no avail.
Questions:
Did I set up the virtualenv wrong?
At the top of the Python script, instead of the “#!/usr/bin/env python3” should I call something else?
If so, where do I find it? Should I start from scratch and will that
help?
Thank you for your help;
PS: I am running Ubuntu16, PHP7 and I need to use Python3
In the virtualenv'ed scripts (i.e. installed via the setuptools' entry-points), you should not touch the shebang (#!... first line). It is populated by the virtualenv & setuptools & related tools.
If you specify your own shebang, then it is not virtualenv'ed script. In that case, call python directly:
exec('/path/to/venv/bin/python3 /var/www/html/components/python/testing.py');
Alternatively, you can put the absolute path to the virtualenv's python binary to the py-script, but this does not look a good idea.
Also, remember that virtualenvs are non-relocatable. So they should stay in the path where they were created.
Also note that exec() returns only the last line of the output. You probably want shell_exec() or exec('...', $output) to get the whole output.
Also, it is unclear what happens with your script, and what is being printed on stderr. Try this command to see what is the error:
exec('/path/to/script 2>&1', $output)
#OR:
exec('/path/to/venv/bin/python3 /path/to/script 2>&1', $output)
OK, I finally figured it out and learned a lot in the process. The newspaper lib that I am using by default tries to write to the base of the users home directory. In this case, it was attempting to write to www-data, /var/www.
To fix this:
Go to the settings.py file in the newspaper library.
Edit the variable DATA_DIRECTORY = '.newspaper_scraper' and change it to DATA_DIRECTORY = '.path/to/writable/directory'
Save the file and you should be good to go.
I have no idea why it was not returning the errors that would have explained this sooner.
Hope this helps anyone else.
Thank you so much Sergey Vasilyev for your help. I appreciate it greatly.
This particular PHP file works perfectly when executed via the browser. However, I'd like it to run on task scheduler in Windows so I set the scheduler to launch php.exe and point it to the correct file.
Task scheduler is basically doing the same thing as if I type it directly into the CLI I believe. Now, it seems to have worked a few times but now it repeatedly fails even when I manually call the task via CLI.
The relevant code is:
include_once("simple_html_dom.php");
$results = ....Some CURL Commands to retrieve data....
$html = str_get_html($results);
foreach($html->find('tr') as $tr)
{
....do stuff....
}
In CLI it says
Fatal error: Call to a member function find() on a non-object in C:\php\report.php on line...
Why does CLI find fault here and browser does not? Again, this has worked once or twice on CLI so it might some kind of time-out setting.
When you run the script on CLI, did you check if file_get_html() is returning FALSE?
If that is the case, maybe the script can't reach the resource from the terminal using curl for some reason (e.g.: proxy settings).
Make sure to check that case on what you get from that function with something like:
$html = str_get_html($results);
if ($html !== FALSE) {
// treat the success case.
}
All your answers led me to figure out the problem. I investigated the permissions angle but that didn't solve it. There is another 'include' file I have called common_functions.php which I also include. Permissions on that also didn't solve the problem.
However, the curl function is actually located in common_functions. Upon investigating that file, it contains references to cookies.txt where the path was not absolute. I had not setup my environment variables correctly so CLI could not find the cookie which made the Curl function fail.....I corrected and it works now.
Lesson learned. Thank you all for the clues you provided.
I've got some PHP code that I want to run as a background process. That code checks a database to see if it should do anything, and either does it or sleeps for awhile before checking again. When it does something, it prints some stuff to stdout, so, when I run the code from the command line, I typically redirect the output of the PHP process to a file in the obvious way: php code.php > code.log &.
The code itself works fine when it's run from the shell; I'm now trying to get it to run when launched from a web process -- I have a page that determines if the PHP process is running, and lets me start or stop it, depending. I can get the process started through something like:
$the_command = "/bin/php code.php > /tmp/code.out &";
$the_result = exec($the_command, $output, $retval);
but (and here's the problem!) the output file-- /tmp/code.out -- isn't getting created. I've tried all the variants of exec, shell_exec, and system, and none of them will create the file. (For now, I'm putting the file into /tmp to avoid ownership/permission problems, btw.) Am I missing something? Will redirection just not work in this case?
Seems like permission issues. One way to resolve this would be to:
rename your echo($data) statements to a function like fecho($data)
create a function fecho() like so
.
function fecho($data)
{
$fp = fopen('/tmp/code.out', 'a+');
fwrite($fp, $data);
fclose($fp);
}
Blurgh. After a day's hacking, this issue is finally resolved:
The scheme I originally proposed (exec of a statement with
redirection) works fine...
...EXCEPT it refuses to work in /tmp. I
created another directory outside of the server's webspace and opened
it up to apache, and everything works.
Why this is, I have no idea. But a few notes for future visitors:
I'm running a quite vanilla Fedora 17, Apache 2.2.23, and PHP 5.4.13.
There's nothing unusual about my /tmp configuration, as far as I know (translation: I've never modified whatever got set up with the basic OS installation).
My /tmp has a large number of directories of the form /tmp/systemd-private-Pf0qG9/, where the latter part is a different set of random characters. I found a few obsolete versions of my log files in a couple of those directories. I presume that this is some sort of Fedora-ism file system juju that I will confess to not understanding, and that these are orphaned files left over from some of my process hacking/killing.
exec(), shell_exec(), system(), and passthru() all seemed to work, once I got over the hump.
Bottom line: What should have worked does in fact work, as long as you do it in the right place. I will now excuse myself to take care of a large bottle of wine that has my name on it, and think about how my day might otherwise have been spent...
I load dynamically PHP class files with autoload.
And those files could be missing or corrupted by some reason.
Autoload will successfully report missing files so application logic could handle that. But if those files are corrupted, then the whole processing halts with blank screen for the user and "PHP Parse error: syntax error" in error log.
Is it possible to check syntax of PHP file from PHP code?
I've looked here: http://us.php.net/manual/en/function.php-check-syntax.php - it's deprecated.
And
exec("php -l $file");
seems to be a wrong way (http://bugs.php.net/bug.php?id=46339)
Thoughts?
You really shouldn't try to check for non-correct PHP files at execution time : it'll kill the response time of your application !
A "better way" would be to use php -l from command line when you're done modifying a PHP script ; or include it in your build process if you're using one ; or plug it as an SVN pre-commit hook if you're using SVN and can define SVN hooks.
In my opinion, almost any given solution would be better than checking that yourself at execution time !
Considering errors like the ones you want to avoid will probably won't happen often, it is probably better to... just let them happen.
ONly thing is : activate logs, and monitor them, the be able to detect quickly when tere is a problem :-)
Of course, this doesn't prevent you from dealing with the case of missing files ; but that's a different matter...
Another way you can make one php file in your root directory called
checkSyntax.php
<?php
for($i=1; $i < count($argv); $i++){
$temp = "php -l " . $argv[$i];
$output = exec($temp);
echo "\n$output";
}
?>
now, open your bashrc file to make a shortcut to run this file.
add below line to run checkSyntax.php
alias checkSyntaxErrors='php /root/checkSyntax.php'
and now goto your source directory do svn st.
it shows you list of files, now easily run the command.
checkSyntaxErrors file1.php file2.php .......
this will check all your files passing as arguments.
enjoy :)
In short: i can't see a way to do this, but have an idea which might be sufficient.
There are log monitoring programs or can filter the logs via standard tools for files with parse errors. If a file appears, you put the villain filename into a black list and your autoloader checks before load against this list.
With this method, at first time you'll serve a blank screen (assumig error reporting to the output are turned on on production servers) but the second will have a page without the faulty component.
In the autoloader you should have a list or naming scheme to always try to loading mandatory classes (other ways your application might be in an inconsistent state)
You could also do some unit testing, where you load the PHP you're dynamically executing and assert that exec("php -l $fileName") is valid. If you did that you'd be able to verify it once in your tests, generating it with appropriate variables, and have a reasonable level of confidence your PHP was good.
This is an old question, but it seems in recent php versions we can do this
try {
include_once($file);
} catch (\ParseError $e) {
// Parse error
} catch (\Throwable $e) {
// Any other error
}
This is driving me crazy. I'm trying to execute a command line statement on a windows box for my PHP web app. It's running on windows XP, IIS5.1. The web app is running fine, but I cannot get #exec() to work with a specific contactenated variable. My command construction looks like this:
$cmd = ($config->svn." cat ".$this->repConfig->svnParams().quote($path).' -r '.$rev.' > '.quote($filename));
This command does not work as is above, when it generates the following string:
svn --non-interactive --config-dir /tmp cat "file:///c:/temp/test/acccount/dbo_sproctest.sql" -r 1 > "C:\Inetpub\sites\websvn\temp\wsv5B45.tmp"
If I copy/paste this to my own command line, it works fine.
If I hard code that very same path instead of adding it with the variable, it works! I've tried with and without quotes around the file name. I've tried with and without quotes around the entire command. I've tried other directories. I've tried passing an output paramter to exec(), and it comes back empty (Array () ). I've tried redirecting the output of the error stream of the command to a file, and that error output file never gets created.
The only thing I can possibly concieve of is that exec() is failing silently. What on earth am I doing wrong here? If I hard code the file path, using the same dir structure and filename, it works fine. If I don't, it doesn't.
Maybe the slashes () in the file path aren't being escaped properly, but when I do it manually with single quotes they are not considered escape sequences??
UPDATE:
I took the # off of exec, and still not seeing any errors.
I gave the full path to SVN, still no luck. It should be noted that the command worked fine before with the non-full path SVN so long as I manually specify the file destination for cat.
Update 2: RE: Kieth
I'm calling exec by trying both:
exec($cmd);
or
exec($cmd, $out);
My php.ini already had safe_mode = 0.
I added error_reporting(E_ALL); and didn't see anything new
If I echo (or print_r) my exec call, I am not actually seing anything
If I echo (or print_r) my exec call when included an output var, I get an empty arr
Update 3
I tried both escapeshellcmd and escapeshellarg to no avail (good idea though).
I should add that the file is being created through invoking
tempnam("temp", "wbsn");
The fact that it works just fine if I manually specify the string instead of letting it be generated by tempname seems to suggests that the source of the problem, but I can't figure out how. I did a comparison of the manual string with the one generated, and it came back as a match.
#exec will always fail silently, because # is PHP's error suppression operator.
Not sure if this will help you since you're on Windows and I'm on Linux, but I ran into this same problem of silent errors from PHP exec(). I figured out that the command I was attempting to issue (nconvert) sends its error messages to the standard error stream, not standard out. So I added
2>&1
at the end of the command line to redirect my error back to the standard stream. Then I could see that nconvert was giving me a permission denied error.
It could be that the PATH isn't the same from your php script vs your user account. Try removing the # and see if it's trying to throw an error.
In addition, you may want to try putting the full filesystem path to the SVN executable.
Well, if removing # isn't working, try including this at the start of the piece of code you're running:
error_reporting(E_ALL);
That'll turn on full error reporting, just in case you have it turned down or disabled in your php.ini file.
I don't suppose you could show us how exactly you're calling exec()? Could you also check to make sure you're not accidentally running the script in safe mode? Are you echoing out what exec() is returning, and if so, what is it returning?
I'm not a big fan of adding another response, but if I just edit my previous response, you may not see it.
PHP has some special escaping commands for shell scripts: escapeshellcmd and escapeshellarg.
I think you should be able to use escapeshellcmd around your entire $cmd, but I'm not sure.
Have you tried echo exec("dir") or something simple to see if exec() is working at all?
I don't know what going on, but I at least have a workaround.
This works:
$tmp = tempnam("./", "wbsn");
$filename = dirname($tmp).'\\temp\\'.basename($tmp);
This, however, does not, but I would have expected it to generate the same path (diff file name since it's a new tempnam()).
$tmp = tempnam("temp", "wbsn");
Also, this does not work, which I also would expect to generate the same thing:
$tmp = tempnam("temp", "wbsn");
$filename = dirname($tmp).'\\'.basename($tmp);
All 3 of these solutions appear to generate the same file paths, but only the first one actually works when used in my exec. I have no clue why it does not.
Visual inspection (echo) of all 3 of these appear to generate the same paths (with the exception of filenames differing, of course). A string comparison of dirname() of each of these 3 shows as a match. I have no clue what the deal is, but the first one is a workaround.
A very useful trick when debugging shell exec problems is to place an "echo" at the beginning of the command. Make sure that you can view the standard output somewhere.
This will let you examine the command for any obvious problems. Perhaps it has a wildcard that is expanding unexpectedly, or perhaps the shell quoting is not exactly right. The echo will let you see this, and you can cut and paste the echoed command into another shell to see if it is working properly.
My advice will be to switch from WIN, IIS to linux, but as alternative you can try this:
function exec_alt($cmd) {
exec($cmd, $output);
if (!$output) {
/**
* FIXME: for some reason exec() returns empty output array #mine,'s machine.
* Somehow proc_open() approach (below) works, but doesn't work at
* test machines - same empty output with both pipes and temporary
* files (not we bypass shell wrapper). So use it as a fallback.
*/
$output = array();
$handle = proc_open($cmd, array(1 => array('pipe', 'w')), $pipes, null, null, array('bypass_shell' => true));
if (is_resource($handle)) {
$output = explode("\n", stream_get_contents($pipes[1]));
fclose($pipes[1]);
proc_close($handle);
}
}
return $output; }