Using PHP to call Virtualenv’ed Python Script - php

Question

Last night I spent 5.5 hours trying make PHP execute and receive the output of Virtualenv’ed Python script. Nothing worked; except for scripts that were not Virtualenv’ed.
What I am trying to do:
I am trying to make PHP call a virtualenv’d install of the Newspaper lib output text when I call it.
What I have now:
PHP: (updated)
<?php
$output = exec('newspaper2/bin/python3 /var/www/html/components/python/test.py 2>&1', $output2);
print_r(error_get_last());
echo $output2;
echo $output;
…this works when using a non-virtualenv script
Python: (updated)
from newspaper import Article
url = 'http://example.com/'
article = Article(url)
article.download()
article.html
article.parse()
article.authors
article.publish_date
string = article.text
print(string)
What the issue is:
I can run the script that PHP is running from the command line and it outputs just fine.
What I have tried:
With PHP, (I have tried all the “exec” calls for PHP) it cannot seem to open the virtual environment and returns nothing.
Before the script I have called “python3” and a few other things to no avail.
Yes, I have chmoded it to be executable…
I feel like this should be so simple.
I have tried suggestions on other posts and all over the web to no avail.
Questions:
Did I set up the virtualenv wrong?
At the top of the Python script, instead of the “#!/usr/bin/env python3” should I call something else?
If so, where do I find it? Should I start from scratch and will that
help?
Thank you for your help;
PS: I am running Ubuntu16, PHP7 and I need to use Python3

Answer 1

In the virtualenv'ed scripts (i.e. installed via the setuptools' entry-points), you should not touch the shebang (#!... first line). It is populated by the virtualenv & setuptools & related tools.
If you specify your own shebang, then it is not virtualenv'ed script. In that case, call python directly:
exec('/path/to/venv/bin/python3 /var/www/html/components/python/testing.py');
Alternatively, you can put the absolute path to the virtualenv's python binary to the py-script, but this does not look a good idea.
Also, remember that virtualenvs are non-relocatable. So they should stay in the path where they were created.
Also note that exec() returns only the last line of the output. You probably want shell_exec() or exec('...', $output) to get the whole output.
Also, it is unclear what happens with your script, and what is being printed on stderr. Try this command to see what is the error:
exec('/path/to/script 2>&1', $output)
#OR:
exec('/path/to/venv/bin/python3 /path/to/script 2>&1', $output)

Answer 2

OK, I finally figured it out and learned a lot in the process. The newspaper lib that I am using by default tries to write to the base of the users home directory. In this case, it was attempting to write to www-data, /var/www.
To fix this:
Go to the settings.py file in the newspaper library.
Edit the variable DATA_DIRECTORY = '.newspaper_scraper' and change it to DATA_DIRECTORY = '.path/to/writable/directory'
Save the file and you should be good to go.
I have no idea why it was not returning the errors that would have explained this sooner.
Hope this helps anyone else.
Thank you so much Sergey Vasilyev for your help. I appreciate it greatly.