executing Python script in PHP and exchanging data between the two - php

Is it possible to run a Python script within PHP and transferring variables from each other ?
I have a class that scraps websites for data in a certain global way. i want to make it go a lot more specific and already have pythons scripts specific to several website.
I am looking for a way to incorporate those inside my class.
Is safe and reliable data transfer between the two even possible ? if so how difficult it is to get something like that going ?

You can generally communicate between languages by using common language formats, and using stdin and stdout to communicate the data.
Example with PHP/Python using a shell argument to send the initial data via JSON
PHP:
// This is the data you want to pass to Python
$data = array('as', 'df', 'gh');
// Execute the python script with the JSON data
$result = shell_exec('python /path/to/myScript.py ' . escapeshellarg(json_encode($data)));
// Decode the result
$resultData = json_decode($result, true);
// This will contain: array('status' => 'Yes!')
var_dump($resultData);
Python:
import sys, json
# Load the data that PHP sent us
try:
data = json.loads(sys.argv[1])
except:
print "ERROR"
sys.exit(1)
# Generate some data to send to PHP
result = {'status': 'Yes!'}
# Send it to stdout (to PHP)
print json.dumps(result)

You are looking for "interprocess communication" (IPC) - you could use something like XML-RPC, which basically lets you call a function in a remote process, and handles the translation of all the argument data-types between languages (so you could call a PHP function from Python, or vice versa - as long as the arguments are of a supported type)
Python has a builtin XML-RPC server and a client
The phpxmlrpc library has both a client and server
There are examples for both, Python server and client, and a PHP client and server

Just had the same problem and wanted to share my solution. (follows closely what Amadan suggests)
python piece
import subprocess
output = subprocess.check_output(["php", path-to-my-php-script, input1])
you could also do: blah = input1 instead of just submitting an unnamed arg... and then use the $_GET['blah'].
php piece
$blah = $argv[1];
if( isset($blah)){
// do stuff with $blah
}else{
throw new \Exception('No blah.');
}

The best bet is running python as a subprocess and capturing its output, then parsing it.
$pythonoutput = `/usr/bin/env python pythoncode.py`;
Using JSON would probably help make it easy to both produce and parse in both languages, since it's standard and both languages support it (well, at least non-ancient versions do). In Python,
json.dumps(stuff)
and then in PHP
$stuff = json_decode($pythonoutput);
You could also explicitly save the data as files, or use sockets, or have many different ways to make this more efficient (and more complicated) depending on the exact scenario you need, but this is the simplest.

For me the escapeshellarg(json_encode($data)) is giving not exactly a json-formatted string, but something like { name : Carl , age : 23 }.
So in python i need to .replace(' ', '"') the whitespaces to get some real json and be able to cast the json.loads(sys.argv[1]) on it.
The problem is, when someone enters a name with already whitespaces in it like "Ca rl".

Related

Python PHP equivalent

I have been using PHP for a while now with my Apache2 web server on my raspberry pi. It works great, but I get tired of always having to think "how do I X in PHP" or "what was the function name for this in PHP".
I am under the strong impression that there should be something equivalent in which I can replace the <?php ?> code with python code, but my search results have been confusing at best.
I am essentially looking for something where I can write whatever python code I want in an HTML script and have it interpreted and executed and its output inserted into the page when it is requested.
For example, to make a table of users from a list in python.
<table><tr><td>User list</td></tr>
<?python
import json
library=json.load(open(some_json_file,'r'));
for user in library:
print "<tr><td>"+user+"</td></tr>"
?>
</table>
I'm under the impression that chameleon can do this with its code blocks as described here,(https://chameleon.readthedocs.io/en/latest/reference.html) but as I look deeper, I get the impression it doesn't work like I am thinking it should. This is the impression I have gotten from all of the template engines I have looked at, as well as WSGI
Are there good drop in python alternatives for PHP? Or are there ways to cleanly wrap semi complex python code into my php in way that doesn't involve writing an additional python script that is called by PHP? I've tried exec() with python -c; but this was less than ideal having to escape all the ' and " characters...
update
The below code works just fine, but can become very slow if run multiple times in a script (takes about 0.4 seconds each time on a raspberry pi3). I have written a program in python that runs in the background and handles requests from php, and runs about 15x faster. I'm now maintaining it here on github.
Original Answer
After messing around I was able to come up with something mostly suitable for what I am trying to do. Inside my php I create a function that executes python scripts.
<?php
function py($s){
exec("python -c '$s'",$arr);
foreach($arr as $v){
echo $v."\n";}
}
?>
Then I use php Heredoc(equivalent to python """ , means I don't have to escape every single double quote) to fill the function:
<?php
py(<<<python
print "Hello world<br>"
s="ello world"
for x in s:
print x+"<br>"
python
);
?>
outputs >>>
Hello world
e
l
l
o
w
o
r
l
d
the only real downside I am experiencing at this point is that this method precludes me from using single quotes anywhere in my python script... :(. I'll get over it.
EDIT
I added a few more tweaks to make this even more useful. The new function is below:
<?php
function py($s,$return=false){
$s=str_replace("'","'\''",$s);
$h=<<<head
def cleanup():
for x in globals().keys():
if not x.startswith("_"):
del globals()[x]
import dill
try:
dill.load_session("pyworking.pkl")
except:
pass
head;
$f=<<<foot
import dill
dill.dump_session("pyworking.pkl")
foot;
if ($return==false){
echo shell_exec("python -c '$h$s$f'");
}
else {
return shell_exec("python -c '$h$s$f'");
}
}
?>
this allows you to use single quotes in the script and invoke the py() function multiple times in the same script and your variables and modules will follow you. At the end of the script you just call the clean up (or using php clear the pyworking.pkl file) and wipe the environment clean.
I also put this function in a file and in my pyp.ini I used the auto_prepend_file=my/file/location to automatically include it, so no need to load it before hand.
Overall I am very happy with this method, especially since I can read php variables inside my python script. Passing objects is as simple as:
<?php
$data_en=json_encode($data);
py(<<<p
import json
data=$data_en
#do something with data
p
);
?>
this would be perfect if I could think of a way to assign values to php variables inside the script, but its not a bad workaround if you want a fusion of php and python or just a way to do everything in python without writing a python webserver (which i have also done).

How to pass the string from php to tcl and execute the script

I want to pass the string from my php like
<?php
str1="string to pass"
#not sure about passthru
?>
And my tcl script
set new [exec $str1]#str1 from php
puts $new
Is this Possible? Please let me know I'm stuck with this
The simplest mechanism is to run the Tcl script as a subprocess that runs a receiving script (that you'd probably put in the same directory as your PHP code, or put in some other location) which decodes the arguments it is passed and which does what you require with them.
So, on the PHP side you might do (note the important use of escapeshellarg here! I advise using strings with spaces in as test cases for whether your code is quoting things right):
<?php
$str1 = "Stack Overflow!!!";
$cmd = "tclsh mycode.tcl " . escapeshellarg($str1);
$output = shell_exec($cmd);
echo $output;
echo $output;
?>
On the Tcl side, arguments (after the script name) are put in a list in the global argv variable. The script can pull them out with any number of list operations. Here's one way, with lindex:
set msg [lindex $argv 0]
# do something with the value from the argument
puts "Hello to '$msg' from a Tcl script running inside PHP."
Another way would be to use lassign:
lassign $argv msg
puts "Hello to '$msg' from a Tcl script running inside PHP."
Note however (if you're using Tcl's exec to call subprograms) that Tcl effectively automatically quotes arguments for you. (Indeed it does that literally on Windows for technical reasons.) Tcl doesn't need anything like escapeshellarg because it takes arguments as a sequence of strings, not a single string, and so knows more about what is going on.
The other options for passing values across are by environment variables, by pipeline, by file contents, and by socket. (Or by something more exotic.) The general topic of inter-process communication can get very complex in both languages and there are a great many trade-offs involved; you need to be very sure about what you're trying to do overall to pick an option wisely.
It is possible.
test.php
<?php
$str1="Stackoverflow!!!";
$cmd = "tclsh mycode.tcl $str1";
$output = shell_exec($cmd);
echo $output;
?>
mycode.tcl
set command_line_arg [lindex $argv 0]
puts $command_line_arg

Get python list in PHP

I am looking for a way to obtain a Python list and display it on my website using PHP.
I've checked out and tried many online help-requests so I was hoping someone would be able to explain to me what it is I am doing wrong.
My Python script scrapes a website and puts the result in a Python list.
What I am trying to achieve is the following:
I want to display (a part) of the list on my website.
I've tried to accomplish this with the following code:
PHP
<?php
$outputArray = [];
$returnStatus;
exec('python ./scrapeWebsite.py', $outputArray, $returnStatus);
var_dump($outputArray);
echo $returnStatus;
?>
Python:
print(newsHeadlines) -> returning a list like this: ['item 1','item 2','item 3','item 4']
However, the array comes back as array(0) { } and the $returnStatus value is 1.
Encoding the list with JSON and writing it to a file on the disk, and then reading the file in PHP and decoding JSON did the trick. To ensure the web-data is up-to-date I'm using exec(); to execute the python file.
If going full Python this is an option you could use.
To help others with a similar problem, this is the code I used:
Python:
myListJson = json.dumps(myList) #Encode our Python list into a JSON string
f = open("./fileName.txt", "w") #Open the file that we want to write to for write access
f.write(myListJson) #Write the JSON String to the file that we have currently open
f.close() #Close the file
PHP:
exec(python3 /path/to/file/script.py); //Will execute the python script, ensure the needed packages are installed on the **SERVER**.
$fileContents = file_get_contents("fileName.txt");
$decodedJson = json_decode($fileContents); //This will now contain your Array like any other PHP Array.
Thanks for the help!

PHP + Python - use PHP to pass string to Python program, and parse output

I have a great Python program on my webserver, which I want to use from inside my PHP web app.
Here's an example of the python command, and output as you would see it in terminal:
>>> print MBSP.parse('I ate pizza with a fork.')
I/PRP/I-NP/O/NP-SBJ-1/O/i
ate/VBD/I-VP/O/VP-1/A1/eat
pizza/NN/I-NP/O/NP-OBJ-1/O/pizza
with/IN/I-PP/B-PNP/O/P1/with
a/DT/I-NP/I-PNP/O/P1/a
fork/NN/I-NP/I-PNP/O/P1/fork ././O/O/O/O/.
You might recognize this as a typical POS tagger.
In any case, I'm confused about how to use a PHP-based web app to send this program a string like "I ate pizza with a fork", and somehow get the response back in a way that can be further parsed in PHP.
The idea is to use PHP to pass this text to the Python program, and then grab the response to be parsed by PHP by selecting certain types of words.
It seems like in PHP the usual suspects are popen() and proc_open(), but popen() is only for sending, or receiving information - not both? Is popen() able to give me access to this output (above) that I'm getting from the Python program? Or is there a better method? What about curl?
Here are all my options in terms of functions in PHP:
http://us.php.net/manual/en/function.proc-open.php
I'm lost on this, so thanks for your wise words of wisdom!
I use exec() for this purpose.
exec($command, $output);
print_r($output);
If you want to get a little heavier / fancier... give your python script an http (or xmlrpc) front end, and call that with a GET/POST. Might not be worth all that machinery though!
You could use popen(), and pass the input to your Python script as a command line argument, then read the output from the file descriptor popen gives you, or proc_open() if you want to interact bi-directionally with the Python script.
Example 1 in the proc_open manual: http://us.php.net/manual/en/function.proc-open.php gives an example of this.
If your Python needs it as stdin, you could try popening a command line:
echo "I ate pizza!"|my_python_progam.py
and just read the output. As usual, do proper input validation before sending it to the command-line.
Something like this would work
$command = '/usr/bin/python2.7 /home/a4337/Desktop/script.py'
$pid = popen('$command',r)
........
........
.........
pclose($pid)

Python's cPickle deserialization from PHP?

I have to deserialize a dictionary in PHP that was serialized using cPickle in Python.
In this specific case I probably could just regexp the wanted information, but is there a better way? Any extensions for PHP that would allow me to deserialize more natively the whole dictionary?
Apparently it is serialized in Python like this:
import cPickle as pickle
data = { 'user_id' : 5 }
pickled = pickle.dumps(data)
print pickled
Contents of such serialization cannot be pasted easily to here, because it contains binary data.
If you want to share data objects between programs written in different languages, it might be easier to serialize/deserialize using something like JSON instead. Most major programming languages have a JSON library.
Can you do a system call? You could use a python script like this to convert the pickle data into json:
# pickle2json.py
import sys, optparse, cPickle, os
try:
import json
except:
import simplejson as json
# Setup the arguments this script can accept from the command line
parser = optparse.OptionParser()
parser.add_option('-p','--pickled_data_path',dest="pickled_data_path",type="string",help="Path to the file containing pickled data.")
parser.add_option('-j','--json_data_path',dest="json_data_path",type="string",help="Path to where the json data should be saved.")
opts,args=parser.parse_args()
# Load in the pickled data from either a file or the standard input stream
if opts.pickled_data_path:
unpickled_data = cPickle.loads(open(opts.pickled_data_path).read())
else:
unpickled_data = cPickle.loads(sys.stdin.read())
# Output the json version of the data either to another file or to the standard output
if opts.json_data_path:
open(opts.json_data_path, 'w').write(json.dumps(unpickled_data))
else:
print json.dumps(unpickled_data)
This way, if your getting the data from a file you could do something like this:
<?php
exec("python pickle2json.py -p pickled_data.txt", $json_data = array());
?>
or if you want to save it out to a file this:
<?php
system("python pickle2json.py -p pickled_data.txt -j p_to_j.json");
?>
All the code above probably isn't perfect (I'm not a PHP developer), but would something like this work for you?
I know this is ancient, but I've just needed to do this for a Django 1.3 app (circa 2012) and found this:
https://github.com/terryf/Phpickle
So just in case, one day, someone else needs the same solution.
If the pickle is being created by the the code that you showed, then it won't contain binary data -- unless you are calling newlines "binary data". See the Python docs. Following code was run by Python 2.6.
>>> import cPickle
>>> data = {'user_id': 5}
>>> for protocol in (0, 1, 2): # protocol 0 is the default
... print protocol, repr(cPickle.dumps(data, protocol))
...
0 "(dp1\nS'user_id'\np2\nI5\ns."
1 '}q\x01U\x07user_idq\x02K\x05s.'
2 '\x80\x02}q\x01U\x07user_idq\x02K\x05s.'
>>>
Which of the above looks most like what you are seeing? Can you post the pickled file contents as displayed by a hex editor/dumper or whatever is the PHP equivalent of Python's repr()? How many items in a typical dictionary? What data types other than "integer" and "string of 8-bit bytes" (what encoding?)?

Categories