I have a python script (analyze.py) which takes a filename as a parameter and analyzes it. When it is done with analysis, it waits for another file name. What I want to do is:
Send file name as a parameter from PHP to Python.
Run analyze.py in the background as a daemon with the filename that came from PHP.
I can post the parameter from PHP as a command line argument to Python but I cannot send parameter to python script that already runs at the background.
Any ideas?
The obvious answer here is to either:
Run analyze.py once per filename, instead of running it as a daemon.
Pass analyze.py a whole slew of filenames at startup, instead of passing them one at a time.
But there may be a reason neither obvious answer will work in your case. If so, then you need some form of inter-process communication. There are a few alternatives:
Use the Python script's standard input to pass it data, by writing to it from the (PHP) parent process. (I'm not sure how to do this from PHP, or even if it's possible, but it's pretty simple from Python, sh, and many other languages, so …)
Open a TCP socket, Unix socket, named pipe, anonymous pipe, etc., giving one end to the Python child and keeping the other in the PHP parent. (Note that the first one is really just a special case of this one—under the covers, standard input is basically just an anonymous pipe between the child and parent.)
Open a region of shared memory, or an mmap-ed file, or similar in both parent and child. This probably also requires sharing a semaphore that you can use to build a condition or event, so the child has some way to wait on the next input.
Use some higher-level API that wraps up one of the above—e.g., write the Python child as a simple HTTP service (or JSON-RPC or ZeroMQ or pretty much anything you can find good libraries for in both languages); have the PHP code start that service and make requests as a client.
Here is what I did.
PHP Part:
<?php
$param1 = "filename";
$command = "python analyze.py ";
$command .= " $param1";
$pid = popen( $command,"r");
echo "<body><pre>";
while( !feof( $pid ) )
{
echo fread($pid, 256);
flush();
ob_flush();
}
pclose($pid);
?>
Python Part:
1. I used [JSON-RPC]: https://github.com/gerold-penz/python-jsonrpc to
create a http service that wraps my python script (runs forever)
2. Created a http client that calls the method of http service.
3. Printed the results in json format.
Works like a charm.
Related
sorry for long text.
I am looking for an idea to trigger execution of php function (in different directory, tested and in same directory) when specific act occurs.
System is Debian and php framework yii.
With API we receive new data from remote server and for specific value in data we receive I need to start function in separate process. No need to wait for its completion.
Because API response time I can't integrate this function into it and integration breaks down API rendering it unusable.
I have read dozens of answers on Stackoverflow and many other examples.
For test I just tried to create new folder at specific location and file wasn't created (permission problem I think but can't confirm since it makes it if done from main code - api).
It is supposed to do following:
pass 2 arguments received from API to function
create folder if it doesn't exist
call internal class that uses fpdf to print pdf file.
save document and mail it with PHPMailer.
Can't use pcntl_fork because it requires additional installation (which I am not allowed to do).
Covered topics:
forking with pcntl_fork (execution stops on reaching it.)
popen/pclose, exec, proc_open/proc_close (no reply and can't confirm it actually entered function).
Of course this situation excludes possibility of use of include and require.
Can't release api code but here is what I was asking it to do:
background.php
$docs_dir='test_folder';
if (!file_exists('/var/www/created_documents/'.$docs_dir))
{
mkdir('/var/www/created_documents/'.$docs_dir, 0777, true);
chmod('/var/www/created_documents/'.$docs_dir, 0777);
}
And it does nothing.
Few examples what I used in api code to jump to it (many others were deleted during work).
$proc = popen('php ' . __DIR__ . '/background.php &', 'r');
return $proc;
while (!feof($proc))
{
$data = fgets($proc);
var_dump($data);
}
exec("php background.php");
$cmd = "php background.php";
$timer = popen("start ". $cmd, "r");
pclose($timer);
exec($cmd . " > /dev/null &");
pclose(popen("php background.php &","r"));
You could make a separate internal http request using curl and its async funcionality.
Or you may use a queuing mechanism where one part is firing an event and the other part is consuming it. RabbitMQ or Gearman could do this for you.
im wondering if theres a way to run a code in a loop as a process and interacting with it from a different script. I know sockets listen to incoming requests but im referring to internal usage, without requests.
Standard approach:
Use pcntl-signal() and posix-kill() functions to interact by standard or user-defined signals.
Pros:
PHP built-in, ready for use functions. No need to reinvent wheels.
POSIX compatibility.
Cons:
You can only send defined signals to a script. Not values.
One-way interaction.
Example of listening script:
<?php
pcntl_signal(SIGTERM, 'sig_handler');
pcntl_signal(SIGUSR1, 'sig_handler');
echo 'Run... PID: ' . getmypid() . PHP_EOL;
$finish = false;
while (!$finish) {
pcntl_signal_dispatch();
}
echo 'Shutdown...' . PHP_EOL;
function sig_handler($signal) {
global $finish;
echo 'Received signal: ' . $signal . PHP_EOL;
switch ($signal) {
case SIGTERM:
$finish = true;
break;
case SIGUSR1:
echo 'Processing SIGUSR1 signal...' . PHP_EOL;
break;
}
}
Non-standard approach:
You can implement interaction with script using tools like database, files, sockets, pipes.
Pros:
Functionality depends only on realization.
Cons:
You need to implement protocol for interaction and support it in your script.
I would say first you set up your script to run in the background. You can implement it yourself (using fork) or use existing libraries.
https://www.google.co.jp/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=php%20daemon
Then you define a protocol to communicate. There are many way to implements that, from simple to complex, depending on your needs.
A simple way for example would be to define a folder somewhere in the server that your script reads on the regular basis (loop + sleep). When a file is added, the script reads it, execute the instruction in it, and delete it.
Nope, you cannot directly access another running script, as each script runs on it's own resource sandbox (memory, file descriptors). You'd also need to know it's PID in order to access it, and then after, you'd need some hacking tools to get into that process space.
This may not full fill your requirement. But will works:
Try logging the status to a text file from the first script and read it frequently from the second script to know what is the current status of first script.
I have a situation, which is with similar approach.
I'm starting independent processes by forking and not wait for them - only start. But I've created control structure in a database. There every children process is storing it's state but also observes a control flag and when this flag is up - then children stops immediately. The children processes stores time-stamped records in another table and that let's me to check what process where is with it's task.
In control structure is stored also processes pids and it is possible to send signals to them.
I think this can be useful in your situation too.
when php process starts it take all its requirements for example variables linked pages etc etc.. and show result after it finishes its execution, and after it is completed execution you cant do anything but to re-execute.
its like going to picnic on moon and coming back to home after its over, so no one can disturb you while you are in your picnic. :D
I'm trying to find a way in which I can echo out the output of an exec call, and then flush that to the screen while the process is running. I have written a simple PHP script which accepts a file upload and then converts the file if it is not the appropriate file type using FFMPEG. I am doing this on a windows machine. Currently my command looks like so:
$cmd = "ffmpeg.exe -i ..\..\uploads\\".$filename." ..\..\uploads\\".$filename.".m4v 2>&1";
exec( $cmd, $output);
I need something like this:
while( $output ) {
print_r( $output);
ob_flush(); flush();
}
I've read about using ob_flush() and flush() to clear the output buffer, but I only get output once the process has completed. The command works perfectly, It just doesn't update the Page while converting. I'd like to have some output so the person knows what's going on.
I've set the time out
set_time_limit( 10 * 60 ); //5 minute time out
and would be very greatful if someone could put me in the right direction. I've looked at a number of solutions which come close one Stackoverflow, but none seem to have worked.
Since the exec call is a blocking call you have no way of using buffers to get status.
Instead you could redirect the output in the system call to a log file. Let the client query the server for progress update in which case the server could parse the last lines of the log file to get information about current progress and send it back to the client.
exec() is blocking call, and will NOT return control to PHP until the external program has terminated. That means you cannot do anything to dump the output on a line-by-line basis because PHP is suspended while the external app is running.
For what you want, you need to use proc_open, which returns a filehandle you can read from in a loop. e.g.
$fh = proc_open('.....');
while($line = fgets($fh)) {
print($line);
flush();
}
There are two problems with this approach:
The first is that, as #Marc B notes, the fact that exec will block until it's finished. You'll have to devise some way of measuring progress.
The second is that using ob_flush() in this way amounts to holding the connection between server & client open and dribbling the data out a little at a time. This is not something that the HTTP protocol was designed for and while it might work sometimes, it's not going to work consistently - different browsers and different servers will time out differently. The better way to do it is via AJAX calls: using Javascript's setTimeout() function (or setInterval()), make a call to the server periodically and have the server send back a progress report.
I want to write an Ajax web application, a game to be specific. Two web clients have to communicate with each other via the PHP server. My approach to solve this is to use Ajax between client and server and server and client. Each client creates a separate server process using Ajax. I want that these two server processes communicate via MySQL and via named pipes. I need the named pipes to get immediate response for the whole application.
I cannot use one server process, which first creates a pipe and then forks into two processes, which use the pipe. Web applications create server processes when the web-browser sends a request. So, I need named pipes, where each process doesn't know more than the file name of the named pipe. They cannot exchange file handles (at least I don't know how).
My problem is that named pipes in the PHP way indeed work as long as they are
used within the same function:
public function writeAndReadPipe_test(){
$pipeA = fopen("testpipe",'r+');
fwrite($pipeA, 'ABCD');
$pipeB = fopen("testpipe",'r+');
$content = fread($pipeB, 4);
echo "[" . $content . "]<br>\n";
}
public function testingPipes_same_function(){
posix_mkfifo("testpipe", 0777);
$this->writeAndReadPipe_test();
}
But, when I use different functions, then the fread($pipeB, 4) command blocks the whole application:
public function writePipe_test(){
$pipeA = fopen("testpipe",'r+');
fwrite($pipeA, 'ABCD');
}
public function readPipe_test(){
$pipeB = fopen("testpipe",'r+');
$content = fread($pipeB, 4);
echo "[" . $content . "]<br>\n";
}
public function testingPipes_different_functions(){
posix_mkfifo("testpipe", 0777);
$this->writePipe_test();
$this->readPipe_test();
}
Does anybody know why? And what can I do to make it work between different functions in the first step? In the second step, it should work even between different processes! I found out that I also get a problem when the writer closes the pipe before the reader reads from
it. I suppose that the function closes it automatically when it ends, but this is only a guess.
If the PHP way does not work, I plan to let PHP open a command line, generate BASH commands and let execute them. This should work in any case as long as my web-server works in a LAMP environment. Disadvantage is that it will not work in WAMP environments.
So, has anybody some ideas to this?
P.S:
I need blocking pipes to let the reader wait until an event is sent. I know that the pipes can work in a non-blocking mode using
stream_set_blocking($pipe,false);
or so, but the whole idea is to do it without polling just using a pipe, which wakes the
reader up as soon as an event is fired.
Just a short statement, that I actually found a nice solution
using named pipes:
public function createPipe_test(){
posix_mkfifo("testpipe", 0777);
}
public function writePipe_test($value){
$pipeA = fopen("testpipe",'w');
$valueString = number_format($value);
$valueLen = number_format(strlen($valueString));
fwrite($pipeA, $valueLen);
fwrite($pipeA, $valueString);
}
public function readPipe_test(){
$pipeB = fopen("testpipe",'r');
$valueLen = fread($pipeB, 1);
return fread($pipeB, $valueLen);
}
I have two processes.
If process 1 calls writePipe_test(), then it waits until process 2 calls
readPipe_test() to read the data out of the pipe.
If process 1 calls readPipe_test(), then it waits until process 2 calls
writePipe_test() to write something into the pipe.
The trick is 'w' and 'r' instead of 'r+'.
When you use the pipes in separate functions like this, the write pipe A would seem to be closed/discarded again (local scope of $pipeA). The assumption would be that the pipe must be opened for reading and/or writing in order to retain any info, which makes sense really. Though I don't know the inner magic.
You can also observe that your blocking read-call succeeds when you feed the pipe from another process (like echo magic >> testpipe). So you'd already have step 2 done, but you need some pipehandle management.
If you change it as follows it'd work:
private $pipeA;
public function writePipe_test(){
$this->pipeA = fopen("testpipe",'r+');
fwrite($this->pipeA, 'ABCD');
}
Edit: or setting $pipeA to have global scope, for that matter..
Im not sure wether I understand your 2nd last post..
But to comment on the last one, if I don't misunderstand, TCP might be even more complex because you will have to establish a connection before you can either read or write, so youve got different overhead
As for the pipehandle closing at the function end, I assume you'll face the same problem with the sockets; but the pipefile remains!
Persistent storage (files,db) would make the clients independent timingwise, if you want to use blocking calls then files might actually be a way to go..
I'll show some code, first:
echo exec("compile\\save.exe Untitled.c tmpUntitled.c");
I have a program, named save.exe and i want to know if it already stopped?
If stopped, ok... Do something...
If not, may be an error, or a loop...
Now: I want to build same way to control the time that program use, and put a limit (time limit exceed, something like this...)
Any one has a sugestion ?
Edit[1]:save.exe is a program wrote on C language, and use two parameters: source and destiny.
popen just don't execute save.exe, i say this because it don't gerenate anymore the destiny (with execit happens);
exec() will suspend your script while the exec'd program is running. Control will be returned only when the external program terminates. If you want to see if the program's hung up or waiting for input or somesuch, you use popen(), which returns a filehandle from which you can read the program's output. At that point you can do a polling loop to see if there's been any output:
$app = popen("your shell command here");
while($output = fgets($app)) {
// handle output
sleep(5); // sleep 5 seconds
}
If you want to send something to the app as input, then you have to use proc_open(), which allows bi-directional communication between your script and the external program.