Is it possible to run two methods in parallel without using any extension like pthread or pcntl?
I tried Symfony/Process but it seems like I can use it only for php cli programing.
My requirement is for php web application. I found amphp/parallel that says it works without any extension. I tried the below example. Though I don't know if the library provides true parallel functionality or something else like queuing tasks? Because it is commented below "without any extension parallel processing is not possible".
Before I try the amphp/parallel I wanted to be sure if the library which I found is the right one for my job. Let's say I want to run two methods test1() and test2() in parallel and use the result from those two methods in next line.
<?php
require __DIR__ . '/../vendor/autoload.php';
use Amp\Parallel\Worker;
use Amp\Promise;
$urls = [
'https://secure.php.net',
'https://amphp.org',
'https://github.com',
];
$promises = [];
foreach ($urls as $url) {
$promises[$url] = Worker\enqueueCallable('file_get_contents', $url);
}
$responses = Promise\wait(Promise\all($promises));
foreach ($responses as $url => $response) {
\printf("Read %d bytes from %s\n", \strlen($response), $url);
test1(){
// do some database query
}
test2(){
// run a lengthy for loop
}
Related
I was reading about amphp, and I had a doubt about parallel
Example code:
<?php
require __DIR__ . '/vendor/autoload.php';
$client = new Amp\Artax\DefaultClient;
$promises = [];
$urls = [
"http://google.com.br",
"http://facebook.com.br",
"http://xat.com/",
"http://kobra.ninja/",
"http://wikipedia.com",
"http://manualdomundo.com.br",
"http://globo.com",
"http://gmail.com"
];
foreach ($urls as $url) {
$promises[$url] = Amp\call(function () use ($client, $url) {
// "yield" inside a coroutine awaits the resolution of the promise
// returned from Client::request(). The generator is then continued.
$response = yield $client->request($url);
// Same for the body here. Yielding an Amp\ByteStream\Message
// buffers the entire message.
$body = yield $response->getBody();
echo $url;
return $url;
});
}
$responses = Amp\Promise\wait(Amp\Promise\all($promises));
Is this code running all curl, or waiting for 1 to perform another?
This is using stream_select functionality. It is not running in parallel in the conventional way you are thinking of. It works by registering a callback to be invoked once streams are readable / writable and then returning when the specific promise you are waiting for is done. All other promises that are resolved in the meantime are completed and cached in their respective promise waiting for you to unwrap the value using yield or Promise::onResolve. Amp's event loop is what is managing the multiple sockets and handling the concurrency for you.
If you want a basic example of how this works, I put a sample project on GitHub that is two classes, but is based on Curl instead of stream_select: https://github.com/kwhat/requestful
The first 7 methods are all that is required to setup the promise interface. All of the magic here is based on the two callbacks passed to the constructor as well as the wrappers around the then/cancel callbacks.
The sendRequestAsync method is how the concurrent Curl requests are created. The magic all happens in the callback for wait() on any promise, which calls an anonymous function that intern calls the tick method in a loop. That tick() method is what resolves all of the promises, regardless which one you call wait on.
As you can see from Amp codebase, it will run all your requests inside the same event loop.
Given your requests are being yielded (yield $client->request($url)), and they are being yielded inside the same event loop, they are being dispatched concurrently.
I recommend you reading this article through, specially the "Async development: how Amp framework works" section. I think it will make it a bit clearer how the engine works behind the scenes.
Hope this was helpful. Cheers! :)
I have some PHP script where I invoke method from the external class. I want to run this method asynchronously. I don't want to block rest of the program. This method does some work in the background and return nothing so there is no need to wait while it finished. Is there a way to do this in PHP?
# get post data from user
$postData = $this->f3->get('POST');
# start of asynchronous part
$obj = new asyncClass();
$obj->init($postData);
# end of asynchronous part
# do some work with post data
$soc = new someOtherClass($postData);
$result = $soc->send();
# assign result of work to variable
$this->f3->set('var', $result);
# show it to user
$this->f3->set('view', 'view.html');
If this can help, I'm using Fat Free Framework and PHP 5.6 Non-Thread Safe
You can use $f3->abort() to send the output/response to the browser and process your other blocking function afterwards. That's not a real asynchron solution but would work. You could also use something like php-icicle to add threads support, but that maybe requires some other php modules being installed.
Use threading.
class PostDataHandlerAsync extends Thread
{
private $postData
public function __construct($postData)
{
$this->postData = $postData;
}
public function run()
{
/*
Your code goes here
*/
}
}
$postData = $this->f3->get('POST');
$obj = new PostDataHandlerAsync($postData);
$obj->run();
Within my Symfony application I need to do several operation with files: list of files from a directory, decrypt them using gpg, parse the output with an external software and encrypt again.
My first question is: is this the right approach for this problem? On another scenario, I'd have written bash/python scripts to do this, but since info (user ids, passphrases, etc) is read from a Symfony API I though it was quite convenient to embed the calls into the application.
My second question is more specific: is there any way to efficiently handle the command line outputs and errors? For instance, when I call 'ls' how can easily convert the output into an array of file names?
private function decryptAction()
{
$user_data_source = '/Users/myuser/datafiles/';
// Scan directory and get a list of all files
$process = new Process('ls ' . $user_data_source);
try {
$process->mustRun();
$files = explode(' ', $process->getOutput());
return $files;
} catch (ProcessFailedException $e) {
return $e->getMessage();
}
}
Found the answer for my second question, but I am still very interested in your thoughts about the entire approach.
// Scan directory and get a list of all files
$process = new Process('ls -1 ' . $user_data_source);
try {
$process->mustRun();
$files = array_filter( explode("\n", $process->getOutput()), 'strlen');
return $files;
} catch (ProcessFailedException $e) {
return $e->getMessage();
}
Unless you really need an immediate response from the call, this kind of tasks are better left to a background process.
So what I would do is write one or more Symfony commands that perform the described processes (read, decrypt, and so on).
Those processes can be executed via crontab, or "daemonized" via another scheduler like Supervisord.
Then, the API call only creates some kind of "semaphore" file that triggers the actual execution, or even better you can use some kind of queue system.
I'm trying to open multiple connections (various devices) to run a command and get the output.
The problem is that i have to run them "all at once"/parallel.
If i wait for one result and then to run the other one it takes way too long
and with a large number of devices that can go very bad.
I'm also using curl which I know that there is curl_multi and I was wondering if there was something similar with SSH for php.
I'm using Net_SSH2 for now.
You'll need to use two PHP libraries: https://robo.li/tasks/Remote/#ssh and https://github.com/cheprasov/php-parallel. Your class method might be something similar to the example below:
function runParallelSSH() {
$parallel = new Parallel(new ApcuStorage());
foreach ($credentials as $user => $host) {
$connection = sprintf('%s#%s', $user, $host);
$connections[] = $connection;
$Parallel->run($connection, function() {
$gitTask = $this->taskGitStack()
->checkout('master')
->pull();
});
}
$results = $parallel->wait($connections);
}
Without using thirdparties like curl_multi you have to use PHP multithreading, for this you need an extension pthreads.
Look in the docs for PHP threading
The most interesting feature is using it like this (code modified from PHP.net)
class My extends Thread {
public function run() {
//curl_exec whatever
}
}
$my = new My();
//start as many as you need
$my->start();
//wait for the threads to finnish and join one thread at a time with main-process-thread:
var_dump($my->join());:
Good Luck!
I want to execute a php-script from php that will use different constants and different versions of classes that are already defined.
Is there a sandbox php_module where i could just:
sandbox('script.php'); // run in a new php environment
instead of
include('script.php'); // run in the same environment
Or is proc_open() the only option?
PS: The script isn't accessible through the web, so fopen('http://host/script.php') is not an option.
There is runkit, but you may find it simpler to just call the script over the command line (Use shell_exec), if you don't need any interaction between the master and child processes.
The is a class on GitHub that may help, early stages but looks promising.
https://github.com/fregster/PHPSandbox
Also, you should look at the backtick operator:
$sOutput = `php script_to_run.php`;
This will allow you to inspect the output from the script you are running. However, note that the script will be run with the privileges you have, but you can circumvent this by using sudo on Linux.
This approach also assumes that you have the PHP CLI installed, which is not always the case.
There is Runkit_Sandbox - you might get it to work, it's a PHP extension. I'd say the way to go.
But you might need to create a "sandbox" your own, e.g. by resetting the global variable state of the superglobals you use.
class SandboxState
{
private $members = array('_GET', '_POST');
private $store = array();
public function save() {
foreach($members as $name) {
$this->store[$name] = $$name;
$$name = NULL;
}
}
public function restore() {
foreach($members as $name) {
$$name = $this->store[$name];
$this->store[$name] = NULL;
}
}
}
Usage:
$state = new SanddboxState();
$state->save();
// compile your get/post request by setting the superglobals
$_POST['submit'] = 'submit';
...
// execute your script:
$exec = function() {
include(func_get_arg(0)));
};
$exec('script.php');
// check the outcome.
...
// restore your own global state:
$state->restore();
dynamic plugin function execution that allows the loaded file and function to execute anything it wants, however it can only take and return variables that can be json_encode'ed.
function proxyExternalFunction($fileName, $functionName, $args, $setupStatements = '') {
$output = array();
$command = $setupStatements.";include('".addslashes($fileName)."');echo json_encode(".$functionName."(";
foreach ($args as $arg) {
$command .= "json_decode('".json_encode($arg)."',true),";
}
if (count($args) > 0) {
$command[strlen($command)-1] = ")";//end of $functionName
}
$command .= ");";//end of json_encode
$command = "php -r ".escapeshellarg($command);
exec($command, $output);
$output = json_decode($output,true);
}
the external code is totally sandboxed and you can apply any permission restrictions you want by doing sudo -u restricedUser php -r ....
I have developed a BSD-licensed sandbox class for this very purpose. It utilizes the PHPParser library to analyze the sandboxed code, check it against user-configurable whitelists and blacklists, and features a wide array of configuration options along with sane default settings. For your needs you can easily redefine classes called in your sandboxed code and route them to different ones.
The project also includes a sandbox toolkit (use only on your local machine!) that can be used to experiment with the sandbox settings, and a full manual and API documentation.
https://github.com/fieryprophet/php-sandbox
i know its not 100% topic related, but maybe useful for somebody n__n
function require_sandbox($__file,$__params=null,$__output=true) {
/* original from http://stackoverflow.com/a/3850454/209797 */
if($__params and is_array($__params))
extract($__params);
ob_start();
$__returned=require $__file;
$__contents=ob_get_contents();
ob_end_clean();
if($__output)
echo $__contents;
else
return $__returned;
};