How to convert code from core PHP to Pthread code
My core PHP code :
require_once 'xyz.php';
$count="0";
foreach($sites as $site)
{
require_once 'allsite/'.$site.'.php';
$siteObj = new $site;
$data[$count] = $siteObj->getsiteData($query,$cat);
$count++;
}
How can i execute foreach loop code in Pthread manner ( Multi-threading )
After foreach loop execution complete normal PHP stuff done (Means No Multi-Threading after foreach Loop)
Note : I already installed and config pthread in PHP
Comments included.
<?php
//Require Core Component
require_once("xyz.php");
$Threads = new StackableArray();
//A Global Storage that will be shared between threads.
$Storage = new Storage();
//Store {count} in Global Storage
$Storage['count'] = 0;
/*
* For Each Site, spawn a new thread, process the site and store result in Global Storage.
*/
foreach($sites as $site)
{
$Thread = new SiteProcess($Storage, $site);
$Thread->start();
$Threads[$re] = $Thread;
}
//When the threads are complete, join them(destroy the threads)
foreach($Threads as $Thread)
{
$Thread->join();
}
//A thread class
class SiteProcess extends Threads
{
private $Storage;
private $site;
public function __construct(Storage $Storage, $site)
{
$this->Storage = $Storage;
$this->site = $site;
}
public function run()
{
require_once("allsite/{$this->site}.php");
$siteObj = new $this->site;
$this->Storage[$this->Storage['count']] = $siteObj->getsiteData($query, $cat);
$this->Storage['count'] = $this->Storage['count'] + 1;
}
}
class StackableArray extends Stackable { public function run() { } };
class Storage extends Stackable { public function run() { } };
Related
I'm using php-zts to perform parallel data processing, using symfony 4 and PThreads
I'm great at running multiple threads, but I'm facing a problem, I need each of the threads to be able to work with doctrine
I need to make sure that each thread is able to work with doctrine
I tried to transfer a container instance directly, but it won't work because it can't be sterilized
/console_comand.php
private function gettingStatistics(){
$pool = new \Pool(4, Autoloader::class, ["vendor/autoload.php"]);
$store = new \Threaded();
$class = new Meta();
$pool->submit(new Task($class,$store));
$pool->collect();
$pool->shutdown();
$listQuotes = array();
foreach ($store as $obj){
foreach ($obj->{'response'} as $exchange => $data){
$listQuotes[$exchange] = $data;
}
}
unset($store);
unset($interface);
return $listQuotes;
}
/Autoloader.php
<?php
namespace App\Worker;
class Autoloader extends \Worker
{
protected $loader;
public function __construct($loader)
{
$this->loader = $loader;
}
/* включить автозагрузчик для задач */
public function run()
{
require_once($this->loader);
}
/* переопределить поведение наследования по умолчанию для нового потокового контекста */
public function start(int $options = PTHREADS_INHERIT_ALL)
{
return parent::start(PTHREADS_INHERIT_NONE);
}
}
/Autoloadable.php
<?php
namespace App\Worker;
/* нормальный, автоматически загруженный класс */
class Autoloadable
{
public $response;
public function __construct($greeting)
{
$this->response = $greeting->job();
}
}
/Task.php
<?php
namespace App\Worker;
class Task extends \Threaded
{
protected $greeting;
protected $result;
public function __construct($greeting,\Threaded $store)
{
$this->greeting = $greeting;
$this->result = $store;
}
public function run()
{
$greeting = new Autoloadable($this->greeting);
$this->result[] = $greeting;
}
}
how do I pass the right doctrine to be able to work with it from the job?
there's a very similar question on github but I can't deal with it.
https://github.com/krakjoe/pthreads/issues/369
Have you tried requiring an ObjectManager instance in the __construct of Task (your last code block)?
Have a read of this article
Cannot test it atm, don't have zts setup, but I've used this to great success in other projects.
I would expect you need to do something like:
$pool = new Pool(4);
for ($i = 0; $i < 15; ++$i) {
$pool->submit(new class($objectManager) extends \Threaded
{
private $objectManager;
public function __construct(ObjectManager $objectManager)
{
$this->objectManager= $objectManager;
}
public function run()
{
// obviously replace the contents of this function
$this->objectManager->performTask;
echo 'Job\'s done.' . PHP_EOL;
}
});
}
while ($pool->collect());
$pool->shutdown();
The instantiation of the new anonymous class takes the $objectManager present in your current instance, like /console_comand.php there, and passes it to this new anonymous class to fulfill the __construct requirements.
The linked article does a better job of explaining it than I do, so please give it a read.
I am working on improving a process in php so I resorted into using multithreading using Worker to manage my threads. But the threads in the worker executes one after the other not concurrently.
This is my sample code
namespace App\Engine\Threads\ThreadClass;
class StatePaperAttendancePDFThread extends \Threaded
{
private $write_folder_name;
private $center_code;
private $paper_id;
private $title;
public function __construct($write_folder_name, $center_code, $paper_id, $title)
{
$this->write_folder_name = $write_folder_name;
$this->center_code = $center_code;
$this->paper_id = $paper_id;
$this->title = $title;
}
public function run(){
echo " Retrieving paper attendance ".$this->center_code." ".$this->paper_id." ".\Thread::getCurrentThreadId()." ".date("H:i:s:u").PHP_EOL;
$artisan = 'C:/xampp/htdocs/attendance/artisan';
$cmd = "php $artisan attendancereport:center $this->write_folder_name $this->center_code $this->paper_id $this->title";
$return = exec($cmd, $output, $return_var);
echo $return;
}
}
foreach ($centers as $i=>$center){
$center_code = $center->getCenterCode();
$thread = new StatePaperAttendancePDFThread($folder_name, $center_code, $paper_id, $title);
$worker->stack($thread);
}
$worker->start(PTHREADS_INHERIT_ALL ^ PTHREADS_INHERIT_CLASSES);
$worker->shutdown();
but when I monitor it from the CLI using the time been printed I can see that none of the threads starts together. They all starts with some seconds interval
Please what am I missing
I was able to solve my problem by creating a customized Thread Pool Executor as below. Please I am open to suggestions and Improvement
class ThreadPoolExecutor{
private $poolSize;
private $threadPool;
private $done = false;
private $workingThreads;
public function __construct($poolSize, array $threadPool)
{
$this->poolSize = $poolSize;
$this->threadPool = $threadPool;
}
public function execute()
{
$this->parametersOk();
try {
while (!empty($this->threadPool)) {
$this->extractThreads();
foreach ($this->workingThreads as $thread) {
$thread->start(PTHREADS_INHERIT_ALL ^ PTHREADS_INHERIT_CLASSES);
}
foreach ($this->workingThreads as $thread) {
$thread->join();
}
}
$this->done = true;
} catch (\Exception $ex) {
var_dump($ex->getMessage());
}
}
private function parametersOk()
{
if (!is_array($this->threadPool))
throw new \RuntimeException("threadPool expected to be an array of threads");
if (count($this->threadPool) <= 0)
throw new \RuntimeException("expected at least an element in threadPool");
foreach ($this->threadPool as $thread) {
if (!is_subclass_of($thread, \Thread::class, false)) {
throw new \RuntimeException(" an element of threadPool does not extend class \\Thread");
}
}
if ($this->poolSize > count($this->threadPool)) {
throw new \RuntimeException("The number of threads set to execute can not be greater than the threadPool");
}
return true;
}
private function extractThreads()
{
$this->workingThreads = [];
$this->workingThreads = array_slice($this->threadPool, 0, $this->poolSize);
for ($i = 0; $i < count($this->workingThreads); $i++) {
array_shift($this->threadPool);
}
}
public function isDone()
{
return $this->done;
}
}
I will appreciate any addition or correction to this.
safe storage of data. I read that for this task suits Stackable.
I inherit Stackable but data in storage is not synchronized.
AsyncOperation -- just incrementing value in storage
AsyncWatcher -- just making echo of value in storage.
Problem : data in storage is not modifying from AsyncOperation thread, storage permanently contains a -1.
I'm using pthreads.
class Storage extends Stackable {
public function __construct($data) {
$this->local = $data;
}
public function run()
{
}
public function getData() { return $this->local; }
}
class AsyncOperation extends Thread {
private $arg;
public function __construct(Storage $param){
$this->arg = $param->getData();
}
public function run(){
while (true) {
$this->arg++;
sleep(1);
}
}
}
class AsyncWatcher extends Thread {
public function __construct(Storage $param){
$this->storage = $param -> getData();
}
public function run(){
while (true) {
echo "In storage ". $this->storage ."\n";
sleep(1);
}
}
}
$storage = new Storage(-1);
$thread = new AsyncOperation($storage);
$thread->start();
$watcher = new AsyncWatcher($storage);
$watcher->start();
As you can see, Stackable class has a lot of methods, mainly used for async operations and they can help you with your issue. You should modify your async classes in this way:
class AsyncOperation extends Thread {
private $arg;
public function __construct(Storage $param){
$this->arg = $param->getData();
}
public function run(){
while (true) {
$this->arg++;
sleep(1);
}
$this->synchronized(function($thread){
$thread->notify();
}, $this);
}
}
and their usage will be like this:
$storage = new Storage();
$asyncOp = new AsyncOperation($storage);
$asyncOp->start();
$asyncOp->synchronized(function($thread){
$thread->wait();
}, $asyncOp);
var_dump($storage);
I have this thread class that utilises the pthread PHP extension:
class Task extends Thread
{
protected $arr = array();
public function run()
{
$this->arr[] = 1;
$this->arr[] = 2;
$this->arr[] = 3;
var_dump($this->arr);
}
}
$thread = new Task();
$thread->start();
$thread->join();
The output inexplicably shows an empty array. Could anybody briefly explain why?
I have a solution but not a solid explanation, so more answers would be very welcome.
Here is my Threaded child (trimmed for the sake of brevity):
class ObjectConstructorThreaded extends Threaded
{
protected $worker;
protected $className;
protected $parameters;
protected $objectKey;
public function __construct($className, $parameters)
{
$this->className = $className;
$this->parameters = $parameters;
}
public function setWorker(\Worker $worker)
{
$this->worker = $worker;
}
protected function getWorker()
{
return $this->worker;
}
public function run()
{
$reflection = new \ReflectionClass($this->className);
$instance = $reflection->newInstanceArgs($this->parameters);
$this->objectKey = $this->getWorker()->notifyObject($instance);
}
public function getObjectKey()
{
return $this->objectKey;
}
}
And the Worker (again trimmed):
class ObjectServer extends Worker
{
protected $count = 0;
protected $objects = array();
public function notifyObject($object)
{
$key = $this->generateHandle();
/*
// Weird, this does not add anything to the stack
$this->objects[$key] = $object;
// Try pushing - fail!
$this->objects[] = $object;
// This works fine? (but not very useful)
$this->objects = array($key => $object);
*/
// Try adding - also fine!
$this->objects = $this->objects + array($key => $object);
return $key;
}
}
Finally, to kick off the thread:
$thread = new ObjectServer();
$thread->start();
$threaded = new ObjectConstructorThreaded($className, $parameters);
$threaded->setWorker($this->worker);
$thread->stack($threaded);
As you can see from the unadulterated comments I wrote at the time, attempts to insert or push to the array failed, but rewriting it (by setting it to a fixed value or a merge of the old value and the new entry) seems to work.
I'm therefore regarding threading as making non-trivial types (arrays and objects) effectively immutable, and that they can only be reset and not modified. I've had the same experience with serialisable classes too.
As to why this is the case, or if there is a better approach, I will update this answer if I find out!
I have run into a situation with pthreads.Consider the following script:
<?php
class W extends \Worker {
public function run()
{
echo "worker started\n";
}
}
clasds S extends \Stackable {
public function run()
{
echo "stackable executed\n";
}
}
class x{
protected $workers;
protected $i;
public function __construct()
{
$this->workers = array();
$this->i = 0;
}
public function go()
{
$thread = $this->spawn();
$j = new S;
$thread->stack($j);
//$thread->shutdown();
}
public function spawn()
{
$this->i !== 0 ? $n=$this->i++ : $n=$this->i;
$thread = new W;
$this->workers[$n] = $thread;
$thread->start();
return $thread;
}
}
$exe = new x;
$exe->go();
php crashes.2 ways to solve the issue, either:
uncommenting
$w->shutdown();
which means the workers one creates are not reusable, or commenting out
$this->c[$n] = $w;
which means not saving workers in an array.
I do not understand what is going on here.must one call shutdown on a worker?why does attempting to store
the worker in an array lead to a crash?