I am working on improving a process in php so I resorted into using multithreading using Worker to manage my threads. But the threads in the worker executes one after the other not concurrently.
This is my sample code
namespace App\Engine\Threads\ThreadClass;
class StatePaperAttendancePDFThread extends \Threaded
{
private $write_folder_name;
private $center_code;
private $paper_id;
private $title;
public function __construct($write_folder_name, $center_code, $paper_id, $title)
{
$this->write_folder_name = $write_folder_name;
$this->center_code = $center_code;
$this->paper_id = $paper_id;
$this->title = $title;
}
public function run(){
echo " Retrieving paper attendance ".$this->center_code." ".$this->paper_id." ".\Thread::getCurrentThreadId()." ".date("H:i:s:u").PHP_EOL;
$artisan = 'C:/xampp/htdocs/attendance/artisan';
$cmd = "php $artisan attendancereport:center $this->write_folder_name $this->center_code $this->paper_id $this->title";
$return = exec($cmd, $output, $return_var);
echo $return;
}
}
foreach ($centers as $i=>$center){
$center_code = $center->getCenterCode();
$thread = new StatePaperAttendancePDFThread($folder_name, $center_code, $paper_id, $title);
$worker->stack($thread);
}
$worker->start(PTHREADS_INHERIT_ALL ^ PTHREADS_INHERIT_CLASSES);
$worker->shutdown();
but when I monitor it from the CLI using the time been printed I can see that none of the threads starts together. They all starts with some seconds interval
Please what am I missing
I was able to solve my problem by creating a customized Thread Pool Executor as below. Please I am open to suggestions and Improvement
class ThreadPoolExecutor{
private $poolSize;
private $threadPool;
private $done = false;
private $workingThreads;
public function __construct($poolSize, array $threadPool)
{
$this->poolSize = $poolSize;
$this->threadPool = $threadPool;
}
public function execute()
{
$this->parametersOk();
try {
while (!empty($this->threadPool)) {
$this->extractThreads();
foreach ($this->workingThreads as $thread) {
$thread->start(PTHREADS_INHERIT_ALL ^ PTHREADS_INHERIT_CLASSES);
}
foreach ($this->workingThreads as $thread) {
$thread->join();
}
}
$this->done = true;
} catch (\Exception $ex) {
var_dump($ex->getMessage());
}
}
private function parametersOk()
{
if (!is_array($this->threadPool))
throw new \RuntimeException("threadPool expected to be an array of threads");
if (count($this->threadPool) <= 0)
throw new \RuntimeException("expected at least an element in threadPool");
foreach ($this->threadPool as $thread) {
if (!is_subclass_of($thread, \Thread::class, false)) {
throw new \RuntimeException(" an element of threadPool does not extend class \\Thread");
}
}
if ($this->poolSize > count($this->threadPool)) {
throw new \RuntimeException("The number of threads set to execute can not be greater than the threadPool");
}
return true;
}
private function extractThreads()
{
$this->workingThreads = [];
$this->workingThreads = array_slice($this->threadPool, 0, $this->poolSize);
for ($i = 0; $i < count($this->workingThreads); $i++) {
array_shift($this->threadPool);
}
}
public function isDone()
{
return $this->done;
}
}
I will appreciate any addition or correction to this.
Related
I have a code for testing and I do not understand why it is halted.
The script only performs 799 transitions.
set_time_limit(0);
ini_set('display_errors', 0);
class
class CheckIpThread extends Thread {
private $ip;
public $data = null;
public function __construct ($ip_to_check) {
$this->ip = $ip_to_check;
}
public function run () {
$this->data = th($this->ip);
$this->kill;//This line probably does not matter
}
}
end class
for($a=0;$a<=2000;$a=$a+100)
{th_($a);}
function th_($co){
$threads = [];
for($a=0;$a<=100;$a++){
$thread = new CheckIpThread($co+$a);
$threads []= $thread;
$thread->start();
}
foreach ($threads as $thread) {
$thread->join();
}
foreach ($threads as $thread) {
echo_($thread->data);
}
}
function th($wsad)
{return $wsad;//}
live viev
function echo_($text,$def=0)
{
ob_flush();
flush();
//usleep(200);
if($def===0)
{var_dump($text);}
if($def===1)
{echo($text);}
echo"</br>";
ob_end_flush();
}
//return int(0) ...int(799) and browser waiting but nothing happens
check apache error log, may be memory overflow.
I'm currently working with pthreads to implement multithreading on a very demanding function. So far I got this working:
class Operation extends Thread {
public function __construct($arg) {
$this->arg = $arg;
}
public function run() {
if ($this->arg) {
$parameters = $this->arg;
echo my_function($parameters[0],$parameters[1]);
}
}
}
$stack = array();
foreach ($work as $operation) { $stack[] = new Operation($operation); };
foreach ($stack as $t) { $t->start(); };
It outputs the results directly.
I'd like to have my results stored one by one in an array (in the same order would be nice) but of course this does not work :
class Operation extends Thread {
public function __construct($arg) {
$this->arg = $arg;
}
public function run() {
if ($this->arg) {
$parameters = $this->arg;
$results[] = my_function($parameters[0],$parameters[1]);
}
}
}
$stack = array();
foreach ($work as $operation) { $stack[] = new Operation($operation); };
foreach ($stack as $t) { $t->start(); };
var_dump($results);
Any help would be appreciated.
Details:
my_function outputs an UTF-8 string.
The basic problem is that arrays are not thread safe, pthreads provides array-like interfaces on all Threaded objects; This means you can use Threaded objects in place of arrays in a multi-threaded context.
<?php
function demanding(...$params) {
/* you have parameters here */
return array(rand(), rand());
}
class Task extends Collectable {
public function __construct(Threaded $result, $params) {
$this->result = $result;
$this->params = $params;
}
public function run() {
$this->result[] =
demanding(...$this->params);
}
protected $result;
protected $params;
}
$pool = new Pool(16);
$result = new Threaded();
while (#$i++<16) {
$pool->submit(
new Task($result, $argv));
}
$pool->shutdown();
var_dump($result);
?>
There isn't a built-in way to do a multi-threaded sort, so the simplest thing to do is sort your result when the threads are finished.
I'm sure someone can do better but it's (apparently) working and it's quite a boost of performance. The "waiting for thread" part is very inefficient and inelegant, any help will be appreciated!
First of all, check if you have pthreads installed with phpinfo() or install it https://php.net/manual/en/pthreads.installation.php .
$key = 0;//We initialise the key to sort our results
foreach($iteration as $parameters) {//We make a loop to create the task list
$arguments[] = array($key,$parameters,$_POST['stuff'],$another_parameter,...);//We index ALL the parameters our function need for each task ($_POST, $_FILES, $_GET, user defined...) in a nice array
++$key;//We increment our key
};
class operation extends Thread {
public $done = 0;//We initialize the "done" indicator
public function __construct($arguments) {
$this->arguments = $arguments;//We put our parameters in a construct
}
public function run() {
if ($this->arguments)
{
$parameters = $this->arguments;//We extract the parameters for this worker
$this->result = array($parameters[0], your_function($parameters[1],$parameters[2],...));//We launch our function and add the key to the result
$this->done = 1;//This thread is done
}
}
}
$stack = array();//Lets initialize our stack
foreach ($arguments as $operation) { $stack[] = new operation($operation); };//We initialize the process
foreach ($stack as $t) { if($t->start()) { $t->join(); }; };//We launch it and wait until all the workers have completed their task
foreach($stack as $strata) {//We get each of the results
while($strate->done == 0) {sleep(1);};//Inefficient way to wait for the thread to complete, ANY HELP WILL BE APPRECIATED
$results[] = $strata->result;//Once it's done we add it to our results array
};
ksort($results);//We sort our array with our key
foreach($results as $line) { $results_sorted[] = $line[1]; };//We delete the key
Now you have your $results_sorted ! Enjoy !
I am running 4 threads running in same time. (Threads are running work() function in same time in this case)
global $i;
$i = 1;
function work($address) {
while($i < 1000) {
$i++;
----
if($i == something) some job...
----
}
}
For some reason this don't do the job.
Threads do sometimes same circle in while, so I have later some duplicate values. (probably they have some critical section)
Any idea how to fix this ?
The counter object must be thread safe, it must also employ synchronized methods.
Follows is an example of such code:
<?php
class Counter extends Threaded {
public function __construct($value = 0) {
$this->value = $value;
}
/** protected methods are synchronized in pthreads **/
protected function increment() { return ++$this->value; }
protected function decrement() { return --$this->value; }
protected $value;
}
class My extends Thread {
/** all threads share the same counter dependency */
public function __construct(Counter $counter) {
$this->counter = $counter;
}
/** work will execute from job 1 to 1000, and no more, across all threads **/
public function run() {
while (($job = $this->counter->increment()) <= 1000) {
printf("Thread %lu doing job %d\n",
Thread::getCurrentThreadId(), $job);
}
}
protected $counter;
}
$counter = new Counter();
$threads = [];
while (($tid = count($threads)) < 4) {
$threads[$tid] = new My($counter);
$threads[$tid]->start();
}
foreach ($threads as $thread)
$thread->join();
?>
work() seems superfluous, this logic should be in the ::run function most likely.
I have run into a situation with pthreads.Consider the following script:
<?php
class W extends \Worker {
public function run()
{
echo "worker started\n";
}
}
clasds S extends \Stackable {
public function run()
{
echo "stackable executed\n";
}
}
class x{
protected $workers;
protected $i;
public function __construct()
{
$this->workers = array();
$this->i = 0;
}
public function go()
{
$thread = $this->spawn();
$j = new S;
$thread->stack($j);
//$thread->shutdown();
}
public function spawn()
{
$this->i !== 0 ? $n=$this->i++ : $n=$this->i;
$thread = new W;
$this->workers[$n] = $thread;
$thread->start();
return $thread;
}
}
$exe = new x;
$exe->go();
php crashes.2 ways to solve the issue, either:
uncommenting
$w->shutdown();
which means the workers one creates are not reusable, or commenting out
$this->c[$n] = $w;
which means not saving workers in an array.
I do not understand what is going on here.must one call shutdown on a worker?why does attempting to store
the worker in an array lead to a crash?
How to interrupt an execution of a thread from the main context?
In the snippet below - how would one go about stopping the action that the thread does without destroying it?
class ReadFileThread extends Thread
{
public function __construct($file, $chunk = 1024)
{
$this->file = $file;
$this->chunk = $chunk;
}
public function run()
{
if(is_file($this->file) && is_readable($this->file))
{
$fh = fopen($this->file, 'rb');
while(!feof($fh))
{
$content = fread($fh, $this->chunk);
}
fclose($fh);
}
}
}
$num = 10;
$threads = [];
for($i = 0; $i < $num; $i++)
{
$thread = new ReadFileThread('/path/to/10gig_file.txt', 1024);
$threads[] = $thread;
// I start the thread, now it's detached from the main context and is reading the file asynchronously
$thread->start();
}
// The interesting part - I want to random 1 thread whose operation of file reading I want to interrupt
$to_interrupt = $threads[rand(0, $num)];
// How to interrupt the thread without destroying it? I want its context preserved
RandomSeeds answer is close, but open to race conditions.
<?php
class FileReader extends Thread {
public $file;
public $pause;
public function __construct($file) {
$this->file = $file;
$this->pause = false;
}
public function run() {
if (($handle = fopen($this->file, "rb"))) {
$len = 0;
do {
$this->synchronized(function(){
if ($this->paused) {
printf(
"\npausing %lu ...\n", $this->getThreadId());
$this->wait();
}
});
$data = fread($handle, 1024);
$len += strlen($data);
if (($len % 2) == 0) {
printf(
"\r\rread %lu", $len);
}
} while (!feof($handle));
fclose($handle);
}
}
public function pause() {
return $this->synchronized(function(){
return ($this->paused = true);
});
}
public function unpause() {
return $this->synchronized(function(){
$this->paused = false;
if ($this->isWaiting()) {
return $this->notify();
}
});
}
}
function do_something($time) {
$start = time();
while (($now = time()) < ($start + $time)) {
usleep(100);
if (($now % 2) == 0) {
echo ".";
}
}
echo "\n";
}
$reader = new FileReader("/path/to/big/file.ext");
$reader->start();
sleep(2);
$reader->pause();
do_something(rand(2, 4));
$reader->unpause();
sleep(2);
$reader->pause();
do_something(rand(2, 4));
$reader->unpause();
sleep(2);
$reader->pause();
do_something(rand(2, 4));
$reader->unpause();
?>
It is important that variables used for purposes of synchronization are only ever access in synchronized blocks, I have omitted the implementation of a stop/quit function but it's logic is much the same, as RandomSeeds example shows.
Race conditions lurk within:
public function mine($data) {
/* anyone can set doSynchronization at any time */
if ($this->doSynchronization) {
$this->synchronize(function(){
/* checking the predicate in here is safer */
$this->wait();
});
}
}
Good:
public function mine($data) {
$this->synchronize(function(){
if ($this->doSynchronization) {
$this->wait();
}
});
}
Extreme:
public function mine($data) {
$this->synchronize(function(){
while ($this->doSynchronization) {
$this->wait();
}
});
}
The posix standard would always have you write it the extreme way, I'm not so fussed, whatever works for you. The reason for this extreme code is, allowances must be made for a thread to receive a signal other than the one it is waiting on, many low level signals may result in a thread waking from a call to pthread_cond_wait; checking the predicate in a loop like that guards against what the specification calls spurious wakeups ... but such extreme measures can also lead to ill side effects; the reason those threads receive the low level signal is because it is necessary for them to take some action of some kind, ignoring that can easily cause a different part of the stack to deadlock because it expected your thread to die (or do something else, die is an example) when it was signalled ...
AFAIK you can't arbitrarily pause a concurrent thread, but you can send a notification to it. The other thread must cooperate and willingly pause itself when it receives the notification.
Example:
<?php
class MyThread extends Thread {
private $pauseRequested = false;
private $stopRequested = false;
public function pause() {
$this->synchronized(function($thread){
$thread->pauseRequested = true;
}, $this);
}
public function resume() {
$this->synchronized(function($thread){
$thread->pauseRequested = false;
$thread->notify();
}, $this);
}
public function stop() {
$this->synchronized(function($thread){
$thread->stopRequested = true;
}, $this);
}
public function run() {
echo 'Thread started!' . PHP_EOL;
while (!$this->stopRequested) {
// do the actual work
echo 'Working...';
sleep(1);
// check if we have been requested to pause
$this->synchronized(function($thread){
if ($this->pauseRequested) {
echo 'Paused...';
$thread->wait(); // this is where the magic happens
}
}, $this);
}
if ($this->stopRequested) {
echo PHP_EOL . 'Stopped!' . PHP_EOL;
}
}
}
$t = new MyThread();
$t->start();
sleep(5);
$t->pause();
sleep(2);
$t->resume();
sleep(5);
$t->stop();
// wait for $t to complete
$t->join();
?>
Never used pthreads, tbh, but have you tried making a public boolean flag inside the thread class ?