Im trying to make a pthreads class that continuously updates a variable that I'll be using elsewhere in the script. The pthreads class should pull the contents of a URL, assign the value to the variable, sleep and repeat.
$pair = array();
class Pair extends Thread {
public function __construct($url){
$this->url = $url;
}
public function run() {
global $pair;
while (true){
$pair = json_decode("https://btc-e.com/api/2/".$this->url."/ticker", true);
sleep(5);
}
}
}
$BTC_USD = new Pair("btc_usd");
while (true){
print_r($pair);
sleep(5);
}
$pair needs to be continuously updated and printed to the screen
<?php
define ('SECOND', 1000000);
class Repeater extends Thread {
public $url;
public $data;
public function __construct($url) {
$this->url = $url;
}
public function run() {
do {
/* synchronize the object */
$this->synchronized(function(){
/* fetch data into object scope */
$this->data = file_get_contents($this->url);
/* wait or quit */
if (!$this->stop)
$this->wait();
});
} while (!$this->stop);
}
public function stop() {
/* synchronize the object */
$this->synchronized(function(){
/* set stop */
$this->stop = true;
/* notify the object (could/should be waiting) */
$this->notify();
});
}
public function getData() {
/* fetch data / wait for it to arrive */
return $this->synchronized(function(){
return $this->data;
});
}
}
/* to provide a limit for the example */
$iterations = 5;
/* the url etc is not important */
$repeat = new Repeater("http://www.google.com");
$repeat->start();
/* this will only return when there is data */
while (($data = $repeat->getData())) {
var_dump(strlen($data));
$repeat->synchronized(function($repeat){
/* notify allows the thread to make he next request */
$repeat->notify();
/* waiting here for three seconds */
$repeat->wait(3 * SECOND);
}, $repeat);
if (!--$iterations) {
/* because examples should be sensible */
$repeat->stop();
break;
}
}
?>
It does not seem like a good use of resources to have a request per thread, you probably don't really want to do that in the real world. sleep is not suitable for use in multi-threading, it does not leave threads in a receptive state.
globals have no effect in the context of threading, they work as they usually do with regard to scope, but it is not inclusive of the scope of a thread.
Related
I am using a library, and it has the following process to attach many operations to one event:
$action = (new EventBuilder($target))->addOperation($Operation1)->addOperation($Operation2)->addOperation($Operation3)->compile();
I am not sure how to dynamically add operations depending on what I need done.
Something like this
$action = (new EventBuilder($target));
while (some event) {
$action = $action->addOperation($OperationX);
}
$action->compile();
I need to be able to dynamically add operations in while loop and when all have been added run it.
Your proposed solution will work. The EventBuilder provides what is known as a Fluent Interface, which means that there are methods that return an instance of the builder itself, allowing you to chain calls to addOperation as many times as you want, then call the compile method to yield a result. However you are free to ignore the return value of addOperation as long as you have a variable containing an instance of the builder that you can eventually call compile on.
Take a walk with me...
// Some boilerplate classes to work with
class Target
{
private ?string $name;
public function __construct(string $name)
{
$this->name = $name;
}
public function getName(): string
{
return $this->name;
}
}
class Operation
{
private ?string $verb;
public function __construct(string $verb)
{
$this->verb = $verb;
}
public function getVerb(): string
{
return $this->verb;
}
}
class Action
{
private ?Target $target;
private array $operations = [];
public function __construct(Target $target, array $operations)
{
$this->target = $target;
$this->operations = $operations;
}
/**
* Do the things
* #return array
*/
public function run(): array
{
$output = [];
foreach ($this->operations as $currOperation)
{
$output[] = $currOperation->getVerb() . ' the ' . $this->target->getName();
}
return $output;
}
}
Here is a basic explanation of what your EventBuilder is doing under the covers:
class EventBuilder
{
private ?Target $target;
private array $operations = [];
public function __construct(Target $target)
{
$this->target = $target;
}
/**
* #param Operation $operation
* #return $this
*/
public function addOperation(Operation $operation): EventBuilder
{
$this->operations[] = $operation;
// Fluent interface - return a reference to the instance
return $this;
}
public function compile(): Action
{
return new Action($this->target, $this->operations);
}
}
Let's try both techniques and prove they will produce the same result:
// Mock some operations
$myOperations = [
new Operation('Repair'),
new Operation('Clean'),
new Operation('Drive')
];
// Create a target
$target = new Target('Car');
/*
* Since the EventBuilder implements a fluent interface (returns an instance of itself from addOperation),
* we can chain the method calls together and just put a call to compile() at the end, which will return
* an Action instance
*/
$fluentAction = (new EventBuilder($target))
->addOperation($myOperations[0])
->addOperation($myOperations[1])
->addOperation($myOperations[2])
->compile();
// Run the action
$fluentResult = $fluentAction->run();
// Traditional approach, create an instance and call the addOperation method as needed
$builder = new EventBuilder($target);
// Pass our mocked operations
while (($currAction = array_shift($myOperations)))
{
/*
* We can ignore the result from addOperation here, just keep calling the method
* on the builder variable
*/
$builder->addOperation($currAction);
}
/*
* After we've added all of our operations, we can call compile on the builder instance to
* generate our Action.
*/
$traditionalAction = $builder->compile();
// Run the action
$traditionalResult = $traditionalAction->run();
// Verify that the results from both techniques are identical
assert($fluentResult == $traditionalResult, 'Results from both techniques should be identical');
// Enjoy the fruits of our labor
echo json_encode($traditionalResult, JSON_PRETTY_PRINT).PHP_EOL;
Output:
[
"Repair the Car",
"Clean the Car",
"Drive the Car"
]
Rob Ruchte thank you for detailed explanation, one thing I did not include was that each operation itself had ->build() call and I needed to move that to each $builder for it to work.
I'm using php-zts to perform parallel data processing, using symfony 4 and PThreads
I'm great at running multiple threads, but I'm facing a problem, I need each of the threads to be able to work with doctrine
I need to make sure that each thread is able to work with doctrine
I tried to transfer a container instance directly, but it won't work because it can't be sterilized
/console_comand.php
private function gettingStatistics(){
$pool = new \Pool(4, Autoloader::class, ["vendor/autoload.php"]);
$store = new \Threaded();
$class = new Meta();
$pool->submit(new Task($class,$store));
$pool->collect();
$pool->shutdown();
$listQuotes = array();
foreach ($store as $obj){
foreach ($obj->{'response'} as $exchange => $data){
$listQuotes[$exchange] = $data;
}
}
unset($store);
unset($interface);
return $listQuotes;
}
/Autoloader.php
<?php
namespace App\Worker;
class Autoloader extends \Worker
{
protected $loader;
public function __construct($loader)
{
$this->loader = $loader;
}
/* включить автозагрузчик для задач */
public function run()
{
require_once($this->loader);
}
/* переопределить поведение наследования по умолчанию для нового потокового контекста */
public function start(int $options = PTHREADS_INHERIT_ALL)
{
return parent::start(PTHREADS_INHERIT_NONE);
}
}
/Autoloadable.php
<?php
namespace App\Worker;
/* нормальный, автоматически загруженный класс */
class Autoloadable
{
public $response;
public function __construct($greeting)
{
$this->response = $greeting->job();
}
}
/Task.php
<?php
namespace App\Worker;
class Task extends \Threaded
{
protected $greeting;
protected $result;
public function __construct($greeting,\Threaded $store)
{
$this->greeting = $greeting;
$this->result = $store;
}
public function run()
{
$greeting = new Autoloadable($this->greeting);
$this->result[] = $greeting;
}
}
how do I pass the right doctrine to be able to work with it from the job?
there's a very similar question on github but I can't deal with it.
https://github.com/krakjoe/pthreads/issues/369
Have you tried requiring an ObjectManager instance in the __construct of Task (your last code block)?
Have a read of this article
Cannot test it atm, don't have zts setup, but I've used this to great success in other projects.
I would expect you need to do something like:
$pool = new Pool(4);
for ($i = 0; $i < 15; ++$i) {
$pool->submit(new class($objectManager) extends \Threaded
{
private $objectManager;
public function __construct(ObjectManager $objectManager)
{
$this->objectManager= $objectManager;
}
public function run()
{
// obviously replace the contents of this function
$this->objectManager->performTask;
echo 'Job\'s done.' . PHP_EOL;
}
});
}
while ($pool->collect());
$pool->shutdown();
The instantiation of the new anonymous class takes the $objectManager present in your current instance, like /console_comand.php there, and passes it to this new anonymous class to fulfill the __construct requirements.
The linked article does a better job of explaining it than I do, so please give it a read.
I am using PThreads for multi-threading in php. I have installed and run it successfully on my XAMPP server on windows.
I have 100K records in my database and I want to run 20 threads in parallel.Every thread will call 5k record from database and process them.Here is my code for this
require('mailscript.php');
class My extends Thread{
function __construct() {
$this->mailscript = new mailscript();
}
function run(){
$this->mailscript->runMailScript(5000);
}
}
for($i=0;$i<20;$i++){
$pool[] = new My();
}
foreach($pool as $worker){
$worker->start();
}
foreach($pool as $worker){
$worker->join();
}
When I run this code it only run for approx 600 records per thread maximum.Is there any limit issue for number of threads in PThreads. What is the issue please help me
hie, here is how I would handle pthreads with your example with a collection method to be used if nescessary. Hope this will help you.
/*pthreads batches */
$batches = array();
$nbpool = 20; // cpu 10 cores
/* job 1 */
$list = [/* data1 */];
$batches[] = array_chunk($list, 5000);
/* job 2 */
$list2 = [/* data2 */];
$batches[] = array_chunk($list2, 10000);
/*final collected results*/
$resultFinal = [];
/* loop across batches */
foreach ($batches as $key => $chunks) {
/* for intermediate collection */
$data[$key] = [];
/* how many workers */
$workCount = count($chunks);
/* set pool job up to max cpu capabilities */
$pool = new Pool($nbpool, Worker::class);
/* pool cycling submit */
foreach (range(1, $workCount) as $i) {
$chunck = $chunks[$i - 1];
$pool->submit(new processWork($chunck));
}
/* on collection cycling */
$collector = function (\Collectable $work) use (&$data) {
/* is worker complete ? */
$isGarbage = $work->isGarbage();
/* worker complete */
if ($isGarbage) {
$data[$key] = $work->result;
}
return $isGarbage;
};
do {
/* collection on pool stack */
$count = $pool->collect($collector);
$isComplete = count($data) === $workCount;
} while (!$isComplete);
/* push stack results */
array_push($resultFinal, $data);
/* close pool */
$pool->shutdown();
}
class processWork extends \Threaded implements \Collectable {
private $isGarbage;
private $process;
public $result;
public function __construct($process) {
$this->process = $process;
}
public function run() {
$workerDone = array();
foreach ($this->process as $k => $el) {
/* whatever stuff with $this->process */
}
$this->result = $workerDone;
$this->isGarbage = true; // yeah, it s done
}
public function isGarbage(): bool {
return $this->isGarbage;
}
}
I`m writing a Timer script,using a signal.Every 1 second send a SIGALRM to the process,then catch the signal and check whether the there is a task to do.
I use pcntl_alarm(1) to send the signal,but the process excute-and-quit,before receive the signal.
I think whether there is a method to set the prcess blocked waiting for the signal.I have used base_event,but its much heavy.
So anybody can help me.Thanks a lot.
That`s my script(Sorry for the chinesse ^_^):
<?php
/**
*定时器
*/
class Timer
{
//保存所有定时任务
public static $task = array();
//定时间隔
public static $time = 1;
/**
*开启服务
*#param $time int
*/
public static function run($time = null)
{
if($time)
{
self::$time = $time;
}
self::installHandler();
pcntl_alarm(1);
do{ sleep(2); }while(true);
}
/**
*注册信号处理函数
*/
public static function installHandler()
{
echo "installHandler\n";
pcntl_signal(SIGALRM, array('Timer','signalHandler'),false);
pcntl_alarm(1);
}
/**
*信号处理函数
*/
public static function signalHandler()
{
self::task();
}
/**
*执行回调
*/
public function task()
{
if(empty(self::$task))
{
return ;
}
foreach(self::$task as $time => $arr)
{
$current = time();
$func = $arr['func'];
$argv = $arr['argv'];
$interval = $arr['interval'];
$persist = $arr['persist'];
if($current == $time)
{
call_user_func_array($func, $argv);
}
if($persist)
{
self::$task[$current+$interval] = $arr;
}
unset(self::$task[$time]);
}
pcntl_alarm(self::$time);
}
/**
*添加任务
public static function add($interval, $func, $argv,$persist = false)
{
if(is_null($interval))
{
return;
}
$time = time()+$interval;
self::$task[$time] = array('func'=>$func, 'argv'=>$argv, 'interval'=>$interval, 'persist'=>$persist);
}
/**
*删除所有定时器任务
*/
public function dellAll()
{
self::$task = array();
}
}
Now, I know the answer even though so many times has passed away for I had other jobs to finish...sorry for the late.So my answer is very simple.
sleep
is right for blocking a process, then the process receives a signal,and what I have missed is never capture it...so use
pcntl_signal_dispatch
function in a loop is exactly the way.
I have a site which uses the SplObserver class to add events and then notify various users by email about those events.
However, I am having a problem with the code:
/**
* Attach a Observer to Offer
* #param SplObserver $obs Observer instance
*
* return void
*/
public function attach(SplObserver $obs)
{
$id = spl_object_hash($obs);
$this->observers[$id] = $obs;
}
/**
* Remove a Observer to Offer
* #param SplObserver $obs Observer instance
*
* #return void
*/
public function detach(SplObserver $obs)
{
$id = spl_object_hash($obs);
unset($this->observers[$id]);
}
/**
* Notify attached Observer
*/
public function notify()
{
if ($this->observers) {
foreach ($this->observers as $obs) {
$this->detach($obs);
$obs->update($this);
}
}
$this->lastEvent = null;
}
public function prepareCommonObservers(MyPDO $DB, Core_Context $context, $extraInfo=null)
{
//Mails
require_once ('class/contrib/phpmailer/class.phpmailer.php');
$notificator = new MailNotificator(new PHPMailer(),new FileReader(),$context);
$mailObs = OfferObserver_OfferMail::getInstance($this->type(), $context, $notificator, $DB);
$mailObs->extraInfo=$extraInfo;
$this->attach($mailObs);
//Business Listing Limits
$keyObs = new OfferObserver_BusinessLimitObserver($context, $notificator, $DB);
$this->attach($keyObs);
//Keywords
$keyObs = new OfferObserver_KeywordObserver($context, $notificator, $DB);
$this->attach($keyObs);
//Banned Words
$bannedWordObs = new OfferObserver_BannedWordsObserver($DB);
$this->attach($bannedWordObs);
//Final value fee
if ($this->item->user->businessSeller) {
$finalFee = BusinessFee::getInstanceFromDB($DB, 'offer/finalValue', $this->originalSiteId, $context->siteOptions['currentChargeStructure']);
} else {
$finalFee = Fee::getInstanceFromDB($DB, 'offer/finalValue', $this->originalSiteId, $context->siteOptions['currentChargeStructure']);
}
if ($finalFee->active) {
$finalValueObs = new OfferObserver_FinalValueFeeObserver($context, $notificator, $DB);
$this->attach($finalValueObs);
}
//Auction Item queue
$itemQueueObs = new OfferObserver_AuctionQueue($DB);
$this->attach($itemQueueObs);
}
In particular, for two sites running the same code on Centos 5, I have had to alter the original:
public function notify()
{
if ($this->observers) {
foreach ($this->observers as $obs) {
$obs->update($this);
}
}
}
to read:
public function notify()
{
if ($this->observers) {
foreach ($this->observers as $obs) {
$this->detach($obs);
$obs->update($this);
}
}
$this->lastEvent = null;
}
Without this change, I was often getting duplicated emails sent out. Both servers are running Centos 5.11, although different PHP versions (5.5 and 5.6).
However, on another site running on Centos 7 (PHP 5.6), I have to revert to the original function, otherwise emails are not sent out...
Any suggestions as to what may be happening?
The answer for me is to ensure that at the end of the update() function all of the observers are nullified - it is interesting that this change was not needed on Centos 7 - I wonder why?
public function notify()
{
if ($this->observers) {
foreach ($this->observers as $obs) {
$obs->update($this);
}
}
$this->observers = null;
}