pthreads destroys array results - php

Question

I have a code that uses pthreads v3.1.6 with PHP 7.0.22. The issue I'm having is that the threads do not return array values. My code is as follows
$threadCount = 1;
$url_idList = range(1,2000);
$url_idListChunked = array_chunk($url_idList, $threadCount);
class WorkerThreads extends Thread {
private $threadName, $url_id;
public function __construct($threadName,$url_id) {
$this->threadName = $threadName;
$this->url_id = $url_id;
$this->result = [];
}
public function run() {
if ($this->threadName && $this->url_id) {
printf('%sLoading URL #: %s' . "\n", $this->threadName, $this->url_id);
$this->result = send_request('GET',$this->url_id,NULL,$this->threadName);
}
}
}
while(count($url_idListChunked)){
$url_idListChunk = array_shift($url_idListChunked);
$workers = [];
foreach (range(0,count($url_idListChunk)-1) as $i) {
$threadName = "Thread #".$i.": ";
$workers[$i] = new WorkerThreads($threadName,$url_idListChunk[$i]);
$workers[$i]->start();
}
foreach (range(0,count($url_idListChunk)-1) as $i) {
$workers[$i]->join();
print_r($workers[$i]);
exit();
echo $workers[$i]['threadName']."Result for URL #: ".$workers[$i]['url_id']."\n";
}
}
function send_request($method,$url_id,$data,$threadName=NULL){
$url = 'https://www.example.tld/?id='.$url_id;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_TIMEOUT, 3);
if(!$data && $method=='POST'){
$data = generate_post_data();
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
}
$response = curl_exec($ch);
while((curl_errno($ch) == 6 OR curl_errno($ch) == 28)){
$response = curl_exec($ch);
echo $threadName.'Curl error #'.curl_errno($ch).' - ' . curl_error($ch)." Retrying.\n";
sleep(2);
}
curl_close($ch);
$result['data'] = $response;
return $result;
}
When I get try to print_t($workers) I get the following error message Uncaught RuntimeException: pthreads detected an attempt to connect to an object which has already been destroyed. Why do I lose the array results? It seems like threads have no problem passing the strings back.

Answer 1

Hie, I am not completely sure using start->join method is the preferred way to go for what you want to achieve? I think you need to use the pool collectable method that is made for in pthreads.
Here is an example that might inspire you in your work if you want to collect results from chunked batches. I personally use it intensively and this is the fastest method to push pthreads into its limits. Careful not to push the pool number over your CPU threads (in this example it s 10 cores).
If I may say regarding your code, never try to output something on screen from a pthreads worker, it will surely mess around. Return the object and echo it on collection. Make sure you have result has public in your class to allow returned object.
Not to mention multi-curl that might be the fastest and appropriate way of doing it.
/* pthreads batches */
$batches = array();
$nbpool = 20; // cpu 10 cores
/* job 1 */
$list = [/* data1 */];
$url_idList[] = array_chunk($list, 5000);
/* job 2 */
$list2 = [/* data2 */];
$url_idList[] = array_chunk($list, 10000);
/* final collected results */
$resultFinal = [];
/* loop across batches */
foreach ($url_idList as $key => $url_idListChunked) {
$url_idListChunk = array_shift($url_idListChunked);
/* for intermediate collection */
$data[$key] = [];
/* how many workers */
$workCount = count($url_idListChunk);
/* set pool job up to max cpu capabilities */
$pool = new Pool($nbpool, Worker::class);
/* pool cycling submit */
foreach (range(1, $workCount) as $i) {
$chunck = $url_idListChunk[$i - 1];
$pool->submit(new WorkerThreads(($i - 1), $chunck));
}
/* on collection cycling */
$collector = function (\Collectable $work) use (&$data) {
/* is worker complete ? */
$isGarbage = $work->isGarbage();
/* worker complete */
if ($isGarbage) {
$result = $work->result;
$info = $work->info;
$data[$key] = $result;
/* echo result info outside worker */
echo($info);
}
return $isGarbage;
};
do {
/* collection on pool stack */
$count = $pool->collect($collector);
$isComplete = count($data) === $workCount;
} while (!$isComplete);
/* push stack results */
array_push($resultFinal, $data);
/* close pool */
$pool->shutdown();
}
class WorkerThreads extends \Threaded implements \Collectable {
private $url_id;
private $isGarbage;
private $threadName;
public $result;
public $info;
public function __construct($i, $url_id) {
$this->threadName = "Thread #" . $i . ": ";
$this->url_id = $url_id;
}
public function run() {
if ($this->threadName && $this->url_id) {
$this->result = send_request('GET', $this->url_id, NULL, $this->threadName);
}
$this->info = $this->threadName . " Result for URL #: " . $this->url_id;
$this->isGarbage = true; // yeah, it s done
}
public function isGarbage(): bool {
return $this->isGarbage;
}
}