Breaking a PHP array into parts and running chunks asynchronously? - php

Question

I would like to take an array and break it up into chunks (closure $r accomplishes this)
/**
* Breaks an array into bits
*
* #param $list
* #param $p
* #return array
*/
$r = function ($list, $p)
{
$ll = count( $list ); $pl = floor( $ll / $p ); $pt = $ll % $p; $r = []; $m = 0;
for ($px = 0; $px < $p; $px++)
{
$inc = ($px < $pt) ? $pl + 1 : $pl; $r[$px] = array_slice( $list, $m, $inc );
$m += $inc;
}
return $r;
};
$hosts = [
'devhost-0',
'devhost-1',
'devhost-2',
'devhost-3',
'devhost-4',
'devhost-5',
'devhost-6',
'devhost-7',
'devhost-8',
'devhost-9',
'devhost-10',
'devhost-11',
'devhost-12',
'devhost-13',
'devhost-14',
'devhost-15',
'devhost-16',
'devhost-17',
'devhost-18',
'devhost-19',
'devhost-20',
'devhost-21',
'devhost-22',
'devhost-23',
'devhost-24',
'devhost-25',
'devhost-26',
'devhost-27',
'devhost-28',
'devhost-29',
'devhost-30',
'devhost-31',
'devhost-32',
'devhost-33',
'devhost-34',
'devhost-35',
'devhost-36',
];
$hosts = $r($hosts, 6); // chunks of six
This will break the above array into chunks of 5-6, from here I would like to run each of the chunks at the same time
with a simple method, lets say this ping closure below.
/**
* Polls host
*
* #param $host
* #param $port
* #param $timeout
* #return bool
*/
$ping = function ($host, $port, $timeout)
{
$errno = $errstr = false; // silence....
return (! #fSockOpen($host, $port, $errno, $errstr, $timeout))
? false
: true;
};
I'm not sure how to accomplish this though? I would assume that I would be using pcntl_fork() or pthreads but am uncertain as to how I should be setting this up? I have read multiple articles on running processes asynchronously but I am having issues finding one using an array as parts in an example.

Answer 1

i think this could work (based off threading example here http://acm.msu.ru/mkoshp/php-chunked-xhtml/pthreads.tutorials.html )
$hosts = [
'devhost-0',
'devhost-1',
'devhost-2',
'devhost-3',
'devhost-4',
'devhost-5',
'devhost-6',
'devhost-7',
'devhost-8',
'devhost-9',
'devhost-10',
'devhost-11',
'devhost-12',
'devhost-13',
'devhost-14',
'devhost-15',
'devhost-16',
'devhost-17',
'devhost-18',
'devhost-19',
'devhost-20',
'devhost-21',
'devhost-22',
'devhost-23',
'devhost-24',
'devhost-25',
'devhost-26',
'devhost-27',
'devhost-28',
'devhost-29',
'devhost-30',
'devhost-31',
'devhost-32',
'devhost-33',
'devhost-34',
'devhost-35',
'devhost-36',
];
class Devhost_worker extends Thread{
public $jobs;
public function __construct($jobs) {
$this->jobs = $jobs;
}
public function run() {
//do your stuff with the $this->jobs here.
}
}
$chunks=array_chunk($hosts,6,true);
$threads=array();
$i=0;
foreach($chunks as $chunk){
$threads[$i]=new Devhost_worker($chunk);
$threads[$i]->start();
++$i;
}

Answer 2

Something that I came up with just a while ago, will not accept but will post as a possible solution to my question. I was able to ping 4k of my servers in about 1 minute or so.
<?php
define('APP_DIR', dirname(__DIR__)); // autloads in non-sample script....
$hosts = explode("\n", file_get_contents(APP_DIR . '/data/test.txt'));
/**
* Processes in Parallel.
*
* Run a function (with no return result) on each item in an array in parallel.
* Note: This function is only useful if order is not important, and you don't
* need any return values from the function (i.e. no inter-process communication).
*
* #param mixed $func A closure function to apply to each item in parallel.
* #param array $arr The array to apply function to.
* #param integer $procs Number of processes to run in parallel.
*
* #return void
*/
function parallelize($func, array $arr, $procs = 4)
{
$chunks = array_chunk($arr, ceil((count($arr) / $procs)));
$pid = -1;
$children = [];
foreach ($chunks AS $items)
{
$pid = pcntl_fork();
switch ($pid)
{
case (-1): die ('Unable to fork');
case (0): // We are the child process. Pass a chunk of items to process.
array_walk($items, $func);
exit(0);
default: // We are the parent.
$children[] = $pid;
break;
}
}
// Wait for children to finish.
foreach ($children AS $pid)
{
// We are still the parent.
pcntl_waitpid($pid, $status);
}
}
/**
* Polls host
*
* #param $host
* #param $port
* #param $timeout
* #return bool
*/
$ping = function ($host, $port, $timeout)
{
$errno = $errstr = false; // silence....
return (! #fSockOpen($host, $port, $errno, $errstr, $timeout))
? false
: true;
};
/**
* Simple true false return on pinging...
*
* #param $host
* #return bool
*/
$tap = function ($host) USE ($ping)
{
if (! $ping($host, 22, 3))
{
echo "[fail] does not exist\n";
return false;
}
else
{
echo "[good] exists\n";
return true;
}
};
parallelize($tap, $hosts, 20); // do work....