I've been using System_Daemon class to create a daemon to send sms.
The script worked perfect with php 5.3.8, but now, with php 5.4.9 it crashes but no error or notice messages is created.
In the function _fork, of System_Daemon class, always return a value that tells it's parent
static protected function _fork()
{
self::debug('forking {appName} daemon');
$pid = pcntl_fork();
if ($pid === -1) {
// Error
return self::warning('Process could not be forked');
} else if ($pid) {
// Parent
self::debug('Ending {appName} parent process');
// Die without attracting attention
exit();
} else {
// Child
self::$_processIsChild = true;
self::$_isDying = false;
self::$_processId = posix_getpid();
return true;
}
}
So, in _summon() function, where it ask about _fork() returned value, always is equal to false.
I've red this post, from another member who has a similar issue:
PHP Pear system_daemon doesn't fork
I've made his suggestions but with no success.
Can somebody, please, give me a hand with this?
I'm so sorry about my english, i made an effort to explain myself.
Related
Problem
The title really says it all. I had a Timeout class that handled timing out using pcntl_alarm(), and during ssh2_* calls, the signal handler simply does not fire when it should.
In its most basic form,
print date('H:i:s'); // A - start waiting, ideally 5 seconds at the most
pcntl_alarm(5);
// SSH attempt to download remote file /dev/random to ensure it blocks *hard*
// (omitted)
print date('H:i:s'); // B - get here once it realizes it's a tripeless-cat scenario
with a signal handler that also outputs the date, I would expect this (rows B and C might be inverted, that does not matter)
"A, at 00:00:00"
"C, at 00:00:05" <-- from the signal handler
"B, at 00:00:05"
but the above will obtain this disappointing result instead:
"A, at 00:00:00"
"C, at 00:01:29" <-- from the signal handler
"B, at 00:01:29"
In other words, the signal handler does fire, but it does so once another, longer, timeout has expired. I can only guess this timeout is inside ssh2_*. Quickly browsing through the source code yielded nothing obvious. What's worse, this ~90 seconds timeout is what I can reproduce by halting a download. In other cases when the firewall dropped the wrong SSH2 packet, I got a stuck process with an effectively infinite timeout.
As the title might reveal, I have already checked other questions and this is definitely not a mis-quoting of SIGALRM (also, the Timeout class worked beautifully). It seems like I need a louder alarm when libssh2 is involved.
Workaround
This modification of the Timeout yields a slightly less precise, but always working system, yet it does so in an awkward, clunky way. I'd really prefer for pcntl_alarm to work in all cases as it's supposed to.
public static function install() {
self::$enabled = function_exists('pcntl_async_signals') && function_exists('posix_kill') && function_exists('pcntl_wait');
if (self::$enabled) {
// Just to be on the safe side
pcntl_async_signals(true);
pcntl_signal(SIGALRM, static function(int $signal, $siginfo) {
// Child just died.
$status = null;
pcntl_wait($status);
static::$expired = true;
});
}
return self::$enabled;
}
public static function wait(int $time) {
if (self::$enabled) {
static::$expired = false;
// arrange for a SIGALRM to be delivered in $time seconds
// pcntl_alarm($time);
$ppid = posix_getpid();
$pid = pcntl_fork();
if ($pid == -1) {
throw new RuntimeException('Could not fork alarming child');
}
if ($pid) {
// save the child's PID
self::$thread = $pid;
return;
}
// we are the child. Send SIGALRM to the parent after requested timeout
sleep($time);
posix_kill($ppid, SIGALRM);
die();
}
}
/**
* Cancel the timeout and verify whether it expired.
**/
public static function expired(): bool {
if (self::$enabled) {
// Have we spawned an alarm?
if (self::$thread) {
// Yes, so kill it.
posix_kill(self::$thread, SIGTERM);
self::$thread = 0;
}
// Maybe.
$status = null;
pcntl_wait($status);
// pcntl_alarm(0);
}
return static::$expired;
}
Question
Can pcntl_alarm() be made to work as expected?
Console script perform data import from external API. For boosting import loading performed in concurrent processes that is created by pcntl_fork command.
For communication with API cUrl is used. Communication performed by https protocol.
By some undefined reason periodically some children becomes zombie. There is no errors/warnings/notices in console and also there is not logs is written. Errors level is configured appropriately.
After investigation I suppose that there problem in curl extension since without it, with fake connection, there is no problems.
Also if run import in single process mode - there is no problems at all.
PHP: 7.2.4,
OS: Debian 9,
Curl: 7.59.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 OpenSSL/1.0.2g zlib/1.2.8 libidn/1.32 librtmp/2.3
Maybe someone encountered similar problem or know possible reasons of this strange behavior?
Pseudo code sample of child logic (main part of child showed):
while (true) {
$socket->writeRawString(Signal::MESSAGE_REQUEST_DATA);
$response = $socket->readRawString();
if (Signal::MESSAGE_TERMINATE_PROCESS === $response) {
break;
}
$response = json_decode($response, true);
if (empty($response) || empty($response['deltaId'])) {
continue;
}
$delta = $this->providerConnection->getChanges($response['deltaId']);
if(empty($delta)) {
continue;
}
$xmlReader = new \XMLReader();
$xmlReader->XML($delta);
$xmlReader->read();
$xmlReader->read();
$hasNext = true;
while ($hasNext && 'updated' !== $xmlReader->name) {
$hasNext = $xmlReader->next();
}
if ('updated' !== $xmlReader->name) {
throw new \RuntimeException('Deltas file do not contain updated date.');
}
if (strtotime($xmlReader->readString()) < $endDateTimestamp) {
$socket->writeRawString(self::SIGNAL_END_DATE_REACHED);
continue;
}
}
posix_kill(\posix_getpid(), SIGTERM);
In providerConnection->getChanges($response['deltaId']); request performed via cUrl. For work with cUrl used Php cUrl class extension
As mentioned in my comments, your problem probably is, that childprocesses that died/finished need to be collected by the parent process, or they remain as zombies.
First solution:
Install a signal handler in the parent. Something like this:
pcntl_signal(SIGCHLD, [$this, 'handleSignals']);
With a signal handler that could look like this:
/**
* #param integer $signal
*/
public function handleSignals($signal) {
switch($signal) {
case SIGCHLD:
do {
$pid = pcntl_wait($status, WNOHANG);
} while($pid > 0);
break;
default:
//Nothing to do
}
}
I normally store the pids of forked children and check them all individually with pcntl_waitpid, but this could get you going.
Second Solution:
Use a double-fork to spawn the child-processes, if the parent does not need to wait for all sub-tasks to finish. A double fork looks like this:
$pid = pcntl_fork();
if ($pid == -1) handleError();
elseif ($pid == 0) { // child
$pid = pcntl_fork();
if ($pid == -1) handleChildError();
elseif($pid == 0) { // second level child
exit(startWork()); // init will take over this process
}
// exit first level child
exit(0);
} else {
// parent, wait for first level child
pcntl_wait($pid, $status); // forked child returns almost immediatly, so blocking wait is in order
}
I give up using cUrl for my task. Today I switched to Guzzle with StreamHandler instead of cUrl and it solved all my problems.
I suppose, that due to some internal errors in cUrl, system was killing my child processes.
This is not answer to my question. It just workaround of my problem for those who may also encounter similar problem.
Topic is still open for possible suggestions/explanations.
I've a problem using elasticsearch in a php application. The application is built with zend and uses a .env to hold the following configuration:
ELASTICSEARCH_MAX_DOCUMENTS=250
ELASTICSEARCH_MAX_BULK_SIZE=3M
ELASTICSEARCH_HOST=my-elasticsearch.intern.rz
ELASTICSEARCH_PORT=80
ELASTICSEARCH_USER=user
ELASTICSEARCH_PASSWORD=pw
The call to index the new files is part of a import service class and looks like this:
public function flush(FlushInterface $flushInterface = null) {
$bulk = $this->getSearchDocumentBulk();
if (!empty($bulk->getActions())) {
$response = $bulk->send();
$this->resetSearchDocumentBulk();
if (0 === $response->count()) {
$data = $response->getData();
throw new BulkException(isset($data['message']) ? strip_tags($data['message']) : '');
}
}
$this->documentCache = [];
if ($flushInterface instanceof FlushInterface) {
$flushInterface->flush();
}
return $this;
}
protected function getSearchDocumentBulk() {
if (!($this->searchDocumentBulk instanceof Bulk)) {
$this->searchDocumentBulk = new Bulk($this->getSearchClient()->getClient());
$this->searchDocumentBulk->setIndex(self::INDEX);
}
return $this->searchDocumentBulk;
}
I know it's only a short snippet but it's quite difficult to pull out the relevant code. So please let me know if I have to post some more methods.
The application is started by a symfony command and I'm able to curl to elasticsearch (version 5.1) without any errors.
My problem is that no document is indexed. If I check elasticsearch-head I see that there was no data transfer anymore.
But there's also no error, no exception or something like that. The process is completed with 100% (100,000 of 100,000 files imported). So I've no idea what happens or how to find out the bug.
How should I multithread some php-cli code that needs a timeout?
I'm using PHP 5.6 on Centos 6.6 from the command line.
I'm not very familiar with multithreading terminology or code. I'll simplify the code here but it is 100% representative of what I want to do.
The non-threaded code currently looks something like this:
$datasets = MyLibrary::getAllRawDataFromDBasArrays();
foreach ($datasets as $dataset) {
MyLibrary::processRawDataAndStoreResultInDB($dataset);
}
exit; // just for clarity
I need to prefetch all my datasets, and each processRawDataAndStoreResultInDB() cannot fetch it's own dataset. Sometimes processRawDataAndStoreResultInDB() takes too long to process a dataset, so I want to limit the amount of time it has to process it.
So you can see that making it multithreaded would
Speed it up by allowing multiple processRawDataAndStoreResultInDB() to execute at the same time
Use set_time_limit() to limit the amount of time each one has to process each dataset
Notice that I don't need to come back to my main program. Since this is a simplification, you can trust that I don't want to collect all the processed datasets and do a single save into the DB after they are all done.
I'd like to do something like:
class MyWorkerThread extends SomeThreadType {
public function __construct($timeout, $dataset) {
$this->timeout = $timeout;
$this->dataset = $dataset;
}
public function run() {
set_time_limit($this->timeout);
MyLibrary::processRawDataAndStoreResultInDB($this->dataset);
}
}
$numberOfThreads = 4;
$pool = somePoolClass($numberOfThreads);
$pool->start();
$datasets = MyLibrary::getAllRawDataFromDBasArrays();
$timeoutForEachThread = 5; // seconds
foreach ($datasets as $dataset) {
$thread = new MyWorkerThread($timeoutForEachThread, $dataset);
$thread->addCallbackOnTerminated(function() {
if ($this->isTimeout()) {
MyLibrary::saveBadDatasetToDb($dataset);
}
}
$pool->addToQueue($thread);
}
$pool->waitUntilAllWorkersAreFinished();
exit; // for clarity
From my research online I've found the PHP extension pthreads which I can use with my thread-safe php CLI, or I could use the PCNTL extension or a wrapper library around it (say, Arara/Process)
https://github.com/krakjoe/pthreads (and the example directory)
https://github.com/Arara/Process (pcntl wrapper)
When I look at them and their examples though (especially the pthreads pool example) I get confused quickly by the terminology and which classes I should use to achieve the kind of multithreading I'm looking for.
I even wouldn't mind creating the pool class myself, if I had a isRunning(), isTerminated(), getTerminationStatus() and execute() function on a thread class, as it would be a simple queue.
Can someone with more experience please direct me to which library, classes and functions I should be using to map to my example above? Am I taking the wrong approach completely?
Thanks in advance.
Here comes an example using worker processes. I'm using the pcntl extension.
/**
* Spawns a worker process and returns it pid or -1
* if something goes wrong.
*
* #param callback function, closure or method to call
* #return integer
*/
function worker($callback) {
$pid = pcntl_fork();
if($pid === 0) {
// Child process
exit($callback());
} else {
// Main process or an error
return $pid;
}
}
$datasets = array(
array('test', '123'),
array('foo', 'bar')
);
$maxWorkers = 1;
$numWorkers = 0;
foreach($datasets as $dataset) {
$pid = worker(function () use ($dataset) {
// Do DB stuff here
var_dump($dataset);
return 0;
});
if($pid !== -1) {
$numWorkers++;
} else {
// Handle fork errors here
echo 'Failed to spawn worker';
}
// If $maxWorkers is reached we need to wait
// for at least one child to return
if($numWorkers === $maxWorkers) {
// $status is passed by reference
$pid = pcntl_wait($status);
echo "child process $pid returned $status\n";
$numWorkers--;
}
}
// (Non blocking) wait for the remaining childs
while(true) {
// $status is passed by reference
$pid = pcntl_wait($status, WNOHANG);
if(is_null($pid) || $pid === -1) {
break;
}
if($pid === 0) {
// Be patient ...
usleep(50000);
continue;
}
echo "child process $pid returned $status\n";
}
Is there any way to determine whether FirePHP is installed on the server (via PEAR)? I'd like to make possible logging in FirePHP but also not to crash the code for everyone without that tool.
The example, how I imagine it should work:
$message = "hello";
function log($message) {
if (library_exists('FirePHPCore/fb.php')) {
require_once('FirePHPCore/fb.php');
ob_start();
\FB::log($message);
} else {
SomeBoringLogger::log($message);
}
}
I haven't found anything like my library_exists method. Is there anything like that in PHP?
#include_once('FirePHPCore/fb.php'); // Ignore any errors here, as we check for existance
if (class_exists('FirePHP')) { // Do something after this
http://php.net/manual/en/function.class-exists.php
FirePHP uses FirePHP as its class name, so if it is available, that class should be defined
For PHP 5.3.2 or later, use zerkms's suggestion:
(!stream_resolve_include_path('FirePHPCore/fb.php')===FALSE)
Use include_once, so it doesn't kill the request. As #Brad suggests, use class_exists afterwards.
$message = "hello";
safe_include_once('FirePHPCore/fb.php');
if (class_exists('FB')) {
function log($message) {
//using FirePHP
}
} else {
function log($message) {
SomeBoringLogger::log($message);
}
}
function safe_include_once($path) {
if ($path = stream_resolve_include_path($path)) {
include_once($path);
}
}
[Edit] Using stream_resolve_include_path in safe_include_path.
[Edit2] Faster runtime logging.
file_exists() can be used for your case.