pthreads access to resource globally in PHP - php

I'm having problems with accessing resources from global context using php pthreads for Windows.
The logic is very simple: there's a Server as a main class, only one, which will:
create logfile
create couple of threads
each thread will need to write something to this logfile
Problem is, the resource of logfile handle is somehow totally messed up from inside the thread. When I create the resource, it's fine, I can write in it. When I try to call the log from inside the running thread, the resource of logfile handler appears to be integer 0, not even a resource.
Here's my code:
$main_server = new CMainServer();
$main_server->init();
$main_server->go();
$main_server->log("All done");
Inside the CMainServer class:
class CMainServer
{
private $logfile = null;
public function init()
{
$this->logfile = fopen('wstext.log', 'w');
}
public function log($str)
{
if ($this->logfile === null)
{
echo "[".date("H:i:s", time())."]: logfile is null<BR />";
return false;
}
if (!is_resource($this->logfile))
{
echo "[".date("H:i:s", time())."]: logfile is NOT a resource, can't write {$str}<BR />";
return false;
}
echo "[".date("H:i:s", time())."]: logfile is resource, not null, writing {$str}<BR />";
flush();
fwrite($this->logfile, "[".date("H:i:s", time())."]: {$str}\r\n");
return true;
}
public function go()
{
$this->log('Before creating a thread');
$first_thread = new CThread();
$first_thread->start(PTHREADS_INHERIT_ALL | PTHREADS_ALLOW_GLOBALS);
$second_thread = new CThread();
$second_thread->start(PTHREADS_INHERIT_ALL | PTHREADS_ALLOW_GLOBALS);
$first_thread->join();
$second_thread->join();
}
public function __destruct()
{
if ($this->logfile)
fclose($this->logfile);
}
}
And, finally, the CThread class:
class CThread extends Thread
{
public function run()
{
global $main_server;
$thread_id = $this->getThreadId();
Thread::globally(function()
{
for ($i = 0; $i < 2; $i++)
{
$main_server->log("({$i}) writing random number ".rand(0, 100)." to log from running thread id={$thread_id}");
sleep(1);
}
});
}
}
Result is sad:
[13:38:10]: logfile is NOT a resource, can't write (0) writing random number 21 to log from running thread id=9080
[13:38:11]: logfile is NOT a resource, can't write (1) writing random number 91 to log from running thread id=9080
[13:38:10]: logfile is NOT a resource, can't write (0) writing random number 16 to log from running thread id=17316
[13:38:11]: logfile is NOT a resource, can't write (1) writing random number 50 to log from running thread id=17316
[13:38:10]: logfile is resource, not null, writing Before creating a thread
[13:38:12]: logfile is resource, not null, writing All done
So while I'm outside the thread, all is fine. However, from within a thread the $logfile is not a resource at all.
I tried different options: tried calling from CThread::run() a global function:
function LogFromThread($i, $thread_id)
{
global $main_server;
$main_server->log("({$i}) writing random number ".rand(0, 100)." to log from running thread id={$thread_id}");
}
The result is the same.
Tried without Thread::globally() at all, but all for no good.
I'm running Apache/2.4.10 (Win32) OpenSSL/1.0.1i PHP/5.6.3, tried pthreads version 2.0.8, 2.0.9. Also tried with PHP 7RC2 and RC3, but there seems to be a problem to start a new thread at all, apache logs an error, so I returned to 5.6.3.
Maybe someone could give me a hint out about this?
Much appreciated! =)

Do not try to use globals in threads.
The PTHREADS_ALLOW_GLOBALS constant and functionality is there for a special use case, it is not intended for use by everyone, in addition globally has been removed in v3.
It is not a good idea to use threads at the frontend of a web application.
You will not be able to load pthreads v3 in Apache any longer
There is a much tidier way of doing what you want to do, that actually works.
Resources are officially unsupported, that doesn't mean you can't use them, it means you shouldn't expect to be able to share them among contexts.
In this case, you don't need to share the resource and so should not try, nor should you try to do anything in the global scope.
Follows is some PHP7 code (which I recommend new projects should use as pthreads v3 is vastly superior to v2):
<?php
class Logger extends Threaded {
public function __construct(string $file) {
$this->file = $file;
}
private function getHandle() {
if (!self::$handle) {
self::$handle = fopen($this->file, "a");
}
return self::$handle;
}
public function log(string $message, ... $args) {
return $this->synchronized(function() use($message, $args) {
return vfprintf($this->getHandle(), $message, $args);
});
}
private $file;
private static $handle;
}
class My extends Thread {
public function __construct(Logger $logger) {
$this->logger = $logger;
}
public function run() {
while (#$i++<100) {
$this->logger->log("Hello %s from %s #%lu\n",
"World", __CLASS__, $this->getThreadId());
/* just simulating work, don't need to wait here */
$this->synchronized(function(){
$this->wait(1000);
});
}
}
private $logger;
}
$logger = new Logger("/tmp/log.txt");
$threads = [];
while (#$i++ < 10) {
$threads[$i] = new My($logger);
$threads[$i]->start();
}
foreach ($threads as $thread)
$thread->join();
?>
In the Logger you will notice that the handle is stored statically, which for pthreads means thread-local.
This means that each thread has a handle to the log, which is how resources are intended to be used in PHP.
You will also notice, the log method is wrapped in a synchronized block, the reason is this: If many threads attempt to write a log at the same time, you will have a log filled with gibberish.
Synchronizing provides mutual exclusion, so that only one thread may write the log at a time, I create ten threads and have them all write to the log one hundred times, no gibberish comes out, it will behave like that everywhere.
A little bit about file locking (flock): On some operating systems, in some circumstances, append is atomic. It's worth mentioning because you should never rely on this, nor should you try to force the writes to be atomic with flock, since flock only places advisory locks on a file: A process executing with the correct permissions is free to ignore flocks and manipulate the file anyway.
Scary stuff, don't rely on append being atomic, and don't rely on flock is the only sensible advice in the context of multi-threading.
On a side note, if you think you have found a bug in pthreads (v3, PHP7), please report it on github.

Related

Background Processing using pthread in php

I am trying to implement muti-threading in php using pthread to send emails. The basic idea here is to send email as a background job so that users dose not have to wait for the task to finish.
I have a service that users PHPMailer to send emails and its working fine. I am using the following code to muti thread
class ThreadWorkerService extends \Thread {
private $_runMethod;
private $_vars;
private $_autoloderPath;
function __construct($vars) {
$this->_vars = $vars;
$reflector = new \ReflectionClass('Composer\Autoload\ClassLoader');
$dir = dirname($reflector->getFileName());
$this->_autoloderPath = $dir . "/../autoload.php";
}
public function setRunMethod($method) {
$this->_runMethod = $method;
}
public function run() {
if (!class_exists('Composer\Autoload\ClassLoader')) {
if (file_exists($this->_autoloderPath )) {
require_once( $this->_autoloderPath );
} else {
echo "autoloder not found";
}
}
$method = $this->_runMethod;
$results = $method($this->_vars);
}
}
$storage = new \Threaded();
$storage['emails'] = $emailArray;
$storage['config'] = $config;
$threadWorker = new ThreadWorkerService($storage);
$threadWorker->setRunMethod(function ($vars) {
$emailArray = $vars['emails'];
$config = $vars['config'];
$emailService = new \MyServices\EmailService($config);
$emailService->sendAllEmails(true, $emailArray);
}
});
$threadWorker->start(PTHREADS_INHERIT_CONSTANTS);
The issue here is that the tread dose not execute if i don't use
$threadWorker->join();
which eliminates the whole purpose of muti-treading in this scenario, what I want to know is, if it is possible to keep the child thread alive even when the parent process is complete. I have even tried detaching the thread with no luck.
I am familiar with on how to do it with messaging services like RabbitMQ, but I want to keep the application independent.
The issue here is that the thread does not execute if I don't use: $threadWorker->join();, which eliminates the whole purpose of multi-threading in this scenario.
It is likely the case that the main thread's stdout, or some other dependency, is being closed or destroyed before the thread gets to execute.
Taken from the PHP manual for Thread:
Warning: Relying on the engine to determine when a Thread should join may cause undesirable behaviour; the programmer should be explicit, where possible.
You don't need to join immediately, but it is advisable to join explicitly.
The following example code illustrates the problem:
<?php
class Test extends Thread {
public function run() {
sleep(1);
echo "Thread\n";
}
}
$test = new Test();
$test->start();
echo "End\n";
?>
CPU speed permitting, the main context will begin to destroy the Thread before the Thread gets to echo.
You will normally get the output:
End
Thread
However, if the code is more complex, for example, code that manipulates dependencies set in the constructor of the Thread, those dependencies might have already been destroyed by the time they are used.
A last ditch attempt to join implicitly is made (in the most recent versions of pthreads), but you cannot rely on destruction order being compatible with your expectations, and so explicit join is preferable.
Noteworthy: The most recent versions of pthreads (PHP7+) prohibit execution anywhere but CLI; If this is code that is being executed at the frontend of a web server, you will need to rethink your architecture.

Right strategy with shared memory and semaphore removing in RAII like php class

When such situation occurs?
If your are using shared memory and semaphores for interpocess locking (with pcntl extension) you should care about semaphore and shared memory segment life circle. For example, you writing backgroud worker application and use master and some child (forked) process for job processing. Using shared memory and semaphores good idea for IPC between them. And RAII like class wrapper around shm_xxx and sem_xxx php functions look`s like good idea too.
Example
class Semaphore
{
private $file;
private $sem;
public function __construct()
{
$this->file = tempnam(sys_get_temp_dir(), 's');
$semKey = ftok($this->file, 'a');
$this->sem = sem_get($semKey, 1); //auto_release = 1 by default
}
public function __destruct()
{
if (is_resource($this->sem) {
sem_remove($this->sem);
}
}
....
}
Not the good choise - after fork we have one instanse in parent and one in child process. And destructor in any of them destroy the semaphore.
Why important
Most of linux systems has limit about semaphore of shared memory count. If you have application which should create and remove many shared memory segfments of semaphores you can`t wait while it be automatically released on process shutdown.
Question
Using с you can use shmctl with IPC_RMID - it marks the segment for removal. The actual removal itself occurs when the last process currently attached to the segment has properly detached it. Of course, if no processes are currently attached to the segment, the removal seems immediate. It works like simple referenc counter. But php do not implements shmctl.
The other strategy - destroy semaphore only in destructor of master process:
class Semaphore
{
...
private $pid;
public function __construct()
{
$this->pid = getmypid();
...
}
public function __destruct()
{
if (is_resource($this->sem) && $this->pid === getmypid()) {
sem_remove($this->sem);
}
}
....
}
So, the questions is
If any way to use IPC_RMID in php?
What strategy should be used in such cases? Destroy in master process only? Other cases?
I checked the current PHP source code and IPC_RMID is not used. However, PHP uses semop() and with it, the SEM_UNDO flag, in case auto_release (see PHP sem_get() manual) is set. But be aware that this works on a per process level. So in case you are using PHP as Apache module, or FCGI or FPM, it might not work as expected. It should work nicely for CLI, though.
For your cleanup, it depends on whether the "master" terminates last or not.
If you do not know, you can implement reference counting yourself.
class Semaphore
{
static private $m_referenceCount = 0;
public function __construct()
{
++self::$m_referenceCount;
// aquire semaphore
}
public function __destruct()
{
if (--self::$m_referenceCount <= 0) {
// clean up
}
}
}
But be aware that the destructor is NOT executed in some circuumstances.

PHP fwrite to null?

My home-made simple logger (just a wrapper around fopen, fwrite, fclose) should be enabled or disabled on the fly. I think the way i'm checking if logging is enable is overkill (every call to a logging method requires to evaluate LOGGING):
config.inc.php
define('LOGGING', true);
logger.php
public function __construct($filename)
{
$this->fp = fopen($filename, 'w+');
}
public function __destruct()
{
fclose($this->fp);
}
public function warn($message)
{
$this->log('WARN', $message);
}
private function log($type, $message)
{
if(!defined(LOGGING) || !LOGGING) return;
fwrite($this->fp, "[$type] $message\n");
}
Question: how to initialize file handler to something like null and remove the check for LOGGING? I mean something like this:
public function __construct($filename)
{
// Create a fake stream if logging is disabled
$this->fp = !defined(LOGGING) || !LOGGING ? null : fopen($filename, 'w+);
}
Future call to $logger->warn('Ops..') should do nothing, without triggering any error or notice.
I mentioned it as a comment above. However, now once again as an answer, because now I think it is one.
You can create a special logger subclass, something like
class NullLogger implements Logger {
public function log ($message) { /* No operation */ }
}
When LOGGING is false, just use this log class and every call with just doing nothing, what will reduce the overhead to the absolute minimum.
This approach has even an own name/pattern: http://en.wikipedia.org/wiki/Null_Object_pattern
You can extend this class and override the log method (leave empty body for example). Then if logging is disabled, create object of child class.
The overhead of file operations is going to VASTLY outweight the overhead of doing if (LOGGING) { ... }. You're optimizing the entirely wrong thing. Checking if LOGGING is true is a simple single value comparison. By comparison, a file operation, even if you're writing to /dev/null, requires invoking file system operations, OS buffers, permissions checks, blah blah blah.
I don't know about you, but I'll do the operation that costs $0.01 each versus one that has the same effect but costs $100 each time.

Does PHP support the RAII pattern? How?

Most resources on PHP never touch memory management because the language itself is pretty good at doing this for you. However, in PHP you often end up dealing with external resources which aren't memory -- database handles, sessions, database transactions, etc. These external resources could be managed most cleanly using some form of RAII object.
I initially thought that PHP used a garbage collection scheme similar to the JVM or the CLR, where the concept of a destructor does not exist. (Remember: Everyone thinks about garbage collection the wrong way -- finalizers are not destructors!) There's the special __destruct method, but I thought that was a "finalizer" similar to a Java or C# finalizer. For this reason, you cannot use RAII on the JVM or the CLR (C#'s using blocks get you about 95% of the way there, but that's a bit different...).
However, Google seems to indicate that PHP supports the RAII pattern, though I cannot find verification of this in the PHP docs. Does the language support this and is putting the cleanup logic in __destruct sufficient for accomplishing RAII tasks?
This is nearly the same question as Is destructor in PHP predictable? and the answer is the same. PHP uses refcounting, and it promises that the destructor will be called immediately as soon as the refcount goes to zero (generally when the object goes out of scope). So if you create an object and take care not to leak it out of scope, RAII is viable.
PHP uses reference counting, so when you're done with a variable it gets cleaned up immediately. (Unless you create cycles.) That frees up resources promptly so you generally don't need to worry about explicit resource management beyond being careful to not create memory cycles.
If you did want to implement any particular strategy, you can do it by making sure that the resource is only used by one variable. Whenever that variable is pointed away from the resource, the resource should be immediately freed up.
The following class ReturnHandler provides automatic invocation of a handler when ReturnHandler's instance goes out of scope. You can have several returns in your function (myfunc) without the need to think of releasing the resource before each of them.
/**
* Automatically calls a handler before returning from a function. Usage:
*
* function myfunc()
* {
* $resource = new Resource();
* $rh = new ReturnHandler( function() use ($resource) { $resource->release(); } );
* // ...
* if(...) {
* return; // look, ma, automatic clean up!
* }
* }
*/
class ReturnHandler
{
private $return_handler;
public function __construct( $return_handler )
{
$this->return_handler = $return_handler;
}
public function __destruct()
{
$handler = $this->return_handler;
$handler();
}
}
Here's a test for it:
class ReturnHandlerTest extends PHPUnit_Framework_TestCase
{
private static function trigger_return_handler(&$var)
{
$rh = new ReturnHandler(function() use (&$var) { $var++; } );
}
public function test()
{
$a = 0;
$this->assertEquals(0, $a);
self::trigger_return_handler($a);
$this->assertEquals(1, $a);
}
}
Slightly offtopic: you can do a using-like pattern with lambdas. Like this:
function WithFile($Name, $Func)
{
$File = fopen($Name, 'r');
$r = $Func($File);
fclose($File);
return $r;
}
And then use it like this
$FileHeader = WithFile('myfile', function($File) {return fread($File, 16);});
Perfectly deterministic. That said, were there a terser syntax for lambdas...

Php Destructors

Please give me some real life examples when you had to use __destruct in your classes.
Ok, since my last answer apparently didn't hit the mark, let me try this again. There are plenty of resources and examples on the internet for this topic. Doing a bit of searching and browsing other framework's code and you'll see some pretty good examples...
Don't forget that just because PHP will close resources on termination for you doesn't mean that it's bad to explictly close them when you no longer need them (or good to not close them)... It depends on the use case (is it being used right up to the end, or is there one call early on and then not needed again for the rest of execution)...
Now, we know that __destruct is called when the object is destroyed. Logically, what happens if the object is destroyed? Well, it means it's no longer available. So if it has resources open, doesn't it make sense to close those resources as it's being destroyed? Sure, in the average web page, the page is going to terminate shortly after, so letting PHP close them usually isn't terrible. However, what happens if for some reason the script is long-running? Then you have a resource leak. So why not just close everything when you no longer need it (or considering the scope of the destructor, when it's no longer available)?
Here's some examples in real world frameworks:
Lithium's lithium\net\Socket class
Kohana's Memcached Driver
Joomla's FTP Implementation
Zend Frameworks's SMTP Mail Transport Class
CodeIgniter's TTemplate Class
A Tidy Filter Helper for Cake
A Google-Groups Thread about using Destructors For the Symfony Session Class
The interesting thing is that Kohana keeps track of the tags, so that it can delete by "namespace" later (instead of just clearing the cache). So it uses the destructor to flush those changes to the hard storage.
The CodeIgniter class also does something interesting in that it adds debugging output to the output stream in the destructor. I'm not saying this is good, but it's an example of yet another use...
I personally use destructors whenever I have long running processes on my master controller. In the constructor, I check for a pid file. If that file exists (And its process is still running), I throw an exception. If not, I create a file with the current processes id. Then, in the destructor I remove that file. So it's more about cleaning up after itself than just freeing resources...
There is another handy use to generate HTML page
class HTMLgenerator {
function __construct() {
echo "<html><body>";
}
function __destruct() {
echo "</body></html>";
}
}
With this class, you can write
$html = new HTMLgenerator();
echo "Hello, world!";
And the result is
<html><body>Hello, world!</body></html>
For example:
<?php
class Session
{
protected $data = array();
public function __construct()
{
// load session data from database or file
}
// get and set functions
public function __destruct()
{
// store session data in database or file
}
};
This is a good why to use destruct. You prevents reading and writing to a session source all the time and do this only at the start and at the end.
I create a php page what will generate a movie information jpg file. This page will have to gather a few information and run inkscape to convert template (an svg file) to a png before converting to jpg. The svg contain relative links to other image which must be a file. So my page download necessary files into a temporary folder, convert the svg file. At the end, the temporary folder must be deleted.
I put the temporary folder deletion into the destructor. Before there can be many reason the page ends unexpected and the only think I can be sure is that destructor will be call when page exit.
Hope this helps.
A destructor is extremely useful if you use a custom database connector/wrapper.
In the constructor, you can pass the connection information. Because you can use a destructor (rather than a finalizer, etc.,) you can rely on that to close the connection for you. It's more of a convenience, but it certainly is useful.
For example, when PHP decides to explicitly "free" the object (i.e., it is no longer used,) it will call the destructor at that time. This is more useful in the scenario I describe as you're not waiting for the garbage collector to run and call the finalizer.
$0.02
Ian
<?php
class Database
{
private $connection;
private $cache = array();
function __construct([$params])
{
//Connection here
}
//Query
public function query(Query $Query)
{
if($this->is_cached($Query->checksum))
{
return $this->get_cache($Query->checksum);
}
//...
}
public function __destruct()
{
unset($this->connection);
$this->WriteCache();
unset($this->cache);
shutdown_log($this,'Destruction Completed');
}
}
?>
theres an example that should make you understand.
If you use handles returned by fopen() for say, logging, you can use __destruct() to make sure fclose() is called on our resources when your class is destroyed.
You are right, __destruct is mostly unnecessary for the short running php scripts. Database connections, file handles and so on close on script exit or sometimes even earlier if variables run out of scope.
One example i can think of is writing logs to the database. Since we didn't want to fire one query per log entry that gets created somewhere in the script we wrote the "write to db" part in the __destruct of the logging class so when the script ends everything gets inserted into the database at one.
Another example: If you allow a user to upload files the destructor is sometimes a nice places to delete the temp file (in case something goes wrong in the script it at least get cleaned up)
But even for filehandles it can be useful. I've worked on a application that did use the old fopen etc. calls wrapped in objects and when using those on large filetrees php would run out of filehandles sooner or later, so cleaning up while the script was running was not only nice but necessary.
I use APC caching for large numbers of "low level" objects, that otherwise would use excessive memory; and I have a cacheCollection object that handles the reading and writing of those "low level" objects to and from APC during execution of the script. When the script terminates, the objects must be cleared down from APC, so I use the cacheCollection __destruct method to perform that function.
I have used __destruct() in a logging class that wrapped a database connection:
<?php
class anyWrap
{
private $obj,$calls,$log,$hooks;
function anyWrap($obj, $logfile = NULL)
{
if(is_null($logfile))
{
$this->log = dirname(__FILE__) . "/../logs/wrapLog.txt";
}
$this->hooks = array();
$this->dbCalls = 0;
$this->obj = $obj;
}
public function __set($attri, $val) {
$this->obj->$attri = $val;
}
public function __get($attri) {
return $this->obj->$attri;
}
public function __hook($method)
{
$this->hooks[] = $method;
}
public function __call($name,$args)
{
$this->calls++;
if(in_array($name,$this->hooks))
{
file_put_contents($this->log,var_export($args,TRUE)."\r\n",FILE_APPEND);
}
return call_user_func_array(array($this->obj,$name),$args);
}
//On destruction log diagnostics
public function __destruct()
{
unset($this->dbReal);
file_put_contents($this->log,$this->calls."\r\n",FILE_APPEND);
}
}
The script hooks into the database calls and logs the prepare statements, then when the script has run to an end (I don't always know when) it will finally log the number of calls to the database to the file. This way I can see how many times certain functions has been called on the database and plan my optimization accordingly.
If you are creating a view using a PHP script in a MySQL database, you must drop that view at the end of the script. Because if not, the next time that script is executed view will not be created, as there is already a view of similar name in the database. For this purpose you can use destructor.
Here's a rather unusual use case for destructors that I think libraries such as pest are using to combine method chaining with functions or in other words, to achieve fluent interface for functions, Which goes like this:
<?php
class TestCase {
private $message;
private $closure;
private $endingMessage;
public function __construct($message, $closure) {
$this->message = $message;
$this->closure = $closure;
}
public function addEndingMessage($message) {
$this->endingMessage = $message;
return $this;
}
private function getClosure() {
return $this->closure;
}
public function __destruct() {
echo $this->message . ' - ';
$this->getClosure()();
echo $this->endingMessage ? ' - ' . $this->endingMessage : '';
echo "\r\n";
}
}
function it($message, $closure) {
return new TestCase($message, $closure);
}
it('ok nice', function() {
echo 'what to do next?';
});//outputs: ok nice - what to do next?
it('ok fine', function() {
echo 'what should I do?';
})->addEndingMessage('THE END');//outputs: ok fine - what should I do? - THE END

Categories