Background Processing using pthread in php - php

I am trying to implement muti-threading in php using pthread to send emails. The basic idea here is to send email as a background job so that users dose not have to wait for the task to finish.
I have a service that users PHPMailer to send emails and its working fine. I am using the following code to muti thread
class ThreadWorkerService extends \Thread {
private $_runMethod;
private $_vars;
private $_autoloderPath;
function __construct($vars) {
$this->_vars = $vars;
$reflector = new \ReflectionClass('Composer\Autoload\ClassLoader');
$dir = dirname($reflector->getFileName());
$this->_autoloderPath = $dir . "/../autoload.php";
}
public function setRunMethod($method) {
$this->_runMethod = $method;
}
public function run() {
if (!class_exists('Composer\Autoload\ClassLoader')) {
if (file_exists($this->_autoloderPath )) {
require_once( $this->_autoloderPath );
} else {
echo "autoloder not found";
}
}
$method = $this->_runMethod;
$results = $method($this->_vars);
}
}
$storage = new \Threaded();
$storage['emails'] = $emailArray;
$storage['config'] = $config;
$threadWorker = new ThreadWorkerService($storage);
$threadWorker->setRunMethod(function ($vars) {
$emailArray = $vars['emails'];
$config = $vars['config'];
$emailService = new \MyServices\EmailService($config);
$emailService->sendAllEmails(true, $emailArray);
}
});
$threadWorker->start(PTHREADS_INHERIT_CONSTANTS);
The issue here is that the tread dose not execute if i don't use
$threadWorker->join();
which eliminates the whole purpose of muti-treading in this scenario, what I want to know is, if it is possible to keep the child thread alive even when the parent process is complete. I have even tried detaching the thread with no luck.
I am familiar with on how to do it with messaging services like RabbitMQ, but I want to keep the application independent.

The issue here is that the thread does not execute if I don't use: $threadWorker->join();, which eliminates the whole purpose of multi-threading in this scenario.
It is likely the case that the main thread's stdout, or some other dependency, is being closed or destroyed before the thread gets to execute.
Taken from the PHP manual for Thread:
Warning: Relying on the engine to determine when a Thread should join may cause undesirable behaviour; the programmer should be explicit, where possible.
You don't need to join immediately, but it is advisable to join explicitly.
The following example code illustrates the problem:
<?php
class Test extends Thread {
public function run() {
sleep(1);
echo "Thread\n";
}
}
$test = new Test();
$test->start();
echo "End\n";
?>
CPU speed permitting, the main context will begin to destroy the Thread before the Thread gets to echo.
You will normally get the output:
End
Thread
However, if the code is more complex, for example, code that manipulates dependencies set in the constructor of the Thread, those dependencies might have already been destroyed by the time they are used.
A last ditch attempt to join implicitly is made (in the most recent versions of pthreads), but you cannot rely on destruction order being compatible with your expectations, and so explicit join is preferable.
Noteworthy: The most recent versions of pthreads (PHP7+) prohibit execution anywhere but CLI; If this is code that is being executed at the frontend of a web server, you will need to rethink your architecture.

Related

Singletton Pattern not respected in PHP

I need a few variables to be shared accross several sessions. To do this, I thought I could use a class with a singleton pattern storing the variables, so each user (session) would just have to include the class and access its data.
I proceed with the following way :
<?php
require_once('team.php');
class Game {
private static $_instance = null;
private $team1;
private $team2;
public function getTeam($num) {
if ($num == 1) return $this->team1;
if ($num == 2) return $this->team2;
}
private function __construct() {
$this->team1 = new Team();
$this->team2 = new Team();
$this->team1->init_team(1);
$this->team2->init_team(2);
}
public static function getInstance() {
if(is_null(self::$_instance)) {
self::$_instance = new self();
}
return self::$_instance;
}
}
?>
Then I try to use the class like this :
$game = Game::getInstance();
$player = new Player();
...
$game->getTeam($team)->addPlayer($player);
Here is the problem : two players from a same team can't see each other when displaying all the members of their teams. Moreover, when I use the following instruction on 2 different sessions :
spl_object_hash(Game::getInstance());
The output is different, whereas I would expect it to be the same, assuming that it works like Java classes's hashcodes.
Any help regarding this issue would be appreciated. Thanks for reading.
When running a PHP application, every request starts a new instance of the application.
Your code IS running how you expect it to, but PHP doesn't run how you expect it to. It is not comparable to the way Java runs.
If you were to start a HTTP server to serve your PHP application and you had 10 concurrent visitors, you would see 10 new processes running on your server - each one containing a different instance of the application for each visitor.
However, if you were to create your HTTP server in Java and have 10 concurrent visitors, you would only see 1 process.
If you are looking to have persistent data in PHP which can be displayed to different users you should look into storage systems such as MySQL.
Every page run is distinct PHP application. This means that every user (on page call) is using his own compiled version of app, which will be deleted afterwards.
Singleton it not for that. It's just to ensure there is only one instance of object. But this object "lives" solely within separately compiled application.
You should use database for this kind of things (or any other kind of storage)

php pthreads and using a class to store data that is shared between threads

I'm fairly new to PHP, and now very new to pthreads.
I'm using the latest PHP7 RC6 build, with pthreads built from git/src to get the latest (and tried the 'official' v3.0.8 one), on Ubuntu 3.13.0-66-generic
I'm trying to write a threaded solution to read in data from a socket and process it. I'm using threading to try to maximize my performance, mainly due to the fact I'm doing operations like http requests (to AWS DynamoDB and other services) and such that are waiting for responses from external systems, and therefore I can benefit from threading.
The real code I have is more complicated than this is. This is a simple example to show my problem.
What I am trying to do is to 'cache' certain information in an 'array' that I get from a database (AWS DynamoDB) so that I can get better performance. I need each thread to be able to use/access and modify this 'global' cache, and with multiple 'records' in the cache.
I had great success with testing and simply storing a string in this way, but now I'm doing it for real, I need to store more complicated data, and I decided to use a little class (cacheRecord) for each record, instead of a simple string of data. But the problem is that when I try to assign a value back to a class member, it seems to not want to 'save', back to the array.
I managed to get it to work by copying the whole 'class' to a tmp variable, modifying that, and then saving back the whole class to the array, but that seems like an overhead of code, and also I would need to wrap it in a ->synchronized to keep integrity between threads.
Is this the only way to do it correctly, with copying it to a tmp and copying it back and using 'synchronized', or am I doing something else wrong/stupid?
Experimenting with it, I made the cacheRecord class 'extends Threaded'. This made the single assign of the member work fine, but this then made it immutable, and I couldn't unset/delete that record in the cache later.
Code to show what I mean:
<?php
class cacheRecord {
public $currentPos;
public $currentRoom;
public $someOtherData;
}
class cache extends Threaded {
public function run() {}
}
class socketThread extends Thread {
public function __construct($myCache) {
$this->cacheData = $myCache;
}
public function run() {
// This will be in a loop, waiting for sockets, and then responding to them, indefinitely.
// At some point, add a record to the cache
$c = new cacheRecord;
$c->currentPos = '1,2,4';
$c->currentRoom = '2';
$this->cacheData['record1'] = $c;
var_dump($this);
// Later on, update the cache record, but this doesnt work
$this->cacheData['record1']->currentRoom = '3';
var_dump($this);
// However this does work, but is this the correct way? Seems like more code to execute, than a simple assign, and obviously, I would need to use synchronized to keep integrity, which would further slow it down.
$tmp = $this->cacheData['record1'];
$tmp->currentRoom = '3';
$this->cacheData['record1'] = $tmp;
var_dump($this);
// Later on some more, remove the record
unset($this->cacheData['record1']);
var_dump($this);
// Also will be using ->synchronized to enforce integrity of certain other operations
// Just an example of how I might use it
/*
$this->cacheData->synchronized(function() {
if ($this->cacheData['record1']->currentRoom == '3') {
$this->cacheData['record1']->Pos = '0,0,0'; // Obviously this wont work as above.
$this->cacheData['record1']->currentRoom = '4';
}
});
*/
}
}
// Main
$myCache = new cache;
for ($th=0;$th<1;$th++) { // Just 1 thread for testing
$socketThreads[$th] = new socketThread($myCache);
$socketThreads[$th]->start();
}
extends \Threaded is the way to go.
However, "anything" in the cache should be extended from this, not only the cache itsef.
It is explained somewhere in the manuals (sorry dont remember exactly where) than only volatile (aka threaded) object will not me immutable.
So if your class cacheRecord is not extended from threaded, it will be immutable, even into another threaded structure.
threaded makes inner attributes array automatically volatile (so thread-usable), but not object if they are not extended from threaded.
Try extending cacheRecord from threaded and tell me if it works.
Phil+

pthreads access to resource globally in PHP

I'm having problems with accessing resources from global context using php pthreads for Windows.
The logic is very simple: there's a Server as a main class, only one, which will:
create logfile
create couple of threads
each thread will need to write something to this logfile
Problem is, the resource of logfile handle is somehow totally messed up from inside the thread. When I create the resource, it's fine, I can write in it. When I try to call the log from inside the running thread, the resource of logfile handler appears to be integer 0, not even a resource.
Here's my code:
$main_server = new CMainServer();
$main_server->init();
$main_server->go();
$main_server->log("All done");
Inside the CMainServer class:
class CMainServer
{
private $logfile = null;
public function init()
{
$this->logfile = fopen('wstext.log', 'w');
}
public function log($str)
{
if ($this->logfile === null)
{
echo "[".date("H:i:s", time())."]: logfile is null<BR />";
return false;
}
if (!is_resource($this->logfile))
{
echo "[".date("H:i:s", time())."]: logfile is NOT a resource, can't write {$str}<BR />";
return false;
}
echo "[".date("H:i:s", time())."]: logfile is resource, not null, writing {$str}<BR />";
flush();
fwrite($this->logfile, "[".date("H:i:s", time())."]: {$str}\r\n");
return true;
}
public function go()
{
$this->log('Before creating a thread');
$first_thread = new CThread();
$first_thread->start(PTHREADS_INHERIT_ALL | PTHREADS_ALLOW_GLOBALS);
$second_thread = new CThread();
$second_thread->start(PTHREADS_INHERIT_ALL | PTHREADS_ALLOW_GLOBALS);
$first_thread->join();
$second_thread->join();
}
public function __destruct()
{
if ($this->logfile)
fclose($this->logfile);
}
}
And, finally, the CThread class:
class CThread extends Thread
{
public function run()
{
global $main_server;
$thread_id = $this->getThreadId();
Thread::globally(function()
{
for ($i = 0; $i < 2; $i++)
{
$main_server->log("({$i}) writing random number ".rand(0, 100)." to log from running thread id={$thread_id}");
sleep(1);
}
});
}
}
Result is sad:
[13:38:10]: logfile is NOT a resource, can't write (0) writing random number 21 to log from running thread id=9080
[13:38:11]: logfile is NOT a resource, can't write (1) writing random number 91 to log from running thread id=9080
[13:38:10]: logfile is NOT a resource, can't write (0) writing random number 16 to log from running thread id=17316
[13:38:11]: logfile is NOT a resource, can't write (1) writing random number 50 to log from running thread id=17316
[13:38:10]: logfile is resource, not null, writing Before creating a thread
[13:38:12]: logfile is resource, not null, writing All done
So while I'm outside the thread, all is fine. However, from within a thread the $logfile is not a resource at all.
I tried different options: tried calling from CThread::run() a global function:
function LogFromThread($i, $thread_id)
{
global $main_server;
$main_server->log("({$i}) writing random number ".rand(0, 100)." to log from running thread id={$thread_id}");
}
The result is the same.
Tried without Thread::globally() at all, but all for no good.
I'm running Apache/2.4.10 (Win32) OpenSSL/1.0.1i PHP/5.6.3, tried pthreads version 2.0.8, 2.0.9. Also tried with PHP 7RC2 and RC3, but there seems to be a problem to start a new thread at all, apache logs an error, so I returned to 5.6.3.
Maybe someone could give me a hint out about this?
Much appreciated! =)
Do not try to use globals in threads.
The PTHREADS_ALLOW_GLOBALS constant and functionality is there for a special use case, it is not intended for use by everyone, in addition globally has been removed in v3.
It is not a good idea to use threads at the frontend of a web application.
You will not be able to load pthreads v3 in Apache any longer
There is a much tidier way of doing what you want to do, that actually works.
Resources are officially unsupported, that doesn't mean you can't use them, it means you shouldn't expect to be able to share them among contexts.
In this case, you don't need to share the resource and so should not try, nor should you try to do anything in the global scope.
Follows is some PHP7 code (which I recommend new projects should use as pthreads v3 is vastly superior to v2):
<?php
class Logger extends Threaded {
public function __construct(string $file) {
$this->file = $file;
}
private function getHandle() {
if (!self::$handle) {
self::$handle = fopen($this->file, "a");
}
return self::$handle;
}
public function log(string $message, ... $args) {
return $this->synchronized(function() use($message, $args) {
return vfprintf($this->getHandle(), $message, $args);
});
}
private $file;
private static $handle;
}
class My extends Thread {
public function __construct(Logger $logger) {
$this->logger = $logger;
}
public function run() {
while (#$i++<100) {
$this->logger->log("Hello %s from %s #%lu\n",
"World", __CLASS__, $this->getThreadId());
/* just simulating work, don't need to wait here */
$this->synchronized(function(){
$this->wait(1000);
});
}
}
private $logger;
}
$logger = new Logger("/tmp/log.txt");
$threads = [];
while (#$i++ < 10) {
$threads[$i] = new My($logger);
$threads[$i]->start();
}
foreach ($threads as $thread)
$thread->join();
?>
In the Logger you will notice that the handle is stored statically, which for pthreads means thread-local.
This means that each thread has a handle to the log, which is how resources are intended to be used in PHP.
You will also notice, the log method is wrapped in a synchronized block, the reason is this: If many threads attempt to write a log at the same time, you will have a log filled with gibberish.
Synchronizing provides mutual exclusion, so that only one thread may write the log at a time, I create ten threads and have them all write to the log one hundred times, no gibberish comes out, it will behave like that everywhere.
A little bit about file locking (flock): On some operating systems, in some circumstances, append is atomic. It's worth mentioning because you should never rely on this, nor should you try to force the writes to be atomic with flock, since flock only places advisory locks on a file: A process executing with the correct permissions is free to ignore flocks and manipulate the file anyway.
Scary stuff, don't rely on append being atomic, and don't rely on flock is the only sensible advice in the context of multi-threading.
On a side note, if you think you have found a bug in pthreads (v3, PHP7), please report it on github.

Right strategy with shared memory and semaphore removing in RAII like php class

When such situation occurs?
If your are using shared memory and semaphores for interpocess locking (with pcntl extension) you should care about semaphore and shared memory segment life circle. For example, you writing backgroud worker application and use master and some child (forked) process for job processing. Using shared memory and semaphores good idea for IPC between them. And RAII like class wrapper around shm_xxx and sem_xxx php functions look`s like good idea too.
Example
class Semaphore
{
private $file;
private $sem;
public function __construct()
{
$this->file = tempnam(sys_get_temp_dir(), 's');
$semKey = ftok($this->file, 'a');
$this->sem = sem_get($semKey, 1); //auto_release = 1 by default
}
public function __destruct()
{
if (is_resource($this->sem) {
sem_remove($this->sem);
}
}
....
}
Not the good choise - after fork we have one instanse in parent and one in child process. And destructor in any of them destroy the semaphore.
Why important
Most of linux systems has limit about semaphore of shared memory count. If you have application which should create and remove many shared memory segfments of semaphores you can`t wait while it be automatically released on process shutdown.
Question
Using с you can use shmctl with IPC_RMID - it marks the segment for removal. The actual removal itself occurs when the last process currently attached to the segment has properly detached it. Of course, if no processes are currently attached to the segment, the removal seems immediate. It works like simple referenc counter. But php do not implements shmctl.
The other strategy - destroy semaphore only in destructor of master process:
class Semaphore
{
...
private $pid;
public function __construct()
{
$this->pid = getmypid();
...
}
public function __destruct()
{
if (is_resource($this->sem) && $this->pid === getmypid()) {
sem_remove($this->sem);
}
}
....
}
So, the questions is
If any way to use IPC_RMID in php?
What strategy should be used in such cases? Destroy in master process only? Other cases?
I checked the current PHP source code and IPC_RMID is not used. However, PHP uses semop() and with it, the SEM_UNDO flag, in case auto_release (see PHP sem_get() manual) is set. But be aware that this works on a per process level. So in case you are using PHP as Apache module, or FCGI or FPM, it might not work as expected. It should work nicely for CLI, though.
For your cleanup, it depends on whether the "master" terminates last or not.
If you do not know, you can implement reference counting yourself.
class Semaphore
{
static private $m_referenceCount = 0;
public function __construct()
{
++self::$m_referenceCount;
// aquire semaphore
}
public function __destruct()
{
if (--self::$m_referenceCount <= 0) {
// clean up
}
}
}
But be aware that the destructor is NOT executed in some circuumstances.

Php Destructors

Please give me some real life examples when you had to use __destruct in your classes.
Ok, since my last answer apparently didn't hit the mark, let me try this again. There are plenty of resources and examples on the internet for this topic. Doing a bit of searching and browsing other framework's code and you'll see some pretty good examples...
Don't forget that just because PHP will close resources on termination for you doesn't mean that it's bad to explictly close them when you no longer need them (or good to not close them)... It depends on the use case (is it being used right up to the end, or is there one call early on and then not needed again for the rest of execution)...
Now, we know that __destruct is called when the object is destroyed. Logically, what happens if the object is destroyed? Well, it means it's no longer available. So if it has resources open, doesn't it make sense to close those resources as it's being destroyed? Sure, in the average web page, the page is going to terminate shortly after, so letting PHP close them usually isn't terrible. However, what happens if for some reason the script is long-running? Then you have a resource leak. So why not just close everything when you no longer need it (or considering the scope of the destructor, when it's no longer available)?
Here's some examples in real world frameworks:
Lithium's lithium\net\Socket class
Kohana's Memcached Driver
Joomla's FTP Implementation
Zend Frameworks's SMTP Mail Transport Class
CodeIgniter's TTemplate Class
A Tidy Filter Helper for Cake
A Google-Groups Thread about using Destructors For the Symfony Session Class
The interesting thing is that Kohana keeps track of the tags, so that it can delete by "namespace" later (instead of just clearing the cache). So it uses the destructor to flush those changes to the hard storage.
The CodeIgniter class also does something interesting in that it adds debugging output to the output stream in the destructor. I'm not saying this is good, but it's an example of yet another use...
I personally use destructors whenever I have long running processes on my master controller. In the constructor, I check for a pid file. If that file exists (And its process is still running), I throw an exception. If not, I create a file with the current processes id. Then, in the destructor I remove that file. So it's more about cleaning up after itself than just freeing resources...
There is another handy use to generate HTML page
class HTMLgenerator {
function __construct() {
echo "<html><body>";
}
function __destruct() {
echo "</body></html>";
}
}
With this class, you can write
$html = new HTMLgenerator();
echo "Hello, world!";
And the result is
<html><body>Hello, world!</body></html>
For example:
<?php
class Session
{
protected $data = array();
public function __construct()
{
// load session data from database or file
}
// get and set functions
public function __destruct()
{
// store session data in database or file
}
};
This is a good why to use destruct. You prevents reading and writing to a session source all the time and do this only at the start and at the end.
I create a php page what will generate a movie information jpg file. This page will have to gather a few information and run inkscape to convert template (an svg file) to a png before converting to jpg. The svg contain relative links to other image which must be a file. So my page download necessary files into a temporary folder, convert the svg file. At the end, the temporary folder must be deleted.
I put the temporary folder deletion into the destructor. Before there can be many reason the page ends unexpected and the only think I can be sure is that destructor will be call when page exit.
Hope this helps.
A destructor is extremely useful if you use a custom database connector/wrapper.
In the constructor, you can pass the connection information. Because you can use a destructor (rather than a finalizer, etc.,) you can rely on that to close the connection for you. It's more of a convenience, but it certainly is useful.
For example, when PHP decides to explicitly "free" the object (i.e., it is no longer used,) it will call the destructor at that time. This is more useful in the scenario I describe as you're not waiting for the garbage collector to run and call the finalizer.
$0.02
Ian
<?php
class Database
{
private $connection;
private $cache = array();
function __construct([$params])
{
//Connection here
}
//Query
public function query(Query $Query)
{
if($this->is_cached($Query->checksum))
{
return $this->get_cache($Query->checksum);
}
//...
}
public function __destruct()
{
unset($this->connection);
$this->WriteCache();
unset($this->cache);
shutdown_log($this,'Destruction Completed');
}
}
?>
theres an example that should make you understand.
If you use handles returned by fopen() for say, logging, you can use __destruct() to make sure fclose() is called on our resources when your class is destroyed.
You are right, __destruct is mostly unnecessary for the short running php scripts. Database connections, file handles and so on close on script exit or sometimes even earlier if variables run out of scope.
One example i can think of is writing logs to the database. Since we didn't want to fire one query per log entry that gets created somewhere in the script we wrote the "write to db" part in the __destruct of the logging class so when the script ends everything gets inserted into the database at one.
Another example: If you allow a user to upload files the destructor is sometimes a nice places to delete the temp file (in case something goes wrong in the script it at least get cleaned up)
But even for filehandles it can be useful. I've worked on a application that did use the old fopen etc. calls wrapped in objects and when using those on large filetrees php would run out of filehandles sooner or later, so cleaning up while the script was running was not only nice but necessary.
I use APC caching for large numbers of "low level" objects, that otherwise would use excessive memory; and I have a cacheCollection object that handles the reading and writing of those "low level" objects to and from APC during execution of the script. When the script terminates, the objects must be cleared down from APC, so I use the cacheCollection __destruct method to perform that function.
I have used __destruct() in a logging class that wrapped a database connection:
<?php
class anyWrap
{
private $obj,$calls,$log,$hooks;
function anyWrap($obj, $logfile = NULL)
{
if(is_null($logfile))
{
$this->log = dirname(__FILE__) . "/../logs/wrapLog.txt";
}
$this->hooks = array();
$this->dbCalls = 0;
$this->obj = $obj;
}
public function __set($attri, $val) {
$this->obj->$attri = $val;
}
public function __get($attri) {
return $this->obj->$attri;
}
public function __hook($method)
{
$this->hooks[] = $method;
}
public function __call($name,$args)
{
$this->calls++;
if(in_array($name,$this->hooks))
{
file_put_contents($this->log,var_export($args,TRUE)."\r\n",FILE_APPEND);
}
return call_user_func_array(array($this->obj,$name),$args);
}
//On destruction log diagnostics
public function __destruct()
{
unset($this->dbReal);
file_put_contents($this->log,$this->calls."\r\n",FILE_APPEND);
}
}
The script hooks into the database calls and logs the prepare statements, then when the script has run to an end (I don't always know when) it will finally log the number of calls to the database to the file. This way I can see how many times certain functions has been called on the database and plan my optimization accordingly.
If you are creating a view using a PHP script in a MySQL database, you must drop that view at the end of the script. Because if not, the next time that script is executed view will not be created, as there is already a view of similar name in the database. For this purpose you can use destructor.
Here's a rather unusual use case for destructors that I think libraries such as pest are using to combine method chaining with functions or in other words, to achieve fluent interface for functions, Which goes like this:
<?php
class TestCase {
private $message;
private $closure;
private $endingMessage;
public function __construct($message, $closure) {
$this->message = $message;
$this->closure = $closure;
}
public function addEndingMessage($message) {
$this->endingMessage = $message;
return $this;
}
private function getClosure() {
return $this->closure;
}
public function __destruct() {
echo $this->message . ' - ';
$this->getClosure()();
echo $this->endingMessage ? ' - ' . $this->endingMessage : '';
echo "\r\n";
}
}
function it($message, $closure) {
return new TestCase($message, $closure);
}
it('ok nice', function() {
echo 'what to do next?';
});//outputs: ok nice - what to do next?
it('ok fine', function() {
echo 'what should I do?';
})->addEndingMessage('THE END');//outputs: ok fine - what should I do? - THE END

Categories