This question already has an answer here:
How does pcntl_fork work in PHP?
(1 answer)
Closed 3 years ago.
I'm trying to run some code asynchronously within my class (see: https://github.com/duncan3dc/fork-helper). I need to call a series of methods that will modify the values of my properties. I can't seem to do so. I'm trying to use the last example of the call() method on this page: https://duncan3dc.github.io/fork-helper/usage/getting-started/
<?php
class foobar
{
private $x = 0;
public function doubler(&$number_to_double)
{
$number_to_double = $number_to_double * 2;
}
public function index()
{
$fork = new \duncan3dc\Forker\Fork;
$this->x = 5;
// outputs 5, as expected
var_dump($this->x);
$fork->call([$this, 'doubler'], $this->x);
$fork->wait();
// does not output 10, which is what I want
var_dump($this->x);
}
}
$my_foobar = new foobar();
$my_foobar->index();
I don't have to pass by reference, like I did in my doubler. Instead, I'm also open to just modifying the member from within the doubler method.
Why isn't my private member x doubling at the second var_dump()?
The linked library seems to be internally using pcntl_fork which performs a true fork of the running PHP process. The library mentions "threading" but this is not correct. Forking is a pretty low level operating system concept whereas a process creates a copy of its memory space and instructions in a new process. That new process becomes a child process of the forking process.
This means that everything, like included code and instantiated objects are copied and therefore a child process cannot directly modify the objects of the parent process. The only ways a child process can communicate with the parent is shared memory. In addition the parent can "wait" for the child process to terminate (failing to do this may result in zombie child processes). The linked library does not seem to implement true shared memory but (if you must) you could probably use the PHP shared memory library.
This however is impractical for simple tasks like the one you're sharing. In your case you need to use a true threading library like e.g. pthreads. Threads, unlike processes, are part of the parent process and share the same memory and data structures and have a significantly lower overhead when context switching.
Note: All the above concepts are pretty low level ones so maybe PHP is not the best choice of language to implement these.
Related
I have a snippet that resembles the following:
while (true) {
$myObject = new Class();
$myOtherObject = $myObject->getSomeOtherObj();
...
$myArray = [1, 2, 3];
...
sleep(1); //Sleep at the end, to save CPU.
}
This snippet should run as a daemon service, but I'm having a lot of trouble making this work.
The issue: each iteration increases the process memory usage. As if at each new iteration a new $myObject is being instantiated, but the previous one remains allocated in memory, and such.
I have tried:
to unset all variables at the end of the loop (right before the sleep()).
Setting all variables to null.
encapsulating them in a separate function (while (true) { doThis(); })
manually calling gc_collect_cycles()
None of those worked to decrease memory usage.
I have no idea how to force all memory to be released.
After much research on the topic, I am finally convinced that there are no ways to manually force the memory to be released or to force object destruction.
However, something that has helped me lower the memory usage (absolutely preventing infinite memory stacking was not possible) was to realize that there are no loop scopes in PHP and that the Garbage Collection happens when switching scopes.
In C# or Java, a variable created within a while (...) {} is only accessible from within the loop. This is not the norm for PHP. A $myObject created from within a while instruction is accessible throughout your entire application.
This means the provided snippet would be better presented as:
while (true) {
myFunc();
}
function myFunc()
{
$myObject = new Class();
$myOtherObject = $myObject->getSomeOtherObj();
...
$myArray = [1, 2, 3];
...
sleep(1); //Sleep at the end, to save CPU.
}
Encapsulating the logic in a function forces the scope to change, which means the Garbage Collector will be called at each iteration. This has not solved my problem, but it has lowered my RAM usage somewhat.
What I have learned from this experience is that PHP is probably not suited to this specific project requirement. I'd need more control over memory, and PHP doesn't provide any kind of control over created/destroyed objects. Some native functions do not release memory properly (specially those that do I/O, database access and memcached).
I'm compiling my previous comments in an answer here. This doesn't explain exactly how you can free allocated memory, but it will guide you through a way to discover what in your application is causing that. With that, you can work on optimizing your code.
Finding memory usage bottlenecks is usually a challenging task. You can start by looking at your I/O-related calls, like database queries, file access, or even networking. Beyond increasing the execution time, sometimes these operations can allocate some amount of memory.
If you're already removing from memory the resources returned by I/O operations and no considerable decrease in allocated memory is noticed, the next step might be profiling your application using a tool like Blackfire (https://blackfire.io/).
Blackfire will give you a detailed view of each function call and its statistics on memory, CPU, and execution time. With that data, it's possible to check which operations are allocating excessive memory. You can find this info when you land your mouse pointer over the memory bar inside the call details, like this:
It is very likely (provided with the information given) that there are still references to the created object which prevent the garbage collector from removing the object from memory. Since it is basically counting references, therefor making sure that no reference is being stored, by either making copies of values or unsetting them carefully this can be avoided.
Normally it is easier when using while(true) constructs to not create objects precisely for this reason and make it as self contained as possible just to make sure that no memory leaks can actually happen.
I know this answer is not very helpfull in a direct manner (and I do not have enough rep to comment on the question) but it might get you on the right track.
The problem is that you are in an infinite loop, with no end at the request. The garbage collector of PHP is designed to be dealt with at the end of the request, and it not accessible to the user otherwise. PHP is designed to be called and discarded, not to be kept alive indefinately. Hence it is not fixable. So, what I would suggest is to create a chron job that restarts the php loop at regular intervals thus ending the request and freeing up the memory. See this document for details.
My best guess (due to lack of knowledge of the internals of the Classes involved) is that either the classes assign other Objects as their properties (or perhaps they have self-references) or that the array used (since the code sample resembles the real case) has a reference to itself, which would explain the memory leak if the size of the array is significant.
If it is of any help, please check out the reference counting fundamentals from php.net:
http://php.net/manual/en/features.gc.refcounting-basics.php
As both #m1lt0n and #MartPluijmaekers has mention above, this might be an issue related to object references.
We don't know what is inside your Class() class and getSomeOtherObj() method, so I can't say anything for sure, however below snippet might be able to help you figure out if that is the case or not.
/* Here is your Class */
class Class {
public function __construct () {
$this->child = new AnotherClass( $this );
}
/* This is what you have to have ... */
protected function __destruct () {
unset( $this->child );
}
}
/* Here is the other class */
class AnotherClass {
public function __construct ( $parent ) {
$this->parent = $parent;
}
}
/* Infinite Loop */
while ( true ) {
$myObject = new Class();
$myOtherObject = $myObject->getSomeOtherObj();
$myArray = [1, 2, 3];
/* manually destroying the object */
$myObject->__destruct();
unset( $myObject );
/* rest of free-ing goes here ... */
sleep(1); //Sleep at the end, to save CPU.
}
The snippet should be pretty much self-explanatory.
I'm currently making a behavior that uses a plugin model internally.
Every time I need to look for a reference I use:
$this->Viewed = ClassRegistry::init('Viewed.Viewed');
So it will init the model that holds the data.
It's a good idea to use
$this->Viewed = ClassRegistry::removeObject('Viewed.Viewed')
to save memory in the excecution time of the script? or will add more time?
This behavior is used a lot in the code... so every gain is necesary.
Here's the code for CakePHP's ClassRegistry::removeObject($key) function:
public static function removeObject($key) {
$_this = ClassRegistry::getInstance();
$key = Inflector::underscore($key);
if (isset($_this->_objects[$key])) {
unset($_this->_objects[$key]);
}
}
Really all it's doing is unsetting the relevant key in an array the presumably holds all CakePHP's objects. The unset() function in PHP will not free up memory immediately, though PHP may reclaim that memory eventually. Since there's not much going on in ClassRegistry::removeObject() I wouldn't be too concerned with it taking up much more processing.
But keep in mind also that once a PHP script is finished being executed, memory will also be reclaimed. So there's not much point in calling ClassRegistry::removeObject() if you can only call it near the end of your script. PHP is not really designed for manual memory management like that. If memory really starts becoming an issue, you could also consider writing something external in another language that your application can call using exec().
I'm currently working with an existing application that defines a couple constants on the login of a user. For example, if Alice logs in SOME_CONSTANT is defined as 1, while if Bob logs in SOME_CONSTANT is defined as 2. Now I'm trying to write a script that will do a couple of things as if it were Alice and a couple things as if it were Bob. By "as if it were" I mean that SOME_CONSTANT is defined one way for one iteration and another way for the next iteration. Unfortunately, constants are not the best at switching values and refactoring the application to change these from being constants is not an option at this time.
One method I had considered was to use pcntl_fork(). I would fork before the time the constants were defined and run a separate process for each constant. However, I would like this script to be able to run on Windows as well as Linux. At the moment the pcntl extension is not directly supported for Windows. And I'm going to try to avoid getting everything working through Cygwin if I can help it.
Another method I had considered was having the script call children scripts using exec("php childscript.php constant_value"). Will this method allow one child script to define a constant one way and another child script define it another way? I think it should, but I haven't tested it yet. Also, is there any other major problems anyone can see with this method?
Is there another method I haven't considered that would be a better choice? Thank you for your time.
As you've already noticed, using a const variable is not a viable method to handle your task. Additionally, even if you didn't need to write a script to do something with multiple users, a single instance of this wouldn't work - you would need to "set the constant" when the user logs in, which you can't do.
If you're looking for a pseudo-readonly implementation, and you are using OOP-style, you can add a private variable and override the __get magic-method. So, whenenver the outside requests SOME_CONSTANT, your class will return the value of _someFakeConstant. Then, in your login() or switchUser() method inside the class you can safely change the value.
Example:
class User {
private $_someFakeConstant = -1;
public function __get($name) {
if ($name == 'SOME_CONSTANT') {
return $this->_someFakeConstant;
}
// handle undefined variables; trigger_error() will work (see example on php.net)
}
public function login() {
// logic to "identify" the user
$this->_someFakeConstant = 1;
}
}
$user = new User();
$user->login();
echo $user->SOME_CONSTANT;
The method of using exec() to call the PHP child script appears to work fine.
There is a class like this in codeigniter framework ( I edited it to be more clear, full function is here http://pastebin.com/K33amh7r):
function &load_class($class, $directory = 'libraries', $prefix = 'CI_')
{
static $_classes = array();
// Does the class exist? If so, we're done...
if (isset($_classes[$class]))
{
return $_classes[$class];
}
is_loaded($class);
$_classes[$class] = new $name();
return $_classes[$class];
}
So, first time when class is loaded ( passed to this function), it will be saved to this static variable. Next time when the same class is loaded, this function checks if class exists already ( if it's already assigned to static, cached, I'm not sure how in memory is this stored) and if it exists, it's loaded ( NOT *instantiated* again )
As far as I can see, the only purpose is to save time or memory and not instantiate the same class twice.
My question here is: Does really instantiating a class can take up memory or consume loading time so it has to be cached like this?
CodeIgniter is is geared for rapid prototyping, and is really not a good example of enterprise patterns in almost any cases. This behavior is related to their design choice of the relationship the "controller" has to almost all other objects; namely that there is exactly one of almost anything (only one instance of controller, only one instance of each library, etc). This design choice is more for rapid development (justified by the developer "not having to keep track of as much" or some such...).
Basically, there is memory saved by not instantiating an object (as much memory as it takes to store the object's instance variables) and if the object's constructor tries to do a fair bit of work, you can save time, too.
However, the appropriateness of the single-instance imperative is clearly not universal; sometimes you really do want a new instance. When you can justify this, choose a better framework.
The resources and time used in instantiating a class are usually negligible. The main reason I usually see singleton classes used is to maintain data integrity. For example, if you have a class that represents data in a database, creating multiple objects for it can cause the data to become out of sync. If one object changes and commits data to the DB, the other objects could have old data.
Its rather a simple concept, utilizing singleton-pattern it makes sure that one class is instantiated only once during an application's execution cycle.
This sort of concept apply for libraries more. Lets see a basic example:
class Authenticate {
public function login($username, $password) {
....
}
public function logout() {
}
}
Now, through a execution of a page, there is hardly any case that the object of the above class, needs to be created more than once. The main thing to understand is Utilization of resources
And YES, instantiating same class over and over again will without a doubt add up in the memory, although it might be negligible like in the example I have shown, but it does affect.
I have asked a similar question to this one already but I think it was badly worded and confusing so hopefully I can make it a bit clearer.
I am programming in a native Linux file system.
I have a class of HelpTopic:
class HelpTopic extends Help{}
And a class of Help:
class Help{}
Now I go to include HelpTopic:
include('HelpTopic.php');
And even though I do not instantiate HelpTopic with new HelpTopic() PHP (in a Linux file system) still reads the class signature and tries to load Help with HelpTopic.
I do not get this behaviour from a cifs file system shared from a Windows System.
My best guess is that there is some oddity with Linux that causes PHP to react this way but not sure what.
Does anyone have any ideas or solutions to this problem?
EDIT:
I have added my loading function to show what I am doing:
public static function import($cName, $cPath = null){
if(substr($cName, -2) == "/*"){
$d_name = ROOT.'/'.substr($cName, 0, -2);
$d_files = getDirectoryFileList($d_name, array("\.php")); // Currently only accepts .php
foreach($d_files as $file){
glue::import(substr($file, 0, strrpos($file, '.')), substr($cName, 0, -2).'/'.$file);
}
}else{
if(!$cPath) $cPath = self::$_classMapper[$cName];
if(!isset(self::$_classLoaded[$cName])){
self::$_classLoaded[$cName] = true;
if($cPath[0] == "/" || preg_match("/^application/i", $cPath) > 0 || preg_match("/^glue/i", $cPath) > 0){
return include ROOT.'/'.$cPath;
}else{
return include $cPath;
}
}
return true;
}
}
I call this by doing glue::inmport('application/models/*'); and it goes through including all the models in my app. Thing is PHP on a linux based file system (not on cifs) is trying to load the parents of my classes without instantiation.
This is a pretty base function that exists in most frameworks (in fact most of this code is based off of yiis version) so I am confused why others have not run into this problem.
And even though I do not instantiate HelpTopic with new HelpTopic() PHP still reads the class signature and tries to load Help with HelpTopic.
Correct.
In order to know how to properly define a class, PHP needs to resolve any parent classes (all the way up) and any interfaces. This is done when the class is defined, not when the class is used.
You should probably review the PHP documentation on inheritance, which includes a note explaining this behavior:
Unless autoloading is used, then classes must be defined before they are used. If a class extends another, then the parent class must be declared before the child class structure. This rule applies to class that inherit other classes and interfaces.
There are two ways to resolve this problem.
First, add a require_once at the top of the file that defines the child class that includes the file defining the parent class. This is the most simple and straight-forward way, unless you have an autoloader.
The second way is to defione an autoloader. This is also covered in the documentation.
The ... thing ... you're using there is not an autoloader. In fact, it's a horrible abomination that you should purge from your codebase. It's a performance sap and you should not be using it. It also happens to be the thing at fault.
We don't have the definition of getDirectoryFileList() here, so I'll assume it uses either glob() or a DirectoryIterator. This is the source of your problem. You're getting the file list in an undefined order. Or, rather, in whatever order the underlying filesystem wants to give to you. On one machine, the filesystem is probably giving you Help.php before HelpTopic.php, while on the other machine, HelpTopic.php is seen first.
At first glance, you might think this is fixable with a simple sort, but it's not. What happens if you create a Zebra class, and then later need to create an AlbinoZebra that inherits from it? No amount of directory sorting is going to satisfy both the "load ASCIIbetical" and the "I need the Zebra to be first" requirements.
Let's also touch on the performance aspect of the problem. On every single request, you're opening a directory and reading the list of files. That's one hell of a lot of stat calls. This is slow. Very slow. Then, one by one, regardless of whether or not you'll need them, you're including the files. This means that PHP has to compile and interpret every single one of them. If you aren't using a bytecode cache, this is going to utterly destroy performance if the number of files there ever grows to a non-trivial number.
A properly constructed autoloader will entirely mitigate this problem. Autoloaders run on demand, meaning that they'll never attempt to include a file before it's actually needed. Good-performing autoloaders will know where the class file lives based on the name alone. In modern PHP, it's accepted practice to name your classes such that they'll be found easily by an autoloader, using either namespaces or underscores -- or both -- to map directory separators. (Meaning namespace \Models; class Help or class Models_Help would live in Models/Help.php)
Unfortunately most examples won't be useful here, as I don't know what kind of weird things your custom framework does. Take a peek at the Zend Framework autoloader, which uses prefix registration to point class prefixes (Model_) at directories.