I was wondering if it is possible to store function variables in memcached. I wrote basic templating system which compiles xml templates into functions. The templates can get very big and I think I could get a performance boost if I could cache it. Would something like this work, or would I just be caching a reference to the function?
$populate_template = function($values){
//compiled template
};
$memcached_object->set("some_key",$populated_template);
EDIT: I do realize that there are php accelerators that do exactly what I want, however it would be a lot easier to be able to use memcached because I won't have to go through the process of getting another technology approved.
I don't think so. Naturally, I looked at serializing it. But:
PHP Warning: Uncaught exception 'Exception' with message 'Serialization of 'Closure' is not allowed' in php shell code:1
With memcached, all you can probably do is store the code as a string, then call create_function after you pull it out.
I don't think there would be a performance boost with "serializing" the function source code as a string and recreating it via create_function. It will rather be hundred times slower than defining the function in PHP, because of the input/output with external medium + doing what would PHP need to do in either case.
You need to go with an opcode cache. But if PHP performance is the only lasting bottleneck in your application, you're a happy man.
Related
I have a PHP script that is failing with a fatal out-of-memory error. It is a script that processes all records in the DB - it works fine up to about 10k records and then hits the memory error.
However, I can't find out what is using up my application's memory.
I've checked the $_GLOBALS array and that accounts for maybe 1 MB or so.
I've checked the call stack at various points and have seen nothing unexpected.
The base memory requirement for PHP plus all relevant class files, etc. is about 7MB.
My feeling is that there is probably somewhere in the code that is resulting in variable references persisting - either deliberately (e.g. via a static cache in some class or other) or by mistake (e.g. resource handles not being freed).
Obviously functions like memory_get_usage() can tell me how much memory is used at any given point in the script, and tracking this is a slow but effective way of debugging. However, is there any way of getting details about what is actually using that memory?
Happy to accept answers that use an external tool (e.g. XDebug) providing they give useful output (i.e. output that identifies the class/variable names rather than using PHPs internal IDs). The output I expect would be something like you get from var_dump/print_r.
[Note that this question is not about how to debug out-of-memory issues in general, but specifically about whether there is a way to expose the details of memory use when debugging.]
Paul Crovella provided a link, but couldn't be bothered to post it as an answer, so I'm doing it for him.
The php-memprof extension (https://github.com/arnaud-lb/php-memory-profiler) provides a number of tools that can be used to provide exactly the information that the question asks for.
From the readme file:
php-memprof profiles memory usage of PHP scripts, and especially can tell which function has allocated every single byte of memory currently allocated.
Memprof can be enabled during script execution by calling memprof_enable().
Then the memory usage can be dumped by calling one of the memprof_dump_ functions. Both tell which functions allocated all the currently allocated memory.
The original question asked for something like print_r(), and this is provided by the memprof_dump_array() function. However, there are a number of other ways of accessing the memory profile which may be more useful depending on what you are trying to achieve, including dumping the entire memory map in callgrind format, for offline analysis.
As it is a PHP extension, it will require access to php.ini in order to install it, so it may not be suitable for debugging issues on live sites (but nobody does that, right?).
Is there a way in PHP to use "out of session" variables, which would not be loaded/unloaded at every connexion, like in a Java server ?
Please excuse me for the lack of accuracy, I don't figure out how to write it in a proper way.
The main idea would be to have something like this :
<?php
...
// $variablesAlreadyLoaded is kind of "static" and shared between all PHP threads
// No need to initialize/load/instantiate it.
$myVar = $variablesAlreadyLoaded['aConstantValueForEveryone'];
...
?>
I already did things like this using shmop and other weird things, but if there is a "clean" way to do this in "pure PHP" without using caching systems (I think about APC, Redis...), nor database.
EDIT 1 :
Since people (thanks to them having spent time for me) are answering me the same way with sessions, I add a constraint I missed to write : no sessions please.
EDIT 2 :
It seems the only PHP native methods to do such a thing are shared memory (shmop) and named pipes. I would use a managed manner to access shared objects, with no mind of memory management (shared memory block size) nor system problems (pipes).
Then, I browsed the net for a PHP module/library which provides functions/methods to do that : I found nothing.
EDIT 3 :
After a few researches on the way pointed out by #KFO, it appears that the putenv / setenv are not made to deal with objects (and I would avoid serialization). Thus, it resolves the problem for short "things" such as strings or numbers but not for more large/comples objects.
Using the "env way" AND another method to deal with bigger objects would be uncoherent and add complexity to the code and maintenability.
EDIT 4 :
Found this : DBus (GREE Lab DBus), but I'm not having tools to test it at work. Has somebody tested it yet ?
I'm open to every suggestion.
Thanks
EDIT 5 ("ANSWER"):
Since DBus is not exactly what I'm looking for (needs to install a third-party module, with no "serious" application evidence), I'm now using Memcache which has already proven its reliability (following #PeterM comment, see below).
// First page
session_id('same_session_id_for_all');
session_start();
$_SESSION['aConstantValueForEveryone'] = 'My Content';
// Second page
session_id('same_session_id_for_all');
session_start();
echo $_SESSION['aConstantValueForEveryone'];
This works out of the box in PHP. Using the same session id (instead of an random user-uniqe string) to initialize the session for all visitors leads to a session which is the same for all users.
Is it really necessary to use session to achieve the goal or wouldn't it better to use constants?
There is no pure PHP way of sharing information across different
threads in PHP! Except for an "external"
file/database/servervariable/sessionfile solution.
Since some commentators pointed out, that there is serialize/unserialize functionality for Session data which might break data on the transport, there is a solution: In PHP the serialize and unserialize functionality serialize_handler can be configured as needed. See https://www.php.net/manual/session.configuration.php#ini.session.serialize-handler It might be also interesting to have a look at the magic class methods __sleep() and __wakeup() they define how a object behaves on a serialize or unserialize request. https://www.php.net/manual/language.oop5.magic.php#object.sleep ... Since PHP 5.1 there is also a predefined Serializable interface available: https://www.php.net/manual/class.serializable.php
You can declare a Variable in your .htaccess. For Example SetEnv APPLICATION_ENVIRONMENT production and access it in your application with the function getenv('APPLICATION_ENVIRONMENT')
Another solution is to wrap your variable in a "persistent data" class that will automatically restore its data content every time the php script is run.
Your class needs to to the following:
store content of variable into file in __destructor
load content of variable from file in __constructor
I prefer storing the file in JSON format so the content can be easily examined for debugging, but that is optional.
Be aware that some webservers will change the current working directory in the destructor, so you need to work with an absolute path.
I think you can use $_SESSION['aConstantValueForEveryone'] that you can read it on every page on same domain.
Consider to refer to it's manual.
I'm trying to figure out why my pages are taking so long to load. I have a page with like 30 calls to get_template_directory_uri() and the implementation of that function makes calls to the database, but the value returned by the function is the same for the entirety of my page generation. Therefore I should probably do something like
$tduri = get_template_directory_uri();
and use $tduri as needed, at least as a good practice, but is this totally necessary or would the value of get_template_directory_uri() have been cached anyways?
PHP does not cache return values of functions by itself. In your case the function is making a call to an external database, so how could PHP even know if the result was the same or different each time the function was called?
Also consider that any call to an external system is highly likely to be slower than storing a variable within a PHP script.
So yes, absolutely, cache the return value somewhere suitable and always test whether it makes a noticeable improvement, sometimes the results can be suprising. There are many good ways to profile PHP, microtime is very quick for ad-hoc testing without requiring additional set-up. Search for 'PHP profiling' to get the more powerful tools.
Is there a way in PHP to use "out of session" variables, which would not be loaded/unloaded at every connexion, like in a Java server ?
Please excuse me for the lack of accuracy, I don't figure out how to write it in a proper way.
The main idea would be to have something like this :
<?php
...
// $variablesAlreadyLoaded is kind of "static" and shared between all PHP threads
// No need to initialize/load/instantiate it.
$myVar = $variablesAlreadyLoaded['aConstantValueForEveryone'];
...
?>
I already did things like this using shmop and other weird things, but if there is a "clean" way to do this in "pure PHP" without using caching systems (I think about APC, Redis...), nor database.
EDIT 1 :
Since people (thanks to them having spent time for me) are answering me the same way with sessions, I add a constraint I missed to write : no sessions please.
EDIT 2 :
It seems the only PHP native methods to do such a thing are shared memory (shmop) and named pipes. I would use a managed manner to access shared objects, with no mind of memory management (shared memory block size) nor system problems (pipes).
Then, I browsed the net for a PHP module/library which provides functions/methods to do that : I found nothing.
EDIT 3 :
After a few researches on the way pointed out by #KFO, it appears that the putenv / setenv are not made to deal with objects (and I would avoid serialization). Thus, it resolves the problem for short "things" such as strings or numbers but not for more large/comples objects.
Using the "env way" AND another method to deal with bigger objects would be uncoherent and add complexity to the code and maintenability.
EDIT 4 :
Found this : DBus (GREE Lab DBus), but I'm not having tools to test it at work. Has somebody tested it yet ?
I'm open to every suggestion.
Thanks
EDIT 5 ("ANSWER"):
Since DBus is not exactly what I'm looking for (needs to install a third-party module, with no "serious" application evidence), I'm now using Memcache which has already proven its reliability (following #PeterM comment, see below).
// First page
session_id('same_session_id_for_all');
session_start();
$_SESSION['aConstantValueForEveryone'] = 'My Content';
// Second page
session_id('same_session_id_for_all');
session_start();
echo $_SESSION['aConstantValueForEveryone'];
This works out of the box in PHP. Using the same session id (instead of an random user-uniqe string) to initialize the session for all visitors leads to a session which is the same for all users.
Is it really necessary to use session to achieve the goal or wouldn't it better to use constants?
There is no pure PHP way of sharing information across different
threads in PHP! Except for an "external"
file/database/servervariable/sessionfile solution.
Since some commentators pointed out, that there is serialize/unserialize functionality for Session data which might break data on the transport, there is a solution: In PHP the serialize and unserialize functionality serialize_handler can be configured as needed. See https://www.php.net/manual/session.configuration.php#ini.session.serialize-handler It might be also interesting to have a look at the magic class methods __sleep() and __wakeup() they define how a object behaves on a serialize or unserialize request. https://www.php.net/manual/language.oop5.magic.php#object.sleep ... Since PHP 5.1 there is also a predefined Serializable interface available: https://www.php.net/manual/class.serializable.php
You can declare a Variable in your .htaccess. For Example SetEnv APPLICATION_ENVIRONMENT production and access it in your application with the function getenv('APPLICATION_ENVIRONMENT')
Another solution is to wrap your variable in a "persistent data" class that will automatically restore its data content every time the php script is run.
Your class needs to to the following:
store content of variable into file in __destructor
load content of variable from file in __constructor
I prefer storing the file in JSON format so the content can be easily examined for debugging, but that is optional.
Be aware that some webservers will change the current working directory in the destructor, so you need to work with an absolute path.
I think you can use $_SESSION['aConstantValueForEveryone'] that you can read it on every page on same domain.
Consider to refer to it's manual.
I am currently debugging a script that constantly runs into OutOfMemory exceptions. It is run as a cronjob and usually runs fine, but when the cronjob wasn't run for a while (for whatever reason) the script has to handle to many elements that queued up and will run into a OutOfMemory exception.
From examining the code I was not able to spot the problem. I believe one of the iterative function calls might leak memory, but I am not sure which one and where.
Is there an option to get PHP to dump the heap, when an OutOfMemory exception occurs? I might be able to spot the problem from there (most likely).
While I was not able to find a "dump heap on Exception" option, I did find get_defined_vars() which is basically a heap dump if called from a global scope. Using this I was able to see that there where hundreds (actually thousands) of still referenced database rows hanging around in my memory. This was due to a not freed mysql result resource somewhere in the infamous function that caused the leak. I found it and fixed it. It runs well now.
Well, easiest approach would be to use a try-catch block around that part of your script where the error possibly occurs and you will have to dump the stack in the catch part. The problem might be that the machine won't be able to react cause the memory is full and it terminates. I do not know if it helps to discard some variables to free up some memory to output some data.
EDIT: For this purpose use the php function debug-backtrace. This will give you a stack trace. So finding the error will be much likely in case the machine is still up.
Just do not load all objects together to memory, but read-as-you-process-them?
I've had lots of problems with simpleXML and memory leaks. They are a pain in the are to track down... took me days to figure out that simpleXML was causing then and then fix them.
As far as i know you cand programatically set a handled for OOM:)
Also, PHP's functions for displaying memory info fails to detect the memory leaks, i had scripts eating up ~1Gb of ram, but PHP's functions reported only 100Mb used:)
This is as good of a 'heap dump' as I'm able to quickly write in PHP. I take the defined variables and functions, then sort by their serialized length. Serialized length isn't a 100% reliable method for getting a variable's size, but it's pretty good, and generally useful for determining which objects are your memory hogs:
$memmap = array_map(function($var) { return strlen(serialize($var)); },
array_merge(get_defined_functions(), get_defined_vars()));
arsort($memmap);
var_dump($memmap);
You may want to tweak the callback function a bit if you'd like your results to be more verbose, or to recurse through the defined variables.
I've never seen PHP provide a native facility for this but a few other things might exist:
Try: https://github.com/mcfunley/php-heap/blob/master/php-heap.py
It could also be possible to write an extension to achieve the same.