What's a good way to profile a PHP page's memory usage? For example, to see how much memory my data is using, and/or which function calls are allocating the most memory.
xdebug doesn't seem to provide memory information in its profiling feature.
xdebug does provide it in its tracing feature. This is pretty close to what I want, except the sheer amount of data is overwhelming, since it shows memory deltas for every single function call. If it were possible to hide calls below a certain depth, maybe with some GUI tool, that would solve my problem.
Is there anything else?
As you probably know, Xdebug dropped the memory profiling support since the 2.* version. Please search for the "removed functions" string here: http://www.xdebug.org/updates.php
Removed functions
Removed support for Memory profiling as that didn't work properly.
So I've tried another tool and it worked well for me.
https://github.com/arnaud-lb/php-memory-profiler
This is what I've done on my Ubuntu server to enable it:
sudo apt-get install libjudy-dev libjudydebian1
sudo pecl install memprof
echo "extension=memprof.so" > /etc/php5/mods-available/memprof.ini
sudo php5enmod memprof
service apache2 restart
And then in my code:
<?php
memprof_enable();
// do your stuff
memprof_dump_callgrind(fopen("/tmp/callgrind.out", "w"));
Finally open the callgrind.out file with KCachegrind
Using Google gperftools (recommended!)
First of all install the Google gperftools by downloading the latest package here: https://code.google.com/p/gperftools/
Then as always:
sudo apt-get update
sudo apt-get install libunwind-dev -y
./configure
make
make install
Now in your code:
memprof_enable();
// do your magic
memprof_dump_pprof(fopen("/tmp/profile.heap", "w"));
Then open your terminal and launch:
pprof --web /tmp/profile.heap
pprof will create a new window in your existing browser session with something like shown below:
Xhprof + Xhgui (the best in my opinion to profile both cpu and memory)
With Xhprof and Xhgui you can profile the cpu usage as well or just the memory usage if that's your issue at the moment.
It's a very complete solutions, it gives you full control and the logs can be written both on mongo or in the filesystem.
For more details see my answer here.
Blackfire
Blackfire is a PHP profiler by SensioLabs, the Symfony2 guys https://blackfire.io/
If you use puphpet to set up your virtual machine you'll be happy to know it's supported ;-)
Well, this may not be exactly what you're looking for, but PHP does have a couple of functions built-in that will output memory usage. If you just wanted to see how much memory a function call is using, you could use memory_get_peak_usage() before and after a call, and take the difference.
You use the same technique around your data using the very similar memory_get_usage().
Pretty unsophisticated approach, but it's a quick way to check out a piece of code. I agree that xdebug mem deltas can be too verbose to be useful sometimes, so I often just use it to narrow down to a section of code, then dump out specific memory usage for small pieces manually.
Xdebug reimplemented memory tracing in 2.6 (2018-01-29) which can be used in Qcachegrind or similar tool. Just make sure to select the memory option :)
From the docs:
Since Xdebug 2.6, the profiler also collects information about how much memory is being used, and which functions aGnd methods increased memory usage.
I'm not familiar with the format of the file, but it's Qcachegrind has worked great for me in tracing a couple memory issues.
http://geek.michaelgrace.org/2012/04/tracing-php-memory-usage-using-xdebug-and-mamp-on-mac/
I'm on a Mac so if you're on Windows you'll have to test this, but this works for me.
I modified my tracefile-analyzer.php file and added the path to the PHP binary at the top so that you could call it in terminal as a normal unix script.
#!/Applications/MAMP/bin/php5.3/bin/php
<?php
if ( $argc <= 1 || $argc > 4 )
{
Don't forget to chmod this file to 755.
You could easily create a ruby watchr script to automatically call the script each time it creates a memory profile file (*.xt). That way you could keep testing and seeing your improvements without having to execute the command over and over.
Related
I have installed PHP5 - PHP5-MEMCACHE - PHP-APC.
Can they work like that together? Will the loading be fast with these modules ?
I tried to use them, I don't "see" particular differences, maybe the CPU is used less with these modules. My website doesn't have high traffic, but If i can save resources is better!
Thank you
APC keeps cache of PHP bytecode. Memcache keeps cache of your vars, that you set.
So answer is Yes, they can. They're made for different things.
They work together very well, you just need to use them properly :
Memcached is a distributed cache system. What that means in a nutshell is that if you have a cluster of servers all of them can access the same cache pool
APC is an opcache and local cache system. Meaning it optimizes the php scripts so when going through the compiler less operations are made and the code is executed way faster. Another use of APC is local cache, which means you can store values in the cache and access them from the machine running the code.
Yes, they can work together. Whether they will on a production system is another story...
Personally, I had to give up trying to get the following to work for any extended period of time:
Ubuntu 10.04
NGINX 0.7.65
PHP 5.3.2
php-apc
php5-memcache
It will run for awhile, but after stress testing php errors out. I can restart php-fastcgi via /etc/init.d/php-fastcgi and things will role along for some time more, but it always crashes again sooner than later.
I can run either/or without issue, but the two together won't cooperate for me. FYI I tried using binaries (apt-get packages), installing as PECL extensions, downloading source, but all roads lead me to the same sad fate. I also tried running the memache daemon local & remotely on my web host, but same outcome.
I'm working on mmo game based on JavaScript and PHP. We are using both of them. I can't tell you more, beacause I am only frontend developer, however I think if APC and memcache were bad we were not using it.
I want to know what functions are being called and what time each request is taking for an application which is running on apache.
Is there any tool or any other way where i can get this data.
I also want to know how much time each function is taking.
The application is running it cannot be stopped.
So i need to get the details in the running environment itself.
Thanks in advance.
One of the most used industry tools for this is: http://www.xdebug.org/
I have used it religiously for a long time now! From it's front-page it does the following:
"The Xdebug extension helps you debugging your script by providing a lot of valuable debug information. The debug information that Xdebug can provide includes the following:
* stack traces and function traces in error messages with:
o full parameter display for user defined functions
o function name, file name and line indications
o support for member functions
* memory allocation
* protection for infinite recursions"
Xdebug can write a profiling file that you can analyze in kcachegrind or wincachegrind.
It depends on whether you want active or passive profiling.
Passive tools such as New Relic work silently in the background and collect a little information about all requests by sacrificing a small amount of compute resources. These generally have more information regarding the entire server stack. These are typically used for production environments, which sounds like what you require.
Active profilers are used for development and will run just once per request, collecting a lot of information specifically about the part of the application you are working on at the cost of a large performance hit. The most common PHP active profiler is probably Xdebug.
Ways to analyse the data from Xdebug:
NOTE: If you use a virtual machine, vagrant, docker etc make sure you set the output dir to a shared volume, accessible outside of the virtual machine
e.g.
(php.ini) xdebug.profiler_output_dir = "/project_root/tmp"
If you use PHPStorm:
Tools > Analyse Xdebug Profiler snapshot...
For those using Xdebug on MacOS:
Using homebrew install qcachegrind and AppViz
brew install qcachegrind
brew install graphviz
Then run it from command line:
qcachegrind cachegrind.out.1394
OR just run qcachegrind and open the cachegrind file generated by xdebug using the GUI navigator
There are couple of PHP profiling tools,
such as
xdebug
Particletree
I was thinking to use opcode caching for performance profit
what is the easiest way for opcode caching with PHP/Apache ?
and what are the performance improvements ?
I have read about xDebug but I was wondering if there are more options ?
I use the APC extension as an opcode cache on both my personnal server, and on the servers we are using at work -- and I've almost never run into any kind of trouble with it.
Installation is pretty easy : depending on your Linux distribution, you might want to use something like :
sudo aptitude install php5-apc
But you'd not necessarily have the last stable version... And I generally prefer using :
sudo pecl install apc
Which will fetch the last version from PECL and compile it (Note : you'll probably need to installed a package called php5-dev).
You'll then have to configure it ; see Runtime Configuration for the directives you can configure.
The performance improvements can vary depending on your server/application, but, on a server that only serves PHP pages (i.e. if your DB is on another machine) you might see a drop in CPU load that's quite important (I've seen CPU load go from 80% to 40-50% on a server, once)
Xdebug is not related to opcode caching nor performance : as its name indicates, it's useful for debugging.
And Xdebug should not be installed on a production server : it can be a pain, when it comes to performances -- I suppose it's because it "hooks" into PHP and add lots of stuff, like logging many informations useful for debugging ; which means more calculations to do -- which takes time and CPU.
Try APC : been using it with Mediawiki and the results are significant.
Other popular options are eAccelerator and APC.
Performance improvements will depend on the web app you're caching. Here are some benchmarks for the Drupal CMS using APC and eAccelerator, and here's the same site's tutorial on installing APC (nice and easy).
On Debian/Ubuntu, try:
sudo aptitude install php5-xcache
A commercial solution is from Zend, http://www.zend.com
Does APC module in PHP when running in CLI mode support code optimization? For example, when I run a file with php -f <file> will the file be optimized with APC before executing or not? Presuming APC is set to load in config file. Also, will the scripts included with require_once be also optimized?
I know optimization works fine when running in fastcgi mode, but I'm wondering if it also works in CLI.
apc_* functions work, but I'm wondering about the code optimization, which is the main thing I'm after here.
Happy day,
Matic
The documentation of apc.enable_cli, which control whether APC should be activated in CLI mode, says (quoting) :
Mostly for testing and debugging.
Setting this enables APC for the CLI
version of PHP. Under normal
circumstances, it is not ideal to
create, populate and destroy the APC
cache on every CLI request, but for
various test scenarios it is useful to
be able to enable APC for the CLI
version of PHP easily.
Maybe APC will store the opcodes in memory, but as the PHP executable dies at the end of the script, that memory will be lost : it will not persist between executions of the script.
So opcode-cache in APC is useless in CLI mode : it will not optimize anything, as PHP will still have to re-compile the source to opcodes each time PHP's executable is launched.
Actually, APC doesn't "optimize" : the standard way of executing a PHP script is like this :
read the file, and compile it into opcodes
execute the opcodes
What APC does is store in opcodes in memory, so the execution of a PHP script becomes :
read the opcodes from memory (much faster than compiling the source-code)
execute the opcodes
But this means you must have some place in memory to store the opcodes. When running PHP as an Apache module, Apache is responsible for the persistence of that memory segment... When PHP is run from CLI, there is nothing to keep the memory segment there, so it is destroyed at the end of PHP's execution.
(I don't know how it works exactly, but it's something like that, at least in the principles, even if my words are not very "technical" ^^ )
Or, by "optimization" you mean something else than opcode cache, like the configuration directive apc.optimization ? If so, this one has been removed in APC 3.0.13
If you have CLI code that generates any configuration based on the environment, then the CLI code will think that APC isn't enabled. For example, when generating Symfony's DI container through the CLI, it will tell Doctrine not to use APC (details).
Also, I have not tested it but there's a chance APC may improve the speed of scripts for files included after a pcntl_fork(). Edit: I've asked the question about APC & pcntl_fork() here.
For completeness, to enable APC on the CLI (in Ubuntu):
echo 'apc.enable_cli = 1' > /etc/php5/cli/conf.d/enable-apc-cli.ini
Well, there's a good reason for APC in CLI Mode:
UnitTesting: I wanna do my unit test using an environment as close to the later production environment as possible. Zend Framework has an internal caching solution, which may use APC's Variable Cache as Storage Backend - and I wanna use this.
There is another reason to use it in CLI mode: some scripts are able to use it as a cache
Is there anything out there freeware or commercial that can facilitate analysis of memory usage by a PHP application? I know xdebug can produce trace files that shows memory usage by function call but without a graphical tool the data is hard to interpret.
Ideally I would like to be able to view not only total memory usage but also what objects are on the heap and who references them similar to Jprofiler.
As you probably know, Xdebug dropped the memory profiling support since the 2.* version. Please search for the "removed functions" string here: http://www.xdebug.org/updates.php
Removed functions
Removed support for Memory profiling as that didn't work properly.
So I've tried another tool and it worked well for me.
https://github.com/arnaud-lb/php-memory-profiler
This is what I've done on my Ubuntu server to enable it:
sudo apt-get install libjudy-dev libjudydebian1
sudo pecl install memprof
echo "extension=memprof.so" > /etc/php5/mods-available/memprof.ini
sudo php5enmod memprof
service apache2 restart
And then in my code:
<?php
memprof_enable();
// do your stuff
memprof_dump_callgrind(fopen("/tmp/callgrind.out", "w"));
Finally open the callgrind.out file with KCachegrind
Using Google gperftools (recommended!)
First of all install the Google gperftools by downloading the latest package here: https://code.google.com/p/gperftools/
Then as always:
sudo apt-get update
sudo apt-get install libunwind-dev -y
./configure
make
make install
Now in your code:
memprof_enable();
// do your magic
memprof_dump_pprof(fopen("/tmp/profile.heap", "w"));
Then open your terminal and launch:
pprof --web /tmp/profile.heap
pprof will create a new window in your existing browser session with something like shown below:
Xhprof + Xhgui (the best in my opinion to profile both cpu and memory)
With Xhprof and Xhgui you can profile the cpu usage as well or just the memory usage if that's your issue at the moment.
It's a very complete solutions, it gives you full control and the logs can be written both on mongo or in the filesystem.
For more details see my answer here.
Blackfire
Blackfire is a PHP profiler by SensioLabs, the Symfony2 guys https://blackfire.io/
If you use puphpet to set up your virtual machine you'll be happy to know it's supported ;-)
I came across the same issue recently, couldn't find any specific tools unfortunately.
But something that helped was to output the xdebug trace in human readable format with mem deltas enabled (an INI setting, xdebug.show_mem_deltas or something I think?). Then run sort (if you are on *nix) on the output:
sort -bgrk 3 -o sorted.txt mytracefile.xt
That sorts on the third col, the mem deltas. You can also sort on the second column, in which case you can find the line at which your app uses the most memory in total.
Of course, this can't detect when an object's memory usage only creeps up in small increments but ends up using a lot of memory overall. I have a fairly dumb method that attempts to do this using a combination of object iteration and serialization. It probably doesn't equate exactly to memory usage, but hopefully gives an idea of where to start looking. Bear in mind it will use up memory itself, and also has not been extensively tested, so buyer beware:
function analyzeMem($obj, $deep=false)
{
if (!is_scalar($obj))
{
$usage = array('Total'=>strlen(serialize($obj)));
while (list($prop, $propVal) = each($obj))
{
if ($deep && (is_object($propVal) || is_array($propVal)))
{
$usage['Children'][$prop] = analyzeMem($propVal);
}
else
{
$usage['Children'][$prop] = strlen(serialize($propVal));
}
}
return $usage;
}
else
{
return strlen(serialize($obj));
}
}
print_r(analyzeMem(get_defined_vars()));
Also, just got suggested this method by a colleague (cheers Dennis ;-) This hides the steps that are below 2 levels of indentation, you can quite easily see the points where the overall memory usage jumps up, and can narrow things down by increasing the indentation:
egrep '[0-9]+ ( ){1,2}-> ' mytracefile.xt
On http://www.xdebug.org/updates.php for Xdebug 2.0.4 they write in section "removed functions": "...Removed support for Memory profiling as that didn't work properly...". Hence xdebug wont be an option
I personally used https://github.com/arnaud-lb/php-memory-profiler
on PHP 5.6 and Ubuntu 18, and Kcachegrind for visualizing.
Kcachegrind is okay, but not the best. I hope to find a better alternative even if it's on Mac or Windows.
With the version 2.6.0 on 2018-01-29 xdebug added support for profiling memory usage. Now you can generate callgrind files with time and memory information. On Mac you can visualize that information for example with Qcachegrind or Profiling Viewer (premium).
A graphical tool for xdebug output is KCacheGrind.
Try webgrind. It gives you the profiling of CacheGrinder in an easy to read, browser based format. I'm on a Mac and it has made profiling a breeze.
phpDesigner 2008 can debug and benchmark websites using xdebug and KCacheGrind. It also has a built-in monitor.