APC is showing 100% fragmentation. Is this bad?
Does it mean that it's not helping at all? What paths do I go down to improve situation?
Thanks in advance.
In my experience, yes. I had a system where APC was showing 100% fragmentation, and performance was bad. I increased APC's memory limit (to 200 MB in my case -- but we had a lot of code) enough to give it some slack room. Fragmentation dropped to zero, and IIRC, CPU usage on the server dropped by 50%.
Also, make sure you're using the apc.php script that comes with APC to monitor fragmentation/utilization. We've even written a nagios check to watch APC, 'cause we have enough traffic that apache locks up entirely when APC fills up.
Moral of the story: give APC enough memory, and monitor utilization.
Fragmentation means that apc often throws out items from it's cache and adds new ones and has trouble finding large enough continous blocks.
There are two main ways to improve performance then
Give more memory to APC. Ideally APC can store your complete scripts in memory.
use apc.filter in php.ini to filter out files which aren't requested often or change often.
Also using apc_store() with a short time to live is bad, as is overwriting using apc_store() often.
[...] Fragmentation is what hurts performance, not the size of memory per se. But it also seems that fragmentation happens when memory is low [...]
Note also that there seems to be a bug with apc.php's graph: http://pecl.php.net/bugs/bug.php?id=13146
Related
I am on a VPS server (Quad core with 768MB of RAM. I am running Apache and using APC as my default caching engine) . I get around 2000 visitors per day with a maximum of 100 concurrent visitors.
According to CakePHP DebugKit, my most memory intensive pages use around 16MB and the other pages average at around 10MB.
Is this memory usage normal?
Is it normal for my memory to bottleneck at 2000 visitors per page?
Should I consider upgrading my plan to 2GB RAM?
I also noticed that the View rendering is taking up most of the memory, around 70% on most pages.
I am monitoring my resource usage, when I have around 50 or more concurrent users, I am getting 0 free MB left.
Thank you
You should also check your other processes. From my experience, MySQL takes up more memory than anything else on any stack that I run. You should also implement better page caching so that PHP doesn't need to be touched when it isn't absolutely necessary. But Apache is also a memory hog that needs to be fine tuned. If you want to stick with Apache, then run Varnish in front of it.
Also, keep APC, but also add Memcached. It's much faster.
If your site has a spike-load that brings it to zero memory, then, if you can, consider launching extra instances of the server and doing a sort of round-robin (if the VPS is cloud-hosted and this is possible). If the load is constant, then definitely upgrade.
#burzum is completely right, however, that you should just switch to nginx. It's far, far better than Apache at this point. But, just to put you on the right track, quite a few people run nginx as a reverse proxy in front of Apache, and while that does speed up the server, it's entirely unnecessary because nginx can do pretty much anything you need it to do without Apache. Don't bother using Varnish in front of nginx either because nginx can act as its own reverse proxy.
Your best bet is to implement nginx with apcu (upgrade php to 5.5 if possible for better performance) and use memcached, and implement nginx's native microcaching abilities. If your site is heavy on read and light on write, then you might notice that everything is taken care of by nginx just retrieving a cached copy from memcache. While this helps with memory, it also helps with processing. My servers' CPUs usually have a 3-5% usage during peaks.
Is this memory usage normal?
Yes, it doesn't seem to be high for a normal CakePHP application.
Is it normal for my memory to bottleneck at 2000 visitors per page?
I guess yes, I'm not a "server guy".
Should I consider upgrading my plan to 2GB RAM?
I would try switching to Nginx first. Apache is a memory eating monster compared to Nginx, just search for a few banchmarks, a random one I've picked by a quick search is from this page.
Overall Nginx should provide you a much better performance. At some stage I would consider updating the memory but try Nginx first.
I made the mistake of getting into the habit of autoloading a bunch of libraries, models..etc, when I don't need to. It is too hard to trace down all the cases to make sure everything is available and not broken. I am estimating the autoloading is causing 1-2 MB of extra resources per script. The total memory usage for my script is around 4Mb. (I used the profiler and disabled autoloading and saw that it dropped 1-2mb)
Is this something to worry about? The server I am running this on has 1gb of ram and doesn't ever seem to be under heady load.
Is this a bad thing? Am I worrying too much?
Is always better to load what you needs.
But 4MB is a normal PHP app memory usage.
I read somewhere, in case of a php optimized app to worry when you exceed 9MB.
Memory usage become important when your server have to respond many requests. So the goal become to use less server ressource as possible to pay less.
Sorry for my poor english.
I'm primarily wondering what the speed difference is in accessing the object cache of APC v. memcached (NOT op-code cache). The primary advantage of memcached is that it is distributed and not restricted to the local machine. However, since it is over the network, there's is some sort of latency involved.
I was wondering whether the speed difference between accessing APC (on the machine) and memcached (on another server) is big enough to warrant having a staged caching scheme, where the program first tries APC, then memcached, and finally the database if all else fails.
Like most everything else: it depends.
If you have a lot of calculations and can store the results then caching will speed things up. If you're just basically storing rows from the database then in memory caching will help but memcached may not add a huge amount of difference vs. a database (assuming the db queries are all simple). On the other hand if you're doing complex queries, or a lot of programmatic work to create something, then caching makes much more sense.
To give you an example, I recently worked on a site that was written by a 3rd party contractor who did not do any performance work during design. It was slow as an ox because it had a lot of unoptimized includes and such. Adding APC basically improved the performance by 10x. Adding memached decreased load times by 10 - 20 ms.
If you're far enough along then do some performance testing (look up xdebug, or another tool) and see where your bottlenecks are, then plan accordingly.
Keep in mind that if you fill up your APC cache with other things then APC will have to re-calculate the op-code for your pages again. This can cause problems if the pages keep removing objects, then once the page runs the objects keep removing pages. Not fun.
Just be safe and don't be tempted to use APC for anything but config values which won't cause your pages to be removed to make space.
TL;DR Once APC gets full your site will slow down and your server will work much harder.
CakePHP offers support for APC, XCache and Memcache in addition to its default caching engine. Having had some problems with my application sporadically caching broken pages for no known reason, I've decided to try another engine to see if that makes a difference.
XCache and Memcache both seem as though they might take a little bit more setup, but APC appears to be literally a case of changing one line in the core.php. My question is, where can I find information about why I should choose APC over the default engine? What are the pros and cons? It can't really be a case of "just try them both and see if one feels better than the other" (can it?), but a basic snoop around hasn't revealed a simple breakdown of the differing merits of cache engines in Cake.
Can anyone explain the mysterious workings of cache engines in Cake to me? Or point me to a resource that does? Bonus points if XCache and Memcache are also compared, because they might be my next port of call...
Files are on disk :
Not quite fast ; and concurrent access are not great at all, if several processes try to read/write at the same time
Local to one server (if you have several servers, you'll have to store the files on each one of them -- NFS being slow)
But you have a lot of space
APC is in memory :
Really fast
But you have less space
And it's local to each server too
memcached is in memory, on a network cluster :
quite fast (a bit less than APC ; but still pretty fast)
Shared between all your servers : each item has to be cached only once, even if you have several webservers.
You can have several servers in your memcached cluster (which means virtually no limit in the size of the cache)
I have an installation of 1.3.2.4 running two Store Views and 2,734 products. The site sees around 15,000 visits a month.
Apache and MySQL (mostly Apache) hovers at around 1.5 GB RAM usage most of the time and peaks over 3 GB. My questions is, considering the stats, is this normal? Seems like a lot.
If that memory usage is in fact abnormal, would an upgrade to 1.4.1.1 help?
If you consider your stores, then you are doing just fine. But regarding the traffic you're getting, it seems that you need to provide some extra features to Magento to let it fire up. For this, you can have some of the following:-
Install APC (Alternative PHP Cache) or XCache (or any other alternative) and configure the use of it in your Magento back-end. It dramatically increases the speed of Magento.
You can have Magento's cache stored in memory (tmpfs in Linux).
You can also tell Magento to save sessions into Memcache so that your sessions are in memory & distributed.
Check your Magento's Index Management section for any requirement of indexes, every month or bi-monthly. If you do find any indexing required, then do it immediately & clear the cache from your Cache Management.
Check your database every week or bi-monthly for any overhead in any of your database's tables. If you do find any overhead, then "optimize" those tables immediately.
Try reading some of these articles, to know more about these.
Also, upgrading to 1.4.1.1 will help you out in terms of features provided by Magento. But for performance, I think it will be best to wait for some more time, until Magento releases its version 2 in the market, in which some performance issues may be taken care of by Magento.
Hope it helps.
1.3.2.4 is a good stable release, upgrading to 1.4.0.1 is very painless and will give you the added benefit of split index management and much faster administration area (mass attribute update is fixed).
Don't be overly concerned about memory usage, depending on the number of Apache modules you have loaded, you should expect to see about 30MB per child. As long as your not swapping or encroaching your limits, you shouldn't have any real concerns over how much is being consumed. Disabling unused modules will help cut down memory - but to be honest, not by any noticeable margin.
You could always throw Nginx in front as a reverse proxy to serve static content requests and pipe PHP/dynamic reqs. back to Apache. That way you can keep the modular Apache build with .htaccess support and cut down your memory overheads significantly.
However, this could do with more information, such as the output from
free -m
To see how some of the memory is being allocated.
I'd probably suggest downloading tuning-primer.sh to run on your MySQL config. It will give a good (entry-level) indication of how efficient your memory allocation is.
Those stats look quite typical for Magento, if you consider a single hit/page load can use upwards of 64MB RAM.
Your Apache settings can also drastically effect the amount of RAM your system uses. Upgrading your Magento installation may give some small performance boost, but don't expect it to do much for memory consumption etc.
If your memory consumption is a real issue for you then you have several possible routes to reduce resource usage, such as:
Install Nginx as a reverse caching proxy to apache (apache is a hog and is poor serving static content).
Use Nginx + PHP Fast CGI and remove apache
Try using worker MPM module for apache, or Fast CGI.
Install caching proxy such as Varnish/Squid.
If you are stuck with apache you can tweek KeepAlive and other settings to allow you to reduce memory usage
Tweek MySQL settings, such as query caching to imporove resource usage / performance
I have found 1. to work very well in reducing cpu/memory usage as it will allow Nginx to serve static images etc without requiring apache to hog RAM trying to serve them.