I am trying to move from the typical combination of APC (for opcode) and Memcache (for my userdata) to a purely apc based cache. Given my usage structure it makes sense and performance is higher.
But unlike before, where the apc cache was limited to a reasonable size and did not affect my data cache, I am now worried that the opcode cache might grow relatively large so that the sum of opcode cache an datacache exceeds the limit. As far as I understand, this would result in a flush of the total cache.
Is there a way to mimic the behaviour, as if apc was only an opcode cache and limit the opcode cache size to a reasonable limit, leaving the rest for user data?
Or should I try setting lower ttl values for the opcode cache, so it always gets flushed first?
It turned out, that my question is irrelevant. Due to the cache fragmentation problems, the cache is quickly marked as full. (see other posts on stackoverflow). As a consequence apc user cache should only be used selectively.
Related
I'm struggling to find any logic in the way APC clears up its old entries.
Especially with user entries, I notice the fragmentation quickly increasing to unexpected levels.
Although apc.ttl and apc.user_ttl are set at 3600, I notice a LOT of old entries in the APC cache which are not getting cleared. I have repeatedly increased the size of the memory for APC, but it only makes it last a little longer before reaching 100% fragmentation.
So, why could this be happening? Actually I would consider this a bug in APC, it's just not normal behaviour. I would expect some APC process to clear out old entries from time to time.
Also, could it lead to the conclusion that it would be better to use another caching system for PHP, and only use APC as an opcode cache (where the lack of cleaning is less of a problem)?
If you want to ensure old values are removed. You need to explicitly remove it.
I use a combination of APC and memcached in my applications. APC provides opcode caching as well as caches local data that is likely to remain unchanged (Like config files). All other objects are cached in memcached using some sort of a read through logic. (Gets cleared if data is changed)
When I do a new release, I restart APC (via apache or php if its a separate process). That will clear the APC cache and the config files will get reloaded shortly afterwards. I will restart memcached if the objects have changed significantly enough to cause issues if its loading old cache data.
APC is showing 100% fragmentation. Is this bad?
Does it mean that it's not helping at all? What paths do I go down to improve situation?
Thanks in advance.
In my experience, yes. I had a system where APC was showing 100% fragmentation, and performance was bad. I increased APC's memory limit (to 200 MB in my case -- but we had a lot of code) enough to give it some slack room. Fragmentation dropped to zero, and IIRC, CPU usage on the server dropped by 50%.
Also, make sure you're using the apc.php script that comes with APC to monitor fragmentation/utilization. We've even written a nagios check to watch APC, 'cause we have enough traffic that apache locks up entirely when APC fills up.
Moral of the story: give APC enough memory, and monitor utilization.
Fragmentation means that apc often throws out items from it's cache and adds new ones and has trouble finding large enough continous blocks.
There are two main ways to improve performance then
Give more memory to APC. Ideally APC can store your complete scripts in memory.
use apc.filter in php.ini to filter out files which aren't requested often or change often.
Also using apc_store() with a short time to live is bad, as is overwriting using apc_store() often.
[...] Fragmentation is what hurts performance, not the size of memory per se. But it also seems that fragmentation happens when memory is low [...]
Note also that there seems to be a bug with apc.php's graph: http://pecl.php.net/bugs/bug.php?id=13146
I have a single server site thats pushing 200k unqiues per day, and the traffic doubles roughly every 40 days (for the last 5 months anyway).
I pretty much only plan to cache the output of mysql_query functions for an hour or so. If cache is older than that, run query, put result back into the cache for another hour.
My mysql DB is about 200mb in size (grows by maybe 10-20mb/month).
Im doing a lot of file caching by writing HTML outputs and using them for a few minutes, and then regenerating the html.
Unfortunately, since its a database site, that allows for many sorting, searching and ordering methods, as well as pagination.... there are over 150,000 cached pages. Im also not caching the search queries, which cause most of the load.
I'd like to implement a caching system, and I wanted to know which one is faster. Would love to see some benchmarks.
A quick Googling says that APC is 5 times faster than Memcached.
My experience say that APC is nearly 7-8 times faster than Memcached.. but, memchached can be accessed by different services (for example, if you run mainly on apache and delegates some traffic, e.g. static contents like images or pure html, to another web-service, like lighttpd), that can be really usefull, if not indispensable.
APC have less feature than memcached and is easly to use and optimize, but this depends on your needs.
Like you mentioned there are a few different aspects of caching. I probably would focus on the following aspects of caching in your php app:
opcode caching which caches the compiled bytecode of php scripts. You can see a benchmark here (albeit an older article): http://itst.net/654-php-on-fire-three-opcode-caches-compared
Note: I strongly recommend using opcode caching.
Caching user data - APC and others do this. This would be your reference data or data that is fairly static and doesn't change often. You can clear the cache every day or trigger a clean cache when this reference data changes. This is also strongly recommended since typically reference data is used frequently and doesn't change often.
Caching sql queries - I know that Zend makes this task easy with a simple setup. Since these queries don't change this is another obvious one (like you mentioned)
Additional (if possible):
caching html pages - obviously caching a static page is faster than a generated one and typically this is hard to do since most pages in apps are so dynamic. Worth it if you can do it although if your queries are cached and your SQL is simple I wouldn't focus on this.
caching sql results - personally I stay away from this. I'll let the database do its work and what it does best since the DBMS typically has caching. I may cache the results for the thread of execution (i.e., I just retrieved this so don't do it again) but I don't go much beyond that.
I've used APC and eAccelerator successfully (I personally like to work with APC and it supposed opcode caching and user data caching for my reference data and sql queries). Use XDebug to profile your code.
You want to compare APC key-value store vs Memcache right? Because APC also does opcode cache, which is a different thing.
Well, on a single machine, APC k-v cache is way faster than memcache. Memcache has more functionality, but is intended for distributed environments, while APC works on single servers only.
I did a benchmark recently to set and then get 1 million keys in both, each key was a sequential integer, and the values were a 32byte string.
Over localhost, memcache could retrieve 12k keys/second in a single thread. APC returned 90K/second. However, if you use multi-threads or "multi_get" with memcache, it gets very close to APC performance.
The benchmark ran on a 1GB vps at slicehost.
in my case apc is 59 times faster than memcache
<?php
ini_set('apc.enable_cli','1'); //if u run in cli you may need to do changes in php.ini
error_reporting(E_ALL);
$mem=new Memcache();
$mem->connect('127.0.0.1',11211);
$mem->replace('testin','something');
$i=0;
$time=time()+microtime();
apc_store ( 'testin','something');
$num=1000000;
while($i<$num){
$mem->get('testin');
$i++;
}
echo "memcache took: ",time()+microtime()-$time," for 1 million gets","\n";
$time=time()+microtime();
$i=0;
print_r(apc_fetch('testin'));
while($i<$num) {
apc_fetch('testin');
$i++;
}
echo "apc took: ",time()+microtime()-$time,"for 1 million gets \n";
here is the output
memcache took: 37.657398939133 for 1 million gets
somethingapc took: 0.64599800109863for 1 million gets
It's almost impossible to accurately predict which would be faster. I would run tests with both in a development environment with similar data.
When performance is of importance, always use a profiler.
Im use IPB 3.1.4 with APC it works justy two times faster then without it.
Requests per second: 43.46 [#/sec] (mean)
Requests per second: 24.23 [#/sec] (mean)
Don't test IPB with memcached yet
I have websites developed in PHP. Im using Opcode cache.
But because Opcode cache like eAccelerator or APC is cached in RAM, I needs too much RAM.
So Im looking for any project or technique which cache the PHP Opcode in hard disk.
Thanks so much
(my website is not generate money, so Im thinking about cheaper solution)
all op-code caches allow you to configure the maximum size of shared memory used (look for a configuration option with shm - for SHared Memory - in the name, eg. apc.shm_size). so you can control that they don't use too much ram.
some caches also allow you to cache on disk instead/besides of caching in ram:
eAccelerator
the question is if a small amount of shared memory or a disk only cache gains you anything in performance compared to plain php without op-code cache. as always when using a cache, you should benchmark this.
Can anyone tell me what is the memory usage overhead associated with PHP opcode cache?
I've seen a lot of reviews of opcode cache but all of them only concentrate on the performance increase. I have a small entry level VPS and memory limits are a concern for me.
Most of the memory overhead will come from the opcode cache size. Each opcode cacher has their own default(e.g. 30MB for APC) which you can change through the config file.
Other than the cache size, the actual memory overhead of the cacher itself is negligible.
In todays world: It's neglectible. I think memory consumption was about 50 MB bigger with eAccelerator then it was without when I did my benchmarks.
If you really need the speed but do have headaches that your RAM might be not enough: grab $40 and buy another GIG of RAM for your server ;)
You can set a limit to memory consumption for APC, but that potentially limits its effectiveness.
If you're just using it for silent opcode caching, then it should be fine. Once the memory allotment is full, no new files will be cached, but everything will work as expected. However, the user-space cache functions like apc_store() and apc_fetch() will fail silently and inexplicably if there is no memory available.
This can be tricky to catch and debug since no error is reported and no exception is thrown.