Is it possible to modify the ttl (timeout) of an APC entry?
For example if I do
apc_store($cache_key, $productInfo, 100);
but within 100 seconds, I want to increase to 200. I want it to be 200 seconds from the original creation date, while maintaining a hit count.
My assumption is that this is possible because there is a last_modified time in the APC cache viewer, but I only know of apc_store.
I don't want to overwrite the entry with another apc_store as this will reset the hit count and creation date. So at time 0 if I did TTL of 100, at time 33 I would have to now make the TTL be a 167 if I wanted it to expire at time 200 (which is what going from 100 to 200 would do), this requires looking up a creation date and overwriting the data (not needed).
Looking for a solution that avoids those issues.
This is not possible unless you store the TTL as part of the data in the key and write your own logic, you will still have to overwrite the entry each time though.
Consider using Memcached instead and you can use touch to achieve this.
Related
From the memcached wiki:
When the table is full, subsequent inserts cause older data to be purged in least recently used (LRU) order.
I have the following questions:
Which data will be purged? The one which is older by insertion, or the one which is least recently used? I mean if recently accessed data is d1 which is oldest by insertion and the cache is full while replacing data will it replace d1?
I am using PHP for interacting with memcached. Can I have control over how data is replaced in memcached? Like I do not want some of my data to get replaced until it expires even if the cache is full. This data should not be replaced instead other data can be removed for insertion.
When data is expired is it removed immediately?
What is the impact of the number of keys stored on memcached performance?
What is the significance of -k option in memcached.conf? I am not able to understand what "lock down all paged memory" means. Also, the description in README is not sufficient.
When memcached needs to store new data in memory, and the memory is already full, what happen is this:
memcached searches for a a suitable* expired entry, and if one is found, it replaces the data in that entry. Which answers point 3) data is not removed immediately, but when new data should be set, space is reallocated
if no expired entry is found, the one that is least recently used is replaced
*Keep in mind how memcached deals with memory: it allocates blocks of different sizes, so the size of the data you are going to set in the cache plays role in deciding which entry is removed. The entries are 2K, 4K, 8K, 16K... etc up to 1M in size.
All this information can be found in the documentation, so just read in carefully. As #deceze says, memcached does not guarantee that the data will be available in memory and you have to be prepared for a cache miss storm. One interesting approach to avoid a miss storm is to set the expiration time with some random offset, say 10 + [0..10] minutes, which means some items will be stored for 10, and other for 20 minutes (the goal is that not all of items expire at the same time).
And if you want to preserve something in the cache, you have to do two things:
a warm-up script, that asks cache to load the data. So it is always recently used
2 expiration times for the item: one real expiration time, let's say in 30 minutes; another - cached along with the item - logical expiration time, let's say in 10 minutes. When you retrieve the data from the cache, you check the logical expiration time and if it is expired - reload data and set it in the cache for another 30 minutes. In this way you'll never hit the real cache expiration time, and the data will be periodically refreshed.
5) What is the significance of -k option in "memcached.conf". I am not
able to understand what does "Lock down all paged memory" means. Also
description in README is also not sufficient.
No matter how much memory you will allocate for memcached, it will use only the amount it needs, e.g. it allocates only the memory actually used. With the -k option however, the entire memory is reserved when memcached is started, so it always allocates the whole amount of memory, no matter if it needs it or not
I am using the Yii framework for a site that will get a lot of hits (hopefully) because each client will be polling every 250 milliseconds. I want to limit the hits to my database and cache the data, but it needs to be close to real time (about 250 milliseconds). I noticed that the $expire parameter in CCache->set() takes an integer for seconds. I tried to go directly to apc_store() and found the same issue. Memcache seems to also define TTL in seconds.
Does anyone know of a PHP data cache that can work with TTL values less than 1 second or another workaround?
Redis has sub-second expiration. See https://github.com/antirez/redis/issues/169.
I have implemented memcache in my PHP-MySQL based app and it gets updated regularly from a backend process.
Due to this some data is conflicting with the expiration time and other backend processes, so I came up with a solution but for that I would have to make the TTL = infinite.
Easy - just write 0 there.
expire
Expiration time of the item. If it's
equal to zero, the item will never
expire. You can also use Unix
timestamp or a number of seconds
starting from current time, but in the
latter case the number of seconds may
not exceed 2592000 (30 days).
You can set the TTL to 0, which means that it should 'never' expire.
But remember that it will never really be infinite. The data is stored in memory and will be lost under some circumstances, the most obvious being the server being rebooted. :)
You should always have the possibility to reconstruct that data when the memcache fails.
More details to be found here.
As far as I know, if you don't set a ttl, it will never expire.
However, there are replacing policies for keys, about which you can read here
I am not sure if this is possible. I was storing some information in a memcache server. The memcache entry i was storing was suppossed to have an expiry of 30 minutes. During that 30 minutes i could update the value of that memcache entry reference by the same key. But when i update the value i do not want to change the expire time. For example:
Key is created and set to expire in 30 minutes
10 minutes goes by and the value of the key is requested and we change the value
i replace the value of they key using memcache replace (i do not provide a new expire time because it is optional), i want the expire time to be 30-10 = 20 because the key was created 10 minutes ago and was set to expire in 30 minutes.
Now since i did not set an expire time it defaults to 0 and the key will never expire.
Now, is there a way of setting items in memcache, setting an expire time, and then getting/replacing the item while keeping the expire time to x minutes after i set the item in cache?
I might possible be able to use unix timestamps instead of seconds to expire when setting into memcache, and also storing that timestamp in memcache and when i set it back into memcache i would just set it to the same timestamp stored in the value. Or is there a better way of doing this?
BTW I an using memcache and not memcached.
I know this question is old, but I thought I'd add a caution to gprime's solution.
It sounds like gprime's "little ugly hack" is to store the expiration as a separate value in memcache. The problem is, memcache may end up purging the expiry value while it is still needed. This can happen even when the memory allocated to memcached is not full.
(See http://sparklewise.com/?p=506 for further explanation.)
This could be a problem if your code doesn't account for the possibility that the previously-stored expiry is gone. Even if you do account for that, you could end up with values living longer than expected in the cache.
It's probably not a huge deal in 99.999% of the cases, but it's one of those gotchas that will cause massive hair-pulling and head-scratching when it does happen. Hopefully this post will help someone avoid that pain. :-)
Essentially Memcache does exactally what you want it to. It does its job very well, getting and setting values... I think the answer your looking for is outside of the default functionality of memcache. I suppose you can put more control on your codebase to check a timestamp that you store with your blob and use that to set expire times for future updates?
I don't know what your using for your non-memcache consistent storage, but I would store an expiry date in that-- then use that value to update your memcache.
I'm caching tweets on my site (with 30 min expiration time). When the cache is empty, the first user to find out will repopulate it.
However, at that time the Twitter API may return a 200. In that case I'd like to prolong the previous data for another 30 mins. But the previous data will already be lost.
So instead I'd like to look into repopulating the cache, say, 5 minutes before expiration time so that I don't lose any date.
So how do I know the expiration time of an item when using php's memcache::get()?
Also, is there a better way of doing this?
In that case, isn't this the better logic?
If the cache is older than 30 minutes, attempt to pull from Twitter
If new data was successfully retrieved, overwrite the cache
Cache data for an indefinite amount of time (or much longer than you intend to cache anyway)
Note the last time the cache was updated (current time) in a separate key
Rinse, repeat
The point being, only replace the data with something new if you have it, don't let the old data be thrown away automatically.
don't store critical data in memcached. it guarantees nothing.
if you always need to get "latest good" cache - you need to store data at any persistent storage, such as database or flat file.
in this case if nothing found in cache - you do twitter api request. if it fails - you read data from persistent. and on another http request you will make same iteration one more time.
or you can put data from persistent into memcache with pretty shor lifetime. few minutes for example (1-5) to let twitter servers time to get healthy. and after it expired - repeat the request.
When you are putting your data into memcache - you are setting also how long the cache is valid. So theoretically you could also put the time when cache was created and/or when cache will expire. Later after fetching from cache you can always validate how much time left till cache will expire and decide what you want to do.
But letting cache to be repopulated on user visit can be still risky at some point - lets say if you would like to repopulate cache when it reaches ~5 min before expiration time - and suddenly there would be no visitors coming in last 6 minutes before cache expires - then cache will still expire and no one will cause it to be repopulated. If you want to be always sure that cache entry exists - you need to do checks periodically - for example - making a cronjob which does cache checks and fill-ups.