I am not sure if this is possible. I was storing some information in a memcache server. The memcache entry i was storing was suppossed to have an expiry of 30 minutes. During that 30 minutes i could update the value of that memcache entry reference by the same key. But when i update the value i do not want to change the expire time. For example:
Key is created and set to expire in 30 minutes
10 minutes goes by and the value of the key is requested and we change the value
i replace the value of they key using memcache replace (i do not provide a new expire time because it is optional), i want the expire time to be 30-10 = 20 because the key was created 10 minutes ago and was set to expire in 30 minutes.
Now since i did not set an expire time it defaults to 0 and the key will never expire.
Now, is there a way of setting items in memcache, setting an expire time, and then getting/replacing the item while keeping the expire time to x minutes after i set the item in cache?
I might possible be able to use unix timestamps instead of seconds to expire when setting into memcache, and also storing that timestamp in memcache and when i set it back into memcache i would just set it to the same timestamp stored in the value. Or is there a better way of doing this?
BTW I an using memcache and not memcached.
I know this question is old, but I thought I'd add a caution to gprime's solution.
It sounds like gprime's "little ugly hack" is to store the expiration as a separate value in memcache. The problem is, memcache may end up purging the expiry value while it is still needed. This can happen even when the memory allocated to memcached is not full.
(See http://sparklewise.com/?p=506 for further explanation.)
This could be a problem if your code doesn't account for the possibility that the previously-stored expiry is gone. Even if you do account for that, you could end up with values living longer than expected in the cache.
It's probably not a huge deal in 99.999% of the cases, but it's one of those gotchas that will cause massive hair-pulling and head-scratching when it does happen. Hopefully this post will help someone avoid that pain. :-)
Essentially Memcache does exactally what you want it to. It does its job very well, getting and setting values... I think the answer your looking for is outside of the default functionality of memcache. I suppose you can put more control on your codebase to check a timestamp that you store with your blob and use that to set expire times for future updates?
I don't know what your using for your non-memcache consistent storage, but I would store an expiry date in that-- then use that value to update your memcache.
Related
I used to clear cash using flush() in PHP every day, like 5-10 times.
I want to clear specific items cache to prevent caching all server cache.
This is the right way:
Use Memcache::delete() to delete item
Memcache::add() the same item
Is it correct?
https://www.php.net/manual/en/book.memcache.php
Yes, that is correct.
But be aware that Memcache::add() will fail if the key already exists.
If you always want to write the data even though it exists already, you can use Memcache::set() instead.
Another a little bit funky thing with the memcache PHP class, is that TTL is measured in seconds, but if it is larger than 30 days it instead is interpreted as a date stamp (unix timestamp).
Is it possible to modify the ttl (timeout) of an APC entry?
For example if I do
apc_store($cache_key, $productInfo, 100);
but within 100 seconds, I want to increase to 200. I want it to be 200 seconds from the original creation date, while maintaining a hit count.
My assumption is that this is possible because there is a last_modified time in the APC cache viewer, but I only know of apc_store.
I don't want to overwrite the entry with another apc_store as this will reset the hit count and creation date. So at time 0 if I did TTL of 100, at time 33 I would have to now make the TTL be a 167 if I wanted it to expire at time 200 (which is what going from 100 to 200 would do), this requires looking up a creation date and overwriting the data (not needed).
Looking for a solution that avoids those issues.
This is not possible unless you store the TTL as part of the data in the key and write your own logic, you will still have to overwrite the entry each time though.
Consider using Memcached instead and you can use touch to achieve this.
I am using CI's sessions in connection with a database. So all of our sessions are in this ci_sessions table on our database and it can get a lot of rows, considering that the session_id keep changing every 5 minutes.
Do we need to empty the table, say every one a month / week maybe?
While what #Marc-Audet said is true, if you take a look at the code, you can see it is a really lousy way to clean up sessions.
The constructor calls the _sess_gc function every time it is initiated. So, basically each request to your server if you have it autoloaded.
Then, it generates a random number below 100 and sees if that's below a certain value (by default it is 5). If this condition is met, then it will remove any rows on the session table with last_activity value less than current time minus your session expiration.
While this works for most cases, it is technically possible that (if the world is truly random) the random number generator does not generate a number below 5 for a long time, in which case, your sessions will not be cleaned up.
Also, if you have your session expiry time set to a long time (if you set to 0, CI will set it to 2 years) then those rows are not going to get deleted anyway. And if your site is good enough to get a decent amount of visitors, your DBA will be pointing fingers at the session table some time soon :)
It works for most cases - but I would not call it a proper solution. Their session id regeneration really should have been built to remove the records pertaining to the previous ids and the garbage collection really should not be left to a random number - in theory, it is possible that the required number is not generated as frequently as you wished.
In our case, I have removed the session garbage collection from the session library and I manually take care of it once a day (with a cron job .. and a reasonable session expiration time). This reduces the number of unnecessary hits to the DB and also does not leave a massive table in the DB. It is still a big table, but lot smaller than what it used to be.
Given the fact that the OP question doesn't have a CodeIgniter 2 tag, I'll answer how to deal with sessions cleanup when the database keeps growing for CodeIgniter 3.
Issue:
When you set (in the config.php file) sess_expiration key too high (let's say 1 year) and sess_time_to_update key low (let's say 5 min), the session table will keep growing as the users browse though your website, until sessions rows will expire and will be garbage collected (which is 1 year).
Solution:
Setting sess_regenerate_destroy key to TRUE (default set to FALSE) will delete an old session when it will regenerate itself with the new id, thus cleaning your table automatically.
No, CodeIgniter cleans up after itself...
Note
According to the CodeIgniter documentation:
The Session class has built-in garbage collection which clears out expired sessions so you do not need to write your own routine to do it.
CodeIgniter's Session Class probably checks the session table and cleans up expired entries. However, the documentation does not say when the clean up happens. Since there are no cron jobs as part of CodeIgniter, the clean up must occur when the Session class is invoked. I suppose if the site remains idle forever, the session table will never be cleared. But, this would be an unusual case.
CodeIgniter implements the SessionHandlerInterface (see the docs for the custom driver).
CodeIgniter defines a garbage collector method named gc() for each driver (database, file, redis, etc) or you can define your custom gc() for your custom driver.
The gc() method is passed to PHP with the session_set_save_handler() function, therefore the garbage collector is called internally by PHP based on session.gc_divisor, session.gc_probability settings.
For example, with the following settings:
session.gc_probability = 1
session.gc_divisor = 100
There is a 1% chance that the garbage collector process starts on each request.
So, you do not need to clean the session table if your settings are properly set.
When you call:
$this->session->sess_destroy();
It deletes the information in database by itself.
Since PHP7, the GC-based method is disabled by default, as per the documentation at https://www.php.net/manual/en/function.session-gc.php Stumbled upon this because a legacy application suddenly stopped working, reaching a system limitation since sessions are never ever cleaned up. A cronjob to clean up the sessions would be a good idea...
It is always good practice to clear the table. Otherwise, if your querying the session data for say creating reports or something, it will be slow and unreliable. Nevertheless, given the performance of mysql, yes do so.
I have implemented memcache in my PHP-MySQL based app and it gets updated regularly from a backend process.
Due to this some data is conflicting with the expiration time and other backend processes, so I came up with a solution but for that I would have to make the TTL = infinite.
Easy - just write 0 there.
expire
Expiration time of the item. If it's
equal to zero, the item will never
expire. You can also use Unix
timestamp or a number of seconds
starting from current time, but in the
latter case the number of seconds may
not exceed 2592000 (30 days).
You can set the TTL to 0, which means that it should 'never' expire.
But remember that it will never really be infinite. The data is stored in memory and will be lost under some circumstances, the most obvious being the server being rebooted. :)
You should always have the possibility to reconstruct that data when the memcache fails.
More details to be found here.
As far as I know, if you don't set a ttl, it will never expire.
However, there are replacing policies for keys, about which you can read here
I'm caching tweets on my site (with 30 min expiration time). When the cache is empty, the first user to find out will repopulate it.
However, at that time the Twitter API may return a 200. In that case I'd like to prolong the previous data for another 30 mins. But the previous data will already be lost.
So instead I'd like to look into repopulating the cache, say, 5 minutes before expiration time so that I don't lose any date.
So how do I know the expiration time of an item when using php's memcache::get()?
Also, is there a better way of doing this?
In that case, isn't this the better logic?
If the cache is older than 30 minutes, attempt to pull from Twitter
If new data was successfully retrieved, overwrite the cache
Cache data for an indefinite amount of time (or much longer than you intend to cache anyway)
Note the last time the cache was updated (current time) in a separate key
Rinse, repeat
The point being, only replace the data with something new if you have it, don't let the old data be thrown away automatically.
don't store critical data in memcached. it guarantees nothing.
if you always need to get "latest good" cache - you need to store data at any persistent storage, such as database or flat file.
in this case if nothing found in cache - you do twitter api request. if it fails - you read data from persistent. and on another http request you will make same iteration one more time.
or you can put data from persistent into memcache with pretty shor lifetime. few minutes for example (1-5) to let twitter servers time to get healthy. and after it expired - repeat the request.
When you are putting your data into memcache - you are setting also how long the cache is valid. So theoretically you could also put the time when cache was created and/or when cache will expire. Later after fetching from cache you can always validate how much time left till cache will expire and decide what you want to do.
But letting cache to be repopulated on user visit can be still risky at some point - lets say if you would like to repopulate cache when it reaches ~5 min before expiration time - and suddenly there would be no visitors coming in last 6 minutes before cache expires - then cache will still expire and no one will cause it to be repopulated. If you want to be always sure that cache entry exists - you need to do checks periodically - for example - making a cronjob which does cache checks and fill-ups.