Time to time, I receive this strange warning message. It is usually gone on page reload. What does that mean. I googled but to no avail.
Warning: include(): GC cache entry '/.../...class.php' (dev=2049 ino=37120489) was on gc-list for 3840 seconds in /.../...class.php on line 111
Definitely this issue goes from APC, source code from package apc-3.1.6-r1. When item is inserted into user cache or file cache, this function is called.
static void process_pending_removals(apc_cache_t* cache TSRMLS_DC)
{
slot_t** slot;
time_t now;
/* This function scans the list of removed cache entries and deletes any
* entry whose reference count is zero (indicating that it is no longer
* being executed) or that has been on the pending list for more than
* cache->gc_ttl seconds (we issue a warning in the latter case).
*/
if (!cache->header->deleted_list)
return;
slot = &cache->header->deleted_list;
now = time(0);
while (*slot != NULL) {
int gc_sec = cache->gc_ttl ? (now - (*slot)->deletion_time) : 0;
if ((*slot)->value->ref_count <= 0 || gc_sec > cache->gc_ttl) {
slot_t* dead = *slot;
if (dead->value->ref_count > 0) {
switch(dead->value->type) {
case APC_CACHE_ENTRY_FILE:
apc_warning("GC cache entry '%s' (dev=%d ino=%d) was on gc-list for %d seconds" TSRMLS_CC,
dead->value->data.file.filename, dead->key.data.file.device, dead->key.data.file.inode, gc_sec);
break;
case APC_CACHE_ENTRY_USER:
apc_warning("GC cache entry '%s'was on gc-list for %d seconds" TSRMLS_CC, dead->value->data.user.info, gc_sec);
break;
}
}
*slot = dead->next;
free_slot(dead TSRMLS_CC);
}
else {
slot = &(*slot)->next;
}
}
}
From APC configuration ( http://cz.php.net/manual/en/apc.configuration.php#ini.apc.gc-ttl )
apc.gc_ttl integer
The number of seconds that a cache entry may remain on the garbage-collection list. This value provides a fail-safe in the event that a server process dies while executing a cached source file; if that source file is modified, the memory allocated for the old version will not be reclaimed until this TTL reached. Set to zero to disable this feature.
We get messages "GC cache entry '%s' (dev=%d ino=%d) was on gc-list for %d seconds" or "GC cache entry '%s'was on gc-list for %d seconds" in this condition:
(gc_sec > cache->gc_ttl) && (dead->value->ref_count > 0)
First condition means, item was deleted later then apc.gc_ttl seconds ago and its still in garbage collector list. Seconds condition means, item is still referenced.
e.g. when process unexpectedly died, reference is not decreased. First apc.ttl seconds is active in APC cache, then is deleted (there isn't next hit on this item). Now item is on garbage collector list (GC) and apc.gc_ttl timeout is running. When apc.gc_ttl is less then (now - item_deletion_time), warning is written and item is completely flushed.
Try to check your logs (web server, php, system/kernel) to critical errors, e.g. php, web server segfault.
Related
I have a couple of functions used to prevent concurrent execution of some PHP scripts that are being run via crontab on a debian 10.13. Essentially, if an instance of that script is already running, the new instance is supposed to quit.
The problem is that every now and then (about 2-3 times per day for a script that runs every 5 minutes so perhaps in 1% of the cases), the semaphore handle seems to be gone at the time the script attempts to release it. This happens even though the script takes less than 30s to run, typically 10-15s.
The setup is as follows, in the script, right after including the library:
if (!singleInstanceHelper()) {
die("The script ".__file__." is already running.\n");
}
In the included library:
function singleInstanceHelper($wait = false, $suffix = '') {
global $semdebug;
$bt = debug_backtrace();
$calling_file = $bt[count($bt)-1]['file'];
$semaphore_key = crc32($calling_file.$suffix); // unique integer key for this semaphore
$semaphore_max = 1; // The number of processes that can acquire this semaphore
$semaphore_permissions = 0666; // Unix style permissions for this semaphore
$semaphore_autorelease = 1; // Auto release the semaphore if the request shuts down
// open a new or get an existing semaphore
$semaphore_handle = sem_get($semaphore_key, $semaphore_max, $semaphore_permissions, $semaphore_autorelease);
$semdebug = "semaphore key $semaphore_key handle ".print_r($semaphore_handle, true);
if(!$semaphore_handle)
{
die("Failed to get semaphore - sem_get().\nCalling script: $calling_file.\n");
}
// acquire exclusive control
if (sem_acquire($semaphore_handle, !$wait)) {
register_shutdown_function('semaphoreRelease', $semaphore_handle);
return true;
}
return false;
}
function semaphoreRelease($semaphore_handle) {
global $semdebug;
if (!sem_release($semaphore_handle)) {
print $semdebug."\n";
system('ipcs -s');
}
}
This is what crond mails me when the failure occurs (the script runs as user, line 1790 is sem_release):
PHP Warning: sem_release(): failed to release key 0x9efb366f: Invalid argument in /path/to/lib.php on line 1790
semaphore key 2667263599 handle Resource id #25
------ Semaphore Arrays --------
key semid owner perms nsems
0x00000000 957349888 user 600 1
0x00000000 1083801604 user 600 1
0x00000000 1083834373 user 600 1
0x00000000 1083867142 user 600 1
0x00000000 1083899911 user 600 1
0x00000000 1083932680 user 600 1
0x1a48fa0e 455868425 root 666 3
0x474b7f2f 470319114 root 666 3
0x30614a01 1002373236 root 666 3
Any idea what might be causing this? Or, perhaps, any suggestions on how to further debug it or how to modify the code so that it no longer happens?
I'm using zend_shm_cache functions as a fast storage of variables. My problem is that the cache doesn't seem to get cleared after TTL is over. Example:
zend_shm_cache_store( $key = 'test3', 'value', 2 );
foreach( range(1,5) as $timer ){
sleep( 1 );
echo $timer.' - ' .zend_shm_cache_fetch( $key ).'<br/>';
}
Returns:
1 - value
2 -
3 - value
4 - value
5 - value
I would expect the cache to be empty after second second. Can you explain what's happening or propose a solution?
I got the answer in the documentation of APCU:
After the ttl has passed, the stored variable will be expunged from
the cache (on the next request).
As my code runs in only one request the cache is never deleted even when its time has expired.
I'm using PHP AWS SDK to communicate with CloudSearch. According to this post, pagination can be done with either cursor or start parameters. But when you have more than 10,000 hits, you can't use start.
When using start, I can specify ['start' => 1000, 'size' => 100] to get directly to 10th page.
How to get to 1000th page (or any other random page) using cursor? Maybe there is any way to calculate this parameter?
I would LOVE there to be a better way but here goes...
One thing I've discovered with cursors is that they return the same value for duplicate search requests when seeking on the same data set, so don't think of them as sessions. Whilst your data isn't updating you can effectively cache aspects of your pagination for multiple users to consume.
I've came up with this solution and have tested it with 75,000+ records.
1) Determine if your start is going to be under the 10k Limit, if so use the non-cursor search, otherwise when seeking past 10K, first perform a search with an initial cursor and a size of 10K and return _no_fields. This gives is our starting offset and the no fields speeds up how much data we have to consume, we don't need these ID's anyway
2) Figure out your target offset, and plan how many iterations it will take to position the cursor just before your targeted page of results. I then iterate and cache the results using my request as the cache hash.
For my iteration I started with a 10K blocks then reduce the size to 5k then 1k blocks as I start getting "closer" to the target offset, this means subsequent pagination are using a previous cursor that's a bit closer to the last chunk.
eg what this might look like is:
Fetch 10000 Records (initial cursor)
Fetch 5000 Records
Fetch 5000 Records
Fetch 5000 Records
Fetch 5000 Records
Fetch 1000 Records
Fetch 1000 Records
This will help me to get to the block that's around the 32,000 offset mark. If I then need to get to 33,000 I can used my cached results to get the cursor that will have returned the previous 1000 and start again from that offset...
Fetch 10000 Records (cached)
Fetch 5000 Records (cached)
Fetch 5000 Records (cached)
Fetch 5000 Records (cached)
Fetch 5000 Records (cached)
Fetch 1000 Records (cached)
Fetch 1000 Records (cached)
Fetch 1000 Records (works using cached cursor)
3) now that we're in the "neighborhood" of your target result offset you can start specifying page sizes to just before your destination. and then you perform the final search to get your actual page of results.
4) If you add or delete documents from your index you will need a mechanism for invalidating your previous cached results. I've done this by storing a time stamp of when the index was last updated and using that as part of the cache key generation routine.
What is important is the cache aspect, you should build a cache mechanism that uses the request array as your cache hash key so it can be easily created/referenced.
For a non-seeded cache this approach is SLOW but if you can warm up the cache and only expire it when there's a change to the indexed documents (and then warm it up again), your users will be unable to tell.
This code idea works on 20 items per page, I'd love to work on this and see how I could code it smarter/more efficient, but the concept is there...
// Build $request here and set $request['start'] to be the offset you want to reach
// Craft getCache() and setCache() functions or methods for cache handling.
// have $cloudSearchClient as your client
if(isset($request['start']) === true and $request['start'] >= 10000)
{
$originalRequest = $request;
$cursorSeekTarget = $request['start'];
$cursorSeekAmount = 10000; // first one should be 10K since there's no pagination under this
$cursorSeekOffset = 0;
$request['return'] = '_no_fields';
$request['cursor'] = 'initial';
unset($request['start'],$request['facet']);
// While there is outstanding work to be done...
while( $cursorSeekAmount > 0 )
{
$request['size'] = $cursorSeekAmount;
// first hit the local cache
if(empty($result = getCache($request)) === true)
{
$result = $cloudSearchClient->Search($request);
// store the results in the cache
setCache($request,$result);
}
if(empty($result) === false and empty( $hits = $result->get('hits') ) === false and empty( $hits['hit'] ) === false )
{
// prepare the next request with the cursor
$request['cursor'] = $hits['cursor'];
}
$cursorSeekOffset = $cursorSeekOffset + $request['size'];
if($cursorSeekOffset >= $cursorSeekTarget)
{
$cursorSeekAmount = 0; // Finished, no more work
}
// the first request needs to get 10k, but after than only get 5K
elseif($cursorSeekAmount >= 10000 and ($cursorSeekTarget - $cursorSeekOffset) > 5000)
{
$cursorSeekAmount = 5000;
}
elseif(($cursorSeekOffset + $cursorSeekAmount) > $cursorSeekTarget)
{
$cursorSeekAmount = $cursorSeekTarget - $cursorSeekOffset;
// if we still need to seek more than 5K records, limit it back again to 5K
if($cursorSeekAmount > 5000)
{
$cursorSeekAmount = 5000;
}
// if we still need to seek more than 1K records, limit it back again to 1K
elseif($cursorSeekAmount > 1000)
{
$cursorSeekAmount = 1000;
}
}
}
// Restore aspects of the original request (the actual 20 items)
$request['size'] = 20;
$request['facet'] = $originalRequest['facet'];
unset($request['return']); // get the default returns
if(empty($result = getCache($request)) === true)
{
$result = $cloudSearchClient->Search($request);
setCache($request,$result);
}
}
else
{
// No cursor required
$result = $cloudSearchClient->Search( $request );
}
Please note this was done using a custom AWS client and not the official SDK class, but the request and search structures should be comparable.
I see 5 output each time when I run this code:
<?php
$v = 5;
apc_store('vwxyz',$v,3);
$before = apc_fetch('vwxyz');
sleep(5);
$after = apc_fetch('vwxyz'); //should be false
echo $before;
echo "<br>";
echo $after;
$later = apc_fetch('vwxyz'); //OK this should definitely be false
echo "<br>";
echo $later;
Shouldn't the cached entry be cleared from the cache and return false to apc_fetch()? The user_ttl setting is 2 for APC. I'm still trying to figure out what user_ttl does (the documentation is quite cryptic).
From the manual:
Time To Live; store var in the cache for ttl seconds. After the ttl
has passed, the stored variable will be expunged from the cache (on
the next request). If no ttl is supplied (or if the ttl is 0), the
value will persist until it is removed from the cache manually, or
otherwise fails to exist in the cache (clear, restart, etc.).
So it says that the item gets removed from the cache after the TTL on the next request. So the item isn't removed from cache until your next request which is why you keep getting 5.
I have a fairly expensive server call that I need to cache for 30 seconds. It seems however that I can not get the cache to expire.
In the code below, after the first time it caches, it will never get past $return->cache_data, even after the time() + 30 seconds.
Note, I can even print $cache->expire and it is definitely set to a time past 30 seconds ago and never updates.
I've manually cleared cache many times to confirm I get the same results.
Does anything look wrong with this?
function mymodule_get_something($id) {
// set the unique cache id
$cid = 'id-'. $id;
// return data if there's an un-expired cache entry
// *** $cache ALWAYS gets populated with my expired data
if ($cache = cache_get($cid, 'cache_mymodule')) {
return $cache->data;
}
// set my super expensive data call here
$something = array('Double Decker Taco', 'Burrito Supreme');
// set the cache to expire in 30 seconds
cache_set($cid, $something, 'cache_mymodule', time() + 30);
// return my data
return $something;
}
There's nothing wrong with your code as such, I think the problem is in how cache_set behaves. From the docs page, passing a UNIX timestamp:
Indicates that the item should be kept at least until the given time, after which it behaves like CACHE_TEMPORARY.
CACHE_TEMPORARY behaves like this:
Indicates that the item should be removed at the next general cache wipe.
My best guess is that because you're not implicitly forcing that general cache wipe (using cache_clear_all()) the cache object will persist.
I think a simple way around it would just be to manually test the expiry time after your cache check, and let it fall through to re-setting that cache object if it has expired:
if ($cache = cache_get($cid, 'cache_mymodule')) {
if ($cache->expire > REQUEST_TIME) {
return $cache->data;
}
}