Let's assume I have these variables saved in apc, memcached and eaccelerator:
article_1_0
article_1_1
article_3_2
article_3_3
article_2_4
How can I delete all cached variables that starts with article_3_ (they can reach up to 10000) ?
is there any way to list the cached variables ?
The slow solution
For APC:
$iterator = new APCIterator('user', '#^article_3_#', APC_ITER_KEY);
foreach($iterator as $entry_name) {
apc_delete($entry_name);
}
For eaccelerator:
foreach(eaccelerator_list_keys() as $name => $infos) {
if (preg_match('#^article_3_#', $name)) {
eaccelerator_rm($name);
}
}
For memcached, look at #rik's answer
The proper solution
The general solution for expiring multiple keys at once is to namespace them. For expiring them, you just have to change the namespace:
Say you have a group of keys "article_3_1", "article_3_2", .... You can store them like this:
$ns = apc_fetch('article_3_namespace');
apc_store($ns."_article_3_1", $value);
apc_store($ns."_article_3_2", $value);
Fetch them like this:
$ns = apc_fetch('article_3_namespace');
apc_fetch($ns."_article_3_1");
And expire them all by just incrementing the namespace:
apc_inc('article_3_namespace');
Although the docs say APCIterator is available in apc >= 3.1.1, I'm on several systems that claim to have apc 3.1.9, however there is no APCIterator present. If you don't have APCIterator at your disposal, give something like this a whirl:
$aCacheInfo = apc_cache_info('user');
foreach($aCacheInfo['cache_list'] as $_aCacheInfo)
if(strpos($_aCacheInfo['info'], 'key_prefix:') === 0)
apc_delete($_aCacheInfo['info']);
In this example we're checking for a prefix in the key, but you could use preg_match et. al and achieve something closer to what APCIterator provides.
There is a way to retrieve all keys from memcache but it's very expensive.
If there is possibility to use alternatives for memcached, scache supports structured keyspaces. With it you could store data to nested paths :
scache_shset($conn, 'article/1/0', $data10);
scache_shset($conn, 'article/3/0', $data30);
scache_shset($conn, 'article/3/1', $data31);
and eventually destroy data by deleting the parent node
scache_shunset($conn, 'article/3');
There is an APCIterator which helps you search through the keys in APC.
Instantiate the APCIterator.
APCIterator::valid() means that there are keys still to iterate trough. APCIterator::key() returns you the apc key. APCIterator::next() moves the iterator position to the next item.
// APC
$iterator = new APCIterator('user', '/^article_3_/');
while($iterator->valid()) {
apc_delete($iterator->key());
// You can view the info for this APC cache value and so on by using
// $iterator->current() which is array
$iterator->next();
}
For memcache you can use Memcached and use getAllKeys method
// Memcached
$m = new Memcached();
$m->addServer('mem1.domain.com', 11211);
$items = $m->getAllKeys();
foreach($items as $item) {
if(preg_match('#^article_3_#', $item)) {
$m->delete($item);
}
}
Related
I am able to connect to my elasticache cluster like so:
$awsElasticache = new ElastiCacheClient(CredentialProvider::atsDefaultConfigConstructor(false, false));
$clusterResult = $awsElasticache->describeCacheClusters(array('CacheClusterId'=>'my_cluster'));
When I print $clusterResult, I get info about the cluster, good.
But how can I actually interact with the endpoint to set key/value pairs?
I am trying this without success:
$this->mem = new Memcached();
$this->mem->addServer($this->endPoint,11211);
$this->mem->set('myKey','myValue',3600);
$result = $this->mem->get('myKey');
echo $result;
I get nothing printed from $result.
I am confused about which object to use to set and get key/value pairs.
To set key/value pair in Memcached, always extend the time of expiry from current time.
Try this
$this->mem = new Memcached();
$this->mem->addServer($this->endPoint,11211);
$expires = Carbon::now()->addMinutes(10);
$this->mem->set('myKey','myValue', $expires);
$result = $this->mem->get('myKey');
echo $result;
NOTE: For some reason, Memcached works best with Carbon time
See https://artisansweb.net/work-php-datetime-using-carbon/ on how to setup and use Carbon on your current project
I want to get the name of all keys for memcached, but Memcached::getAllKeys method always return false.
use contos 6.5 + memcached 1.4.31 + php-memcached-2.2.0 PECL
It looks like newer versions of memcached don't like nor support the getAllKeys method.
However it looks like someone made it work by setting
Memcached::OPT_BINARY_PROTOCOL = false
If it does not work I think you have to fall back to 1.4.23 version or install REDIS :P
I had the same problem on my live server. I was preparing a test to show the techs there how to replicate my problem:
$m = new Memcached();
$m->addServer(MEMCACHED_SERVER, MEMCACHED_PORT);
echo "added ". MEMCACHED_SERVER. ":". MEMCACHED_PORT. PHP_EOL;
$keys = [];
$stop = 100;
foreach( $m->getAllKeys() as $k){
array_push( $keys, $k );
if( --$stop == 0 ) break;
}
var_dump( $keys );
this would return 100 keys on my local R&D server, but an empty list on the live server. To show them there was definitely something in there I echoed a dump of a key in there that I knew for sure was there:
var_dump( $m->get( "cache:pool:70:230" ));
that line showed there was a key, but it also made getAllKeys to return a list of 100 entries more! I still believe this is a bug, but there is a workaround.
Edit: Turns out any redundant call before getallkeys fixes this: $m->getVersion(); would also make getAllKeys works
During deep researching about hash and zval structure and how arrays are based on it, faced with strange insert time.
Here is example:
$array = array();
$someValueToInsert = 100;
for ($i = 0; $i < 10000; ++$i) {
$time = microtime(true);
array_push($array, $someValueToInsert);
echo $i . " : " . (int)((microtime(true) - $time) * 100000000) . "</br>";
}
So, I found that every 1024, 2024, 4048... element will be inserted using much more time(>~x10).
It doesn't depends will I use array_push, array_unshift, or simply $array[] = someValueToInsert.
I'm thinking about that in Hash structure:
typedef struct _hashtable {
...
uint nNumOfElements;
...
} HashTable;
nNumOfElements has default max value, but it doesn't the answer why does it took more time to insert in special counters(1024, 2048...).
Any thoughts ?
While I would suggest double checking my answer on the PHP internals list, I believe the answer lay in zend_hash_do_resize(). When more elements are needed in the hash table, this function is called and the extant hash table is doubled in size. Since the table starts life at 1024, this doubling explains the results you've observed. Code:
} else if (ht->nTableSize < HT_MAX_SIZE) { /* Let's double the table size */
void *old_data = HT_GET_DATA_ADDR(ht);
Bucket *old_buckets = ht->arData;
HANDLE_BLOCK_INTERRUPTIONS();
ht->nTableSize += ht->nTableSize;
ht->nTableMask = -ht->nTableSize;
HT_SET_DATA_ADDR(ht, pemalloc(HT_SIZE(ht), ht->u.flags & HASH_FLAG_PERSISTENT));
memcpy(ht->arData, old_buckets, sizeof(Bucket) * ht->nNumUsed);
pefree(old_data, ht->u.flags & HASH_FLAG_PERSISTENT);
zend_hash_rehash(ht);
HANDLE_UNBLOCK_INTERRUPTIONS();
I am uncertain if the remalloc is the performance hit, or if the rehashing is the hit, or the fact that the whole block is uninterruptable. Would be interesting to put a profiler on it. I think some might have already done that for PHP 7.
Side note, the Thread Safe version does things differently. I'm not overly familiar with that code, so there may be a different issue going on if your using ZTS.
I think it is related to implementation of dynamic arrays.
See here "Geometric expansion and amortized cost" http://en.wikipedia.org/wiki/Dynamic_array
To avoid incurring the cost of resizing many times, dynamic arrays resize by a large amount, **such as doubling in size**, and use the reserved space for future expansion
You can read about arrays in PHP here as well https://nikic.github.io/2011/12/12/How-big-are-PHP-arrays-really-Hint-BIG.html
It is a standard practice for dynamic arrays. E.g. check here C++ dynamic array, increasing capacity
capacity = capacity * 2; // doubles the capacity of the array
The question title basically says it all..
If I perform:
$Cache = new Memcache;
$Cache->connect('HOST');
$Cache->set('Information', 'array(
"Testing" => "Value,
"Anther_Test" => "Another Value"
)');
and leave the information there, would the information be flushed/deleted after an interval?
or will it retain within the server until I call:
$Cache->flush();
What #hakre wrote in a comment plus you should get how to use cache.
Cache is not a date storage, it is a cache. You should never relay on what is there and what is not.
Typically you have a limit of a cache size and when it gets full it purges old values.
Typical usage case:
function get_something() {
$Cache = new Memcache;
$something = $Cache->get('something');
if ($something === false) {
$something = //get from db, prepare or whatever
$Cache->set('something', $something);
}
return $something;
}
TLDR; no
I have a list of MAC addresses coming from a database. I would like to lookup the vendor for each MAC address and then have a count of devices on network by vendor in the end.
I believe I could do it the dirty way which would be to parse the vendor prefixes from the file available here http://standards.ieee.org/develop/regauth/oui/oui.txt.
But I'm wondering if there is a better way ?
There is a library in Pear, but it does have substantial overhead involved in that the vendor lookup requires a relational database that's been loaded with the vendor data. However, considering the alternative this might be worth exploring.
http://pear.php.net/manual/en/package.networking.net-mac.php
The package provides a loader for the list maintained by wireshark: https://code.wireshark.org/review/gitweb?p=wireshark.git;a=blob_plain;f=manuf
If all you care about is getting the manufacturer of the device based on the mac address then you can simply copy and paste the list on this website here (unto 200 at a time). It's very quick:
www.admin-toolkit.com/mac-address-lookup-manufacturer.html
But I'm wondering if there is a better way ?
If you can use other languages like ruby, there's a gem called macvendors
There is no need to use an external api or get stopped by rate limit.
You can use it from your command line:
gem install macvendors
macvendors find 98:e0:d9:a5:61:eb
Apple, Inc.
You can use it in your ruby code:
require 'macvendors'
MacVendors.setup #for the first time
puts MacVendors.find("98:e0:d9:a5:61:eb")
I have made an SQlite (Macvendors.db) from the Wireshark manuf source. I use it via PDO with a simple query.
Here is a method I write to show you how you could use it:
public function getAllCompaniesFromMacs($macs)
{
try {
// connect to the SQLite file
$conn = new \PDO("sqlite:/path/to/sqlite/Macvendors.db");
$conn ->setAttribute(\PDO::ATTR_ERRMODE, \PDO::ERRMODE_EXCEPTION);
// aux vars
$macsParaBuscar = array();
$macsIN = "";
// output vars
$salida = array();
// Clean repeated MACs and remove the ":" chars
foreach ($macs as $unaMac) {
$unaMac = str_replace(":", "", $unaMac);
$firstChars = substr($unaMac, 0, 6);
$macsParaBuscar[$firstChars] = $firstChars;
}
// Create a IN statment for the WHERE
$macsIN = "( 'HOLA'";
foreach ($macsParaBuscar as $unaMacBuscar) {
$macsIN .= ", '" . $unaMacBuscar . "'";
}
$macsIN .= ")";
// Prepare and execute the query
$stm = $conn->prepare("SELECT mac, vendor FROM macvendor WHERE mac IN " . $macsIN);
$stm->execute();
// Put results in output var
while( $row = $stm->fetch() ) {
$auxMac = $row["mac"];
$salida[$auxMac] = $row["vendor"];
}
return $salida;
} catch (\Exception $e) {
throw new \Exception("Ha ocurrido un error", 1);
}
}
Sorry about the unbeauty example code.
Have fun!
Update
There is another library that doesn't depend on any API but the XML file database which is Cisco vendorMacs.xml
The package is hosted here
Yes here is one I wrote, it depends on the online API of Mac address vendor website
Here is the repository with examples