Storing sessions in disk very slow and painful for me. I'm having very high traffic. I want to store session in Advanced PHP Cache, how can I do this?
<?php
// to enable paste this line right before session_start():
// new Session_APC;
class Session_APC
{
protected $_prefix;
protected $_ttl;
protected $_lockTimeout = 10; // if empty, no session locking, otherwise seconds to lock timeout
public function __construct($params=array())
{
$def = session_get_cookie_params();
$this->_ttl = $def['lifetime'];
if (isset($params['ttl'])) {
$this->_ttl = $params['ttl'];
}
if (isset($params['lock_timeout'])) {
$this->_lockTimeout = $params['lock_timeout'];
}
session_set_save_handler(
array($this, 'open'), array($this, 'close'),
array($this, 'read'), array($this, 'write'),
array($this, 'destroy'), array($this, 'gc')
);
}
public function open($savePath, $sessionName)
{
$this->_prefix = 'BSession/'.$sessionName;
if (!apc_exists($this->_prefix.'/TS')) {
// creating non-empty array #see http://us.php.net/manual/en/function.apc-store.php#107359
apc_store($this->_prefix.'/TS', array(''));
apc_store($this->_prefix.'/LOCK', array(''));
}
return true;
}
public function close()
{
return true;
}
public function read($id)
{
$key = $this->_prefix.'/'.$id;
if (!apc_exists($key)) {
return ''; // no session
}
// redundant check for ttl before read
if ($this->_ttl) {
$ts = apc_fetch($this->_prefix.'/TS');
if (empty($ts[$id])) {
return ''; // no session
} elseif (!empty($ts[$id]) && $ts[$id] + $this->_ttl < time()) {
unset($ts[$id]);
apc_delete($key);
apc_store($this->_prefix.'/TS', $ts);
return ''; // session expired
}
}
if (!$this->_lockTimeout) {
$locks = apc_fetch($this->_prefix.'/LOCK');
if (!empty($locks[$id])) {
while (!empty($locks[$id]) && $locks[$id] + $this->_lockTimeout >= time()) {
usleep(10000); // sleep 10ms
$locks = apc_fetch($this->_prefix.'/LOCK');
}
}
/*
// by default will overwrite session after lock expired to allow smooth site function
// alternative handling is to abort current process
if (!empty($locks[$id])) {
return false; // abort read of waiting for lock timed out
}
*/
$locks[$id] = time(); // set session lock
apc_store($this->_prefix.'/LOCK', $locks);
}
return apc_fetch($key); // if no data returns empty string per doc
}
public function write($id, $data)
{
$ts = apc_fetch($this->_prefix.'/TS');
$ts[$id] = time();
apc_store($this->_prefix.'/TS', $ts);
$locks = apc_fetch($this->_prefix.'/LOCK');
unset($locks[$id]);
apc_store($this->_prefix.'/LOCK', $locks);
return apc_store($this->_prefix.'/'.$id, $data, $this->_ttl);
}
public function destroy($id)
{
$ts = apc_fetch($this->_prefix.'/TS');
unset($ts[$id]);
apc_store($this->_prefix.'/TS', $ts);
$locks = apc_fetch($this->_prefix.'/LOCK');
unset($locks[$id]);
apc_store($this->_prefix.'/LOCK', $locks);
return apc_delete($this->_prefix.'/'.$id);
}
public function gc($lifetime)
{
if ($this->_ttl) {
$lifetime = min($lifetime, $this->_ttl);
}
$ts = apc_fetch($this->_prefix.'/TS');
foreach ($ts as $id=>$time) {
if ($time + $lifetime < time()) {
apc_delete($this->_prefix.'/'.$id);
unset($ts[$id]);
}
}
return apc_store($this->_prefix.'/TS', $ts);
}
}
I tried to lure better answers by offering 100 points as a bounty, but none of the answers were really satisfying.
I would aggregate the recommended solutions like this:
Using APC as a session storage
APC cannot really be used as a session store, because there is no mechanism available to APC that allows proper locking, But this locking is essential to ensure nobody alters the initially read session data before writing it back.
Bottom line: Avoid it, it won't work.
Alternatives
A number of session handlers might be available. Check the output of phpinfo() at the Session section for "Registered save handlers".
File storage on RAM disk
Works out-of-the-box, but needs a file system mounted as RAM disk for obvious reasons.
Shared memory (mm)
Is available when PHP is compiled with mm enabled. This is builtin on windows.
Memcache(d)
PHP comes with a dedicated session save handler for this. Requires installed memcache server and PHP client. Depending on which of the two memcache extensions is installed, the save handler is either called memcache or memcached.
In theory, you ought to be able to write a custom session handler which uses APC to do this transparently for you. However, I haven't actually been able to find anything really promising in a quick five-minute search; most people seem to be using APC for the bytecode cache and putting their sessions in memcached.
Simply putting your /tmp disk (or, wherever PHP session files are stored) onto a RAM disk such as tmpfs or ramfs would also have serious performance gains, and would be a much more transparent switch, with zero code changes.
The performance gain may be significantly less, but it will still be significantly faster than on-disk sessions.
Store it in cookies (encrypted) or MongoDB. APC isn't really intended for that purpose.
You can store your session data within PHP internals shared memory.
session.save_handler = mm
But it needs to be available: http://php.net/manual/en/session.installation.php
Another good solution is to store PHP sessions in memcached
session.save_handler = memcache
Explicit Session Closing immediately following Session Starting, Opening and Writing should solve the locking problem in Unirgy's Answer(where session access is always cyclic(start/open-write-close). I also Imagine a Second class - APC_journaling or something similar used in conjunction with Sessions would be ultimately better.... A session starts and is written to with a unique external Id assigned to each session, that session is closed, and a journal (array in apc cache via _store & _add) is opened/created for any other writes intended to go to session which can then be read, validated and written to the session(identified by that unique id!) in apc at the next convenient opportunity.
I found a good blog post Explaining that the Locking havoc Sven refers to comes from the Session blocking until it's closed or script execution ends. The session being immediately closed doesn't prevent reading just writing.
http://konrness.com/php5/how-to-prevent-blocking-php-requests - link to the blog post.
Hope this helps.
Caching external data in PHP
Tutorial Link - http://www.gayadesign.com/diy/caching-external-data-in-php/
How to Use APC Caching with PHP
Tutorial Link - http://www.script-tutorials.com/how-to-use-apc-caching-with-php/
Related
In php, i create a cache file in order to stock complexe results variables. One variable, one cache file. Well done its work fine.
The problem lies in the term of the cache. For the moment i put into the file the timeout and the variable, but its not optimized because i must open the file to check the timeout.
I want to (if its possible) check the timeout to a file property (like date last modified with the function filemtime()). Can we add custom property to a file?
The other way is to add the timeout in the filename, not my favorite solution.
[Edit]
final class Cache_Var extends Cache {
public static function put($key, $value, $timeout=0) {
// different timeout by variable (if 0, infinite timeout)
}
public static function get($key) {
// no timeout to get a var cache
// return null if file not found, or if timeout expire
// return var otherwise
}
}
filectime()can really help you
$validity = 60 * 60; // 3600s = 1 hour
if(filectime($filename) > time() - $validity) {
// cache is valid
} else {
// cache is invalid: recreate it
}
There are some caching fdrameworks around that use exactly this mechanism.
Edit:
If you need different timeouts per cache item than use touch()to set modification time of cache files. You could even set modification time to a future value and directly compare filectime with current time.
final class Cache_Var extends Cache {
public static function put($key, $value, $timeout=0) {
// different timeout by variable (if 0, infinite timeout)
// ...
touch($filename, time() + $timeout);
// For static files with unlimited lifetime I would simply store
// them in a separate folder
}
}
We have a PHP application, and were thinking it might be advantageous to have the application know if there was a change in its makeup since the last execution. Mainly due to managing caches and such, and knowing that our applications are sometimes accessed by people who don't remember to clear the cache on changes. (Changing the people is the obvious answer, but alas, not really achievable)
We've come up with this, which is the fastest we've managed to eke out, running an average 0.08 on a developer machine for a typical project. We've experimented with shasum,md5 and crc32, and this is the fastest. We are basically md5ing the contents of every file, and md5'ing that output. Security isnt a concern, we're just interested in detecting filesystem changes via a differing checksum.
time (find application/ -path '*/.svn' -prune -o -type f -print0 | xargs -0 md5 | md5)
I suppose the question is, can this be optimised any further?
(I realise that pruning svn will have a cost, but find takes the least amount of time out of the components, so it will be pretty minimal. We're testing this on a working copy atm)
You can be notified of filesystem modifications using the inotify extension.
It can be installed with pecl:
pecl install inotify
Or manually (download, phpize && ./configure && make && make install as usual).
This is a raw binding over the linux inotify syscalls, and is probably the fastest solution on linux.
See this example of a simple tail implementation: http://svn.php.net/viewvc/pecl/inotify/trunk/tail.php?revision=262896&view=markup
If you want a higher level library, or suppot for non-linux systems, take a look at Lurker.
It works on any system, and can use inotity when it's available.
See the example from the README:
$watcher = new ResourceWatcher;
$watcher->track('an arbitrary id', '/path/to/views');
$watcher->addListener('an arbitrary id', function (FilesystemEvent $event) {
echo $event->getResource() . 'was' . $event->getTypeString();
});
$watcher->start();
Instead of going by file contents, you can use the same technique with filename and timestamps:
find . -name '.svn' -prune -o -type f -printf '%m%c%p' | md5sum
This is much faster than reading and hashing the contents of each file.
Insteading of actively searching for changes, why not getting notified when something changes. Have a look at PHP's FAM - File Alteration Monitor API
FAM monitors files and directories, notifying interested applications of changes. More information about FAM is available at » http://oss.sgi.com/projects/fam/. A PHP script may specify a list of files for FAM to monitor using the functions provided by this extension. The FAM process is started when the first connection from any application to it is opened. It exits after all connections to it have been closed.
CAVEAT: requires an additional daemon on the machine and the PECL extension is unmaintained.
We didn't want to use FAM, since we would need to install it on the server, and thats not always possible. Sometimes clients are insistent we deploy on their broken infrastructure. Since it's discontinued, its hard to get that change approved red tape wise also.
The only way to improve the speed of the original version in the question is to make sure your file list is as succinct as possible. IE only hash the directories/files that really matter if changed. Cutting out directories that aren't relevant can give big speed boosts.
Past that, the application was using the function to check if there were changes in order to perform a cache clear if there were. Since we don't really want to halt the application while its doing this, this sort of thing is best farmed out carefully as an asynchronous process using fsockopen. That gives the best 'speed boost' overall, just be careful of race conditions.
Marking this as the 'answer' and upvoting the FAM answer.
since you have svn, why don't you go by revisions. i realise you are skipping svn folders but i suppose you did that for speed advantage and that you do not have modified files in your production servers...
that beeing said, you do not have to re invent the wheel.
you could speed up the process by only looking at metadata read from the directory indexes (modification timestamp, filesize, etc)
you could also stop once you spotted a difference (should theoretically reduce the time by half in average) etc. there is a lot.
i honestly think the best way in this case is to just use the tools already available.
the linux tool diff has a -q option (quick).
you will need to use it with the recursive parameter -r as well.
diff -r -q dir1/ dir2/
it uses a lot of optimisations and i seriously doubt you can significantly improve upon it without considerable effort.
Definitely what you should be using is Inotify its fast and easy to configure, multiple options directly from bash or php of dedicate a simple node-inotify instance for this task
But Inotify does not worn on windows but you can easy write a command line application with FileSystemWatcher or FindFirstChangeNotification and call via exec
If you are looking for only PHP solution then its pretty difficult and you might not get the performance want because the only way is to scan that folder continuously
Here is a Simple Experiment
Don't use in production
Can not manage large file set
Does not support file monitoring
Only Support NEW , DELETED and MODIFIED
Does not support Recursion
Example
if (php_sapi_name() !== 'cli')
die("CLI ONLY");
date_default_timezone_set("America/Los_Angeles");
$sm = new Monitor(__DIR__ . "/test");
// Add hook when new files are created
$sm->hook(function ($file) {
// Send a mail or log to a file
printf("#EMAIL NEW FILE %s\n", $file);
}, Monitor::NOTIFY_NEW);
// Add hook when files are Modified
$sm->hook(function ($file) {
// Do monthing meaningful like calling curl
printf("#HTTP POST MODIFIED FILE %s\n", $file);
}, Monitor::NOTIFY_MODIFIED);
// Add hook when files are Deleted
$sm->hook(function ($file) {
// Crazy ... Send SMS fast or IVR the Boss that you messed up
printf("#SMS DELETED FILE %s\n", $file);
}, Monitor::NOTIFY_DELETED);
// Start Monitor
$sm->start();
Cache Used
// Simpe Cache
// Can be replaced with Memcache
class Cache {
public $f;
function __construct() {
$this->f = fopen("php://temp", "rw+");
}
function get($k) {
rewind($this->f);
return json_decode(stream_get_contents($this->f), true);
}
function set($k, $data) {
fseek($this->f, 0);
fwrite($this->f, json_encode($data));
return $k;
}
function run() {
}
}
The Experiment Class
// The Experiment
class Monitor {
private $dir;
private $info;
private $timeout = 1; // sec
private $timeoutStat = 60; // sec
private $cache;
private $current, $stable, $hook = array();
const NOTIFY_NEW = 1;
const NOTIFY_MODIFIED = 2;
const NOTIFY_DELETED = 4;
const NOTIFY_ALL = 7;
function __construct($dir) {
$this->cache = new Cache();
$this->dir = $dir;
$this->info = new SplFileInfo($this->dir);
$this->scan(true);
}
public function start() {
$i = 0;
$this->stable = (array) $this->cache->get(md5($this->dir));
while(true) {
// Clear System Cache at Intervals
if ($i % $this->timeoutStat == 0) {
clearstatcache();
}
$this->scan(false);
if ($this->stable != $this->current) {
$this->cache->set(md5($this->dir), $this->current);
$this->stable = $this->current;
}
sleep($this->timeout);
$i ++;
// printf("Memory Usage : %0.4f \n", memory_get_peak_usage(false) /
// 1024);
}
}
private function scan($new = false) {
$rdi = new FilesystemIterator($this->dir, FilesystemIterator::SKIP_DOTS);
$this->current = array();
foreach($rdi as $file) {
// Skip files that are not redable
if (! $file->isReadable())
return false;
$path = addslashes($file->getRealPath());
$keyHash = md5($path);
$fileHash = $file->isFile() ? md5_file($path) : "#";
$hash["t"] = $file->getMTime();
$hash["h"] = $fileHash;
$hash["f"] = $path;
$this->current[$keyHash] = json_encode($hash);
}
if ($new === false) {
$this->process();
}
}
public function hook(Callable $call, $type = Monitor::NOTIFY_ALL) {
$this->hook[$type][] = $call;
}
private function process() {
if (isset($this->hook[self::NOTIFY_NEW])) {
$diff = array_flip(array_diff(array_keys($this->current), array_keys($this->stable)));
$this->notify(array_intersect_key($this->current, $diff), self::NOTIFY_NEW);
unset($diff);
}
if (isset($this->hook[self::NOTIFY_DELETED])) {
$deleted = array_flip(array_diff(array_keys($this->stable), array_keys($this->current)));
$this->notify(array_intersect_key($this->stable, $deleted), self::NOTIFY_DELETED);
}
if (isset($this->hook[self::NOTIFY_MODIFIED])) {
$this->notify(array_diff_assoc(array_intersect_key($this->stable, $this->current), array_intersect_key($this->current, $this->stable)), self::NOTIFY_MODIFIED);
}
}
private function notify(array $files, $type) {
if (empty($files))
return;
foreach($this->hook as $t => $hooks) {
if ($t & $type) {
foreach($hooks as $hook) {
foreach($files as $file) {
$info = json_decode($file, true);
$hook($info['f'], $type);
}
}
}
}
}
}
I've one page where i do a long polling i've to use at the begin of this page this
session_start();
session_write_close();
Because :
to prevent concurrent writes only one script may operate on a session at any time
So if i do not and the long polling is running the user will not be able to load another page.
So accessing to my data in session from this polling page is possible but at some point in my script i've to save my session back to the server because i made some change in it.
What's the way to do it?
That will be very nice it'll be a way to do something like
session_write_open();
//do stuff
session_write_close();
But the session_write_open() doesn't exist!
Thanks
Before you make some change to the session, call session_start again. Make the changes, and if you still do not want to exit call session_write_close once more. You can do this as many times as you like.
The previous solution will create a session ids and cookies... I wouldn't use it as is:
Session is created every time you call session_start(). If you want
to avoid multiple cookie, write better code. Multiple session_start()
especially for the same names in the same script seems like a really
bad idea.
see here : https://bugs.php.net/bug.php?id=38104
I am looking for a solution right now too and I can't find one. I agree with those who say this is a "bug".
You should be able to reopen a php session, but as you said session_write_open() does not exist...
I found a workaround in the above thread. It involves sending a header specifying manually the session id's cookie after processing the request. Luckily enough I am working with a home-brewed Front Controller that works so that no sub-controller will ever send data on its own.
In a nutshell, it works perfectly in my case. To use this you might just have to use ob_start() and ob_get_clean(). Here's the magic line:
if (SID) header('Set-Cookie: '.SID.'; path=/', true);
EDIT : see CMCDragonkai's answer below, seems good!?
All of the answers here seem to be saying to use the session methods in ways that they were clearly not intended to be used...namely calling session_start() more than once.
The PHP website offers an example SessionHandlerInterface implementation that will work just like existing sessions but without locking the file. Just implementing their example interface fixed my locking issue to allow for concurrent connections on the same session without limiting my ability to add vars to the session. To prevent some race conditions, since the app's session isn't fully stateless, I did have to make a way to save the session mid-request without closing it so that important changes could save immediately after change and less important session vars could just save at the end of the request. See the below example for usage:
Session::start();
echo("<pre>Vars Stored in Session Were:\n");print_r($_SESSION);echo("</pre>");
$_SESSION['one'] = 'one';
$_SESSION['two'] = 'two';
//save won't close session and subsequent request will show 'three'
Session::save();
$_SESSION['three'] = 'three';
If you replace that Session::start() with session_start() and Session::save() with session_write_close(), you'll notice that subsequent requests will never print out the third variable...it will be lost. However, using the SessionHandler (below), no data is lost.
The OOP implementation requires PHP 5.4+. However, you can provide individual callback methods in older versions of PHP. See docs.
namespace {
class Session implements SessionHandlerInterface {
/** #var Session */
private static $_instance;
private $savePath;
public static function start() {
if( empty(self::$_instance) ) {
self::$_instance = new self();
session_set_save_handler(self::$_instance,true);
session_start();
}
}
public static function save() {
if( empty(self::$_instance) ) {
throw new \Exception("You cannot save a session before starting the session");
}
self::$_instance->write(session_id(),session_encode());
}
public function open($savePath, $sessionName) {
$this->savePath = $savePath;
if (!is_dir($this->savePath)) {
mkdir($this->savePath, 0777);
}
return true;
}
public function close() {
return true;
}
public function read($id) {
return (string)#file_get_contents("$this->savePath/sess_$id");
}
public function write($id, $data) {
return file_put_contents("$this->savePath/sess_$id", $data) === false ? false : true;
}
public function destroy($id) {
$file = "$this->savePath/sess_$id";
if (file_exists($file)) {
unlink($file);
}
return true;
}
public function gc($maxlifetime) {
foreach (glob("$this->savePath/sess_*") as $file) {
if (filemtime($file) + $maxlifetime < time() && file_exists($file)) {
unlink($file);
}
}
return true;
}
}
The other answers here present pretty good solutions. As mentioned by #Jon, the trick is to call session_start() again before you want to make changes. Then, when you are done making changes, call session_write_close() again.
As mentioned by #Armel Larcier, the problem with this is that PHP attempts to generate new headers and will likely generate warnings (e.g. if you've already written non-header data to the client). Of course, you can simply prefix the session_start() with "#" (#session_start()), but there's a better approach.
Another Stack Overflow question, provided by #VolkerK reveals the best answer:
session_start(); // first session_start
...
session_write_close();
...
ini_set('session.use_only_cookies', false);
ini_set('session.use_cookies', false);
//ini_set('session.use_trans_sid', false); //May be necessary in some situations
ini_set('session.cache_limiter', null);
session_start(); // second session_start
This prevents PHP from attempting to send the headers again. You could even write a helper function to wrap the ini_set() functions to make this a bit more convenient:
function session_reopen() {
ini_set('session.use_only_cookies', false);
ini_set('session.use_cookies', false);
//ini_set('session.use_trans_sid', false); //May be necessary in some situations
ini_set('session.cache_limiter', null);
session_start(); //Reopen the (previously closed) session for writing.
}
Original related SO question/answer: https://stackoverflow.com/a/12315542/114558
After testing out Armel Larcier's work around. Here's my proposed solution to this problem:
ob_start();
session_start();
session_write_close();
session_start();
session_write_close();
session_start();
session_write_close();
session_start();
session_write_close();
if(SID){
$headers = array_unique(headers_list());
$cookie_strings = array();
foreach($headers as $header){
if(preg_match('/^Set-Cookie: (.+)/', $header, $matches)){
$cookie_strings[] = $matches[1];
}
}
header_remove('Set-Cookie');
foreach($cookie_strings as $cookie){
header('Set-Cookie: ' . $cookie, false);
}
}
ob_flush();
This will preserve any cookies that were created prior to working with sessions.
BTW, you may wish to register the above code as function for register_shutdown_function. Make sure to run ob_start() before the function, and ob_flush() inside the function.
We have a PHP application, and were thinking it might be advantageous to have the application know if there was a change in its makeup since the last execution. Mainly due to managing caches and such, and knowing that our applications are sometimes accessed by people who don't remember to clear the cache on changes. (Changing the people is the obvious answer, but alas, not really achievable)
We've come up with this, which is the fastest we've managed to eke out, running an average 0.08 on a developer machine for a typical project. We've experimented with shasum,md5 and crc32, and this is the fastest. We are basically md5ing the contents of every file, and md5'ing that output. Security isnt a concern, we're just interested in detecting filesystem changes via a differing checksum.
time (find application/ -path '*/.svn' -prune -o -type f -print0 | xargs -0 md5 | md5)
I suppose the question is, can this be optimised any further?
(I realise that pruning svn will have a cost, but find takes the least amount of time out of the components, so it will be pretty minimal. We're testing this on a working copy atm)
You can be notified of filesystem modifications using the inotify extension.
It can be installed with pecl:
pecl install inotify
Or manually (download, phpize && ./configure && make && make install as usual).
This is a raw binding over the linux inotify syscalls, and is probably the fastest solution on linux.
See this example of a simple tail implementation: http://svn.php.net/viewvc/pecl/inotify/trunk/tail.php?revision=262896&view=markup
If you want a higher level library, or suppot for non-linux systems, take a look at Lurker.
It works on any system, and can use inotity when it's available.
See the example from the README:
$watcher = new ResourceWatcher;
$watcher->track('an arbitrary id', '/path/to/views');
$watcher->addListener('an arbitrary id', function (FilesystemEvent $event) {
echo $event->getResource() . 'was' . $event->getTypeString();
});
$watcher->start();
Instead of going by file contents, you can use the same technique with filename and timestamps:
find . -name '.svn' -prune -o -type f -printf '%m%c%p' | md5sum
This is much faster than reading and hashing the contents of each file.
Insteading of actively searching for changes, why not getting notified when something changes. Have a look at PHP's FAM - File Alteration Monitor API
FAM monitors files and directories, notifying interested applications of changes. More information about FAM is available at » http://oss.sgi.com/projects/fam/. A PHP script may specify a list of files for FAM to monitor using the functions provided by this extension. The FAM process is started when the first connection from any application to it is opened. It exits after all connections to it have been closed.
CAVEAT: requires an additional daemon on the machine and the PECL extension is unmaintained.
We didn't want to use FAM, since we would need to install it on the server, and thats not always possible. Sometimes clients are insistent we deploy on their broken infrastructure. Since it's discontinued, its hard to get that change approved red tape wise also.
The only way to improve the speed of the original version in the question is to make sure your file list is as succinct as possible. IE only hash the directories/files that really matter if changed. Cutting out directories that aren't relevant can give big speed boosts.
Past that, the application was using the function to check if there were changes in order to perform a cache clear if there were. Since we don't really want to halt the application while its doing this, this sort of thing is best farmed out carefully as an asynchronous process using fsockopen. That gives the best 'speed boost' overall, just be careful of race conditions.
Marking this as the 'answer' and upvoting the FAM answer.
since you have svn, why don't you go by revisions. i realise you are skipping svn folders but i suppose you did that for speed advantage and that you do not have modified files in your production servers...
that beeing said, you do not have to re invent the wheel.
you could speed up the process by only looking at metadata read from the directory indexes (modification timestamp, filesize, etc)
you could also stop once you spotted a difference (should theoretically reduce the time by half in average) etc. there is a lot.
i honestly think the best way in this case is to just use the tools already available.
the linux tool diff has a -q option (quick).
you will need to use it with the recursive parameter -r as well.
diff -r -q dir1/ dir2/
it uses a lot of optimisations and i seriously doubt you can significantly improve upon it without considerable effort.
Definitely what you should be using is Inotify its fast and easy to configure, multiple options directly from bash or php of dedicate a simple node-inotify instance for this task
But Inotify does not worn on windows but you can easy write a command line application with FileSystemWatcher or FindFirstChangeNotification and call via exec
If you are looking for only PHP solution then its pretty difficult and you might not get the performance want because the only way is to scan that folder continuously
Here is a Simple Experiment
Don't use in production
Can not manage large file set
Does not support file monitoring
Only Support NEW , DELETED and MODIFIED
Does not support Recursion
Example
if (php_sapi_name() !== 'cli')
die("CLI ONLY");
date_default_timezone_set("America/Los_Angeles");
$sm = new Monitor(__DIR__ . "/test");
// Add hook when new files are created
$sm->hook(function ($file) {
// Send a mail or log to a file
printf("#EMAIL NEW FILE %s\n", $file);
}, Monitor::NOTIFY_NEW);
// Add hook when files are Modified
$sm->hook(function ($file) {
// Do monthing meaningful like calling curl
printf("#HTTP POST MODIFIED FILE %s\n", $file);
}, Monitor::NOTIFY_MODIFIED);
// Add hook when files are Deleted
$sm->hook(function ($file) {
// Crazy ... Send SMS fast or IVR the Boss that you messed up
printf("#SMS DELETED FILE %s\n", $file);
}, Monitor::NOTIFY_DELETED);
// Start Monitor
$sm->start();
Cache Used
// Simpe Cache
// Can be replaced with Memcache
class Cache {
public $f;
function __construct() {
$this->f = fopen("php://temp", "rw+");
}
function get($k) {
rewind($this->f);
return json_decode(stream_get_contents($this->f), true);
}
function set($k, $data) {
fseek($this->f, 0);
fwrite($this->f, json_encode($data));
return $k;
}
function run() {
}
}
The Experiment Class
// The Experiment
class Monitor {
private $dir;
private $info;
private $timeout = 1; // sec
private $timeoutStat = 60; // sec
private $cache;
private $current, $stable, $hook = array();
const NOTIFY_NEW = 1;
const NOTIFY_MODIFIED = 2;
const NOTIFY_DELETED = 4;
const NOTIFY_ALL = 7;
function __construct($dir) {
$this->cache = new Cache();
$this->dir = $dir;
$this->info = new SplFileInfo($this->dir);
$this->scan(true);
}
public function start() {
$i = 0;
$this->stable = (array) $this->cache->get(md5($this->dir));
while(true) {
// Clear System Cache at Intervals
if ($i % $this->timeoutStat == 0) {
clearstatcache();
}
$this->scan(false);
if ($this->stable != $this->current) {
$this->cache->set(md5($this->dir), $this->current);
$this->stable = $this->current;
}
sleep($this->timeout);
$i ++;
// printf("Memory Usage : %0.4f \n", memory_get_peak_usage(false) /
// 1024);
}
}
private function scan($new = false) {
$rdi = new FilesystemIterator($this->dir, FilesystemIterator::SKIP_DOTS);
$this->current = array();
foreach($rdi as $file) {
// Skip files that are not redable
if (! $file->isReadable())
return false;
$path = addslashes($file->getRealPath());
$keyHash = md5($path);
$fileHash = $file->isFile() ? md5_file($path) : "#";
$hash["t"] = $file->getMTime();
$hash["h"] = $fileHash;
$hash["f"] = $path;
$this->current[$keyHash] = json_encode($hash);
}
if ($new === false) {
$this->process();
}
}
public function hook(Callable $call, $type = Monitor::NOTIFY_ALL) {
$this->hook[$type][] = $call;
}
private function process() {
if (isset($this->hook[self::NOTIFY_NEW])) {
$diff = array_flip(array_diff(array_keys($this->current), array_keys($this->stable)));
$this->notify(array_intersect_key($this->current, $diff), self::NOTIFY_NEW);
unset($diff);
}
if (isset($this->hook[self::NOTIFY_DELETED])) {
$deleted = array_flip(array_diff(array_keys($this->stable), array_keys($this->current)));
$this->notify(array_intersect_key($this->stable, $deleted), self::NOTIFY_DELETED);
}
if (isset($this->hook[self::NOTIFY_MODIFIED])) {
$this->notify(array_diff_assoc(array_intersect_key($this->stable, $this->current), array_intersect_key($this->current, $this->stable)), self::NOTIFY_MODIFIED);
}
}
private function notify(array $files, $type) {
if (empty($files))
return;
foreach($this->hook as $t => $hooks) {
if ($t & $type) {
foreach($hooks as $hook) {
foreach($files as $file) {
$info = json_decode($file, true);
$hook($info['f'], $type);
}
}
}
}
}
}
In ASP.NET if I declare a variable (or object) static (or if I make a singleton) I can have it persist across multiple sessions of multiple users (it it registered in a server scope) so that I don't have to initialize it at every request.
Is there such a feature in PHP? Thanks
You can set up APC and use the apc_store and apc_fetch functions.
http://us.php.net/manual/en/book.apc.php
You can do that with a PHP extension (written in C).
But if you want to write it in PHP, no. The best alternative is to write the variable to a file (file_put_contents()) at the end of each request, and open it at the start of each request (file_get_contents()).
That alternative isn't going to work for high volume sites because the processes will be doing read/write at the same time and the world will go all BLAAA-WOOO-EEE-WOHHH-BOOOM.
That doesn´t exist in PHP, however, you can serialize the data and put it either in a file on your hard drive or in /dev/shm/. You can also use memcache.
If you put your data in /dev/shm/ or use memcache the data will disappear on reboot.
Sadly, no. PHP's static keyword is limited to the current script instance only.
To persist data across script instances for the same session, you would use the session handling features.
To persist data across sessions, you would have to use something like memcache, however that requires additional set-up work on server side.
Symfony and other frameworks uses "PHPFastCache" who supports a wide range of drivers for caching data including APC, SQLite, MongoDB or simply your file system.
You can donwnload it at https://github.com/PHPSocialNetwork/phpfastcache
Here is an example with file caching :
use Phpfastcache\Helper\Psr16Adapter;
$defaultDriver = 'Files';
$Psr16Adapter = new Psr16Adapter($defaultDriver);
// Setter action
if(!$Psr16Adapter->has('test-key')) {
$data = 'lorem ipsum';
$Psr16Adapter->set('test-key', 'lorem ipsum', 300); // kept in cache for 300 seconds (5 minutes)
}
// Getter action
else {
$data = $Psr16Adapter->get('test-key');
}
You can use the Session Storage for this purpose, if you use the same sessionId for all sessions.
session_id('xyz');
session_start();
for ($i=0; $i < 100000; $i++) {
$_SESSION['counter'] = isset($_SESSION['counter']) ? $_SESSION['counter'] + 1 : 0;
}
echo "<br>session_id(): ".session_id() . "<br>counter: ".$_SESSION["counter"];
Try this script with 2 browsers and you will see that this method shares the data across both browsers - and is very, very fast.
you could store serialized copies of an object inside session
class test{
private static $instance;
public property;
private __construct(){}
public getInstace(){
if(!self::$instance){
self::$instance = new test;
}
return self::$instance;
}
}
$p = test->getInstance();
$p->property = "Howdy";
$_SESSION["p"] = $p;
next page
$p = $_SESSION["p"];
echo $p->property; // "Howdy"