Recently, in one of our projects that use Laravel 5.4, we have noticed that some data is being cached in /storage/framework/cache/data - we are using file cache. The contents of the files in the cache are things like: 1529533237i:1;. Several files are created in the cache throughout the day with content similar to that. So many files are created that we have to clean this cache periodically in order not to run into disk space issues by running out of inodes.
I know that an alternative to using file cache are things like Redis or Memcache, but the issue is, we're not sure what is this data being cached or what component of the project is caching it. We do use several external libraries so it could be one of many, but we don't know for sure what. I've already looked into all configuration files of the project, but couldn't identify anything that is obviously controlling data caching.
Are there any recommendations on trying to identify which piece of code is writing this data so we can better handle the caching of this data, whatever it may be?
Laravel has several events that dispatch during caching.
Create a new listener that listens on the Illuminate\Cache\Events\KeyWritten event. You could log the backtrace to see exactly what leads to specific items being cached.
Related
I have a music site developed on CodeIgniter with youtube API without database. Recently I have noticed that too many files are getting generated in system/cache folder.
How can I stop generating this cache files? Note that I am not using any cache method.
system/cache is NOT default codeigniter cache directory at first. I would not store cache in there, as its framework main folder. Default is application/cache.
By default, CI does NOT cache anything. So your application is build with caching.
You told you don't use database, so it's not DB cache I assume.
Check in your app for somethign like "$this->load->driver('cache'".
Caching can be loaded WITHOUT additional parameters like
$this->load->driver('cache'); OR with parameters like
$this->load->driver('cache',array('adapther'=>'xxx'));
https://www.codeigniter.com/userguide3/libraries/caching.html
Now, in your app search for $this->cache->save OR $this->cache->file->save
if you found this, it means you are using CI caching.
Problem is, you cannot just remove cache loading, as app initiates cache object, and your app will fail, unless you rewrite all places where caching is used.
Now, you have few choices:
1.just clean cache dir with some script periodically via cron.
you can change cache folder permissions to NON writable, which will generate warnings in your logs, so logging should be disabled. This is not the right way IMHO, as can cause fatal errors/blank pages but just one of possible solutions. If file caching is used, this should not cause issues, while in other cases it could.
you can extend caching library, and simply create empty cache SAVE function. In this case your files will not be saved.
you can cache to memcached, if you have it on your server. Well, if your caching is written like $this->cache->file->{operation}, then you will need update all those to $this->cache->memcached->{operation}. If caching is written like $this->cache->{operation}, you can just adjust configuration something like
$this->load->driver('cache',array('adapther'=>'memcached'));
and set memcached server info in config file. (config/memcached.php)
You told you are not using any caching method. So you should not find any of code I put above.
The last thing I can think about is
$this->output->cache(xxx);
where xxx is cache time in minutes.
it forces entire generated page to be cached;
if you find such lines, you can try comment them out and see what happens
https://www.codeigniter.com/user_guide/general/caching.html
there is a good note: If you change configuration options that might affect your output, you have to manually delete your cache files.
If absolutely none from the examples above is not found, you might use some custom make caching.
Good luck!
Put this in your common controller
$this->output->delete_cache();
I have an application built on TYPO3 CMS and its hosted on AWS. The architecture is like this:
Auto scaling Group
Load Balancer
Two instances hosting the application
sometimes when opening the application, we have a PHP error :
The temporary cache file /var/www/htdocs/typo3temp/Cache/Code/fluid_template/file.tmp could not be written
The Exception is generated by the file FileBackEnd.PHP
if ($result === false) {
throw new \TYPO3\CMS\Core\Cache\Exception('The cache file "' . $cacheEntryPathAndFilename . '" could not be written.', 1222361632);
}
The full content of the file HERE.
I guess the reason of this error, is because the load balancer is sending traffic to the other instance where the file was not generated. AM I right?
To resolve this error, I am thinking of instead of storing the temporary files in the volumes of the instances, we should store them on a shared EFS.
Is that technically, TYPO3 wise, possible ?
P.S: TYPO3 v6.2
Thank you.
This answer applies mostly to the specific Fluid caching but can also be used on other caches, but should not be used on caches like "cache_core" which is essential. You've got a couple of options:
You can mark the particular cache "frozen" which means no new entries will be allowed. An exception will be thrown if trying to set in a frozen cache.
You can distribute the specific filesystem location containing the files (but you should be careful not to distribute too much, as this can negatively impact performance on some scaling setups).
I think you want the second option here which allows expired or new Fluid cache entries to be written, but distributes the resulting class files so they may be loaded on any of the slaves.
The frozen cache is only an option if you are able to 100% pre-generate all the compiled Fluid classes (which depending on your setup may not even be possible with a full crawl of the site).
Unfortunately you are on TYPO3 6.2 - had you been on v8 I would certainly recommend https://github.com/NamelessCoder/typo3-cms-fluid-precompiler-module as a nice way to control where those classes get compiled and stored in a cache, and catching all templates (when they exist in standard paths).
In the fileadmin file storage record, you may specify the path for temporary files - and use a different file storage for them.
So create a new AWS file storage, and set the fileadmin temp directory to a directory in that new file storage. See the documentation at https://docs.typo3.org/typo3cms/FileAbstractionLayerReference/singlehtml/Index.html#processed-files
I've been struggling to put SilverStripe behind a load balancer and I've been fixing multiple problems with rsyncing the instances and using shared storage and have almost got it stable, however I've found another issue which breaks the CMS.
Specifically when you try to add a link in the CMS in the TinyMCE editor, when the pop-up screen shows to select the page/file the JavaScript throws an exception that tinyMCE.activeEditor returns null.
I've rsynced the cache directory silverstripe-cache between the two servers and still there is a discrepancy between the m=timestamp of only a few seconds, but I'm guessing this is enough to cause tiny_mce_gzip.php to be forced to load again.
I have a shared redis cache for session storage, shared db, have rsynced the cache directory and use CodeDeploy to deploy the app so it should all be in sync. What other storage areas could cause the different m timestamp? Has anyone had success with SilverStripe CMS being used behind a load balancer without sticky sessions?
You can disable the gzip version of the HTMLEditor. I've seen this happen before. Try adding the following to your config/config.yml:
HTMLEditorField:
use_gzip: false
After that, do a full flush and try again?
Another option, is the javascript not syncing correctly. For that, you'll need to change the way the ?m=12345 is built. By default, it's built based on the timestamp.
I'll see if I can dig up the md5-based one, which might otherwise solve your problem.
*edit
Here ya go, try creating this somewhere in your project, and add the following to _config.php
Requirements::set_backend(new MysiteRequirementsBackend());
https://gist.github.com/Firesphere/794dc0b5a8508cd4c192a1fc88271bbf
Actual work is by one of my colleagues, when we ran into the same issue.
I have a Symfony2 website that I'm testing in production. I went ahead and cleared its cache because I've made and will probably make more modifications, however there is a small problem:
While the cache is being cleared and say, afterwards I want to warm it up, someone that accesses the website rebuilds the cache. That creates a small problem as the cache is being built, but not completely, while half of it gets deleted because the deletion is still in progress.
What happens afterwards is, the cache is built, but only a part of it. Symfony thinks that the cache is built entirely, and runs without trying to build it anymore, but it runs on a half-built cache. The deletion process is a bit long (~15 sec), so in this timeframe nobody must try and create the cache by accessing the website.
Either that, or the cache is completely built, it overwrites the old cache, and the system treats these new files as old ones, deletes part of them and some others remain. Not entirely sure, I'm not sure how to check this.
For instance, one of the errors that I'd get is
The directory "D:\xampp\htdocs\med-app\app\app\cache\dev/jms_diextra/metadata" does not exist.
If I wouldn't use that bundle I'd get another cache problem from Doctrine. This appears at every website access until I delete the cache again WITHOUT anyone accessing the website. it completely blocks access to the website and makes it non-functional.
Also, what about the warmup? That takes a while, too. What if someone accesses the website while the cache is being warmed up? Doesn't that create a conflict, too?
How to handle this problem? Do I need to close the apache service, clear and warm cache and then restart apache? How is this handled with a website in production?
EDIT
Something interesting that I have discovered. The bug occurs when I delete the cache/prod folder. If I delete the contents of the folder without deleting the folder itself, it seems the bug does not occur. I wonder why.
Usually it is good practice to lock the website into maintenance mode if you're performing updates, or clearing the cache for any other reason in the production. Sometimes web hosting services have this option to handle this for you, or there is a nice bundle for handling maintenance easily from the command line.
This way you can safely delete the cache and be sure no-one visits the page and rebuilds the cache incorrectly.
Usually if you have to clear the Symfony cache it means you're updating to a new version - so not only are you having to clear the cache, but you're probably having to dump assets and perform other tasks. In this case what I've done in the past that has worked very well is to treat each production release as its own version n its own folder - so when you install a new version you do it unconnected from the webserver, and then just change your webserver to point to the new version when you are done. The added benefit is if you mess something up and have to perform a rollback, you just immediately link back to the previous version.
For example, say your Apache config has DocumentRoot always points to a specific location:
DocumentRoot /var/www/mysite/web
You would make that root a symlink to your latest version:
/var/www/mysite/web -> /var/www/versions/1.0/web
Now say you have version 1.1 of your site to install. You simply install it to /var/www/versions/1.1 - put the code there, install your assets, update the cache, etc. Then simply change the symlink:
/var/www/mysite/web -> /var/www/versions/1.1/web
Now if the site crashes horribly you can simply point the symlink back. The benefit here is that there is no downtime to your site and it's easy to rollback if you made a mistake. To automate this I use a bash script that installs a new version and updates the symlinks with a series of commands connected via && so if one step of the install fails, the whole install fails and you're not stuck between version limbo.
Granted there are probably better ways to do all of the above or ways to automate it further, but the point is if you're changing production you'll want to perform the Symfony installation/setup without letting users interfere with that.
I have installed the ZendSkelletonApp on my webserver, which runs with php-fpm (5.5+, so opcache is enabled) and apache.
However, response time is - for the out of the box example application - 110ms, which seems like a lot to me. A "static" php-file is served in ~30ms. I am not saying that this should be possible with a php framework looping through listeners and whatnot, but serving a static controller & template with > 100ms is really slow.
Even with generating class- and templatemaps ( http://akrabat.com/zend-framework-2/using-zendloaderautoloader/ ) and enabling module and configuration caching in the application.config.php , I couldn't get below the 100ms mark.
Are there any other ways of enhancing performance for zf2?
ZF2, due to its nature, has a lot of file-IO for every request. A single page load request to load a data set from a database with doctrine and display the results can result in the opening around 200 php files. (Run an xdebug cache grind and you can see how much time is spent checking for, and opening files. It can be substantial.)
Most of what's being opened is "small," and executes very quickly once it's been read off-disk, but the actual file-io itself can cause significant delays.
A couple things you need to go with a ZF2 app in PRODUCTION:
1) Run "composer dump-autoload -o" which will cache a full auto-load map for the vendor directory. This keeps the autoload system from having to run a "file_exists()" before including a needed file.
2) Generate an autoload classmap for your project itself and make sure the project is configured to use it.
3) Make sure you've set up a template map in your config so ZF2 doesn't have to "assume" the location of your templates, which results in disk IO.
4) Make sure you have an opcode caching solution in place such as Zend Opcache or APC (depending on your PHP version). You will want it set up to have a medium-term cache timeout (an hour or more), and file stat should be disabled in production. You should hard-clear this cache every time you deploy code (can be accomplished via apache restart, a script, etc).
5) If you're using anything that depends on Annotations, such as Doctrine, etc, you MUST make sure the annotations are cached. APC is a good solution for this, but even a file cache is much better than no cache at all. Parsing these annotations is very expensive.
This combination resulted in "instantaneous" page loads for ZF2 for me.
While developing, don't sweat it too much. Install opcode caching if you want, but make sure it will stat files to check if they're changed...otherwise it'll ignore changes you make to the files.