I have a function that generates a table with contents from the DB. Some cells have custom HTML which I'm reading in with file_get_contents through a templating system.
The small content is the same but this action is performed maybe 15 times (I have a limit of 15 table rows per page). So does file_get_contents cache if it sees that the content is the same?
file_get_contents() does not have caching mechanism. However, you can use write your own caching mechanism.
Here is a draft :
$cache_file = 'content.cache';
if(file_exists($cache_file)) {
if(time() - filemtime($cache_file) > 86400) {
// too old , re-fetch
$cache = file_get_contents('YOUR FILE SOURCE');
file_put_contents($cache_file, $cache);
} else {
// cache is still fresh
}
} else {
// no cache, create one
$cache = file_get_contents('YOUR FILE SOURCE');
file_put_contents($cache_file, $cache);
}
UPDATE the previous if case is incorrect, now rectified by comparing to current time. Thanks #Arrakeen.
Like #deceze says, generally the answer is no. However operating system level caches may cache recently used files to make for quicker access, but I wouldn't count on those being available. If you'd like to cache a file that is being read multiple times per request, consider using a static variable to act as a cache inside a wrapper function.
function my_file_read($filename) {
static $file_contents = array();
if (!isset($file_contents[$filename])) {
$file_contents[$filename] = file_get_contents($filename);
}
return $file_contents[$filename];
}
Calling my_file_read($filename) multiple times will only read the file from disk a single time, subsequent calls will read the value from the static variable within the function. Note that you shouldn't count on this approach for large files or ones used only once per page, since the memory used by the static variable will persist until the end of the request. Keeping the contents of files unnecessarily in static variables is a good way to make your script a memory hog.
The correct answer is yes. All the PHP file system functions do their own caching, and you can use the "realpath_cache_size = 0" directive in PHP.ini to disable the caching if you like. The default caching timeout is 120 seconds. This is separate from the caching typically done by browsers for all GET requests (the majority of Web accesses) unless the HTTP headers override it. Caching is not a good idea during development work, since your code may read in old data from a file whose contents you have changed.
Related
I would like to limit the number of times a page is generated in a certain time frame, but am unsure of how to even approach the problem.
Previously, I have solved similar problems by scheduling the pages to be generated and saved in a cron job, but this will not allow me to do this dynamically.
Here is the scenario:
I have n number of user created rooms. I would like to build a JSON api to allow the users to access room information for use on their own pages. However, generating a new JSON result for every request would be extremely inefficient and impose security risks on the database server. I would like to limit it, so that all requests in x amount of time use the same JSON result rather than new results each time.
a simple caching system can be done like this
if (!file_exists($cacheFile) || filemtime($cacheFile) - time() > $cacheLifetime){
//generate json and save to $cacheFile
}
header("Content-Type: application/json");
header("Content-Length: ".filesize($cacheFile));
readfile($cacheFile);
This problem is a typical use case for a cache.
The general algorithm is (when a resource is requested):
if (resource is cached) {
return cached contents
} else {
construct the resource
store resource in cache
return the resource
}
If you want to implement a time-based invalidation, the algorithm becomes:
if (resource is cached) {
if (cached resource older than threshold) {
remove resource from cache
} else {
return cached contents
}
}
construct the resource
store resource in cache
return the resource
The "age" of the cached contents is calculated from the date of creation of this content, that you need to store along with the cache entry.
Depending on how dynamic your users web-sites are, you may also want to not regenerate the JSON if "nothing has changed" since the last generation. Your algorithm would then become:
if (resource is cached) {
if (cached resource older than threshold AND something changed) {
remove resource from cache
} else {
return cached contents
}
}
...
The meaning of "nothing / something has changed" is up to you, it could be the user posts new contents, or some statistics are updated or whatever is used to build your resource. Set a flag on the cache when such an event happens.
The cache can take any form: a flat file on the server, a database row, a session variable, etc.
Use a database to cache the json strings (dont forget a timestamp). Then you are able to search your database first and provide the already generated JSON-string :)
I have the following question: how can I run a php script only once? Before people start to reply that this is indeed a similar or duplicate question, please continue reading...
The situation is as follows, I'm currently writing my own MVC Framework and I've come up with a module based system so I can easily add new functionality to my framework. In order to do so, I created a /ROOT/modules directory in which one could add the new modules.
So as you can imagine, the script needs to read the directory, read all the php files, parse them and then is able to execute the new functionality, however it has to do this for all the webbrowsers requests. This would make this task about O(nAmountOfRequests * nAmountOfModules) which is rather big on websites with a large amount of user requests every second.
Then I figured, what if I would introduce a session variable like: $_SESSION['modulesLoaded'] and then simply check if its set or not. This would reduce the load to O(nUniqueAmountOfRequests * nAmountOfModules) but this is still a large Big O if the only thing I want to do is read the directory once.
What I have now is the following:
/** Load the modules */
require_once(ROOT . DIRECTORY_SEPARATOR . 'modules' . DIRECTORY_SEPARATOR . 'module_bootloader.php');
Which exists of the following code:
<?php
//TODO: Make sure that the foreach only executes once for all the requests instead of every request.
if (!array_key_exists('modulesLoaded', $_SESSION)) {
foreach (glob('*.php') as $module) {
require_once($module);
}
$_SESSION['modulesLoaded'] = '1';
}
So now the question, is there a solution, like a superglobal variable, that I can access and exists for all requests, so instead of the previous Big Os, I can make a Big O thats only exists of nAmountOfModules? Or is there another way of easily reading the module files only once?
Something like:
if(isFirstRequest){
foreach (glob('*.php') as $module) {
require_once($module);
}
}
At the most basic form, if you want to run it once, and only once (per installation, not per user), have your intensive script change something on the server state (add a file, change a file, change a record in a database), then check against that every time a request to run it is issued.
If you find a match, it would mean the script was already run, and you can continue with the process without having to run it again.
when called, lock the file, at the end of the script, delete the file. only called once. and as so not needed any longer, vanished in nirvana.
This naturally works the other way round, too:
<?php
$checkfile = __DIR__ . '/.checkfile';
clearstatcache(false, $checkfile);
if (is_file($checkfile)) {
return; // script did run already
}
touch($checkfile);
// run the rest of your script.
Just cache the array() to a file and, when you upload new modules, just delete the file. It will have to recreate itself and then you're all set again.
// If $cache file does not exist or unserialize fails, rebuild it and save it
if(!is_file($cache) or (($cached = unserialize(file_get_contents($cache))) === false)){
// rebuild your array here into $cached
$cached = call_user_func(function(){
// rebuild your array here and return it
});
// store the $cached data into the $cache file
file_put_contents($cache, $cached, LOCK_EX);
}
// Now you have $cached file that holds your $cached data
// Keep using the $cached variable now as it should hold your data
This should do it.
PS: I'm currently rewriting my own framework and do the same thing to store such data. You could also use a SQLite DB to store all such data your framework needs but make sure to test performance and see if it fits your needs. With proper indexes, SQLite is fast.
On my website there is a php function func1(), which gets some info from other resources. It is very costly to run this function.
I want that when Visitor1 comes to my website then this func1() is executed and the value is stored in $variable1=func1(); in a text file (or something, but not a database).
Then a time interval of 5 min starts and when during this interval Visitor2 visits my website then he gets the value from the text file without calling the function func1().
When Visitor3 comes in 20 min, the function should be used again and store the new value for 5 minutes.
How to make it? A small working example would be nice.
Store it in a file, and check the file's timestamp with filemtime(). If it's too old, refresh it.
$maxage = 1200; // 20 minutes...
// If the file already exists and is older than the max age
// or doesn't exist yet...
if (!file_exists("file.txt") || (file_exists("file.txt") && filemtime("file.txt") < (time() - $maxage))) {
// Write a new value with file_put_contents()
$value = func1();
file_put_contents("file.txt", $value);
}
else {
// Otherwise read the value from the file...
$value = file_get_contents("file.txt");
}
Note: There are dedicated caching systems out there already, but if you only have this one value to worry about, this is a simple caching method.
What you are trying to accomplish is called caching. Some of the other answers you see here describe caching at it's simplest: to a file. There are many other options for caching depending on the size of the data, needs of the application, etc.
Here are some caching storage options:
File
Database/SQLite (yes, you can cache to a database)
MemCached
APC
XCache
There are also many things you can cache. Here are a few:
Plain Text/HTML
Serialized data such as PHP objects
Function Call output
Complete Pages
For a simple, yet very configurable way to cache, you can use the Zend_Cache component from the Zend Framework. This can be used on it's own without using the whole framework as described in this tutorial.
I saw somebody say use Sessions. This is not what you want as sessions are only available to the current user.
Here is an example using Zend_Cache:
include ‘library/Zend/Cache.php’;
// Unique cache tag
$cache_tag = "myFunction_Output";
// Lifetime set to 300 seconds = 5 minutes
$frontendOptions = array(
‘lifetime’ => 300,
‘automatic_serialization’ => true
);
$backendOptions = array(
‘cache_dir’ => ‘tmp/’
);
// Create cache object
$cache = Zend_Cache::factory(‘Core’, ‘File’, $frontendOptions, $backendOptions);
// Try to get data from cache
if(!($data = $cache->load($cache_tag)))
{
// Not found in cache, call function and save it
$data = myExpensiveFunction();
$cache->save($data, $cache_tag);
}
else
{
// Found data in cache, check it out
var_dump($data);
}
In a text file. Oldest way of saving stuff (almost). Or do a cronjob to run the script with the function each 5 minutes independently on the visits.
Use caching, such as APC!
If the resource is really big, this may not be the best option and a file may then indeed be better.
Look at:
apc_store
apc_fetch
Good luck!
I have a simple problem. I use php as server part and have an html output. My site shows a status about an other server. So the flow is:
Browser user goes on www.example.com/status
Browser contacts www.example.com/status
PHP Server receives request and ask for stauts on www.statusserver.com/status
PHP Receives the data, transforms it in readable HTML output and send it back to the client
Browser user can see the status.
Now, I've created a singleton class in php which accesses the statusserver only 8 seconds. So it updates the status all 8 seconds. If a user requests for update inbetween, the server returns the locally (on www.example.com) stored status.
That's nice isn't it? But then I did an easy test and started 5 browser windows to see if it works. Here it comes, the php server created a singleton class for each request. So now 5 Clients requesting all 8 seconds the status on the statusserver. this means I have every 8 second 5 calls to the status server instead of one!
Isn't there a possibility to provide only one instance to all users within an apache server? That would be solve the problem in case 1000 users are connecting to www.example.com/status....
thx for any hints
=============================
EDIT:
I already use a caching on harddrive:
public function getFile($filename)
{
$diff = (time()-filemtime($filename));
//echo "diff:$diff<br/>";
if($diff>8){
//echo 'grösser 8<br/>';
self::updateFile($filename);
}
if (is_readable($filename)) {
try {
$returnValue = #ImageCreateFromPNG($filename);
if($returnValue == ''){
sleep(1);
return self::getFile($filename);
}else{
return $returnValue;
}
} catch (Exception $e){
sleep(1);
return self::getFile($filename);
}
} else {
sleep(1);
return self::getFile($filename);
}
}
this is the call in the singleton. I call for a file and save it on harddrive. but all the request call it at same time and start requesting the status server.
I think the only solution would be a standalone application which does an update every 8 seconds on the file... All request should just read the file and nomore able to update it.
This standalone could be a perl script or something similar...
Php requests are handled by different processes and each of them have a different state, there isn't any resident process like in other web development framework. You should handle that behavior directly in your class using for instance some caching.
The method which query the server status should have this logic
public function getStatus() {
if (!$status = $cache->load()) {
// cache miss
$status = // do your query here
$cache->save($status); // store the result in cache
}
return $status;
}
In this way only one request of X will fetch the real status. The X value depends on your cache configuration.
Some cache library you can use:
APC
Memcached
Zend_Cache which is just a wrapper for actual caching engines
Or you can store the result in plain text file and on every request check for the m_time of the file itself and rewrite it if more than xx seconds are passed.
Update
Your code is pretty strange, why all those sleep calls? Why a try/catch block when ImageCreateFromPNG does not throw?
You're asking a different question, since php is not an application server and cannot store state across processes your approach is correct. I suggest you to use APC (uses shared memory so it would be at least 10x faster than reading a file) to share status across different processes. With this approach your code could become
public function getFile($filename)
{
$latest_update = apc_fetch('latest_update');
if (false == $latest_update) {
// cache expired or first request
apc_store('latest_update', time(), 8); // 8 is the ttl in seconds
// fetch file here and save on local storage
self::updateFile($filename);
}
// here you can process the file
return $your_processed_file;
}
With this approach the code in the if part will be executed from two different processes only if a process is blocked just after the if line, which should not happen because is almost an atomic operation.
Furthermore if you want to ensure that you should use something like semaphores to handle that, but it would be an oversized solution for this kind of requirement.
Finally imho 8 seconds is a small interval, I'd use something bigger, at least 30 seconds, but this depends from your requirements.
As far as I know it is not possible in PHP. However, you surely can serialize and cache the object instance.
Check out http://php.net/manual/en/language.oop5.serialization.php
We would like to implement a method that checks mysql load or total sessions on server and
if this number is bigger than a value then the next visitor of the website is redirected to a static webpage with a message Too many users try later.
One way I implemented it in my website is to handle the error message MySQL outputs when it denies a connection.
Sample PHP code:
function updateDefaultMessage($userid, $default_message, $dttimestamp, $db) {
$queryClients = "UPDATE users SET user_default_message = '$default_message', user_dtmodified = '$dttimestamp' WHERE user_id = $userid";
$resultClients = mysql_query($queryClients, $db);
if (!$resultClients) {
log_error("[MySQL] code:" . mysql_errno($db) . " | msg:" . mysql_error($db) . " | query:" . $queryClients , E_USER_WARNING);
$result = false;
} else {
$result = true;
}
}
In the JS:
function UpdateExistingMsg(JSONstring)
{
var MessageParam = "msgjsonupd=" + JSON.encode(JSONstring);
var myRequest = new Request({url:urlUpdateCodes, onSuccess: function(result) {if (!result) window.open(foo);} , onFailure: function(result) {bar}}).post(MessageParam);
}
I hope the above code makes sense. Good luck!
Here are some alternatives to user-lock-out that I have used in the past to decrease load:
APC Cache
PHP APC cache (speeds up access to your scripts via in memory caching of the scripts): http://www.google.com/search?gcx=c&ix=c2&sourceid=chrome&ie=UTF-8&q=php+apc+cache
I don't think that'll solve "too many mysql connections" for you, but it should really really help your website's speed in general, and that'll help mysql threads open and close more quickly, freeing resources. It's a pretty easy install on a debian system, and hopefully anything with package management (perhaps harder if you're using a if you're using a shared server).
Cache the results of common mysql queries, even if only within the same script execution. If you know that you're calling for certain data in multiple places (e.g. client_info() is one that I do a lot), cache it via a static caching variable and the info parameter (e.g.
static $client_info;
static $client_id;
if($incoming_client_id == $client_id){
return $client_info;
} else {
// do stuff to get new client info
}
You also talk about having too many sessions. It's hard to tell whether you're referring to $_SESSION sessions, or just browsing users, but too many $_SESSION sessions may be an indication that you need to move away from use of $_SESSION as a storage device, and too many browsing users, again, implies that you may want to selectively send caching headers for high use pages. For example, almost all of my php scripts return the default caching, which is no cache, except my homepage, which displays headers to allow browsers to cache for a short 1 hour period, to reduce overall load.
Overall, I would definitely look into your caching procedures in general in addition to setting a hard limit on usage that should ideally never be hit.
This should not be done in PHP. You should do it naturally by means of existing hard limits.
For example, if you configure Apache to a known maximal number of clients (MaxClients), once it reaches the limit it would reply with error code 503, which, in turn, you can catch on your nginx frontend and show a static webpage:
proxy_intercept_errors on;
error_page 503 /503.html;
location = /503.html {
root /var/www;
}
This isn't hard to do as it may sound.
PHP isn't the right tool for the job here because once you really hit the hard limit, you will be doomed.
The seemingly simplest answer would be to count the number of session files in ini_get("session.save_path"), but that is a security problem to have access to that directory from the web app.
The second method is to have a database that atomically counts the number of open sessions. For small numbers of sessions where performance really isn't an issue, but you want to be especially accurate to the # of open sessions, this will be a good choice.
The third option that I recommend would be to set up a chron job that counts the number of files in the ini_get('session.save_path') directory, then prints that number to a file in some public area on the filesystem (only if it has changed), visible to the web app. This job can be configured to run as frequently as you'd like -- say once per second if you want better resolution. Your bootstrap loader will open this file for reading, check the number, and give the static page if it is above X.
Of course, this third method won't create a hard limit. But if you're just looking for a general threshold, this seems like a good option.