Let's say I want to have a variable which should keep it's value.
Simple example:
<?php
$files = array('file1.txt','file2.txt','file3.txt'); // list of the files to process
$file = file_get_contents($files[$i]);
process_the_file($file);
$i++;
exit();
So I'd like to process only one file per script run. And in my case the index of the currently processed file is $i.
From my pont of view there are two methods available:
Store variable in a file. I can serialise (and unserialize upon initialization) any data and store in a text file. However, I should keep an eye on the script termination to be sure that the needed values will be stored before the script terminated.
Use database. It looks bulky and as previously described - I should keep an eye on storing.
So both of the methods looks too comlicated. Is there any better solution?
The solution depends on the task:
Use a cache system (memcached, redis, etc.) - when you need a runtime storage (i.e. values may be deleted and will not break an application logic).
Use DB(MySQL, Mongo) - when you need a permanent storage for values (i.e. values may not be deleted - this will break an application logic).
Use Files for both variants.
Sessions is good enough when you need to store data for one user between requests.
The best solution is the solution that the most suitable for your task. I would recommend to use either 1 or 2 variant: in projects where I am working we use memcached for a runtime storage and mongodb for a permanent storage.
Related
I need to hold a semi-static large object in cache so I don't need to request it every time from database. Something like $_SESSION, but not tied to a session, because the data are common to all users.
I can cache client side that data, once I got it, but I would like to avoid disturbing the database with select queries of large data that (almost) never changes.
Also, I cannot add modules (like APC cache) in this environment.
I could store my data into a file, say a JSON, which I read with php instead of querying db, but accessing filesystem is also disturbing if php needs to do it many times per seconds AND filesize is not tiny.
Is there a built in way in php to store objects in memory, common to all php instances?
EDIT: Could I use $_session as storing space, forcing session_id to be always the same? Is it dangerous? I don't use sessions for the application itself. I tried and it works
Most Operating systems will store the result of reading from disk in its cache.
This means that the disk will not be hit each time. File based storage is actually pretty quick for multiple reads of the same file as its really just coming direct from memory.
as long as "pretty large" still means fits in memory this way should be fine
I have considered about it for a long time.
I think I can't use all the API/PHP extension (e.g. memcache, APC, Xcache) that need to install something in my remote Linux server, as my web host server is a shared server, what I just can do is to place files/scripts in the httpdocs folder.
Is there any suggestion for me that can let me programmatically use caching and access the memory?
Actually what I aim at is to find a "place" to save some data, that can be accessed in higher speed than entering the DB to fetch data, and also to reduce the loading to DB.
That means, it is not a must to use memory, if someone can give any other effective suggestions. e.g. will using text file be a good choice?(actually I am just guessing it)
The PHP version of mine is 5.2.17. And I am using MySQL DB.
Hope someone can give me suggestions
Flat files will always be the EASIEST way for caching, but it will be slower than accessing data directly from memory. You can use MySQL tables that are stored in memory. you need to change the engine used by tables to memory. NOTE that this will work only if your db is on the same server as web server.
Set up an in memory table with two columns key and value. variable name will be a key and its contents are values. if you need to cache array, objects then serialize the data before storing it.
If you need to limit the size of in memory table add one more column hitCount. for each read increase the count by one. while inserting new row, check for max number of rows and if its reached a limit delete the row with lowest hitCount.
To check which one is faster (file caching or in memory cache) use following code
<?php
function getTime()
{
$a = explode (' ',microtime());
return(double) $a[0] + $a[1];
}
?>
<?php
$Start = getTime();
//Data fetching tasks comes here
$end = getTime();
echo "time taken = ".number_format(($End - $Start),2)."seconds";
?>
If possible let us know how efficient it is... Thanks
You could very easily just use flat text files as a cache if your DB queries are expensive. Just like you would use memcache with a key/value system, you can use filenames as keys and the context of the files as values.
Here's an example that caches the output of a single page in a file; you could adapt it to suit your needs: http://www.snipe.net/2009/03/quick-and-dirty-php-caching/
Flat file is the easiest way to cache business logic, queries etc on a shared server.
To cache any DB requests your best bet is to fetch the results, serialize them and store them in a file with a possible expiry date (if required). When you need to fetch those results again just pull in the file and unserialize the previously serialized data.
Also if the data is user based cookies and sessions will work too, for as long as the user stays on the application at least. If your pulling a lot of data it would still be better to go with the first option and just save the files based on a user/session id.
Depends on the size of data to cahce.
Based on the restriction of your server environment:
Use flat file( or maybe sqlite db) to cache your data for large data set (e.g., user
preference, user activity logs.)
Use share memory to cache your data for the smaller data set (e.g., system counter, system
status.)
Hope this helps.
I want to add some static information associated with string keys to all of my pages. The individual PHP pages use some of that information filtered by a query string. Which is the better approach to add this information? Generate a 100K (or larger if more info is needed later) PHP file with an associated array or add an other DB table with this info and query that?
The first solution involves loading the 100K file every time even if I use only some of the information on the current page. The second on the other hand adds an extra database call to the rendering of every page.
Which is the less costly if there are a large number of pages? Loading a PHP file or making an extra db call?
Unless it is shown to really be a bottleneck (be it including the php file or querying the database), you should choose the option that is best maintainable.
My guess is that it is the second option. Store it in a database.
Storing it in a database is a much better plan. With the database you can provide better data constraints, more easily cross reference with other data and create strong relationships. You may or may not need that at this time, but it's a much more flexible solution in the end.
What is the data used for? I'm wondering if the data you need could be stored in a session variable/cookie once it is pulled from the database which would allow you to not query the db on the rendering of every page.
If you were to leverage a PHP file then utilizing APC or some other opcode cache will mitigate performance concerns as your PHP files will only be loaded each time the file changes.
However, as others have noted, a database is the best place to store this stuff as it is much easier to maintain (this should be your priority to begin with).
Having ensured ease of maintenance and a working application, should you require a performance boost then generally accepted practice would be to cache this static data in an in-memory key/value store such as memcached. This will give you rapid access to your static values (for most requests).
I wouldn't call this information "static".
To me, it's just a routine call to get dome information from the database, among other calls being made to assemble whole page. What I am missing?
And I do agree with Dennis, all optimizations should be based on real needs and profiling. Otherwise it's effect could be opposite.
If you want to utilize some caching, consider to implement Conditional GET for the whole page.
I have some small sets of data from the database (mysql) who are seldom updated.
Basically 3 or 4 small bi dimensional arrays (50-200 items).
This is the ideal case for memcached, but I'm on a shared server and can't install anything.
I only have PHP and MySQL.
I'm thinking about storing the arrays on file and regenerate the file via a cron job every 2-3 hours.
Any better idea or suggestion about this approach?
What's the best way to store those arrays?
If you're working with an overworked MySQL server then yes, cache that data into a file. Then you have two ways to update your cache: either via a cron job, unconditionally, every N minutes (I wouldn't update it less frequently than every hour) or everytime the data changes. The best approach depends on your specific situation. In general, the cron job way is the simplest but the on-change way pretty much guarantees that you won't ever use stale data.
As for the storage format, you could just serialize() the array and save the string to a file. With big arrays, unserialize() is faster than a big array(...) declaration.
As said in the comments, it would be better to check whether the root of the problem can't be fixed first. A roundtrip that long sounds like a network configuration problem.
Otherwise, if the DB simply is that slow, nothing speaks against a filesystem based cache. You could turn each query into an md5() hash, and use that as a file name. Serialize() the result set into the file and fetch it from there. Use filemtime() to determine whether the cache file is older than x hours. If it is, regenerate the query - or in fact, to avoid locking problems on the cache files, use a cron job to regenerate it.
Just note that this way, you would be dealing with whole result sets that you have to load into your script's memory all at once. You wouldn't have the advantage of being able to query a result set row by row. This can be done too in a cached way, but it's more complicated.
My english is not good, sorry.
Some times I have read about any alternative to memcache. Is complex, but I think that you can use http://www.php.net/manual/en/ref.sem.php acceding to shared memory.
A simple class example used for storing data is here:
http://apuntesytrucosdeprogramacion.blogspot.com/2007/12/php-variables-en-memoria-compartida.html
Is written in spanish, sorry, but the code is easy to understand (Eliminar=delete)
I never have test this code!! and I don't know if it's viable in a shared server.
I'm using the session array to cache chunks of information retrieved from the db:
$result = mysql_query('select * from table');
array_push($_SESSION['data'],new Data(mysql_fetch_assoc($result)));
My question is, is there a limit/a sizeable amount of information that can/should be passed around in a session? Is it ill advised or significantly performance hindering to do this?
By default, $_SESSION data is stored on disk in the /tmp directory of your server. As long as you have enough room in there AND you aren't hitting your PHP memory limit, you're fine.
However, if you're attempting to cache a query that is the SAME for a larger number of users, you might want to use something like APC or memcache that isn't tied to the individual user. Otherwise, your essentially going to cache the same result 1x for each user, and not leveraging a cache across all users.
I think the answer would depend on where you are storing your data and how fast you can transfer it there.
If the data is 44 MB big, and you are on a 1000base-T network, you can expect it to take 1 second to actually transfer THERE. And 1 second to transfer back..
If you use local memory, then you have a finite amount of memory the machine.
If you use disk, then you have load/save times (disk is slow).
But also keep in mind, PHP has a finite amount of memory it allows a script to use. I think the default setting is 8 MB.
If you are talking about large blocks of data, you may want to consider Redis, Tokyo Cabinet or other key/value stores. Or even a backend interface to manipulate the data/cache it for you without transferring it through PHP.
Because Session data is stored in a file (or database record) on your server, it shouldn't matter too much how much data you store in it. I would just advise against huge objects.
You might want to look at APC or memcached to cache the results instead, as it is not a per-user cache, and it uses the memory instead of files.
The session is serialized and written to disk by default, so depending on the size and the amount of users things can become slow. However both things can be changed (read the session manual under http://php.net/session for all details) like using memcache for in-memory storage of the data. Best thing is to try it out under an environment as similar as possible tothe live system and check the resulting load and throughput.
Mmm, tricky. I think you could save it in the session. The real question is: do you want that all that information serialize and unserialize every time a client make a request?
I think it would be OK to save it in there if you will use all that information in every page of your website, but this is unprobable. It would be better if you save that information in a directory like /temptables/sometable/ and each file have the name of the session. You can use session_id to get it, and save and load the information in the pages you have to use with:
$info = unserialize(file_get_contents('/templatebles/sometable/'.session_id().'.ser'));
and saving with:
file_put_contents('/temptables/sometable/'.session_id().'.ser'), serialize($info));
But you need a cron job to clean that directory for old file. You can do it getting the session from the filename and ask for some variable, like 'itsalive', using session_start() or doing something like file_exists(session_save_path().'/sess_'.$session_name) to check if you should delete the temporary file.