How to save a variable at application level(same for all users) in php which will get updated after some time?
I've tried to find about it. I've found the following solutions:
Implement using file handling.
Cache (Memcache or APC)
Implement using Database Support
2 is considered as best (AFAIK). (I'm not allowed to install anything on the Server)
What about other two (mentioned above) or any other options and how I can implement those? I'm bit concerned because traffic is moderately high (but the bad thing is that I still can't use any cache mechanism). We just need to save the contents of buffer of around 255 bytes at application level.
Any snippets, pointers or help of any sort would be highly appreciated.
Thanks.
You need a permanent storage, not cache or something like that.
If your application doesn't use a database already, there are several options you can choose from:
write to a text file, a simple one line entry or preferably in a format like xml, json
write to a light storage engine like sqlite, simple storage (Amazon S3)
If your app uses a database already, why not store that data in a separate table?
This is what databases are for. If you don't want to spend a lot of time setting up a large database application, try out sqlite.
Some caches (memcache in particular) are lossy, and most won't survive being restarted. Use a database.
If you do not have the option to use databases you can consider writing the data to a file on disk.
Related
How to save a variable at application level(same for all users) in php which will get updated after some time?
I've tried to find about it. I've found the following solutions:
Implement using file handling.
Cache (Memcache or APC)
Implement using Database Support
2 is considered as best (AFAIK). (I'm not allowed to install anything on the Server)
What about other two (mentioned above) or any other options and how I can implement those? I'm bit concerned because traffic is moderately high (but the bad thing is that I still can't use any cache mechanism). We just need to save the contents of buffer of around 255 bytes at application level.
Any snippets, pointers or help of any sort would be highly appreciated.
Thanks.
You need a permanent storage, not cache or something like that.
If your application doesn't use a database already, there are several options you can choose from:
write to a text file, a simple one line entry or preferably in a format like xml, json
write to a light storage engine like sqlite, simple storage (Amazon S3)
If your app uses a database already, why not store that data in a separate table?
This is what databases are for. If you don't want to spend a lot of time setting up a large database application, try out sqlite.
Some caches (memcache in particular) are lossy, and most won't survive being restarted. Use a database.
If you do not have the option to use databases you can consider writing the data to a file on disk.
I have php db driven website that uses a lot of flash for user interaction.
I need to make it multilangual like 20+ languages.
Site is quite large and has a lot of users coming to it every day.
Other developer i work with saying we should store translation in local files e.g. /lang/english.php /lang/german.php etc.
I was thinking since database is on the same dedicated server there should not be a slow down, which way you think will work is faster?
I don't know if it's an option, but you could also use gettext().
That way your translations are stored in local files (faster than a database) and you have the advantage that there are programs like poedit (takes some getting used to...) that you or a translator can use to automatically generate the translation files so it's a bit easier to maintain then php files.
Local files are a LOT faster than DB content (Although you can save the DB output in a local cache, like files or even memcache or APC), probably not that easy to translate, but it will help you with the basic speed of implementation too, You should take a look at:
http://framework.zend.com/manual/en/zend.translate.html
You can use only this part of the framework and it will give you a HUGE boost, it supports DB based translation or local files (a lot of adapters)
UPDATE:
thanks Corbin, you are right, it's better to have the direct link.
Fictious Background
I get 100 hits a minute for "the hottest car"
The hottest car always changes by the minute and its currently: "A Pinto"
Everytime I receive what the current hottest car is, I save it into a MySQL Database.
Situation
Everytime I get a hit for "Whats the hottest car", I need to return the answer. I feel confident that retrieving the answer from a file vs. a DB will be faster and less work for the processor due to PHP storing the file into memory. My concern is that if I get a new file, how do I make sure i'm returning the information in the new file and not the old information stored in memory.
P.S. If my assumptions are wrong and there is a faster way, please let me know.
Thanks
Be careful with your assumptions. It can be tempting to assume that file access is faster, since you just have to read a file instead of connect to, query, and retrieve from a database. But bear in mind that databases are designed from the ground up for this kind of fast access to rapidly-changing information, and they have a lot of optimizations built in.
So look into caching, it is often a win, but I would not assume that file access is always faster. You can of course profile the different approaches to see if you have a bottleneck.
Your assumption is most probably going to be invalid. MySQL has a query cache which keeps your query in memory. Even if not I don't think you should but using the filesystem or unless you are using /dev/shm because that's mapped to memory. I would use a library like Cache_Lite to ease the pain of caching.
But if you want to make your site really fast you should install APC. You should always install APC if you want your website to be fast because caching the compiled bytecode of PHP scripts. Or use an in-memory databases redis or memcached, because these are even better in memory databases. redis is the easiest to install only using make and you don't need any ROOT permission either.
P.S: You should check out this redis tutorial because it is really powerful in-memory database.
I'm pretty sure it would be a bad idea to store that information in a file. The biggest problem is file read locks. If one person tries to get the file while another person is getting it, there's a conflict and a fatal error.
You really should go the database route, especially if you're planning on persisting the older "hottest cars". And if performance is a concern, you should look into PHP caching (see #Andrew's comment).
Instead of using a DB/file it should be the fastest way if you directly access the memory: http://www.php.net/manual/en/book.shmop.php
Memcache, memcache, memcache!
http://us2.php.net/memcached
We need a caching solution that essentially caches data (text files) anywhere from 3 days up to a week based on user preferences and criteria. In this case memory based caching does not make sense to us. We were referred to MemcacheDB however I also thought of some NO SQL solutions.
Our current application uses RDMS (MYSQL) and I guess it makes sense to use MemcacheDB however NOSQL does appeal as it is something more on the horizon. However we have not deployed a production level application under NOSQL and the beta stuff does not settle well with management/investors. Any how what are your thoughts and how would you address it?
Thank You
CouchDB and MongoDB are both great databases, but they are terrible choices for a cache layer on top of your existing RDBMS. Besides the fact that they are still fairly immature, they don't suit the purpose at all. Also, speed-wise you'd be better off going without a cache layer than using CouchDB or MongoDB--they are both slower for simple read/writes than even MySQL. Yes, the NoSQL databases are "cool", but that does not mean you should use them for something they were not meant to do.
I'd go with Memcached, as it's just about the fastest and lightest thing you'll find, and it's well-known and well-supported.
If you're worried about the appeal to management and investors, and the current system (you mention MySQL) works, why would you change? You're moving from a fairly stable project to projects still in beta, and what value are you adding if the current system already works?
As mentioned above, all CouchDB resources contain etags.
What wasn't mentioned is that you can put any HTTP caching solution in front of CouchDB and have it do etag based caching. This way you can use Varnish, nginx, whatever you want.
I'd also take a look at Cassandra ( http://cassandra.apache.org/ ). I've tried MemcacheDB and CouchDB, somehow found Cassandra more appealing (Dunno about PHP since i work with Coldfusion). Here's related question Cassandra PHP module
CouchDB does already some caching: when you get a document the server also sends the HTTP ETag header (it's the same as the document revision in CouchDB).
The next time the browser asks for the same document it sends the Etag received. If the document hasn't been modified the server responds with the HTTP code 304 Not Modified and your browser retrieves the document from its local cache.
However if you have to cache files for different times based on user preferences, even if the text file changes, probably your best option is to write custom code that sends the approriate HTTP caching headers based on the user preferences.
For completeness another good option is Redis. You get performance comparable to Memcache but Redis also supports various data structures (hashes, lists, set, sorted sets) and atomic operations.
If you memcached with persistence, you should check out Redis. It has all the memecached functionality (and more) along with persistence.
I have not tried it myself, but I do remember reading that Redis also supported the memcached API as well.
I'm working on a PHP content management system and, in testing, have noticed that quite a few of the system's MySQL tables are queried on almost every page but are very rarely written to. What I'm wondering is will this start to weigh heavily on the database as site traffic increases, and how can I solve/prevent this?
My initial thoughts were to start storing some of the more static data in files (using PHP serialization) but does this actually reduce server load? What I'm worried about is that I'd be simply transferring the high load from the database to the file system!
If somebody could clue me in on the better approach, that would be great. In case the volume of data itself has a large effect, I've detailed some of the data I'll be storing below:
Full list of Countries (including ISO country codes)
Site options (skin, admin email, support URLs etc.)
Usergroups (including permissions)
You have to remember that reading a table from a database on a powerful server and on a fast connection is likely to be faster than reading it from disk on your local machine. The database will cache the entirety of these small, regularly accessed tables in memory.
By implementing the same functionality yourself in the file system, there is only a small possible speed up, but a huge chance to mess it up and make it slower.
It's probably best to stick with using the database.
Optimize your queries (using mysql slow query log) and EXPLAIN function.
If tables are really rarely written to you can use native MySQL caching. You have nothing to change in you code, just enable mysql caching in my.conf.
Try out using template engine like Smarty (smarty.net). It has it's own caching system that works pretty well and will REALLY reduce server load.
You can also use Memcache, but it is really worth using only with really high load websites. (I think that Smarty will be enough.)
Databases are much better at handling large data volumes than the native file system.
Don't worry about optimizing your site to reduce server load, until you actually have a server load problem. :-)
The tables you mentioned (countries and users) will normally be cached in memory by MySQL directly unless you are expecting quite a few millions of records in these tables.
In case where these tables will not fit in memory, you may want to consider a general-purpose distributed memory caching system, such as memcached.
If your database is properly indexed, it will be much faster to query data from the database. If you want to speed that up, look into memcached or similar.
Databases are exactly for this purpose.. To store and provide data. Filesystem is for scripts and programming.
If you encounter load problems, consider using Memcached or another utility for database.
You may also consider trying to cache different parts of your page directly into database as whole sections (eg. a sidebar, that doesn't change too much, generated header section, ..)
you could cache output (flush(), ob_flush() etc.) to a file and include that instead of having multiple MySQL reads. caching is definitely faster than accessing MySQL multiple time.
reading a static file is much faster than adding overhead via php and mysql processing.
You need to evaluate the performance via load testing to avoid prematurely optimising.
It would be foolish and quite possibly increase overall load to store data in files with serialization, databases are really good at retrieving data.
If after analysis there is a true performance hit (which I doubt unless you are talking about massive loading), then caching is a better solution.
It's more important to have a well designed system that facilitates changes as needs arise.
Here's a link to a couple script that will essentially do what dusoft is talking about and cache the output buffer to a file:
http://www.addedbytes.com/articles/caching-output-in-php/
Used this way, it's more of a bolt-on-after-the-fact type of solution, but this same behavior can certainly be implemented in a more integrated fashion if considered earlier in the process. Many frameworks also have this kind of thing built in.