We are developing a high end web application with php/mysql and would like to explore more into memcached usage.
From the tutorials we understand that we need to write to memcached server along with writing to mysql tables. But we are confused how to use this if we are loading a list of data with a pagination. In that page we might need to filter data based on different fields. In this scenario, can we rely on memcached along without using mysql database. If not, how can memcached help to scale php application.
first of all, memcached does not persist your data to the disk, so relying on it to keep your data is not a good practice
memcached helps to scale your php application by minimizing the load from your db by serving results from queries or even better minimizing the load off the web server by storing the entire html if you can (there are actually many good use cases for memcached)
consider the follwing flow:
1. client browsing to your page
2. php application requests the data from memcached by a key (the key can be the sql query, the url ... it depends)
3. if memcached has the data use it
4. if not the php application should query the data from mysql
5. after the data is retrieved save it in memcached so next request it will be available in 2
to handle updates, make sure to define ttl to each item you store in the memcached (of course there are also other ways to invalidate the cache)
Related
I want to use session values in my script which are stored at the server using php can any one kindly explain the process to achieve this.
I want to build a chat app for this am planning to use those session values.
Assume usera and userb are logged in and their userid is sessioned based on this scenario i want to do a chat app.
Now i had done the app but i had used setinterval function of Javascript and am calling the chats i want to avoid the database hits on every 3 mill sec.
Kindly Help me out
Thanks In Advance
You're basically attempting to use PHP session files as a file cache.
Instead, you should use an object caching system such as Memcached or Redis. If memory caching isn't an option (shared hosting, etc), then you could implement your own file cache (or you could use something like PHPFastCache, which supports file caching).
Note: File caching for a chat app may or may not speed up your application. It depends on how you implement it and a number of other factors.
Hi put the session value in input box,
<input type='hidden' id='session_value' value='<?php $_SESSION['value']?>'>
Using the id fetch the session value in script,
<script>
var session_value = document.getElementById ( "session_value" );
</script>
3ms is insanely short delay to run a polling chat system. I suggest increasing it to at least 200ms but preferably around 1000ms.
$_SESSION values are per user and not recommended for viewing a chat stream for a number of reasons. Instead it sounds like you are looking more to just update the chat feed.
The database unless it is hosted on another server and $_SESSION will be the equivalent, since the database is effectively files as well. The database will actually generally be faster than reading raw file storage since Queries are normally cached and Indexing helps lookup records quicker. In addition you won't have to worry about concurrent connections to the files either.
If anything enable OPCache and install APCu for your PHP installation, to help aid the serving of requests. OPCache will cache your compiled OP code into memory so that subsequent requests to the file won't need to be recompiled.
APCu will act as your file cache, again storing your rendered data in memory.
Additionally many Database Frameworks such as Doctrine can also utilize APC caching for query and result caching.
Instead of using a InnoDB or MyISAM storage engines for your chat messages I suggest trying the MEMORY storage engine.
So instead of accessing the File System I/O your database would instead be utilizing the Memory I/O. The general concept is few writes, many reads. Since one person writes to the database, requires everyone to read the data. Just keep in mind that the Memory storage engine is temporary and is lost if the server restarts or power is lost.
For more information see: https://dev.mysql.com/doc/refman/5.6/en/memory-storage-engine.html
Overall if you are able, I would suggest look at using Socket IO (Websockets) instead of either database or file based caching. This puts the load on the clients instead of the server, and everything occurs in real-time instead of polling for changes.
For some examples see:
Ratchet http://socketo.me/
React http://reactphp.org/
Node.js http://tutorialzine.com/2014/03/nodejs-private-webchat/
I am a student of Computing, I work as a web developer using PHP. I have worked on a web application which has a lot of SQL queries that might be sent to the database server +-20 times to query the same information per page load.
For this reason. I decided to go with my own custom File Based Caching by extending the codeIgiter caching library which is unusable due to its reliance on the URL/URI. I basically took out the URL/URI dependency and used SHA-256 for access keys. I store all my cached data as encrypted data using the default CodeIgniter encryption library just to add extra security in case someone opens the cache file, I did not want to store plain text data.
Is the encryption for the cached data expensive/resource heavy on server?
Is file based caching faster than the Mysql/MariaDB/PostGres?
File Based Caching works great for me to minimise database queries, but can this really improve the speed on my web application? How fast is can this be, versus DB?
I got carried away with excitement, as a result my web application is now a farm of cache is this good or bad for speed? I need advise from people with more experience on this topic.
My Custom File Based caching has these basic functions:
save($key,$value)
get($key)
delete($key) or delete($key,$value)
metadata($key)
Thank you.
You can use memcached or redish for the caching which will reduce the number of queries on the database and it will improve the speed of the site.
Apart from the that check the queries and optimize them.
I am using sessions in PHP to track if a user is logged in. I do not use it to store any other data about the user; essentially it is like checking a hash table to see if the user has authenticated.
Would there be some advantage to using redis instead of native PHP sessions?
I'm curious about performance, scalability, and security (not really concerned with code complexity).
Using something like Redis for storing sessions is a great way to get more performance out of load balanced servers.
For example on Amazon Web Services, the load balancers have what's called 'sticky sessions'. What this means is that when a user first connects to your web app, e.g. when logging in to it, the load balancer will choose one of your app servers and this user will continue to be served from this server until they exit your application. This is because the sessions used by PHP, for example, will be stored on the app server that they first start using.
Now, if you use Redis on a separate server, then configure your PHP on each of your app servers to store it's sessions in Redis, you can turn this 'sticky sessions' off. This would mean that any of your servers can access the sessions and, therefore, the user be served from a different server with every request to your app. This ultimately makes for more efficient use of your load balancing set-up.
You want the session save handler to be fast. This is due to the fact that a PHP session will block all other concurrent requests from the same user until the first request is finished.
There are a variety of handlers you could use for PHP sessions across multiple servers: File w/ NFS, MySQL Database, Memcache, and Redis.
The database method (using InnoDB) was the slowest in my experience followed by File w/ NFS. Locking and write contention are the main factors. Memcache and Redis provide similar performance and are by far the better alternatives since all operations are in RAM. Redis is my choice because you can enable disk persistence, and Memcache is only memory based.
I explain Redis Sessions in PHP with Kohana if you want more detail. Here is our dashboard for managing Redis keys:
I don't really think you need to worry much about sessions unless you get MASSIVE ammounts of traffic, PHP handle sessions nicely, and if you store only that little data, it should be fine even with a lot of requests, and about performance it should be close, as redis is not native to PHP.
With 10k users, if each user uses like 1kb data of sessions, it would consume 10,000kb or 10~mb, which is not much; PHP is smart enough to use a good enough data structure to hold and quickly write and read those values. The problem is if the session data is too big, or for some reason the server consumes too many resources reading the session data, but that's normally if it's the data is too big.
I have installed redis on my server and implemented object caching for data returned within a PHP based web application. The php model essentially executes a reasonably complex query and returns a detailed array of data. I have tested the caching and found everything to be working as expected. I first check to see if the key exists in redis. If it does, redis returns the data, the model unserializes and returns the previously cached data. If the cache has expired, the model executes the sql query, returns the data and sets the key and serialized value in redis.
So here are my questions.
I'm not sure how to really benchmark this as it is all browser based. What tools are there out there that would allow me to get a reasonable benchmark to compare caching and not caching. I'm thinking of perhaps a php script that calls the api 1000 times via curl.
I implemented this in redis because I once read that caching with redis will work across multiple sessions or ip addresses accessing the site. For example, if the api is accessed 1000 time an hour by multiple ip addresses/users, I am assuming this approach will reduce the load on the mysql server and let redis do the work of returning the cached data instead. Can anyone shed some light on this? Are my assumptions valid?
All comments are welcome!
Thanks!
Dave
To benchmark the web site, I would use something like Siege rather than writing a specific PHP script.
Regarding Redis usage, caching things in in-memory stores like memcached or Redis is now very common. Both memcached and Redis are suitable for this purpose, although for pure caching, memcached is arguably easier to setup. 1000 times an hour represents only 3.6 TPS - any data store (including MySQL) will support such traffic without any issue. Now, multiply this traffic by 100 or 1000, and the caching layer (memcached or Redis) will become mandatory to protect your database.
To use Redis for caching, you may want to check the EXPIRE command and have a look at the maxmemory-policy parameter in the configuration file.
I have done extensive testing of cache backends for the Zend_Cache library. The tests were done using multiple php-cli processes and randomized data and considered read performance, write performance and cache tag cleaning performance. If testing just the cache backend the web server performance is not relevant so I recommend testing via CLI to simplify the testing. Also, testing with only one process will not give you an accurate picture of a backend's characteristics under heavy load.
MySQL is very fast itself and if you are doing single-record indexed queries then MySQL's own query cache will be very fast. I'd only recommend adding an additional caching layer for things that are slow (aggregated results of multiple queries or generating chunks of HTML). You can use Zend_Cache without including the entire Zend framework so I highly recommend you check out both Cm_Cache_Backend_Redis and Cm_Cache_Backend_File.
Here's a little background, currently i have
3 web servers
one db server which also host memcache for php sessions for the 3 web servers.
I have the php configs on the 3 servers to point to the memcache server for sessions. It was working fine until alot of connections were being produced for reads etc, which then caused connection timeouts.
So I'm currently looking at clustering the memcache on each web server for sessions, my only concern is how to go about making sure that memcache on all the servers have the same information for sessions.
Someone guided me to http://github.com/trs21219/Memcached-Library because i am using codeigniter but how do i converge my php sessions onto this since memcache seems as a key-value store? Thanks in advance.
Has anyone checked out http://repcached.sourceforge.net/ and does it work?
I'm not sure you have the same expectations of memcache that its designers had.
First, however, memcache distribution works differently than you expect: there is no mechanism to replicate stored information. Each memcache instance is a simple key-value store, as you've noticed. The distribution is done by the client code which has a list of all configured memcache instances and does a hash of the key to direct it to one of the instances. It is possible for the client to store it everywhere and retrieve it locally, or for it to hash it multiple times for redundancy, but these are not straightforward exercises.
But the other issue is that memcache is designed for reasonably short-lived data that memcache is allowed to throw away at any time. This makes it really good for caching frequently accessed data that can be a little stale (say up to a few minutes old) but might be expensive to retrieve (such as almost a minute to generate from a query).
PHP sessions don't really qualify for this, in my experience. A database can easily support many thousands of PHP sessions with barely visible traffic, but you need a lot of memcache storage to support the same number: 50k per session and 5000 sessions means close to 256Mb, and then there is all the other data you want to put in there. Not enough storage and you get lots of unexplained logouts (as memcache discards session data when under memory pressure) and thus lots of annoyed users who have to keep logging in again.
We've found GREAT advantage applying MongoDB instead of MySQL for most things, including session handling. It's far faster, far smaller, far easier. We keep MySQL around for transactional needs, but everything else goes into Mongo now. We've relegated memcache to simply caching pages and other data that isn't critical if it's there or not, something like smarty does.
There is no need to use some 3rd party libraries to organize memcached "cluster".
http://ru.php.net/manual/en/memcached.addserver.php
Just use this function to add several servers into the pool and after that data will be stored and distributed over those servers. The server for storing/retrieving the data for the specific key will be selected according to consistent key distribution option.
So in this case you don't need to worry about "how to go about making sure that memcache on all the servers have the same information for sessions"