Here's a little background, currently i have
3 web servers
one db server which also host memcache for php sessions for the 3 web servers.
I have the php configs on the 3 servers to point to the memcache server for sessions. It was working fine until alot of connections were being produced for reads etc, which then caused connection timeouts.
So I'm currently looking at clustering the memcache on each web server for sessions, my only concern is how to go about making sure that memcache on all the servers have the same information for sessions.
Someone guided me to http://github.com/trs21219/Memcached-Library because i am using codeigniter but how do i converge my php sessions onto this since memcache seems as a key-value store? Thanks in advance.
Has anyone checked out http://repcached.sourceforge.net/ and does it work?
I'm not sure you have the same expectations of memcache that its designers had.
First, however, memcache distribution works differently than you expect: there is no mechanism to replicate stored information. Each memcache instance is a simple key-value store, as you've noticed. The distribution is done by the client code which has a list of all configured memcache instances and does a hash of the key to direct it to one of the instances. It is possible for the client to store it everywhere and retrieve it locally, or for it to hash it multiple times for redundancy, but these are not straightforward exercises.
But the other issue is that memcache is designed for reasonably short-lived data that memcache is allowed to throw away at any time. This makes it really good for caching frequently accessed data that can be a little stale (say up to a few minutes old) but might be expensive to retrieve (such as almost a minute to generate from a query).
PHP sessions don't really qualify for this, in my experience. A database can easily support many thousands of PHP sessions with barely visible traffic, but you need a lot of memcache storage to support the same number: 50k per session and 5000 sessions means close to 256Mb, and then there is all the other data you want to put in there. Not enough storage and you get lots of unexplained logouts (as memcache discards session data when under memory pressure) and thus lots of annoyed users who have to keep logging in again.
We've found GREAT advantage applying MongoDB instead of MySQL for most things, including session handling. It's far faster, far smaller, far easier. We keep MySQL around for transactional needs, but everything else goes into Mongo now. We've relegated memcache to simply caching pages and other data that isn't critical if it's there or not, something like smarty does.
There is no need to use some 3rd party libraries to organize memcached "cluster".
http://ru.php.net/manual/en/memcached.addserver.php
Just use this function to add several servers into the pool and after that data will be stored and distributed over those servers. The server for storing/retrieving the data for the specific key will be selected according to consistent key distribution option.
So in this case you don't need to worry about "how to go about making sure that memcache on all the servers have the same information for sessions"
Related
I will like to know how memcached manage cache for php sessions i mean. I would like to design a php app that scale out and in each http-PHP server include a memcached layer for (db,app cache and session caching), but if memcached dont replicate de data when a user come to webserver1 dont see the same session in webserver2.
memcached1 and memcached2 need to be replicated to handle php sessions
thanks in advance.
regards.
While I agree there is no question here we could try to help the OP understand how memcache works.
When you use memcache which is an in-memory cache how you set it up is determined upon your current infrastructure.
For instance if you only have 1 web server you could install memcache on that same machine along with the database layer being on that machine as well. This works for increasing performance of the site because the site can get data from memcache (in memory) rather than from the database (on disk, and slower to read). Using it in this manner is good but as your site requires better performance or scalability you would probably start up a cluster of web servers behind a load balancer.
This is when things can get a bit tricky. You have all these machines and you are thinking that you need to have memcache on every machine so how do we replicate these instances? The simple answer is you don't. If you have multiple web servers the best method is to put memcache on it's own server (or cluster behind a load balancer), this way every web server is hitting the same IP address for the memcache server(s).
You do not need to worry about keeping anything in sync because the way memcache works is it creates a hash that specifies which server the key has been assigned to (when you have a cluster of memcache servers).
Based on this question it would appear that you would need to do one of the following:
1.) Read up on system architecture
2.) Hire someone to architect your systems layer.
My best suggestion would be to use a single server for your memcache instance and set the web servers to use that for memcache rather than trying to run memcache on each of the web servers.
Joseph.
I undertand your point, I already test the architecture with a separate memcached server (and redis too). My intentions is to "pack" the application server in a unit (docker) and the measure the load parameters to deploy a new instance, to scale out the infraestructure.
I found this.
https://www.digitalocean.com/community/tutorials/how-to-share-php-sessions-on-multiple-memcached-servers-on-ubuntu-14-04
thanks for your reply!
regards.
I am using sessions in PHP to track if a user is logged in. I do not use it to store any other data about the user; essentially it is like checking a hash table to see if the user has authenticated.
Would there be some advantage to using redis instead of native PHP sessions?
I'm curious about performance, scalability, and security (not really concerned with code complexity).
Using something like Redis for storing sessions is a great way to get more performance out of load balanced servers.
For example on Amazon Web Services, the load balancers have what's called 'sticky sessions'. What this means is that when a user first connects to your web app, e.g. when logging in to it, the load balancer will choose one of your app servers and this user will continue to be served from this server until they exit your application. This is because the sessions used by PHP, for example, will be stored on the app server that they first start using.
Now, if you use Redis on a separate server, then configure your PHP on each of your app servers to store it's sessions in Redis, you can turn this 'sticky sessions' off. This would mean that any of your servers can access the sessions and, therefore, the user be served from a different server with every request to your app. This ultimately makes for more efficient use of your load balancing set-up.
You want the session save handler to be fast. This is due to the fact that a PHP session will block all other concurrent requests from the same user until the first request is finished.
There are a variety of handlers you could use for PHP sessions across multiple servers: File w/ NFS, MySQL Database, Memcache, and Redis.
The database method (using InnoDB) was the slowest in my experience followed by File w/ NFS. Locking and write contention are the main factors. Memcache and Redis provide similar performance and are by far the better alternatives since all operations are in RAM. Redis is my choice because you can enable disk persistence, and Memcache is only memory based.
I explain Redis Sessions in PHP with Kohana if you want more detail. Here is our dashboard for managing Redis keys:
I don't really think you need to worry much about sessions unless you get MASSIVE ammounts of traffic, PHP handle sessions nicely, and if you store only that little data, it should be fine even with a lot of requests, and about performance it should be close, as redis is not native to PHP.
With 10k users, if each user uses like 1kb data of sessions, it would consume 10,000kb or 10~mb, which is not much; PHP is smart enough to use a good enough data structure to hold and quickly write and read those values. The problem is if the session data is too big, or for some reason the server consumes too many resources reading the session data, but that's normally if it's the data is too big.
Could anyone explain disadvantages of storing large amounts of data within the session or point me to some reading ?
I would also be interested if there is any difference between storing data in a session and reading data from datafiles ?
If you store a large amount of data within a session, you'll have input/output performance drops since there's going to be a lot of reading/writing.
Sessions in PHP, by default, are stored in /tmp directory in a flat file. Therefore, your session data is written in a data-file of some sort.
PHP allows you to override its default session handler by session_set_save_handler() function where you can redefine the way sessions are read / written / maintained.
You can also override this via php.ini file where you specify it via session.save_handler directive.
Now, the implication of having a large number of sessions storing large data is that a lot of files will be created and it will take some time to find them due to the ways hard drives operate (mechanical ones of course, which are the common ones still).
The more you have, the longer it takes to find it. The larger they are, longer it takes to read it. If you have a lot of them and they are large - double the trouble, a change in approach is needed.
So what's the solution?
Usually, when met with performance drop - people load balance their websites. That doesn't work with sessions unfortunately because load balancing is choosing which computer to use that will serve the current request. That means that different computers will serve pages you browse at some website. Which means, if those computers use default mechanism of session storage (/tmp directory), the sessions will not be preserved across the servers since they cannot access each other's /tmp directory.
You can solve this by mounting a NAS and making it globally visible to all of the computers in the cluster, but that's both expensive and difficult to maintain.
The other option is to store the sessions in a database. A database is accessible from any of the computers in our fictional cluster. Usually, there are specialised databases used for handling sessions, specialised in sense of being separate from the database storing your website content or whatever.
In the time of NoSQL popularity - in my opinion, NoSQL is great for handling sessions. They scale easily, they are faster in writing the data to storage devices than RDBMSs are.
Third option is to boost all of this, ditch hard drives as permanent storage solution and just use your server's memory for session storage.
What you get is incredible performance, however all your RAM might be quickly gone.
You can also create a cluster of computers that store sessions in their RAM.
Redis and Memcache are great for this task, googling a bit will give you good resources that explain how to use Redis or Memcache to store sessions.
Bottom line of all this is: don't store too much data in your sessions.
According to your needs and budget - there are 3 options available how to store and work with sessions.
This is a good link: http://tuxradar.com/practicalphp/10/1/0
Session data is very expensive workload too. The best way to do it is to store a cookie, or session_id and use that to look up what you need from a dbfile/rdbms. This also allows your site to run across a multi-server environment where as session data is limited to a single.
Currently we are having a site which do a lot of api calls from our parent site for user details and other data. We are planning to cache all the details on our side. I am planning to use memcache for this. as this is a live site and so we are expecting heavier traffic in coming days(not that like FB but again my server is also not like them ;) ) so I need your opinion what issues we can face if we are going for memcache and cross opinions of yours why shouldn't we go for it. Any other alternative will also help.
https://github.com/steveyen/community-site/blob/master/db_doc/main/WhyNotMemcached.wiki
Memcached is terrific! But not for every situation...
You have objects larger than 1MB.
Memcached is not for large media and streaming huge blobs.
Consider other solutions like: http://www.danga.com/mogilefs
You have keys larger than 250 chars.
If so, perhaps you're doing something wrong?
And, see this mailing list conversation on key size for suggestions.
Your hosting provider won't let you run memcached.
If you're on a low-end virtual private server (a slice of a machine), virtualization tech like vmware or xen might not be a great place to run memcached. Memcached really wants to take over and control a hunk of memory -- if that memory gets swapped out by the OS or hypervisor, performance goes away. Using virtualization, though, just to ease deployment across dedicated boxes is fine.
You're running in an insecure environment.
Remember, anyone can just telnet to any memcached server. If you're on a shared system, watch out!
You want persistence. Or, a database.
If you really just wish that memcached had a SQL interface, then you probably need to rethink your understanding of caching and memcached.
You should implement a generic caching layer for the API calls first. Within the domain of the caching layer you can then change the strategy which backend you want to use. If you then see that memcache is not fitting you can actually switch (and/or testwise monitor how it works compared with other backends).
Even better, you can first code this build upon the filesystem quite easily (which has multiple backends, too) without the hurdle to rely on another daemon, so already get started with caching - probably file system is already enough for your caching needs?
Memcache is fast, but it also can use a lot of memory if you want to get the most out of it. Whenever you hit the disk for I/O, you're increasing the latency of your application. Pull items that are frequently accessed and put them on memcache. For my large scale deployments, we cache sessions there because DB is slow as well as filesystem session storage.
A recommendation to add to your stack is APC. It caches PHP files and lessens the overall memory usage per page.
Alternative: Redis
Memcached is, obviously, limited by your available memory and will start to jettison data when memory thresholds are reached. You may want to look redis which is as fast (faster in some benchmarks) as memcached but allows the use of both volatile and non-volatile keys, more complex data structures, and the option of using virtual memory to put Least Recently Used (LRU) key values to disk.
We are developing a web site in PHP, and we have to use sessions. The site will be published in a server cluster. How can we make that work?
Thanks.
Yes this is possible, you need to store your sessions in a central location like a database though. This is pretty simple and just requires you to make some changes to session_set_save_handler - there's a good example of the process you need to follow here
I would use memcache to store your sessions. It will be much faster than storing them in a database or disk.
Database storage is good but you will need more databases when your site becomes very high traffic. Sessions on disk will also cause a lot of IO issues when your site gets a lot of traffic. Memcache on the other hand scales much better than a DB and files.
I personally use memecache and the sites i work on get millions of hits a day. I have never had any issues with storing sessions in memcache.
If you've got multiple PHP boxes, you'll want a central session store.
Your best choices are probably database (that link from seengee's answer is a good explanation) or a dedicated memcache box.
A shared NFS mount for the session directory would be an option, though I've always found nfs performance a bit slow. Alternatives are to write your own session handler using memcache or database for the sessions.
An alternative option is to load balance your web servers using sticky sessions, which will ensure that requests from the same client always go to the same server during the course of the session.