I'm working on a website, using PHP and Codeigniter, configured to use Memcache for session storage. I'm using this library to replace Codeigniter's built-in session storage in order to use PHP's native sessions, and then I configured PHP to use Memcache to store session data. (I originally had the library connecting to Memcache directly, but I couldn't get the cookie to persist the way I wanted.) So it's all set up and working. But there's a problem, obviously. For some reason, the site will, at unpredictable intervals, drop user sessions. It's very erratic and hard to duplicate, and I'm not sure if it's losing the session entirely or just losing all the userdata - for my application, there's no difference visible to the user. It's definitely not related to inactivity - I've seen it happen overnight, but I've had reports of it occurring after sitting at one page for only a couple of minutes, too.
Now, I know what you're thinking. Read this before you answer. My first thought was, oh, Memcache is running out of space and it's doing its eviction thing. So I checked, and that doesn't seem to be the problem. Memcache is sitting pretty with tons of free space, and evictions are holding steady at zero. Unfortunately, that leaves me with no idea what's going on. Has anyone experienced this kind of problem?
Related
I've seen many sites give up the use of the default handling of sessions in PHP for their own method and I still have no clue why.
They are definitely running PHP and it just seems pointless to me that people would design their own method. Is there some sort of limitation that I do not know of or is it purely so they have control of everything?
(I tried asking them and yeah they either didn't have a way of contacting them or they "saw something somewhere against using PHP sessions" without knowing what it actually was)
Default sessions are stored on the hard drive, usually in the /tmp directory.
When your site gets larger, 1 computer isn't sufficient to run it.
Therefore, people resort to load balancing (among other solutions).
Load balancer effectively switches between a cluster of computers. Therefore, if by any chance you got served by computer #1 on your first request and then by computer #2 at your second request - the second computer cannot read the session since it's not in its /tmp folder.
This is a simplified scenario of course since there's much more to application scaling but this is one of the reasons why people resort to overriding the default session mechanism.
The other thing of interest is storing sessions in the db thus making them searchable and what not. You can also create an interface for effectively forcefully logging people out, which is something that the default mechanism cannot provide.
I would have thought a principal reason for rolling your own session-handling functionality is for the purposes of testing. If you're running unit tests, you won't necessarily have a browser environment going. You won't be able to set cookies, and so PHP won't set $_SESSION variables for you.
If, however, you wrote your own session handling class(es), then you could create a mock class for running unit tests. The object would behave like a "real" session, but you won't have to faff about with browsers, cookies and human beings.
Well with the standard setup you are tied to using the file system, saving session data unencrypted etc.
Writing your own session handling using session_set_save_handler you can adjust the sesssion management to your needs ... applying encryption, saving session in a database, synchronizing the sessions with separate software systems ...
1) Session are still widely used. They works and do the work, so there is not point to change it unless a special case.
2) However, Session is weak, it relies in a single PHP (that can be stolen). However, it is possible to protect a session using different method such cookie + ip + expiration.
So yes and no. Session are still widely used but require a fine tune.
Here's a little background, currently i have
3 web servers
one db server which also host memcache for php sessions for the 3 web servers.
I have the php configs on the 3 servers to point to the memcache server for sessions. It was working fine until alot of connections were being produced for reads etc, which then caused connection timeouts.
So I'm currently looking at clustering the memcache on each web server for sessions, my only concern is how to go about making sure that memcache on all the servers have the same information for sessions.
Someone guided me to http://github.com/trs21219/Memcached-Library because i am using codeigniter but how do i converge my php sessions onto this since memcache seems as a key-value store? Thanks in advance.
Has anyone checked out http://repcached.sourceforge.net/ and does it work?
I'm not sure you have the same expectations of memcache that its designers had.
First, however, memcache distribution works differently than you expect: there is no mechanism to replicate stored information. Each memcache instance is a simple key-value store, as you've noticed. The distribution is done by the client code which has a list of all configured memcache instances and does a hash of the key to direct it to one of the instances. It is possible for the client to store it everywhere and retrieve it locally, or for it to hash it multiple times for redundancy, but these are not straightforward exercises.
But the other issue is that memcache is designed for reasonably short-lived data that memcache is allowed to throw away at any time. This makes it really good for caching frequently accessed data that can be a little stale (say up to a few minutes old) but might be expensive to retrieve (such as almost a minute to generate from a query).
PHP sessions don't really qualify for this, in my experience. A database can easily support many thousands of PHP sessions with barely visible traffic, but you need a lot of memcache storage to support the same number: 50k per session and 5000 sessions means close to 256Mb, and then there is all the other data you want to put in there. Not enough storage and you get lots of unexplained logouts (as memcache discards session data when under memory pressure) and thus lots of annoyed users who have to keep logging in again.
We've found GREAT advantage applying MongoDB instead of MySQL for most things, including session handling. It's far faster, far smaller, far easier. We keep MySQL around for transactional needs, but everything else goes into Mongo now. We've relegated memcache to simply caching pages and other data that isn't critical if it's there or not, something like smarty does.
There is no need to use some 3rd party libraries to organize memcached "cluster".
http://ru.php.net/manual/en/memcached.addserver.php
Just use this function to add several servers into the pool and after that data will be stored and distributed over those servers. The server for storing/retrieving the data for the specific key will be selected according to consistent key distribution option.
So in this case you don't need to worry about "how to go about making sure that memcache on all the servers have the same information for sessions"
We have a problem with PHP session when APC is enabled on our server.
The app works great without APC. However, since we enabled APC, the sessions seems to be getting mixed up when the server experiences heavy load, i.e. users are randomly logging on as another. Everything reverts back to normal once we disabled APC. We can't seem to find anyone with the same problem, except a related problem with these guys (set-cookie was being cached in MS ASP): http://msdn.microsoft.com/en-us/magazine/cc163577.aspx#S2
Anyone else has similar experience? Can you recommend any suggestions?
PS: We have all our sessions handled by files in php.ini. We are also running apache2.
We're having a similar issue here. APC is only a primary suspect at this point because it's been difficult to reproduce.
We're using Zend Framework w/ session management and the theory is that Zend's code is cached in APC and when the system is under severe load the code is using a previously cached SID instead of the current one.
Our safeguard is to save the session ID inside the session data and compare the values when the session is retrieved. If the IDs are different we destroy the session and exit.
well please verify that apc really mixes up data...
the only possible i can think of when this could happen is when it gets full and does a stackoverflow.
please check the usage and maby increase the cache size.
There is a problem that crops up with APC where custom session handling objects get destroyed before the session data is saved. It could be unrelated to your problem, but an explicit call to session_write_close before the regular PHP shutdown fixes that one.
Your problem sounds a bit different though, but I can confirm that issues do exist.
Just to complete the post, we had success in reducing this problem with three things:
1) We made extra checks to make sure that the IP is matched to the session ID, and logout the user otherwise. We could then use this to track how often the problem occurs.
2) We switched to XCache and immediately saw lower number of confused session IDs. However, under very heavy load the problem rears its ugly head again.
3) We then double the memory for Xcache in the php config (xcache.size and xcache.var_size) and now the problem is gone.
So we suspect either APC or Xcache running out of memory was the problem. We're still waiting to see if this is a permanent solution.
I had some thoughts back ago about using memcached for session storage, but came to the conclusion that it wouldn't be sufficient in the event of one or more of the servers in the memcached pool were about to go down.
A hybrid version is to save the main database (mySQL) from load caused by reads would be to work out a function that tries to fetch the data from the cache pool, and if that fails gets it from the database.
After putting some more thought into it, I started to think about using APC cache for session related data. If our web server would go down, sessions would be lost either way, so storing them in a local APC or a localhost memcached server maybe isn't that bad?
What's your experiences?
Generally, session data is something which should be treated as volatile in any situation. The user can always choose to eliminate the cookie themselves at any point (if you are using cookies, of course). For this reason, I see no problem with using memcached for session data.
For me, I'd just keep it simple - no need for a DB fallback unless you absolutely must never lose the user's session in the event of a memcached server failure. As I said at the beginning, I always treat sessions as purely volatile in any case and don't really store anything of any significance in them.
That's my two cents anyways.
OK, so I've got this totally rare an unique scenario of a load balanced PHP website. The bummer is - it didn't used to be load balanced. Now we're starting to get issues...
Currently the only issue is with PHP sessions. Naturally nobody thought of this issue at first so the PHP session configuration was left at its defaults. Thus both servers have their own little stash of session files, and woe is the user who gets the next request thrown to the other server, because that doesn't have the session he created on the first one.
Now, I've been reading PHP manual on how to solve this situation. There I found the nice function of session_set_save_handler(). (And, coincidentally, this topic on SO) Neat. Except I'll have to call this function in all the pages of the website. And developers of future pages would have to remember to call it all the time as well. Feels kinda clumsy, not to mention probably violating a dozen best coding practices. It would be much nicer if I could just flip some global configuration option and Voilà - the sessions all get magically stored in a DB or a memory cache or something.
Any ideas on how to do this?
Added: To clarify - I expect this to be a standard situation with a standard solution. FYI - I have a MySQL DB available. Surely there must be some ready-to-use code out there that solves this? I can, of course, write my own session saving stuff and auto_prepend option pointed out by Greg seems promising - but that would feel like reinventing the wheel. :P
Added 2: The load balancing is DNS based. I'm not sure how this works, but I guess it should be something like this.
Added 3: OK, I see that one solution is to use auto_prepend option to insert a call to session_set_save_handler() in every script and write my own DB persister, perhaps throwing in calls to memcached for better performance. Fair enough.
Is there also some way that I could avoid coding all this myself? Like some famous and well-tested PHP plugin?
Added much, much later: This is the way I went in the end: How to properly implement a custom session persister in PHP + MySQL?
Also, I simply included the session handler manually in all pages.
You could set PHP to handle the sessions in the database, so all your servers share same session information as all servers use the same database for that.
A good tutorial for that can be found here.
The way we handle this is through memcached. All it takes is changing the php.ini similar to the following:
session.save_handler = memcache
session.save_path = "tcp://path.to.memcached.server:11211"
We use AWS ElastiCache, so the server path is a domain, but I'm sure it'd be similar for local memcached as well.
This method doesn't require any application code changes.
You don't mentioned what technology you are using for load balancing (software, hardware etc.); but in any case, the solution to your problem is to employ "sticky sessions" on the load balancer.
In summary, this means that when the first request from a "new" visitor comes in, they are assigned a specific server from the cluster: all future requests for the lifetime of their session are then directed to that server. In practice this means that applications written to work on a single server can be up-scaled to a balanced environment with zero/few code changes.
If you are using a hardware balancer, such as a Radware device, then the sticky sessions is configured as part of the cluster setup. Hardware devices usually give you more fine-grained control: such as which server a new user is assigned to (they can check for health status etc. and pick the most healthy / least utilised server), and more control of what happens when a server fails and drops out of the cluster. The drawback of hardware balancers is the cost - but they are worth it imho.
As for software balancers, it comes down to what you are using. For Apache there is the stickysession property on mod_proxy - and plenty of articles via google to get this working with the php session ( for example )
Edit:
From other comments posted after the original question, it sounds like your "balancing" is done via Round Robin DNS, so the above probably won't apply. I'll refrain from commenting further and starting a flame against round robin dns.
The easiest thing to do is configure your load balancer to always send the same session to the same server.
If you still want to use session_set_save_handler then maybe take a look at auto_prepend.
If you have time and you still want to check more solutions, take a look at
http://redis4you.com/articles.php?id=01..
Using redis you are fault tolerant. From my point of view, it could be better than memcache solutions because of this robustness.
If you are using php sessions you could share with NFS the /tmp directory, where I think the sessions are stored, between all the servers in the cluster. That way you don't need database.
Edited: You can also use an external service like memcachedb (persistent and fast) and store the session info in the memcachedb index and indentify it with a hash of the content or even the session ID.
When we had this situation we implemented some code that lives in a common header.
Essentially for each page we check if we know the session Id. If we dont we check if we're in the situation whehich you describe, by checking if we have stored sesion data in the DB.Otherwise we just start a new session.
Obviously this requires all relevant data to be copied to the DB, but if you encapsulate your session data in a seperate class then it works OK.
you could also try using memcache as session handler
Might be too late, but check this out: http://www.pureftpd.org/project/sharedance
Sharedance is a high-performance server to centralize ephemeral key/data
pairs on remote hosts, without the overhead and the complexity of an SQL
database.
It was mainly designed to share caches and sessions between a pool of web
servers. Access to a sharedance server is trivial through a simple PHP API and
it is compatible with the expectations of PHP 4 and PHP 5 session handlers.
When it comes to php session handling in the Load Balancing Cluster, it's best to have Sticky Sessions. For that ask the network of datacenter who is maintaining the load balancer to enable the sticky session. Once that is enabled you'll don't need worry about sessions at php end