Question basically says it all. I get a lot of traffic, about 200k hits a day. I want to store the original referrer (where they came from) in a session variable for various purposes. Is this a good idea or should I stick in a database instead?
You can do both at once :). PHP allows you define the storage logic of your sessions in scripts. This way it is possible to store sessions in a database as well. Check the manual of set_session_save_handler()
Using a database would have its advantages if you use load balancing (or plan to do it once). This way all web servers could read the session data from the same database (or cluster) and the load balancer would not have to worry about which request should be forwarded to which web server. If session data is stored in files, which is the default mechanism, then a load balancer has to forwared each request of a session to the same physical web server, which is much more complex, as the load balancer has to work on HTTP level.
You could just store the information in a cookie if you only need it for the user's current session. Then you don't need to store it at all on your end.
There are a few down sides as well:
They may have cookies disabled, so you may not be able to save it.
If you need the information next time you may not be able to get it, as it could have been deleted.
Not super secure so don't save passwords, bank info, etc.
So if needing this information is required no matter what, maybe its not the way to go. If the information is optional, then this will work.
The default PHP session handler is the file handler. So, the pertinent questions are:
Are you using more than 1 webserver without sticky sessions (load balancing)?
Are you running out of disk space?
Do you ever intend to do those?
If yes (to any), then store it in a database. Or, even better, calculate the stuff on every request (or cache it somewhere like Memcached). You could also store the stuff in a signed cookie (to prevent tampering).
Related
I know there are hundreds of these questions but what I am asking however is slightly different.
When the user logs in I would like to get all their data from each table in a database and store it in a session variable (obviously not sensative data such as encrypted password/salts etc basically data that would be useless or have no value to a hacker!!), and whilst the user uses the website the relevant data stored in the session will be used as opposed to accessing the database everytime. Moreover when the data is changed or added this will be written or added to the session file, and upon a major action such as "saving" or "loggin out" the new/changed data will be written to the database.
The reason I wish to do this is simply for efficieny, I want my application to not only be fast but less resource consuming. I am no expert on either which may explain why my idea makes no differnece or is more resource intensive.
If there is an alternative to my solution please let me know or if there is something to improve on my solution I will be glad to hear it.
Thank you.
My application is using PHP and mysql.
If any of these don't apply to your app, then please ignore. In general, I'm against using sessions as caches (especially if anything in the session is going to be written back to the DB). Here's why.
Editing the session requires a request from the user. Editing a php session outside of the request-response cycle is very difficult. So if a user Alice makes a change which affects Bob, you have no way to dirty Bob's cache
You can't assume users will log out. They may just leave so you have to deal with saving info if the session times out. Again, this is difficult outside of the request-response cycle and you can't exactly leave session files lying around forever until the user comes back (php will gc them by default)
If the user requires authentication, you're storing private information in the session. Some users may not be happy about that. More importantly, a hacker could imploy that private information to conduct a social engineering attack against the end-user.
Mallory (a hacker) might not be able to use the information you put in the session, but she can poison it (ie. cache poisoning), thereby causing all sorts of problems when you write your cache to your permanent storage. Sessions are easier to poison then something like redis or memcache.
TL;DR Lots of considerations when using a session cache. My recommendation is redis/memcache.
You can also go for local-storage in HTML5, check The Guide and THE PAST, PRESENT & FUTURE OF LOCAL STORAGE FOR WEB APPLICATIONS
Local Storage in HTML5 actually uses your browsers sqlite database that works as cookies but it stores data permanently to your browser
unless someone by force remove the data from the browser finding the data files
Or if someone remove/uninstall browser completely,
or if someone uses the application in private/incognito mode of the browser,
What you need to do
Copy the schema for required tables and for required columns and update data at a regular interval
you dont have to worry about user's state, you only have to update the complete data from the localStorage to mysql Server (and from the mysql server to localStorage if required) every time user backs to your application and keep updating the data at regular interval
Now this is turning out to be more of localStorage but I think this is one of the best solution available for me.
redis is a good solution if it is available for you (sometimes developers can't install external modules for some reason) what I would do is either go with your Session approach but with encoded/encrypted and serialized data. Or, which I really prefer is to use HTML5 data properties such as:
<someElement id="someId" data-x="HiX" data-y="Hi-Y" />
which BTW works fine with all browsers even with IE6 but with some tweaks, specially if your application uses jquery and ajax. this would really be handful.
You need to use Memcache for this kind of work. To solve the problem of keeping the updated data everywhere you can create functions for fetching the data, for example when the user logs in you, authenticate the user and after that insert all the user data into the memcache with unique keys like :-
USER_ID_USERNAME for user's username
USER_ID_NAME for user's name
etc...
Now create some more functions to fetch all this data whenever you need it. For ex
function getName($user_id){
if(Memcache::get($user_id."_name"){
return Memcache::get($user_id."_name");
} else {
//Call another function which will fetch the data from the DB and store it in the cache
}
}
You will need to create functions to fetch every kind of data related to the user. And as you said you want to update this data on some major event. You can try updating the data using CRON or something like that, because as tazer84 mentioned users may never log out.
I also use what the OP described to avoid calls to db. For example, when a user logs-in, i have a "welcome-tip" on their control panel like
Welcome, <USERS NAME HERE>
If i stored only his user_id on $_SESSION then in every pageview i would have to retrieve his information from the database just to have his name available, like SELECT user_name FROM users WHERE user_id = $_SESSION['user']['user_id'] So to avoid this, i store some of his information in $_SESSION.
Be careful! When there is a change on data, you must modify the data in db and if successfull also modify the $_SESSION.
In my example, when a user edits his name (which i also store in $_SESSION so i can use it to welcome-tip), i do something like:
If (UpdateCurrentUserData($new_data)) // this is the function that modifies the db
{
$_SESSION['user']['user_name']=$new_data['user_name']; // update session also!
}
Attention to:
session.gc_maxlifetime in your php.ini
This value says how much time the $_SESSION is protected from being erased by the garbage collector (the file that exists on your disk in which the $_SESSION data are stored)
If you set this very low, users may start getting logged-out unexpectedly if they are idle more than this amount of time because garbage collector will delete their session file too quickly
if you set this very high, you may end up with lots of unused $_SESSION files of users that have left your website a long time ago.
also i must add that gc_maxlifetime works together with session.gc_probability where in general you need lower probability for high-traffic websites and bigger probability for lower traffic since for each pageview there is a session.gc_probability that garbage collector will be activated.
A nice more detailed explanation here http://www.appnovation.com/blog/session-garbage-collection-php
I know this sounds stupid but ....
If ur data is not sensitive the best way to make it accessible faster is to store it in hidden variables inside the forms itself. You can save comma separated or values in an array.
I use cookies for page cache, is that okay?
To show partially static contents( updated very rarely from database),
i don't like to interact much with database so can i use cookie to store that results in client's computer and display data directly from it?
For data authenticity and integrity, i created a crc for data and stored along with data in that cookie.
THe algorithm of crc calculation is hardcoded(assume), so
Does the idea has any demerits?
Cookies should not be used for caching, since your cookie is sent from the browser to the server on each request!! This means you send more bytes, which means higher latency. So you are actually slowing your system down, not helping it, by using cookies in this way. The proper way to cache data on a website is to use localStorage (or one of the other HTML5 storage classes) to cache it.
This sounds like a bad idea to me. Is the cached data user-specific or global? For global data you'd be much better off caching the entire generated page on the server side and responding with that for future requests. Using cookies you'd still need to dynamically generate a page on each request.
For user-specific data, it depends on the page, but you generally want to store thing like a user's name in the session. Caching something like a user's profile page won't be very helpful since it doesn't get hit very often.
In either case, you're going to end up wasting a lot of bandwidth receiving all that extra information on each request and you still have to dynamically generate a page for every request. There are also security implications if your checksum algorithm is compromised and you store anything important in the cookie like an account balance or even a username. Further, how do you invalidate the data cached in the cookie? There would need to be a mechanism on every request to check if the data is old, which defeats the purpose.
Cookies should be used to store small strings like session ids and usernames.
You're better off writing the static content to a local file on your server (or to a database) and serving the static content until you decide to expire it - then repeating that process.
There are servers, such as memcached, which are very efficient at doing this but the psuedocode is the same:
if (is_cached('recent_articles') {
return cached_content('recent_articles');
else {
save('recent_articles', $articles)
return cached_content('recent_articles')
}
Just make sure to expire content after a certain amount of time or when you know its stale - whichever makes the most sense.
I'm running a web application that allows a user to log in. The user can add/remove content to his/her 'library' which is displayed on a page called "library.php". Instead of querying the database for the contents of the users library everytime they load "library.php", I want to store it globally for PHP when the user logs in, so that the query is only run once. Is there a best practice for doing this? fx. storing their library in an array in a session?
Thanks for your time
If you store each user's library in a $_SESSION as an array, as you suggested (which is definitely possible) you will have to make sure that any updates the user makes to the library are instantly reflected to that session variable.
Honestly, unless there is some seriously heavy querying going on to fetch a library, or you have tons of traffic, I would just stick to 'execute query whenever the user hits library.php'.
Consider the size of the data. Multiply that by the maximum number of concurrent users.
Then compare that the to memory avaiable on your server. Also consider whether or not this is a shared server; other sites needs resources too.
Based on this, it is probably best to either create a file that can be used (as per Remi's comment), or remain in the default stateless form and read every time. I doubt that reading the data each time is creating much of an overhead.
When the user login you can generate a xml file (USER_ID.xml for instance) that you display with xslt.
http://php.net/manual/en/book.xslt.php
Each PHP script dies when it completes, so data can not be kept permanentely live in a web application as you would do in a PC application.
One way could be sessions, but it depends on the amount of data you want to save. According to your example you are talking about a library, so it sounds to me like big quantity of data need to be saved, in such case the DB is the way to go, and yes you have to query it each time.
Another way could be to save them in an array inside a php file, but in the same way you have to query the DB each time, you would have to include such php file each time.
Since this is a db performance optimization, I would suggest that you take a look at memcached which matches your problem perfectly:
memcached is [..] intended for use in speeding
up dynamic web applications by
alleviating database load.
I think it would be best to store it in a Session.
It the user logs in, the Session is being created and you can save data in it using the superglobal:
$_SESSION['key'] = "value";
You can also store Arrays or everything else there and it can be cleared if the user logs out.
you care for performance; Please note:
Session may use database or file to store data.
database is here to be used instead of files, for it's performance and abilities.
use database, it is designed to be used exactly in such situations!
I have a social network type app I have been working on for at least 2 years now, I need it to scale well so I have put a lot of effort into perfecting the code of this app. I use sessions very often to cache database results for a user when a user logs into the site I cache there userID number, username, urer status/role, photo URL, online status, last activity time, and several other things into session variables/array. Now I currently have 2 seperate servers to handle this site, 1 server for apache webserver and a seperate server for mysql. Now I am starting to use memcache in some areas to cut down on database load.
Now my sessions are stored on disk and I know some people use a database to store sessions data, for me it would seem that storing session data that I cached from mysql would kind of defeat the purpose if I were to switch to storingsessions in mysql. SO what am I missing here? Why do people choose to use a database for sessions?
Here are my ideas, using a database for sessions would make it easiar to store and access sessions across multiple servers, is this the main reason for using a database?
Also should I be using memcache to store temp variables instead of storing them into a session?
PHP has the ability to use memcached to store sessions.
That may just be the winning ticket for you.
Take a look at this google search.
One part of the Zend Server package is a session daemon.
Be careful using memcache for that purpose. Once the memory bucket is full, it starts throwing away stuff in a FIFO fashion.
Found this on slideshare about creating your own session server with php-cli.
The single best reason to store sessions in the database is so you can load-balance your website. That way it doesn't matter which server hands out the next page because they are all using the same database for storing their sessions.
Have a look at PHP's set_save_handler() for how to install a custom session handler. It takes about 30 lines to set one up that puts the session in the database, though that doesn't count the lines to make a decent database handler. :-) You will need to do:
ini_set('session.save_handler', 'user');
ini_set('session.auto_start', '0');
... although session.auto_start will need to be in your php.ini (and set to 0).
If the database access is going to be a bit expensive, there are some other things you can do to mitigate that. The obvious one is to have a DB server that is just for sessions. Another trick is to have it poke stuff into both memcache and the DB, so when it checks, if the memcache record is missing, it just falls back to the DB. You could get fancy with that, too, and split the session up so some of it is in memcache but the rest lives in the database. I think you'd need to put your own access functions on top of PHP's session API, though.
The main reason to store session data in a database is security because otherwise you have no way to validate it. You'd store the session ID along with the data in the database and match them to see if the session has been tampered with but you can't use the server's (apache) default session mechanism anymore.
For Storing variables in memcache instead of the session.. have you set up your database query cache? I'd have a look there instead first as it's far easier to deal with than with memcache.
I'm making a php web application which stores user specific information that is not shared with other users.
Would it be a good idea to store some of this information in the $_SESSION variable for caching? For example: cache a list of categories the user has created for their account.
This would be an appropriate use of the session mechanism as long as you keep this in mind:
Session does not persist for an indefinite amount of time.
When pulling from session, ensure you actually got a result (ASP.NET will return NULL if the Session has expired/cleared)
Server restarts may wipe the session cache.
Do this for convenience, not performance. For high-performance caching, choose an appropriate mechanism (i.e. memcached)
A good usage pattern would be like this (ether cookies or session):
User logs in
Store preferences (background color, last 10 records looked at, categories) in session/cookie.
On rendering of a page, refer to the Session/Cookie values (ensuring they are valid values and not null).
Things not to do in a cookie
Don't store anything sensitive (use session).
A cookie value should not grant/deny you access to anything (use session).
Trap errors, assume flags and strings may not be what you expect, may be missing, may be changed in transit.
I'm sure there is other things to consider too, but this is just off the top of my head here.
That could work well for relatively small amounts of data but you'll have to take some things into consideration:
$_SESSION is stored somewhere between requests, file on disk or database or something else depending on what you choose to use (default to file)
$_SESSION is local to a single user on a single machine.
sessions have a TTL (time to live), they dissapear after a set amount of time (which you control)
Under certain circumstances, sessions can block (rarely an issue, but i've run into it streaming flash)
If the data you mean to cache is to be accessed by multiple users you're quickly better off caching it seperately.
If you only want this data available during their session, then yes. If you want it available tomorrow, or 4 hours from now, you need to save it to a database.
Technically you can modify the sessions to have a very long lifespan, but realize if they use a different computer, a different browser or flush their cookies they will loose the link to their session, therefore anything serious you should create a type of user account in your application, link the session to their account and save the data in a permeate place.