I'm developing a project of mine with scalability in mind and I've come to a crossroad. On my website I would like to detect if a user is online or not. And I can't quite think of the best way to handle this. The way I was thinking would be something along these lines(in psuedocode):
// SQL user table:
user {
"name": "blah blah",
"email": "derpy#derpyderp.com",
"online": false
}
So whenever the user logs in I could update his online column to true. However that would eventually lead to SQL queries happening every time a user logs in and if it happens that I get say, 10 logins per second, well, that's a lot of queries happening. Another way I figured I could do the same thing but in a different table:
// Activity table:
activity {
"user_id": 2,
"online": true
}
For some reason I believe that would lead to less memory consumption because of the separation from the user table. However I'm not sure if it would have any actual effect on performance.
So if you could bless me with your insight I would be more then grateful, thank you.
Generally speaking it's a common practice to add a column to the users table to store the lastActivity time. Anytime the user logs in, or accesses a page, store the current time in that field. If you want to know whether or not they are online, see if the last recorded time is within a certain window - say, five minutes. You can query all rows to see how many users are currently online as a result.
I wouldn't be too worried about running queries every few seconds - your server can handle it (assuming these are well-written and not very verbose).
you can use datetime for field type and don't forget to record user IP so you can track time o
Depending on how you want it to work you basically have two options:
Define a timeout after which you consider a user logged out
Use ajax/websockets/whatever to poll user
1: Timeout
This is the simpler use case. Every time the user requests a page, you update a timestamp in your database.
To find out how many users are online, you would do a query against this database and do a COUNT of users who have been active in the last N minutes.
This way you will get a relatively accurate idea of how many people are actively using the site at the moment.
2: Constant polling
This is a bit more complex to implement due to having to update the server with Ajax. Otherwise it works in a similar fashion to #1.
Whenever a user is on a page, you can keep a websocket open or do ajax requests every N seconds to the server.
This way you can get a pretty good idea of how many people have pages open on your site currently, but if a user leaves the page open in their browser and doesn't do anything, it would still count them as being online.
A slight modification to the idea would be to use a script on the client to monitor mouse movement. If the user doesn't move the mouse on your page for say 10 minutes, you would stop the polling or disconnect the websocket. This would fix the problem of showing users who are idle as being online.
Related
I just want a approach on how to build a database with live records, so don't just downvote. I don't expect any code.
At the moment I have a MySql database with about 2 thousand users, they are are getting more though. Each player/user has several points, which are increasing or decreasing by certain actions.
My goal is that this database gets refreshed about every second and the user with more points move up and others move down... and so on
My question is, what is the best approach for this "live database" where records have to be updated every second. In MySql I can run time based actions which are executing a SQL command but this isn't the greatest way I think. Can someone suggest a good way to handle this? E.g. other Database providers like MongoDB or anything else?
EDIT
This doesn't work client side, so I can't simply push/post it into the databse due some time based events. For explanation: A user is training his character in the application. This training (to get 1 level up) takes 12 hours. After the time is elapsed the record should be updated in the database AUTOMATICALLY also if the user doesn't send a post request by his self (if the user is not logged in) other users should see the updated data in his profile.
You need to accept the fact that rankings will be stale to some extent. Your predicament is no different than any other gaming platform (or SO rankings for that matter). Business decisions were put in place and constantly get reviewed for the level of staleness. Take the leaderboards on tags here, for instance. Or the recent change that has profile pages updated a lot more frequently, versus around 4AM GMT.
Consider the use of MySQL Events. It is built-in functionality that replaces the need for cron tasks. I have 3 event-related links off my profile page if interested. You could calculate ranks on a timed schedule (your tolerance for staleness) and the users' requests for them would be fast (faster than the below from Gordon). On the con-side, they are stale.
Consider not saving (writing) rank info but rather focus just on filling in the slots of your other data. And get your rankings on the fly. As an example, see this rankings answer here from Gordon. It is dynamic, runs upon request with at least at that moment non-staleness, and would not require Events.
Know that only you should decide what is tolerable for the UX.
I'm working on a basic lamp(willing to change) website , and I currently need a way to run some function on the server that runs for several hours per user, and every X hours it needs to query the mysql database to see if the value for that user has been updated, if it hasn't it need it to insert a new record in the database...I also should mention that the 'every X hours' can change per user too, and the total runtime of the function per user can also vary.
So basically I need a function that runs continuously on the server for few hours per user. What is the best way to do this? I want the site to be able to support many users (like 10000 +).
I'm willing to try new technologies for every aspect of the site, I'm still in the design phase and I was looking for some input.
I've looked at cron but not really sure how well it would work when dealing with so many users...
edit: Here is a typical scenario of events;
User presses button on the website and closes the browser.
Server starts a timer from when they pressed the button, now
the server will check if that user has pressed a different button within a given time frame (time frame can change per user), say within 30 minutes. If they didn't press the other button then the server needs to automatically insert a new record in the database.
The script will need to continue running, checking every 30 mins for say the next 5 hours.
Thank you!
Cron would work as well as you can code the page it will run. It's not a cron limitation.
The question is ambiguous btw. Maybe explaining your full scenario would help.
Meanwhile, my suggestion would be to set up a scrip that allows you to manually check what you need to check.
You definitely need the DB to be InnoDB optimized with proper indexes to be able to support 1000 plus users.
To alleviate the number of calls to the database, a common practice is to run scripts only on what you are interested (so in the case of users you would only select those who have logged on in say the past 3 hours)
That's achievable in 2 ways, a simple select statement, or by adding entries to a specific table on the login page, and remove them after the automated script has finished running.
All of this is pure theory without understanding exactly what you need to do though.
You are telling what/how you want to do, but not why you want to do it. Maybe letting us know why could lead to a different how ;)
However, what you can do is still use cron (or anything similar). The trick is to have
a last_interaction timestamp column
a maximum_interval column
a daily_runtime column
in your users database. Not optimized but you are in the design phase so you shouldn't pay too much attention to the performance aspect (except is explicitly required).
I am creating a web application named Online Exam using PHP+MySQL and AngularJS. Now I am getting some trouble on project like changing the user looged in status. Let us take this condition as a example:
Suppose a authorized user/student successfully logged in online exam section(After successfully logged current time will be inserted in the db in exam_start_time column as unix timestamp format and exam_status will be set as 'ACTIVE`.
1hr(60 min) countdown timer is initialize for him/her as per the inserted exam_start_time in db.
Now suppose after 15 min the system shuts down automatically, then if user logged in again(In same system or other) then the countdown timer should be set for 45 minutes only.
Previously I was updating the last_activity_time in our table in every 10 sec(using ajax calls). but now I want to change this way, Is there any way like(socket or network programming using PHP) to update the column.
Here is my table structure which is managing user logged in status
Please give me some suggestions on it.
A Php socket server programming tutorial : http://www.christophh.net/2012/07/24/php-socket-programming/
Sockets, as Pascal Le Merrer mentioned, is IMO your best option. But beware of Apache! Every WebSocket holds one Apache thread, and Apache wasn't designed to do that. when too many (and by too many I mean few dozen) clients connect to your site, it will crash. (I've been there while trying to implement long polling/comet, ended up using NodeJS. If you're using nginx, it is more likely that it will become low on resources and effective, but there are also other ways. Take a look here:
Using WebSocket on Apache server
If you find this uncomfortable/hard to learn, try also another idea:
try to add hidden iFrame to your exam page, pointing to prepared site that updates database row. Use javascript to refresh this page every 10-15 seconds. Every refresh causes update of specific row in DB, using current date and time. It should work (not tested, but give it a try).
I want to show users who else is logged in as part of a comment system. What are best practices for keeping track of users? For example:
Do you keep track of all sessions and then mark them as closed. Or do you delete users upon logout, keeping track only of active users.
I'm thinking I should create a table with userid, time logged in, time logged out and/or status. Is that the way to go or is there some alternative approach of tracking session ids. If using a table, is there value in keeping sessionid. Should I delete row when session no longer active, negating need for whenloggedout field.
There is a login so easy to keep track of users logging in. However, it is harder to track users logging out since their session may be broken by browser crashing etc.
Is it best practice to consider users logged in as long as they have not destroyed session... for example, FB and Gmail will leave you logged in almost indefinitely--or should there be a time limit since last activity? The idea of saving to this table every time there is activity on site is not appealing.
Right now, I'm thinking of following:
create table loggedin (userid (int), whenloggedin (datetime), whenlogged out (datetime), loggedin(tinyint))
with the latter going to 0 either if whenloggedout not null or after some long time limit like 24 hours. I imagine FB while leaving you logged in for long periods of time, also keeps track of activity for purposes of chat etc. but not sure. I'm also thinking of letting the table expand, rather than deleting closed sessions but maybe that's a mistake.
Would this approach be considered adequate or is there a better way. Many thx for advice on this.
Depending on how you want it to work you basically have two options:
Define a timeout after which you consider a user logged out
Use ajax/websockets/whatever to poll user
1: Timeout
This is the simpler use case. Every time the user requests a page, you update a timestamp in your database.
To find out how many users are online, you would do a query against this database and do a COUNT of users who have been active in the last N minutes.
This way you will get a relatively accurate idea of how many people are actively using the site at the moment.
2: Constant polling
This is a bit more complex to implement due to having to update the server with Ajax. Otherwise it works in a similar fashion to #1.
Whenever a user is on a page, you can keep a websocket open or do ajax requests every N seconds to the server.
This way you can get a pretty good idea of how many people have pages open on your site currently, but if a user leaves the page open in their browser and doesn't do anything, it would still count them as being online.
A slight modification to the idea would be to use a script on the client to monitor mouse movement. If the user doesn't move the mouse on your page for say 10 minutes, you would stop the polling or disconnect the websocket. This would fix the problem of showing users who are idle as being online.
To circumvent the problem with knowing if a user has logged out or browser crash ect, is to use a heartbeat/polling of sorts here is a stripped down example of how you can do that with jQuery
function heartbeat(){
setTimeout(function(){
$.ajax({ url: "http://example.com/api/heartbeat", cache: false,
success: function(data){
//Next beat
heartbeat();
}, dataType: "json"});
}, 10000);//10secs
}
$(document).ready(function(){
heartbeat();
});
http://example.com/api/heartbeat would keep the session alive & update a timestamp in your db, then on each page load you would check the time stamp with current time ect and if its lower then say 15 seconds then you would log them out.
Constant Polling and using heartbeat are a good idea, but for some scenarios they may create useless load on server. I think you should think about the importance of keeping track of your users and use it very appropriately, especially considering the impacts your changes may have on load time.
It could be a really easy thing to do but I am getting pretty confused by it.
Basicly, I want to get the amount of people currently browsing my website. The problem is, I can't count sessions. Many of them are created for different purposes. (e.g $_SESSION['age'] for the users who logged in.)
Also, there are "logged_in" and "not_logged_in" people.
In short, how can I count the users in my website in the most professional way?
Ps. Please note that I will run the function every 1-5 minutes, get the amount, store it to be used as cache. After 5 minutes, cache will be replaced again, and so on. Hence, db connections will be a possibility if you think that would be the most professional way.
One of the issues at hand is, your PHP doesn't know when your user closes the browser.
I've done something similar with a combination of a "last_used"-field in the user database (timestamp/datetime) and a "heartbeat"-function (jscript), which regularly sends an "I'm still alive"-command to the server (every x seconds), where then this last_used-field is updated. It ain't a high traffic site, but it does the job very reliable.
Then just count the amount of users which a very current heartbeat in the database:
SELECT COUNT(*)
FROM your_user_table
WHERE last_used > TIMESTAMPADD(NOW(), INTERVAL -30 SECOND)
You don't need a user table for this, you can use a session table, too. Such a mechanism can then be used for other features, too, like giving the users new messages on the fly and similar.