We're setting up a system which allows a department to make edits to a record here. Their division of labor isn't clear, and they've had problems in the past where more than one individual loads data into a web form, makes edits, and then sends those edits to the record. Inevitably, the slower editor over-writes the faster editor's freshly edited data with the old data that had been loaded when the page loaded.
Currently, we have a white-board solution that would use changes to the last modified time of the data to reject the second request to write data and deliver an error message when that data POSTED.
The members of the department, however, would prefer a file-lock styled system--one where they were notified that another user was in the dataset prior to being allowed to access the data. Timeouts will inevitably be too short or too long depending on the day, and these particular users cannot be relied upon to "log out" somehow.
I'm just wondering if anyone else has implemented a solution to this, and, if so, how?
We're running PHP 5.6 on Apache 2.2 and building in Zend Framework 2.4. Not that any of that has any bearing on the question, but someone will inevitably ask.
Add 2 columns to your table locked_by_user_id and locked_time.
Before you allow a user to enter the "edit" view, check if those values are set and if locked_time is within the past 10 minutes. The reason you should record the locked time instead of a binary flag is because, as you say, some people forget to log out or might not log out cleanly (for example, they could just close their browser).
When someone is able to acquire a lock, set up a setInterval that runs every 5 minutes that reacquires the lock via an ajax call. Someone still might forget to leave the "edit" screen but in that case you can allow someone else to override a lock and if that happens you can have the ajax call exit the "edit" screen when it fails to reacquire the lock.
Why not set a flag on the table row for edit_in_progress? If a user clicks to edit and that flag is already set, fail out with a message saying someone else is editing it (perhaps consider also setting a field for WHO is editing it, so they can go back in before they've saved and continue their edits). Once saved, unset both fields, and allow the next user to lock the row.
Assuming that you're using a database, but not knowing the database schema you're using, this is going to be a generic answer.
You need to give each record an identifier which is set as non-unique. By this, I mean each record could be identified as record_1, record_2 ... record_n but this identifier can appear multiple times in the table.
When a record is edited, don't update the record directly but create a new record which is
timestamped
has the original record_n identifier
Also, add a field to the record to give its current state (e.g. stable, editing?) and a field which gives the edit start date/time if it is being edited
With this, when someone wants to edit a record (e.g., record_2), you retrieve the most recent data for this record and check its state (if marked as editing then report this and prevent concurrent editing).
When they submit changes, a new timestamped record is created in the database, you mark the old and new records as stable.
Using this, you also create a paper-trail for auditing changes
With regards to people who wander off/retire/die and leave records in an editied state, create a scheduled job which resets states to "stable" after a preset number of minutes (60?)
Related
I'm working on a basic lamp(willing to change) website , and I currently need a way to run some function on the server that runs for several hours per user, and every X hours it needs to query the mysql database to see if the value for that user has been updated, if it hasn't it need it to insert a new record in the database...I also should mention that the 'every X hours' can change per user too, and the total runtime of the function per user can also vary.
So basically I need a function that runs continuously on the server for few hours per user. What is the best way to do this? I want the site to be able to support many users (like 10000 +).
I'm willing to try new technologies for every aspect of the site, I'm still in the design phase and I was looking for some input.
I've looked at cron but not really sure how well it would work when dealing with so many users...
edit: Here is a typical scenario of events;
User presses button on the website and closes the browser.
Server starts a timer from when they pressed the button, now
the server will check if that user has pressed a different button within a given time frame (time frame can change per user), say within 30 minutes. If they didn't press the other button then the server needs to automatically insert a new record in the database.
The script will need to continue running, checking every 30 mins for say the next 5 hours.
Thank you!
Cron would work as well as you can code the page it will run. It's not a cron limitation.
The question is ambiguous btw. Maybe explaining your full scenario would help.
Meanwhile, my suggestion would be to set up a scrip that allows you to manually check what you need to check.
You definitely need the DB to be InnoDB optimized with proper indexes to be able to support 1000 plus users.
To alleviate the number of calls to the database, a common practice is to run scripts only on what you are interested (so in the case of users you would only select those who have logged on in say the past 3 hours)
That's achievable in 2 ways, a simple select statement, or by adding entries to a specific table on the login page, and remove them after the automated script has finished running.
All of this is pure theory without understanding exactly what you need to do though.
You are telling what/how you want to do, but not why you want to do it. Maybe letting us know why could lead to a different how ;)
However, what you can do is still use cron (or anything similar). The trick is to have
a last_interaction timestamp column
a maximum_interval column
a daily_runtime column
in your users database. Not optimized but you are in the design phase so you shouldn't pay too much attention to the performance aspect (except is explicitly required).
What would be the best way to achieve an undo function in a PHP CRUD application? The only solution I've been able to come up with is using a sort of buffer table in my database that is periodically wiped.
If the user clicks the "Undo" button after deleting a record for example, the id of the last change will be passed to a handler which will pull the buffer record and reinstate it into the main table for that data type. If the "Undo" is not done in say, 4 or 5 minutes, a reaper script will drop the entry.
Does this sound feasible? Is there a better way of accomplishing this?
You could use a flag field in your database to mark a row for delete.
And you can setup task (crontab in linux) to delete all rows with delete flag set to true and time difference > to 5 mins.
I've learned to not delete anything, but simply do as Ignacio Ocampo stated by using a flag column in your DB such as status. By default set the status column to open. If your client clicks your delete button, just update that records status column to void, or deleted..
In doing this, you'll need to update your data request to pull only those records with the status column set to open. This allows the data to not be lost, but also not seen.
all undo(s) or redo(s) if applicable can reset the open status to - or + 1 record sorted by a timestamp column.
If db space is at a premium, and you need to remove old data then crontab does work, but I prefer the simplicity phpmyadmin conjob to loop a file that will wipe all void or deleted records older than time()-'(last cron run).
Depending on what and how you're building, you might also want to consider using one of the following solutions.
1) A pure PHP CRUD solution would be something along the lines you've mentioned, with also possibly storing cookies on the client side to track which actions are being done. Every action a new cookie is created, then your application will only have to sort the cookies by date and time. You could also set the cookies to be automatically expire after x amount of time. (Although I would expire after a x amount of steps, instead of time)
2) If you are able to use HTML5 local storage (http://www.w3schools.com/html/html5_webstorage.asp) along with some Javascript would be perfect for this, since you wouldn't have to wait around for the server to respond everytime 'undo' is clicked since all the processing would be handled locally.
I am making a session system for my website using PHP and MySQL. The idea is that a user session will last for around 5 minutes if they are inactive, and a CronJob runs every now and then and checks to see if sessions are expired, and if they are, removes the session.
The issue:
Every time someone loads their page it has to check the database to see if their session is still valid. I am wondering if in that CronJob task, I could make it find that users PHP Session and change a variable like $_SESSION['isValidSession'] and set it to false.
So once they load the page it just checks if that variable if the session is valid.
Sorry for the wall of text!
TL;DR: I want to modify session variables of different specified sessions.
Thanks.
Every time someone loads their page it has to check the database to
see if their session is still valid. I am wondering if in that CronJob
task, I could make it find that users PHP Session and change a
variable like $_SESSION['isValidSession'] and set it to false.
You have to do this regardless. When the users load their page, the system must verify whether the session exists in the database (I assume that you're using a DB).
If you run the cron job every minute, and expire all sessions older than five (which seems rather excessive? I often stay inactive on a site for five, ten, even fifteen minutes if I am reading a long page), this will automatically "mark invalid" (actually remove) the sessions.
Normally you would keep a TIMESTAMP column with the time of last update of that row (meaning that session), and the cron job would DELETE all rows with timestamp older than five minutes ago. When reloading the page, the system would no longer find the relevant session row, and deduce (correctly) that the session has expired.
However, what you want (reading a session knowing its SessionID) can be accomplished by reading in the session by the cron job (you can code the job in PHP) either loading as extant session given its ID, or by reading the DB column holding the serialized data with a SELECT SessionData FROM SessionTable WHERE id = 'SessionId'; and de-serializing it. Then you modify the inflated object, re-serialize it and store it back in the database with SQL UPDATE. Hey presto!, session has now been modified.
But be aware that this will likely cause concurrency problems with active clients, and cannot be done in SQL in one fell swoop - you can't execute UPDATE Sessions SET isInactive = 1 WHERE expiry... directly. Normally you need to read the rows of interest one by one, unserialize them and store them back, processing them with PHP code.
You can do it indirectly with two different workarounds.
One, you change your session code to use unserialized data. This will impact maintainability and performance (you can't "just add" something to a session: you have to create a column for it).
Two: you take advantage of the fact that in serialized form, "0" and "1" have the same length. That is, the serialized session containing isValidSession (name of 14 characters) will contain the text
...{s:14:"isValidSession";b:1;}...
and you can change that piece of string with {s:14:"isValidSession";b:0;}, thus making isValidSession become False. This is not particularly good practice - you're messing with the system's internals. Of course, I don't think anybody expects PHP's serialized data syntax to change anytime soon (...or do they?).
<?php var_dump($_SESSION); ?>
You should store the time of last request of the users in the database.
In the cornjob you should check users last view time and compare to current time, then check which user time has been expired.
And then update the column of database as false for expired users.
After than you can easily find out which user should be log out just by checking that colmn in database.
I have log table for all users of website
I'm recording various data about user righ after successfull login.
If signout_dt field not filled and status is 1 for some user_id, website prevents login automatically.
For that who have cookies - there is no problem.
The problem is,lets say user signed in without cookies: only sessions variables. I have no idea, how can I update db table and signout user let's say after 30 minute inactivity. Note that I can't create cron job or something serverside, because using shared hosting.
Heard that, it's possible to create some script like heartbeat that continously sends some data about user activity. But I think this will heavily load the server especially if there are more than 1000 users.. Any suggestion, tutorial, article, something else?
Update
Deceze tried to explain but I really need better explanation (better idea), with code.
To "timeout" a user, simply note the time he was last seen. Then, when necessary, check if the last time you've seen the user was over x minutes/hours/days, and consider the last session timed out. You don't need to run a cron job or anything that cleans up after users in realtime, you only need to be able to determine if some information should be considered stale when you need that information.
You may want to occasionally run a cron job or something to clean out old, unnecessary data, but that doesn't need to happen in realtime. You could even run this as part of a regular page request:
if (mt_rand(1, 1000) == 1) {
mysql_query('DELETE FROM `table` WHERE `last_seen` < some point in time');
}
To note the last seen time, just run this query on each page load:
UPDATE `table` SET `last_seen` = NOW() WHERE `user_id` = ...
To avoid thrashing the database with these queries, you can also just do it every so often. Keep a "last_seen_last_updated" timestamp in the user's session, then on each page load check if you might want to update the database:
if ($_SESSION['last_seen_last_updated'] < strtotime('-5 minutes')) {
mysql_query(...);
$_SESSION['last_seen_last_updated'] = time();
}
That gives you 5 minutes of jitter, but that's usually perfectly acceptable.
Your management of sessions is broken and does not conform to accepted stateless behaviour - in as much as you apparently require the user to sign out, which rarely is the case in web applications -- most people just closes the browser, and the cookies will just float around and appear next time the user accesses the website. If the system wants the user to sign in again, then the web server will have to validate the session -- for example using a timestamp and/or cookie signing etc, and invalidate the cookie to force the user to re-login if needed.
Hence you should treat cookies and sessions variables the same -- that is; have your server side generate a unique signed value. Keep an expiration time (for example now()+20min) either in the cookie/session variable or keep the expiration time in the database.
At each access check that the cookie/session-variable is correctly signed, and check that it is not beyond the expiration time, and update the expiration time to allow another 20min.
If the access is past the expiration time -- i.e. the user has been idle for too long, then clear the cookie/session-variable and force the user to login again.
If you keep the expiration time in the database, you simply write a small program which once and day or once an hour run though all records and remove those which you deem too old.
As per my understanding of your question, you want to address following things:
a. If for a given period of time, a user is inactive then he should be logged out and your database table gets updated. Here being inactive means, user has not used keyboard/mouse for a given period of time.
b. If a user closes the browser without logging out, then he should be forcefully logged out and database table gets updated.
Both these things can be accomplished using Javascript Functions and Ajax. Following is the flow which we have in our application for addressing above issues:
Create a Javascript function, say logoutUser(), which will send an Ajax request for updating the database tables and destroying the session.
Use Javascript function - setTimeOut - to call logoutUser() function after time period you have set for inactivity.
Use Javascript events to catch mouse movement and keyboard activity and in every such event call use successively clearTimeOut (in order to remove the old time for execution of logoutUser()) and setTimeOut (for setting the new time of execution of logoutUser()). This way you would be able to catch the inactivity and logout the user after a period of time.
For taking care of the issue related to closing of browser window use 'onbeforeunload' event of javascript and in this event send the Ajax request for updating the database tables.
As our application uses ExtJS, thus, we have used ExtJs library functions to detect events. You can also prefer using some Javascript library for catching the events and implemeting the above solution.
Hope this helps.
I am currently working on a web application where I have encountered a little problem. In this system, multiple users can log onto the same page and update the data (a series of checkboxes, dropdowns, and text fields).
The issue is that data might get overwritten if one user was already on a page where old data was loaded, and has since been updated, and submits their changes, which update everything.
Any suggestions on how to solve this problem? I am currently just working with plain-text files.
I am currently just working with plain-text files.
Suggestion 1. Use a database.
Suggestion 2. Use a lock file. Use OS-level API calls to open a file with an exclusive
lock. The first user to acquire this file has exclusive access to the data. When that
user finishes their transaction, close the file, release the OS-level lock.
Suggestion 3. Don't "update" the file. Log the history of changes. You can then read usernames and timestamps from the log to find the latest version.
If you do this, you need to make each request do something like this.
When getting the current state, read the last line from the file. Also, get the file size and last modification time. Keep the size and last modified time in the session. Display the current state in the form.
When the user's change is being processed, check the file size and last modification time. If the file is different from what was recorded in the session, this user is attempting an update to data which was changed by someone else. Read the last line from the file. Also, get the file size and last modification time. Keep the size and last modified time in the session. Display the current state in the form.
In addition, you might want to have two files. One with "current" data, the other with the history of changes. This can make it faster to find the current data, since it's the only record in the current state file.
Another choice is to have a "header" in your file that is a fixed-size block of text. Each time you append, you also seek(0,0) and refresh the header with the offset to the last record as well as the timestamp of the last change.
When saving new data you could compare the date that data has been modified with the time the user started editing.
If there have been modifications while the user was making changes you could then show a message to the user and ask them which version to take or allow him to merge the two versions.
This problem has been addressed by revision systems, like svn, git, etc. in the very same fashion.
You can make an additional table, and store there all information as well as userID, so you will be able to access using joins all data that users inserted.