I am working on a web based application using Zend framework.
I am looking for a way to keep history of updates made to a bunch of columns.
Use-case:
User can add an item with 5 properties. The user is allowed to update the 5 properties. I need to track all the updates he makes to those 5 properties.
On way I thought of was adding a new table named log which stores the old value, new value, column name, itemID and timestamp.
What is the best way to do this? Are there any existing methods/options/examples ?
Thanks,
What you're probably looking for is an audit log. You can create one using Triggers on your MySQL database.
An example of how to do this is here: http://ronaldbradford.com/blog/auditing-your-mysql-data-2008-07-15/
In your comment below you added that the database is not 'fixed'. In that case, you'd have to replicate any alters to the table so that they are applied to the audit table in such a way that any newly added columns are added to the log as well.
You can consider logging this in your application (as your tags chosen for this question seem to suggestion) - but keep in mind that this means there can be situations where your log does not provide the complete answer. Triggers, stored procedures and any manual interventions in the database will not be logged... so be careful if you choose to go down that path.
Triggers are the most common way to do auditing and the only really reliable way to capture what was done whether done from the user interface or elsewhere. They do vary by database in how they would be written though. If you know the possible types of database backends you will support, you could write separate triggers for each.
If you must handle this without triggers, then your best bet is to have a process that writes to the audit table as well as makes the update change. It might be complex enough to warrant a stored proc called by the Zend framework rather than relying on the framework itself to do. (I'm not familair with Zend so I don't know if this is something that could be set up, I know a stored proc could handle this.)
Here is a better one..
See Pop On the Audit Trail
I just created a new table called it Comp_Hist_Log and then defined the old data in the
BEFORE UPDATE hook
$oldData = $array('fieldname1', 'fieldname2')
Then at the AFTER_UPDATE hook in my database gui hook file.. I added this code
sql("INSERT INTO Comp_Hist_Log (Com_Rec_Id, old_data, new_data, ChangedDate, ChangedBy)
VALUES('{$data['Record_Id']}', '{$oldData}', '{$messageData}', '{$data['LastUpdated']}', '{$memberInfo['username']}')", $eo);
return TRUE;
Hope it helps.. it does work.
Related
I'm building a web application where several groups have their own page but if they want to modify it, an administrator has to validate it before.
For example, can change to change its logo, post new photo, change their phone number, their name, their location etc... Basically they can edit a value in the database but only if the administrator accepts it. The administrator has to validate every modification because... our customer asked us to.
That's why we have to create a system that could be called "pending queries" management.
At the beginning I thought that keeping the query in the database and executing when an administrator validate it was a good idea, but if we choose this option we can't use PDO to build prepared statements since we have to concatenate string to build our own statement, wich obvious security issues.
Then we thought that we should keep PHP code that calls the right methods (that use PDO) in our database and that we will execute with eval() when the administrator validates it. But again, it seems that using eval() is a very bad idea. As says this Rasmus Lerford's quote : "If eval() is the answer, you're almost certainly asking the
wrong question".
I thought about using eval because I want to call methods that uses PDO to deal with the database.
So, what is the best way to solve this problem ? It seems that there is no safe way to implements it.
Both your ideas are, to be frank, simply weird.
Add a field in a table to tell an approved content from unapproved one.
Here's one possible approach, with an attempt to keep the things organised to an extent, as the system begins to scale:
Create a table called PendingRequests. This will have to have most of the following fields and maybe quite a few more:
(id, request_type, request_contents, reqeust_made_by, request_made_timestamp,
request_approved_by, request_approved_timestamp, ....)
Request_contents is a broad term and it may not just be confined to one column alone. How you gather the data for this column will depend on the front-end environment you provide to the users (WYSIWYG, etc).
Request_approved_by will be NULL when the data is first inserted in the table (i.e. user has made an initial request). This way, you'll know which requests to present in the administration panel. Once an admin approves it, this column will be updated to reflect the id of the admin that approved it and the approved changes could eventually go live.
So far, we've only talked about managing the requests. Once that process is established, then the next question would be to determine how to finally map the approved requests to users. As such, it'd actually require a bit of study of the currently proposed system and its workflow. Though, in short, there may be two school of thoughts:
Method 1:
Create a new table each for everything (logo, phone number, name, etc) that is customisable.
Or
Method 2:
Simply add them as columns in one of your tables (which would essentially be in a 1:1 relationship with the user table, as far as attributes such as logo, name, etc. are concerned).
This brings us to Request_type. This is the field that will hold values / flags for the system to determine which field or table (depending on Method 1 or Method 2) the changes will be incident upon - after an admin has approved the changes.
No matter what requirement or approach it is to go about database management, PHP and PDO are both flexible enough to help write customisable and secure queries.
As an aside, it might be a good idea to maintain a table for history of all the changes / updates made. By now, it should probably be apparent that the number of history tables will once again depend on Method 1 or Method 2.
Hope that helps.
I am creating an application with a click to call button on an html page.
There will be one person manning the phone. I want this person to be able to set a variable with a boolean value on my server: 1 is available, 0 is unavailable.
I could create a single field SQL table but this feels like overkill, or I could read and write to a text file containing just one character.
What is the most correct way to store a single value?
I know it seems like overkill to use a small database table for this.
If your application already uses a database, this is by far the best way to proceed. Your database technology has all kinds of support for storing data so it doesn't get lost. But, don't stand up a database and organize your application to use it just for this one data point; a file will be easier in that case.
(WordPress does something similar; it uses a table called wp_options containing a lot of one-off settings values.)
I suggest your table contain two columns (or maybe more), agent_id and available. Then, if you happen to add another person taking telephone calls, your app will be ready to handle that growth. Your current person can have agent_id = 0.
If you have a DB set up, I'd use it.
That's what DB's are for, persisting changeable data.. otherwise you are basically writing your own separate DB system for the sake of one setting, which would be uberkill in my eyes!
There is value in consistency and flexibility.. what if I suddenly need to store an expected return time? How do I do this in a text-file, how do I differentiate the column? How do I manipulate the data? MySQL already answers all these questions for you.
As a team member, I'd expect most of my dev colleagues (and new hires) to know how to use MySQL.. I wouldn't want them to have to work with, extend or debug a separate bespoke file persistence system that I had tacked on.
If you are worried about having lots of one row tables dotted about, you could use a single table for miscellaneous singular config variables which need updating regularly.
We have a table like this:
Table: `setting`
Columns: `key_string` VARCHAR, `value` VARCHAR
And could store your variable as
['key_string' => 'telephone_service_available', 'value' => '1']
In this specific case a simple file check (Exist a file or not) is probably the most simple way you can do here. And it also has the benefit to easily check if the file exist or not, you don't have to read file contents.
But if you need just one more information, you have to go a complete other way.
Depends on what you try to do afterwards with the information.
If you use it within a web-application store it in the session.
Or try a flatfile-database like SQLite (no active DBMS needed). Its easy and you can extend it very easy.
Or just a bipolar information with creating a file. If the file is not there is is off.
The problem I'm here to talk about and (ask about of course) is not new. I searched web and stack overflow and I got ideas to many part of this problem (pros and cons) but there is still some part missing in my mind. So I thought it would be a good idea to share in one place (of course it will be more complete with others' ideas) and ask for it.
The problem is clear: "We Want to log every single action of user" - probably when we solve the big problem, smaller ones (like logging only one action would be piece of cake).
First from what I read over the web and stack overflow:
Use DB instead of File: That's a good advice although it always depends on situation. But because of many benefits of DB, in long term and in general, it's the better solution.
DB Layer or Application Layer: Actually it depends. For example If you want really monitor everything(I mean really every single rows that changes in Database, it seems we will have one choice "Using Database Triggers". Although there are many discussions around MySQL that says, triggers slowdown DB and they advised not to use it. So it depends on the level of details you need, you can put your logging system in DB Layer or Application Layer(for exam some common function call $logClass->logThis()).
Use Observers: Clean codes are always better. If you are familiar with observers, you can use them to do things for you when an action is happened so you don't have to add $logClass->logThis() every time a CRUD happens in your application.
What To Log: Simple and short answer is: Based on your needs, but there are some common fields you will need:
user_id (if a unique user ID is available)
timestamp (unix maybe)
ip (not everyone know how to fake it in first place so use it, even faking it give you some insight about user behavior)
action_id (should be predefined actions for better unifying in queries and reports)
object_id (the unique row ID of a record that changes had made on)
action (which my question is about this part)
and etc...
I would appreciate if anyone correct me if I made mistake in any part or add other useful information to this post, so it would become one of good references for other users.
And now my question: How to Store actions?. For better understanding, consider following scenario.
I have a table named "product" and a table named "companies". From the business logic we want to assign products to companies, which we ended up in a table "company_product". Now when a user insert new product and simultaneously assign it's companies, 2 table will be affected (the same goes for delete and update): "product" and "company_product" and we want to know:
what's inserted?
what's deleted?
what's updated to what?
For performance issue and because I don't have enough knowledge about triggers, I want to use logging in Application Layer, so I ended up with this idea that I can, save action fields of database in array or json structure. But as I developed my solution I encountered a problem: How to make this log understandable for non technical users? Because for example I want to save something like this in action field of database when delete(insert) product with id 20:
action : [{id: 20, product_id:2, company_id: 1},{id: 21, product_id:2, company_id: 2}]
And this is not something easy for every one to read and understand. Actually I can use this json more readable and make it something like this:
action : {'Product A Deleted From Company X', 'Product A Deleted From Company Y'}
and save the previous action in technical_action field for further diagnose, But it needs additional works and more query to run for something that is not always needed to be considered(log)
I would appreciate any additional information on this article (I'm definitely sure that there exist other criteria that can be discussed), and answer to my question.
You are actually going to gather details for analytics kind of stuffs.
It will be good if you go for flat tables rather than going to relational tables.
Because if you want to do more analysis your relational table will not be a good choice as it lacks in performance.
I have a dilemma, which I hope you will have some expert opinions on.
I have a table called CARDS with a column STATUS. If a record's status changes from 'download' to 'publish', I have to insert the record reference into another table called CARD_ASSIGNMENTS. Additionally, the record needs to be added into CARD_ASSIGNMENTS as many times as there are active records in SCANNERS.
In other words, if there are two active scanners, I will end up with two records in CARD_ASSIGNMENTS as below:
ID CARD_ID SCANNER_ID STATUS_ID
1 1 1 4
2 1 2 4
My dilemma is that I'm not quite sure what would be the most efficient way to execute the above. I've considered the following options:
From PHP - Do one UPDATE query and then the INSERT queries.
Create a stored procedure, which will take care of updating the CARDS record and adding records into the CARD_ASSIGNMENTS. Then, just call that stored procedure from PHP.
Create an ON UPDATE trigger for the CARDS table which will take care of processing INSERTS into the CARD_ASSIGNMENTS table.
PS. A simplified version of my database is available on MySQL Fiddle
Thanks,
Kate
Interesting question.
I'm going to give you clues about how to approach the problem.
So, you have to start by defining precisely three things:
the expected functionality
the access policy to the functionality
the technical upgrade policy
Here I'll detail these points.
So, the first point is that you have to define your functionality. By doing so, you will be able to tell whether adding a card implies always, in all the possible paradigms (sorry for the pedantic word I can't find a more proper one) of your information system, that this card MUST exist in the other table according to the specifications you provided. This 1-1 functional link must be said TRUE or FALSE. This is really important.
Said with other words, if there's at least one possibility that one day you don't want to copy that record to the other table, it means the trigger is a wrong solution, or at least it should be thought with an emergency mode (for example a variable inside that allows it to not get executed in some conditions) setup on.
Then comes the second point, about the access policy. You have to know whether the allowed accessing systems will do so by using your application layer or if they could develop their own (SAAS style). If so, your php layer will be useless and the stored procedure is an excellent option, since every single technical and business layer will go trough it yes or yes.
The last thing to know is whether you're possibly going to upgrade your php layer one day. In most of the cases the answer is yes. If so, you might have to modify the part containing this sql logic you're talking about. Then, having everything into a stored procedure vs storing it hardcoded into the php will definitely save you time, and improve stability.
Left brain right brain, I'm going to tell you my personal opinion afterall. I really love going with stored procedures but not using any triggers. If the environment allows it, I would go for an underlying batch, calling a set of defined stored procedures, concentrating the activity outside of the online scope.
The advantages are the following:
none or less risks of interruption of the online workflow since you reduce the number of operations
different schedule to alliviate the database load
more secure policy since executing the stored procedure requires only one grant, while using the same sql with php would require insert/update grants
better logging quality: you can have a log per job
better emergency response: when a job fails (if well thought) you can restart it, and that's it.
Long post, but that was interesting and I really wanted to share these ideas.
Cheers!
I would use triggers. Some developers say, that if you have too many triggers and stored procedures, the database lives its own life, that means you never know what is going to happen on insert, update etc. But in my opinion, triggers may help you a lot to keep database consistent, so even if someone inserts data directly from some administration tool, the integrity is still kept, because all necessary commands are executed. If you choose stored procedures, you would still have to know, that you need to call this procedure to insert any new data.
I want to create a simple auditing system for my small CodeIgniter application. Such that it would take a snapshot of a table entry before the entry has been edited. One way I could think of would be to create a news_audit table, which would replicate all the columns in the news table. It would also create a new record for each change with the added column of date added. What are your views, and opinions of building such functionality into a PHP web application?
There are a few things to take into account before you decide which solution to use:
If your table is large (or could become large) your audit trail needs to be in a seperate table as you describe or performance will suffer.
If you need an audit that can't (potentially) be modified except to add new entries it needs to have INSERT permissions only for the application (and to be cast iron needs to be on a dedicated logging server...)
I would avoid creating audit records in the same table as it might be confusing to another developer (who might no realize they need to filter out the old ones without dates) and will clutter the table with audit rows, which will force the db to cache more disk blocks than it needs to (== performance cost). Also to index this properly might be a problem if your db does not index NULLS. Querying for the most recent version will involve a sub-query if you choose to time stamp them all.
The cleanest way to solve this, if your database supports it, is to create an UPDATE TRIGGER on your news table that copies the old values to a seperate audit table which needs only INSERT permissions). This way the logic is built into the database, and so your applications need not be concerned with it, they just UPDATE the data and the db takes care of keeping the change log. The body of the trigger will just be an INSERT statement, so if you haven't written one before it should not take long to do.
If I knew which db you are using I might be able to post an example...
What we do (and you would want to set up archiving beforehand depending on size and use), but we created an audit table that stores user information, time, and then the changes in XML with the table name.
If you are in SQL2005+ you can then easily search the XML for changes if needed.
We then added triggers to our table to catch what we wanted to audit (inserts, deletes, updates...)
Then with simple serialization we are able to restore and replicate changes.
What scale are we looking at here? On average, are entries going to be edited often or infrequently?
Depending on how many edits you expect for the average item, it might make more sense to store diff's of large blocks of data as opposed to a full copy of the data.
One way I like is to put it into the table itself. You would simply add a 'valid_until' column. When you "edit" a row, you simply make a copy of it and stamp the 'valid_until' field on the old row. The valid rows are the ones without 'valid_until' set. In short, you make it copy-on-write. Don't forget to make your primary keys a combination of the original primary key and the valid_until field. Also set up constraints or triggers to make sure that for each ID there can be only one row that does not have it's valid_until set.
This has upsides and downsides. The upside is less tables. The downside is far more rows in your tables. I would recommend this structure if you often need to access old data. By simply adding a simple WHERE to your queries you can query the state of a table at a previous date/time.
If you only need to access your old data occasionally then I would not recommend this though.
You can take this all the way to the extreme by building a Temportal database.
In small to medium size project I use the following set of rules:
All code is stored under Revision Control System (i.e. Subversion)
There is a directory for SQL patches in source code (i.e. patches/)
All files in this directory start with serial number followed by short description (i.e. 086_added_login_unique_constraint.sql)
All changes to DB schema must be recorded as separate files. No file can be changed after it's checked in to version control system. All bugs must be fixed by issuing another patch. It is important to stick closely to this rule.
Small script remembers serial number of last executed patch in local environment and runs subsequent patches when needed.
This way you can guarantee, that you can recreate your DB schema easily without the need of importing whole data dump. Creating such patches is no brainer. Just run command in console/UI/web frontend and copy-paste it into patch if successful. Then just add it to repo and commit changes.
This approach scales reasonably well. Worked for PHP/PostgreSQL project consisting of 1300+ classes and 200+ tables/views.