I asked myself what would be the best way to create a history table for a website, I'm only aware of two choices:
Use triggers
Add an extra insert statement to the code when an insert/update/delete statement related to it is used
They said that triggers would be a better way since the load would be on the database and not in the program. But since my website has multiple admins, I would also need to track who modified the content, which I think would only be possible by creating a modified_by column in the history table and manually insert values to it by inserting the session user to that column using the second option together with the time_modified, modified_from, modified_tovalues.
I need to find out if would this be an acceptable reason to use the second option? Are there any more options? Will the second option create any problems in the future?
There is a Rails gem https://github.com/airblade/paper_trail that implements your mentioned functionality comprehensively. I would assume it is doing it purely at the application code level and not the database(triggers) level. This could further indicate that option 2 is better. My thoughts are:
If you use triggers, you will have a serious performance trade off as your site traffic grows and CRUD operations go up. A lot of triggers will go off then.
Implementing some part of the business logic at the database level might be tempting but I would like to keep it all in one place that is in my application code.
You will have some serious thinking to do if you wish to keep your application database agnostic. You will need to re-implement the triggers if you use a different database server.
History keeping increases table size and might become a bottleneck for database performance. You have the option of keeping history for a limited time interval and then archive it to keep the database table nice and clean. Also, you can have a separate database server responsible for history related tables only. These things will be complex to do with triggers.
I can suggest solution # 2. Because i like when all business logic contained in one place (in PHP backend). This code will be more supportable and reusable. And you can save only changed fields into some JSON format. So it keep your HDD place and will work fast. I think triggers is very bad stuff, because you don't know what happens in next time in DB (you must always remember all your triggers):) So i don't use it.
Only one problem you will have - a lot of records in history table. So, i recommend remove records oldest that 3 month
I'd go with option 2. If you are comfortable with ORMs, I'd recommend using one here: a good deal of the history collection code is already written and tested for you.
For example, if you were to use Propel, it comes with a versionable behaviour, which manages a separate versions table, and version numbers per row. (Aside: I believe version 2 hasn't been released as stable yet, though it is linked from the project home page. I use version 1.7, which is still excellent. Both versions have this feature as far as I know).
Doctrine has a similar feature, Versionable, though it looks like that is deprecated in favour of EntityAudit.
Related
I recently started working with Yii PHP MVC Framework. I'm looking for advice on how should I continue working with the database through the framework: should I use framework's base class CActiveRecord which deals with the DB, or should I go with the classic SQL query functions (in my case mssql)?
Obviously or not, for me it seems easier to deal with the DB through classic SQL queries, but, at some point, I imagine there has to be an advantage in using framework's way.
Some SQL queries will get pretty complex pretty often. I just can't comprehend how the framework could help me and not make things more complicated than they actually are.
Very General rule from my experience with Yii and massive databases:
Use Yii Active Record when:
You want to retrieve and post single to a few rows in the database (e.g. user changing his/her settings, updating users balance, adding a vote, getting a count of users online, getting the number of posts under a topic, checking if a model exists)
You want to rapidly design a hierarchical model structure between your tables, (e.g. $user->info->email,$user->settings->currency) allowing you to quickly adjust displayed currency/settings per use.
Stay away from Yii Active Record when:
You want to update several 100 records at a time. (too much overhead for the model)
Yii::app()->db->command()
allows you to avoid the heavy objects and retrieves data in simple arrays.
You want to do advanced joins and queries that involve multiple tables.
Any batch job!! (e.g. checking a payments table to see which customers are overdue on their payments, updating database values etc.)
I love Yii Active Record, but I interchange between the Active Record Model and plain SQL (using Yii::app()->db) based on the requirement in the application.
At the end I have the option whether I want to update a single users currency
$user->info->currency = 'USD';
$user->info->save();
or if I want to update all users currencies:
Yii::app()->db->command('UPDATE ..... SET Currency="USD" where ...');
In any language when dealing with the database a framework can help you by providing an abstraction over the database.
Here is a scenario I know I found myself in many times during my earlier development days:
I have an application that needs a database.
I write a ton of code.
I put the SQL statements in the code along with everything else.
The database changes somehow.
I'm stuck with having to go back and make 100 changes to all my SQL statements.
It's very frustrating.
Another scenario I found:
I write a ton of code against a database.
Bugs come in. Lots of bugs. I can't figure them all out.
I'm asked to write tests for my code.
This is impossible because all my code relies on a direct implementation of the database. How do you test SQL statements when they're with the actual code?
So my advice is to use the framework because it can provide an abstraction over the database. This gives you two really big advantages:
You can potentially swap out the database later and your code stays the same! If you're using interfaces/some framework, then most likely you're dealing with objects and not SQL statements directly. A given implementation might know how to write to MySQL or SQL Server, but in general your code just says "Write this object", "Read that list."
You can test your code! A good framework that deals with data will let you mock the database so you can test it easily.
Try to avoid writing SQL statements directly in the application. It'll save you pain later.
I'm unfamiliar with the database system bundled with Yii, but would advise you to use it a little bit to start with. My experience is with Propel, a popular PHP ORM. In general, ORM systems have a class per table (Propel has three per table).
Now, there'll probably be a syntax to do lookups and joins etc, but the first thing to do is to work out how to use raw SQL in your queries (for any of the CRUD operations). Put methods to do these queries in your model classes, so at least you will be benefitting from centralisation of code.
Once you've got that working, you can migrate to the recommended approach at a later time, without getting overwhelmed with the amount of material you have to learn in one go. Learning Yii (especially how to share code amongst controllers, and to write maintainable view templates) takes a while, so it may be sensible not to over-complicate it with many other things as well.
Why to use Yii:
Just imagine that you have many modules and for each module you have to write a pagination code; writing in old fashion style, will need a lot of time;
Why not use Yii ClistView widget? Oh, and this widget comes with a bonus: the data provider and the auto checking for the existance of the article that is about to be printed;
When using Yii CListView with results from ... Sphinx search engine, the widget will check if the article do really exists, because the result may not be correct
How long will it take for you to write a detection code for non existing registration?
And when you have different types of projects will you addapt the methods?
NO! Yii does this for you.
How long would it take for you to write the code in crud style ? create, read, update, delete ?
Are you going to adapt the old code from another project ?
Yii has a miracle module, called Gii, that generates models, modules, forms, controllers, the crud ... and many more
at first it might seem hard, but when you get experienced, it's easy
I would suggest you should use CActiveRecord.It will give many advantages -
You can use many widgets within yii directly as mentioned above.(For paginations,grids etc)
The queries which are generated by the Yii ORM are highly optimized.
You dont need to put the results extracted from SQLs in your VO objects.
If the tables for some reason modified(addition/deletion of column,changing data type), you just need to regenerate the models using the tool provided by yii.Just make sure you try to avoid doing any code changes in the models generated by yii, that will save your merging efforts.
If you plan to change the DB from MYSQL to other vendor in futur, it would be just config change for you.
Also you and your team would save your precious development time.
I quite often see in PHP, WordPress plugins specifically, that people write SQL directly in their plugins... The way I learnt things, everything should be handled in layers... So that if one day the requirements of a given layer change, I only have to worry about changing the layer that everything interfaces.
Right now I'm writing a layer to interface the database so that, if something ever changes in the way I interact with databases, all I have to do is change one layer, not X number of plugins I've created.
I feel as though this is something that other people may have come across in the past, and that my approach my be inefficient.
I'm writing classes such as
Table
Column
Row
That allow me to create database tables, columns, and rows using given objects with specific methods to handle all their functions:
$column = new \Namespace\Data\Column ( /* name, type, null/non-null, etc... */ );
$myTable = new \Namespace\Data\Table( /* name, column objects, and indexes */ );
\Namespace\TableModel.create($myTable);
My questions are...
Has someone else already written something to provide some separation between different layers?
If not, is my approach going to help at all in the long run or am I wasting my time; should I break down and hard-code the sql like everyone else?
If it is going to help writing this myself, is there any approach I could take to handle it more efficiently?
You seem to be looking for an ORM.
Here is one : http://www.doctrine-project.org/docs/orm/2.0/en/tutorials/getting-started-xml-edition.html
To be honest, I'd just hard-code the SQL, because:
Everyone else does so too. Big parts of WordPress would need to be rewritten, if they would ever wish to change from MySQL to something else. It would just be a waste of time to write your perfect layer for your plugin, if the rest of the whole system still only works with hard-coded SQL.
We don't live in a perfect world. Too much abstraction will - soon or late - end up in performance and other issues, which I don't even think of yet. Keep it simple. Also, using SQL you can benefit from some performance "hacks", which maybe won't work for other systems.
SQL is a widely accepted standard and can already be seen as abstraction layer. for example there's even the possibility to access Facebook's Graph via SQL-like syntax (see FQL). If you want to change to another data-source, you'll probably find some layer wich supports SQL-syntax anyways! In that sense, you could even say SQL already is some kind of abstraction layer.
But: if you decide to use SQL, be sure to use WordPress' $wpdb. Using that, you're on the safe side, as WordPress takes care of connecting to the database, forming the queries, etc. If, one day, WordPress will decide to change from databases to something else, they'll need to create a $wpdb-layer to that new source - for backwards compatibility. Also, many general requests already are in $wpdb as functions (such as $wpdb->insert()), so there's no direct need to hard-code SQL.
If however, you decide to use such an abstraction layer: Wikipedia has more information.
Update: I just found out that the CMS Drupal uses a database abstraction layer - but they still use SQL to form their queries, for all the different databases! I think that shows pretty clearly, how SQL can already be used as an abstraction layer.
I have a design discussion with a collegue.
We are implementing a PHP MySQL database application. In the first instance we have written the Insert Get Update Delete, SelectAll and Search functions for a particular table, writting the table and fieldnames in the code, with several php files, one for the object class, one to draw the HTML table for that table, one for editing a row of that table, one containing the above functions, etc.
The disagreement comes as I have now written generic functions that read/write from the database and draw the HTML taking the table name as a parameter, letting these functions discovers the fieldnames from the database or class. So now that code can be used for any table, with any fields without having to manually go in change each function that needs alteration. I understand there will be cases where more table specific functionality is needed, and this I think that should be done as requirements arise, integrating common parts where possible.
My collegue on the other hand is adamant we should keep a set of files separate for each table, i.e. around 5 php files for each table. The read/write functions written differently in each case, with the need for any changes required for all tables to be affecting 5 x number of tables amount of times.
I can certainly say there will be more than main 15 tables in the database that will at least need basic funcionality.
What approach do you think is most appropriate?
One of the important principles in programming is DRY : Don't Repeat Yourself. So, everything common to several usecases should be written once, in a single location.
Now, I've never had to develop an application where each database table had the same, generic, crud pages. If it were the case, it wouldn't be a functional application, but a database management application. Are you sure you aren't redeveloping phpMyAdmin?
If you need the same code to handle several base operations on several tables, I would say you shouldn't write that code more than once : no need to repeat yourself :
it would take more time writing the application
it would take more time (and possible cause more bugs) when maintaining it
A possible solution could be to put as much common code as possible into a single class, common to many tables ; and, then, have a specific class for each table, that would extend that common class.
This way :
The common code would be written only once
Each table could have its specific code
The common code could be overridden for a specific table, if necessary.
About that, see the Object Inheritance section of the manual.
About that common class idea, you could even go farther, and write pretty much no code at all, if your CRUD is generic enough : there are ORM frameworks that might do that for you.
For instance, you might want to take a look at Doctrine.
We're on the first stages of development, and we don't have the complete functional specifications for the web application we're developing. Yes, we know, but it's not our fault.
So, we're building some parts keeping them pretty simple and straight-forward so we can build on top of that when we have more details on what to build.
We have a section for clients, for ads, for users, ... and I wanted to keep things separate because we don't know what's coming in the future. Yes, at the moment we have only a few fields and some basic listings and editing pages, but all that will grow.
It's not that I don't want to implement some generic code that we can reuse. It's that we don't know yet what will be the limitations in the near future, and I don't want to write generic code that we'll have to parametrize intensely.
For example, Alex built a generic Update method to which you pass an object and it will create an UPDATE SQL statement and execute it. OK, that's cool, but that doesn't work for the Users section of the web app because we store the password encoded. First, it won't encode the password. Second, if you edit a user and don't enter anything on the password and password-confirmation fields, the old password will remain. So, we have a problem with the generic Update method, and as I see it there are two possible solutions:
a) Parametrize the Update method so if it is modifying a user, keep the password if the password on the object is blank. And encode the password, of course.
b) Override the Update method for the child class.
Alex's implementation didn't use inheritance and he used the generic methods in a static class he'd call this way DataAccess::Update($object);. The method takes the table name from the class name as he modified the database to make them match (I prefer "Clients" for the table and "Client" for the class). So, option b is not possible with Alex's implementation.
The way I was trying to build it was keeping separate Update methods for each table. Yes, I was repeating myself but, as I said before, we don't have a full specification, so we don't know how it's going to grow. We have an idea, but we don't have the exact details.
So, the point here is that I don't want to write generic code until we have a much more detailed specification so we can evaluate what can and what cannot be shared between the parts.
Not all sections of the web app work the same and, as JB Nizet said: "If it were the case, it wouldn't be a functional application, but a database management application."
And I can tell you for sure this is not a database management application, even though Alex would say "we're just building a database app." Well, maybe, but a database application is not only showing/modifying the tables. And now, views won't solve all problems.
Again, as JB Nizet said: "If it were the case, it wouldn't be a functional application, but a database management application."
And now I'm repeating myself again, but this time there's no reason for that.
Thanks for your time.
I am working on a web application which involves create list of Restaurants in various lists like "Joe's must visit places". Now for each Restaurant and list, I have display on website which calculates
Calculating popularity of a Restaurant
Popularity of a list
Number of lists a Restaurant is present in
Currently I am using MySQL statements in PHP for this but planning to switch to MySQL VIEWS and do a simple select statement in PHP...
my question is,
What is Advantage/Disadvantage of using VIEWS over writing sql queries in PHP?
Using views adds a level of abstraction : you may later change the structure of your tables, and you will not have to change the code that displays the information about the lists, because you will still be querying the view (the view definition may change, though).
The main difference is that views are updated after each insertion, such that the data is "ready" whenever you query the view, whereas using your custom query will have MySQL compute everything each time (there is some caching, of course).
The bottom line is that if your lists are updated less frenquently than they are viewed, you will see some gains in performance in using views.
My complete answer would depend upon several things (from your application's perspective):
do you plan to allow users to create and share such lists?
can users create lists of any kind, or just by plugging values into existing query templates?
Assuming you have a couple of pre-defined lists to display:
Use of views offers a couple of advantages:
your code will be cleaner
the query to generate the views will not have to be parsed each time by mysql.
I'm not sure about this: I don't think mysql caches views as Tomasz suggests - I don't think views contain "already preparted data".
One disadvantage is that the logic involved in creating the list goes into the database instead of living in your PHP code - something I'm very averse to. In my world databases are for data, and code is for logic.
Cheers
The original question was about pros and cons, but not seeing much about disadvantages in the answers so far.
Isn't one disadvantage of views that they can give you the false comfort of running a simple query?
For instance, SELECT username FROM myview WHERE id='1'
That looks simple, but what if "myview" is a really complex SELECT... Perhaps even built on other views? You end up having a simple-looking query that, in the background, takes a whole lot more work than if you had written your query from the ground up.
I've been experimenting with views, and despite the benefits, have not yet been fully sold.
I'd be interested in hearing what others perceive about the disadvantages of Views, rather than just the party line about why views are so great. Might still make the switch, but would like to understand more about performance.
If that tables you are trying to make view from are not subject to a frequent change, definitely you gain performance, as you are only doing simple select from already prepared data. But be aware of the fact, that view is not something that is made "once and forever" - every change of a content of one of the tables will make database engine do "view refreshing", so another query (query you are making view from) must be called to taki into account changes that were made. To sum up:
Infrequent changes? Performance. Frequent / constant changes (community adding, commenting, rating your restaurants) - better go with SQL queries.
Disadvantages:
In my opinion databases are used for data layer and it is not that proper to put business code inside them. It both reduces maintainability and it contradicts clean separation of layers. The same applies to including business code and calculations in java scripts of web pages. For java script it is even more serious since it creates security threats. Source control for the code inside database is also another issue.
Now that code is inside database, the security and access complications (to views and stored procedures) is also added.
Migrating an application from one database engine to another will be much more difficult (since in addition to simple queries the stored procedures/views etc. are possibly different too). If the database is only about data then an abstraction layer could allow changing the database engine (at least on at some extent).
Advantages:
Slight performance gains (since data is not coming out of the database for processing, it is processed right inside the database).
Code will seem cleaner (since the dirtiness is hidden inside the database views, stored procedures etc.).
I'm creating a movies website, IMDB.com like.. I'm really new to PHP and programming at all but I have some books and StackOverflow of course :)
I have already done lot of the work, but now I have more than 600 lines of code (PHO only) per page and more than 20 database tables only for storing and geting the movie data (many-to-many relationships)
Most of the code are MySQLi queries (prepared statements) to insert/get data and loops. Nothing fancy, just basic PHP/MySQL.
I have some questions:
It is recommended to use MySQLi prepared statements for every SQL query or is better just to use old MySQL for internal PHP/database queries and leave prepared statements for user input only?
To get all the movie data and show it I need to get the data from more than 16 different tables. I use one MySQL query per table (somethimes more) and lots of loops. This is the correct way a website need to work?? I mean.. this is normal???
How can I simplify the code to simplify the reading? Can I separete all those queries to external files?? Can I create functions to handle each table query??
Hope you can give me a hand and maybe you have some more suggestions for correctly building such a website.
Thanks!!
As mentioned in the anwsers above I would also point you to using a framework that impliments the MVC design pattern. Along with that most of the frameworks have an ORM built in but if they do not you can look at Symphony or EZPDO is another good ORM to tie into your Model in (M)VC.
CodeIgniter is a very fast and light weight MVC framework that would allow you to bootstrap pretty quickly but you may also want to look into ZF (ZendFramework). ZF has a great framework feature set and is pretty flexible overall.
Beyond that be sure to seperate your reads and your writes in either or Model or your calls to the ORM of choice. This will allow you to slave data to multiple MySQL Boxes for bigger performance but allows you to use one database engine to start out.
Add in the ability to use Memcached so that you can cache your data/objects vs hitting the database.
When using cache put some thought into how you would expire cache for a database update. In other words if your selecting data from a database to display in a view and that data is has not changed you should not need to hit the database every time but rather pull it from memory. Once the data actually does change you'd want to invalidate that cache so it's not stale and then re-cache the new data.
memcached: http://www.danga.com/memcached/'
-facebook also has a version
CodeIgniter - http://codeigniter.com/
EZPDO - http://www.ezpdo.net/blog/?p=2
ZendFramework -http://framework.zend.com/
Consider looking at or using a web framework for your website.
symfony
CakePHP
CodeIgniter
Are some of the more mainstream ones. Learn from them if anything.
Prepared statements are just fine for your own internal queries to. You'll have a strutured approach for all queries.
Well it depends on what you're showing. But I would say that you normally could use joins to get the data you need from more tables. You'll have a lot less quering to get the correct data, and it sounds like all your data is connected somehow to the one movie you're showing.
As Peter D comments, I would recommend using a web framework to learn how to seperate out the database handling from the view. Are you using an object oriented approach now? Look at the MVC pattern that some of these frameworks implement, that'll get you going.
Like Peter D mention before add this one to the list of framework to use. Zend Framework http://framework.zend.com
Open source and free.
It is recommended to use MySQLi ...
Definitely MySQLi, but it's a big question by itself, if you start coding you'll need to grasp the basic of T-SQL to understand the difference.
To get all the movie data ...
It depends on alot of things. Size of database. Wanted results i.e. the information that need to be displayed, response time of the queries vs displaying in user view. Do you know about JOIN, UNION?
How can I simplify the code to ...
Yes to all theses questions. www.w3schools.com/php/ if it can be of any help and learn the MVC pattern. Useful to alot of programming language these days. Maybe a framework would help you here
To this questions:
"To get all the movie data and show it
I need to get the data from more than
16 different tables. I use one MySQL
query per table (somethimes more) and
lots of loops. This is the correct way
a website need to work?? I mean.. this
is normal???"
No. If I understand you correctly, you should be using some type of JOIN depending on the data you're retrieving from the database. Getting results for huge amounts of data, and then picking out only the pieces you want in PHP is much slower than letting the database do the work of sorting/retrieving only the records/info you want to show.
I highly recommend a somewhat dated but very easy to grasp book that covers PHP and MySQL/Databases in general: http://www.dmcinsights.com/phpmysql2/ - It covers a lot of "in practice" techniques along with the code, so it'd be great to learn from.
There is a apparently a third edition with updated info, but I have not looked at it.