I am working on a web application which involves create list of Restaurants in various lists like "Joe's must visit places". Now for each Restaurant and list, I have display on website which calculates
Calculating popularity of a Restaurant
Popularity of a list
Number of lists a Restaurant is present in
Currently I am using MySQL statements in PHP for this but planning to switch to MySQL VIEWS and do a simple select statement in PHP...
my question is,
What is Advantage/Disadvantage of using VIEWS over writing sql queries in PHP?
Using views adds a level of abstraction : you may later change the structure of your tables, and you will not have to change the code that displays the information about the lists, because you will still be querying the view (the view definition may change, though).
The main difference is that views are updated after each insertion, such that the data is "ready" whenever you query the view, whereas using your custom query will have MySQL compute everything each time (there is some caching, of course).
The bottom line is that if your lists are updated less frenquently than they are viewed, you will see some gains in performance in using views.
My complete answer would depend upon several things (from your application's perspective):
do you plan to allow users to create and share such lists?
can users create lists of any kind, or just by plugging values into existing query templates?
Assuming you have a couple of pre-defined lists to display:
Use of views offers a couple of advantages:
your code will be cleaner
the query to generate the views will not have to be parsed each time by mysql.
I'm not sure about this: I don't think mysql caches views as Tomasz suggests - I don't think views contain "already preparted data".
One disadvantage is that the logic involved in creating the list goes into the database instead of living in your PHP code - something I'm very averse to. In my world databases are for data, and code is for logic.
Cheers
The original question was about pros and cons, but not seeing much about disadvantages in the answers so far.
Isn't one disadvantage of views that they can give you the false comfort of running a simple query?
For instance, SELECT username FROM myview WHERE id='1'
That looks simple, but what if "myview" is a really complex SELECT... Perhaps even built on other views? You end up having a simple-looking query that, in the background, takes a whole lot more work than if you had written your query from the ground up.
I've been experimenting with views, and despite the benefits, have not yet been fully sold.
I'd be interested in hearing what others perceive about the disadvantages of Views, rather than just the party line about why views are so great. Might still make the switch, but would like to understand more about performance.
If that tables you are trying to make view from are not subject to a frequent change, definitely you gain performance, as you are only doing simple select from already prepared data. But be aware of the fact, that view is not something that is made "once and forever" - every change of a content of one of the tables will make database engine do "view refreshing", so another query (query you are making view from) must be called to taki into account changes that were made. To sum up:
Infrequent changes? Performance. Frequent / constant changes (community adding, commenting, rating your restaurants) - better go with SQL queries.
Disadvantages:
In my opinion databases are used for data layer and it is not that proper to put business code inside them. It both reduces maintainability and it contradicts clean separation of layers. The same applies to including business code and calculations in java scripts of web pages. For java script it is even more serious since it creates security threats. Source control for the code inside database is also another issue.
Now that code is inside database, the security and access complications (to views and stored procedures) is also added.
Migrating an application from one database engine to another will be much more difficult (since in addition to simple queries the stored procedures/views etc. are possibly different too). If the database is only about data then an abstraction layer could allow changing the database engine (at least on at some extent).
Advantages:
Slight performance gains (since data is not coming out of the database for processing, it is processed right inside the database).
Code will seem cleaner (since the dirtiness is hidden inside the database views, stored procedures etc.).
Related
I have a bunch of legacy code without a Model layer separating it from the database. SQL statements abound. I would like to normalize tables in the database, yet not have to rewrite all this embedded SQL (eventually I will). I've tried updatable views in MySQL, but for anything of any complexity, it will perform a full table scan.
Can anyone suggest a clever way to hide database schema changes from clients that heavily rely on those very schemas?
There's no clever way to do it all at once. It's bound to be a difficult, meticulous process. Here are some tips:
Refactor your application code so that you don't have SQL code littered around the whole codebase. Refactor so that all the literal SQL is consolidated in a few "data access" classes, with sensible APIs. This is something you can do gradually.
Develop a thorough suite of system tests that you can run against your application so as you change the database schema, you can re-run the tests to ensure you didn't break something. This is also something you can develop gradually.
Use read-only views where you can. You can't make views for complex SQL queries as updatable views, but at least you can use views for the reading queries. I'm not sure why that would cause a table-scan where the original query did not. Sometimes views cause a temporary table, which is also costly.
In some cases, it's easier to make a view for the new tables. In other cases, it's easier to make a view for the old tables. Keep your options open to do it either way, on a case-by-case basis.
In your data access classes, you may have to double the database work. For example, saving data changes to the old tables and the new refactored tables as part of the same operation. Once you have tested to make sure all the database queries are using your data access class, you can then take out the code that maintains the legacy tables.
Eventually, you will have to bite the bullet and make some schema changes that require a "big bang integration". That's the term I've heard for a change that causes a lot of changes across all your code. It's a lot of work to do all the testing necessary to ensure nothing breaks. Sorry, there's no magic to avoid that work. But you can prepare for it with your data access layer and lots of system tests.
This is probably a question that the Stack Overflow community will close as "opinion-based".
I asked myself what would be the best way to create a history table for a website, I'm only aware of two choices:
Use triggers
Add an extra insert statement to the code when an insert/update/delete statement related to it is used
They said that triggers would be a better way since the load would be on the database and not in the program. But since my website has multiple admins, I would also need to track who modified the content, which I think would only be possible by creating a modified_by column in the history table and manually insert values to it by inserting the session user to that column using the second option together with the time_modified, modified_from, modified_tovalues.
I need to find out if would this be an acceptable reason to use the second option? Are there any more options? Will the second option create any problems in the future?
There is a Rails gem https://github.com/airblade/paper_trail that implements your mentioned functionality comprehensively. I would assume it is doing it purely at the application code level and not the database(triggers) level. This could further indicate that option 2 is better. My thoughts are:
If you use triggers, you will have a serious performance trade off as your site traffic grows and CRUD operations go up. A lot of triggers will go off then.
Implementing some part of the business logic at the database level might be tempting but I would like to keep it all in one place that is in my application code.
You will have some serious thinking to do if you wish to keep your application database agnostic. You will need to re-implement the triggers if you use a different database server.
History keeping increases table size and might become a bottleneck for database performance. You have the option of keeping history for a limited time interval and then archive it to keep the database table nice and clean. Also, you can have a separate database server responsible for history related tables only. These things will be complex to do with triggers.
I can suggest solution # 2. Because i like when all business logic contained in one place (in PHP backend). This code will be more supportable and reusable. And you can save only changed fields into some JSON format. So it keep your HDD place and will work fast. I think triggers is very bad stuff, because you don't know what happens in next time in DB (you must always remember all your triggers):) So i don't use it.
Only one problem you will have - a lot of records in history table. So, i recommend remove records oldest that 3 month
I'd go with option 2. If you are comfortable with ORMs, I'd recommend using one here: a good deal of the history collection code is already written and tested for you.
For example, if you were to use Propel, it comes with a versionable behaviour, which manages a separate versions table, and version numbers per row. (Aside: I believe version 2 hasn't been released as stable yet, though it is linked from the project home page. I use version 1.7, which is still excellent. Both versions have this feature as far as I know).
Doctrine has a similar feature, Versionable, though it looks like that is deprecated in favour of EntityAudit.
I recently started working with Yii PHP MVC Framework. I'm looking for advice on how should I continue working with the database through the framework: should I use framework's base class CActiveRecord which deals with the DB, or should I go with the classic SQL query functions (in my case mssql)?
Obviously or not, for me it seems easier to deal with the DB through classic SQL queries, but, at some point, I imagine there has to be an advantage in using framework's way.
Some SQL queries will get pretty complex pretty often. I just can't comprehend how the framework could help me and not make things more complicated than they actually are.
Very General rule from my experience with Yii and massive databases:
Use Yii Active Record when:
You want to retrieve and post single to a few rows in the database (e.g. user changing his/her settings, updating users balance, adding a vote, getting a count of users online, getting the number of posts under a topic, checking if a model exists)
You want to rapidly design a hierarchical model structure between your tables, (e.g. $user->info->email,$user->settings->currency) allowing you to quickly adjust displayed currency/settings per use.
Stay away from Yii Active Record when:
You want to update several 100 records at a time. (too much overhead for the model)
Yii::app()->db->command()
allows you to avoid the heavy objects and retrieves data in simple arrays.
You want to do advanced joins and queries that involve multiple tables.
Any batch job!! (e.g. checking a payments table to see which customers are overdue on their payments, updating database values etc.)
I love Yii Active Record, but I interchange between the Active Record Model and plain SQL (using Yii::app()->db) based on the requirement in the application.
At the end I have the option whether I want to update a single users currency
$user->info->currency = 'USD';
$user->info->save();
or if I want to update all users currencies:
Yii::app()->db->command('UPDATE ..... SET Currency="USD" where ...');
In any language when dealing with the database a framework can help you by providing an abstraction over the database.
Here is a scenario I know I found myself in many times during my earlier development days:
I have an application that needs a database.
I write a ton of code.
I put the SQL statements in the code along with everything else.
The database changes somehow.
I'm stuck with having to go back and make 100 changes to all my SQL statements.
It's very frustrating.
Another scenario I found:
I write a ton of code against a database.
Bugs come in. Lots of bugs. I can't figure them all out.
I'm asked to write tests for my code.
This is impossible because all my code relies on a direct implementation of the database. How do you test SQL statements when they're with the actual code?
So my advice is to use the framework because it can provide an abstraction over the database. This gives you two really big advantages:
You can potentially swap out the database later and your code stays the same! If you're using interfaces/some framework, then most likely you're dealing with objects and not SQL statements directly. A given implementation might know how to write to MySQL or SQL Server, but in general your code just says "Write this object", "Read that list."
You can test your code! A good framework that deals with data will let you mock the database so you can test it easily.
Try to avoid writing SQL statements directly in the application. It'll save you pain later.
I'm unfamiliar with the database system bundled with Yii, but would advise you to use it a little bit to start with. My experience is with Propel, a popular PHP ORM. In general, ORM systems have a class per table (Propel has three per table).
Now, there'll probably be a syntax to do lookups and joins etc, but the first thing to do is to work out how to use raw SQL in your queries (for any of the CRUD operations). Put methods to do these queries in your model classes, so at least you will be benefitting from centralisation of code.
Once you've got that working, you can migrate to the recommended approach at a later time, without getting overwhelmed with the amount of material you have to learn in one go. Learning Yii (especially how to share code amongst controllers, and to write maintainable view templates) takes a while, so it may be sensible not to over-complicate it with many other things as well.
Why to use Yii:
Just imagine that you have many modules and for each module you have to write a pagination code; writing in old fashion style, will need a lot of time;
Why not use Yii ClistView widget? Oh, and this widget comes with a bonus: the data provider and the auto checking for the existance of the article that is about to be printed;
When using Yii CListView with results from ... Sphinx search engine, the widget will check if the article do really exists, because the result may not be correct
How long will it take for you to write a detection code for non existing registration?
And when you have different types of projects will you addapt the methods?
NO! Yii does this for you.
How long would it take for you to write the code in crud style ? create, read, update, delete ?
Are you going to adapt the old code from another project ?
Yii has a miracle module, called Gii, that generates models, modules, forms, controllers, the crud ... and many more
at first it might seem hard, but when you get experienced, it's easy
I would suggest you should use CActiveRecord.It will give many advantages -
You can use many widgets within yii directly as mentioned above.(For paginations,grids etc)
The queries which are generated by the Yii ORM are highly optimized.
You dont need to put the results extracted from SQLs in your VO objects.
If the tables for some reason modified(addition/deletion of column,changing data type), you just need to regenerate the models using the tool provided by yii.Just make sure you try to avoid doing any code changes in the models generated by yii, that will save your merging efforts.
If you plan to change the DB from MYSQL to other vendor in futur, it would be just config change for you.
Also you and your team would save your precious development time.
OOP principles were difficult for me to grasp because for some reason I could never apply them to web development. As I developed more and more projects I started understanding how some parts of my code could use certain design patterns to make them easier to read, reuse, and maintain so I started to use it more and more.
The one thing I still can't quite comprehend is why I should abstract my data layer. Basically if I need to print a list of items stored in my DB to the browser I do something along the lines of:
$sql = 'SELECT * FROM table WHERE type = "type1"';'
$result = mysql_query($sql);
while($row = mysql_fetch_assoc($result))
{
echo '<li>'.$row['name'].'</li>';
}
I'm reading all these How-Tos or articles preaching about the greatness of PDO but I don't understand why. I don't seem to be saving any LoCs and I don't see how it would be more reusable because all the functions that I call above just seem to be encapsulated in a class but do the exact same thing. The only advantage I'm seeing to PDO are prepared statements.
I'm not saying data abstraction is a bad thing, I'm asking these questions because I'm trying to design my current classes correctly and they need to connect to a DB so I figured I'd do this the right way. Maybe I'm just reading bad articles on the subject :)
I would really appreciate any advice, links, or concrete real-life examples on the subject!
Think of a abstracting the data layer as a way to save time in the future.
Using your example. Let's say you changed the names of the tables. You would have to go to each file where you have a SQL using that table and edit it. In the best case, it was a matter of search and replace of N files. You could have saved a lot of time and minimized the error if you only had to edit one file, the file that had all your sql methods.
The same applies to column names.
And this is only considering the case where you rename stuff. It is also quite possible to change database systems completely. Your SQL might not be compatible between Sqlite and MySQL, for example. You would have to go and edit, once again, a lot of files.
Abstraction allows you to decouple one part from the other. In this case, you can make changes to the database part without affecting the view part.
For very small projects this might be more trouble than it is worth. And even then, you should still do it, at least to get used to it.
I'm NOT a php person but this is a more general question so here goes.
You're probably building something small, sometimes though even something small/medium should have an abstracted data layer so it can grow better.
The point is to cope with CHANGE
Think about this, you have a small social networking website. Think about the data you'll store, profile details, pictures, friends, messages. For each of these you'll have pages like pictures.php?&uid=xxx.
You'll then have a little piece of SQL slapped in there with the mysql code. Now think of how easy/difficult it would be to change this? You would change 5-10 pages? When you'll do this, you'll probably get it wrong a few times before you test it thoroughly.
Now, think of Facebook. Think of the amount of pages there will be, do you think it'll be easier to change a line of SQL in each page!?
When you abstract the data access correctly:
Its in one place, its easier to change.
Therefore its easier to test.
Its easier to replace. (Think about what you'd have to do if you had to switch to another Database)
Hope this Helps
One of the other advantage of abstracting the data layer is to be less dependent on the underlying database.
With your method, the day you want to use something else than mysql or your column naming change or the php API concerning mysql change, you will have to rewrite a lot of code.
If all the database access part was neatly abstracted, the needed changes will be minimal and restricted to a few files instead of the whole project.
It is also a lot easier to reuse code concerning sql injection or others utility function if the code is centralized in one place.
Finally, it's easier to do unit testing if everything goes trough some classes than on every pages from your project.
For example, in a recent project of mine (sorry, no code sharing is possible), mysql related functions are only called in one class. Everything from query generation to object instantiation is done here. So it's very for me to change to another database or reuse this class somewhere else.
In my opinion, the data access is one of the most important aspects to separate / abstract out from the rest of your code.
Separating out various 'layers' has several advantages.
1) It neatly organises your code base. If you have to make a change, you'll know immediately where the change needs to be made and where to find the code. This might not be so much of a big deal if you're working on a project on your own but with a larger team the benefits can quickly become obvious. This point is actually pretty trivial but I added it anyway. The real reason is number 2..
2) You should try to separate things that might need to change independently of each other. In your specific example, it is conceivable that you would want to change the DB / data access logic without impacting the user interface. Or, you might want to change the user interface without impacting on the data access. Im sure you can see how this is made impossible if the code is mixed in with each other.
When your data access layer, has a tightly defined interface, you can change its inner workings however you want, and as long as it still adheres to the interface you can be pretty certain it wont have broken anything further up. Obviously this would still need verifying with testing.
3) Reuse. Writing data access code can get pretty repetitive. It's even more repetitive when you have to rewrite the data access code for each page you write. Whenever you notice something repetitive in code, alarm bells should be ringing. Repetitiveness, is prone to errors and causes a maintenance problem.
I'm sure you see the same queries popping up in various different pages? This can be resolved by putting those queries lower down in your data layer. Doing so helps to ease maintenance; whenever a table or column name changes, you only need to correct the one place in your data layer that references it instead of trawling through your entire user interface and potentially missing something.
4) Testing. If you want to use automated tool to carry out unit testing you will need everything nicely separated. How will you test your code to select all Customer records when this code is scattered all throughout your interface? It is much easier when you have a specific SelectAllCustomers function on a data access object. You can test this once here and be sure that it will work for every page that uses it.
There are more reasons that I'll let other people add. The main thing to take away is that separating out layers allows one layer to change without letting the change ripple through to other layers. As the database and user interface are areas of an application / website that change the most frequently it is a very good idea to keep them separate and nicely isolated from everything else and each other.
In my point of view to print just a list of items in a database table, your snippet is the more appropriate: fast, simple and clear.
I think a bit more abstraction could be helpful in other cases to avoid code repetitions with all the related advantages.
Consider a simple CMS with authors, articles, tags and a cross reference table for articles and tags.
In your homepage your simple query will become a more complex one. You will join articles and users, then you will fetch related tag for each article joining the tags table with the cross reference one and filtering by article_id.
You will repeat this query with some small changes in the author profile and in the tag search results.
Using a abstraction tool like this, you can define your relations once and use a more concise syntax like:
// Home page
$articles = $db->getTable('Article')->join('Author a')
->addSelect('a.name AS author_name');
$first_article_tags = $articles[0]->getRelated('Tag');
// Author profile
$articles = $db->getTable('Article')->join('Author a')
->addSelect('a.name AS author_name')->where('a.id = ?', $_GET['id']);
// Tag search results
$articles = $db->getTable('Article')->join('Author a')
->addSelect('a.name AS author_name')
->join('Tag')->where('Tag.slug = ?', $_GET['slug']);
You can reduce the remaining code repetition encapsulating it in Models and refactoring the code above:
// Home page
$articles = Author::getArticles();
$first_article_tags = $articles[0]->getRelated('Tag');
// Author profile
$articles = Author::getArticles()->where('a.id = ?', $_GET['id']);
// Tag search results
$articles = Author::getArticles()
->join('Tag')->where('Tag.slug = ?', $_GET['slug']);
There are other good reasons to abstract more or less, with its pros and cons. But in my opinion for a big part the web projects the main is this one :P
I'm creating a movies website, IMDB.com like.. I'm really new to PHP and programming at all but I have some books and StackOverflow of course :)
I have already done lot of the work, but now I have more than 600 lines of code (PHO only) per page and more than 20 database tables only for storing and geting the movie data (many-to-many relationships)
Most of the code are MySQLi queries (prepared statements) to insert/get data and loops. Nothing fancy, just basic PHP/MySQL.
I have some questions:
It is recommended to use MySQLi prepared statements for every SQL query or is better just to use old MySQL for internal PHP/database queries and leave prepared statements for user input only?
To get all the movie data and show it I need to get the data from more than 16 different tables. I use one MySQL query per table (somethimes more) and lots of loops. This is the correct way a website need to work?? I mean.. this is normal???
How can I simplify the code to simplify the reading? Can I separete all those queries to external files?? Can I create functions to handle each table query??
Hope you can give me a hand and maybe you have some more suggestions for correctly building such a website.
Thanks!!
As mentioned in the anwsers above I would also point you to using a framework that impliments the MVC design pattern. Along with that most of the frameworks have an ORM built in but if they do not you can look at Symphony or EZPDO is another good ORM to tie into your Model in (M)VC.
CodeIgniter is a very fast and light weight MVC framework that would allow you to bootstrap pretty quickly but you may also want to look into ZF (ZendFramework). ZF has a great framework feature set and is pretty flexible overall.
Beyond that be sure to seperate your reads and your writes in either or Model or your calls to the ORM of choice. This will allow you to slave data to multiple MySQL Boxes for bigger performance but allows you to use one database engine to start out.
Add in the ability to use Memcached so that you can cache your data/objects vs hitting the database.
When using cache put some thought into how you would expire cache for a database update. In other words if your selecting data from a database to display in a view and that data is has not changed you should not need to hit the database every time but rather pull it from memory. Once the data actually does change you'd want to invalidate that cache so it's not stale and then re-cache the new data.
memcached: http://www.danga.com/memcached/'
-facebook also has a version
CodeIgniter - http://codeigniter.com/
EZPDO - http://www.ezpdo.net/blog/?p=2
ZendFramework -http://framework.zend.com/
Consider looking at or using a web framework for your website.
symfony
CakePHP
CodeIgniter
Are some of the more mainstream ones. Learn from them if anything.
Prepared statements are just fine for your own internal queries to. You'll have a strutured approach for all queries.
Well it depends on what you're showing. But I would say that you normally could use joins to get the data you need from more tables. You'll have a lot less quering to get the correct data, and it sounds like all your data is connected somehow to the one movie you're showing.
As Peter D comments, I would recommend using a web framework to learn how to seperate out the database handling from the view. Are you using an object oriented approach now? Look at the MVC pattern that some of these frameworks implement, that'll get you going.
Like Peter D mention before add this one to the list of framework to use. Zend Framework http://framework.zend.com
Open source and free.
It is recommended to use MySQLi ...
Definitely MySQLi, but it's a big question by itself, if you start coding you'll need to grasp the basic of T-SQL to understand the difference.
To get all the movie data ...
It depends on alot of things. Size of database. Wanted results i.e. the information that need to be displayed, response time of the queries vs displaying in user view. Do you know about JOIN, UNION?
How can I simplify the code to ...
Yes to all theses questions. www.w3schools.com/php/ if it can be of any help and learn the MVC pattern. Useful to alot of programming language these days. Maybe a framework would help you here
To this questions:
"To get all the movie data and show it
I need to get the data from more than
16 different tables. I use one MySQL
query per table (somethimes more) and
lots of loops. This is the correct way
a website need to work?? I mean.. this
is normal???"
No. If I understand you correctly, you should be using some type of JOIN depending on the data you're retrieving from the database. Getting results for huge amounts of data, and then picking out only the pieces you want in PHP is much slower than letting the database do the work of sorting/retrieving only the records/info you want to show.
I highly recommend a somewhat dated but very easy to grasp book that covers PHP and MySQL/Databases in general: http://www.dmcinsights.com/phpmysql2/ - It covers a lot of "in practice" techniques along with the code, so it'd be great to learn from.
There is a apparently a third edition with updated info, but I have not looked at it.