Please, correct me if I'm wrong:
If we use a Dao/Vo pattern or a TDG pattern we will have a nice code organization by having for each (or at least for a lot of) tables a related class.
The problem with this approach is that or data IS NOT closed inside a given table. We have some domain specific data, like findDogBreed(); or findBookBestSellerAuthor(); and the above patterns don't seem to deal with this nicely.
Once solution is to use Mappers. Mappers will contain a set of methods and properties related to one table BUT they will not be closed to that table only nor will they be related to a specific SQL Schema.
The problem is, if we start to abstract all those things, we will NOT have access to SQL syntax. What if we need our database administrator to work on it ? And on more complex queries, using mappers could lead to a really messy abstraction "thing".
Is this correct ? If so, I'm wondering what paths do we have in order to find a middle term here.
You don't have to lose the option to write SQL manually when you abstract the functionality, even on multiple levels abstraction.
E.g. look at Doctrine, which is Hibernate-inspired ORM for PHP. It allows you to write queries in DQL (Doctrine Query Language) that translates to SQL and automatically maps your entities, but you can also write native SQL (most often for performance optimization), but you need to define the result mapping by yourself.
Related
I cant seem to find an acceptable answer to this.
There are two big things I keep seeing:
1) Don't execute queries in the controller. That is the responsibility of business or data.
2) Only select the columns that you need in a query.
My problem is that these two things kind of butt heads since what is displayed in the UI is really what determines what columns need to be queried. This in turn leads to the obvious solution of running the query in the controller, which you aren't supposed to do. Any documentation I have found googling, etc. seems to conveniently ignore this topic and pretend it isn't an issue.
Doing it in the business layer
Now if I take it the other way and query everything in the business layer then I implicitly am making all data access closely reflect the ui layer. This is more a problem with naming of query functions and classes than anything I think.
Take for example an application that has several views for displaying different info about a customer. The natural thing to do would be to name these data transfer classes the same as the view that needs them. But, the business or service layer has no knowledge of the ui layer and therefore any one of these data transfer classes could really be reused for ANY view without breaking any architecture rules. So then, what do I name all of these variations of, say "Customer", where one selects first name and last name, another might select last name and email, or first name and city, and so on. You can only name so many classes "CustomerSummary".
Entity Framework and IQueryable is great. But, what about everything else?
I understand that in entity framework I can have a data layer pass back an IQuerable whose execution is deferred and then just tell that IQueryable what fields I want. That is great. It seems to solve the problem. For .NET. The problem is, I also do PHP development. And pretty much all of the ORMs for php are designed in a way that totally defeat the purpose of using an ORM at all. And even those dont have the same ability as EF / IQueryable. So I am back to the same problem without a solution again in PHP.
Wrapping it up
So, my overall question is how do I get only the fields I need without totally stomping on all the rules of an ntier architecture? And without creating a data layer that inevitably has to be designed to reflect the layout of the UI layer?
And pretty much all of the ORMs for php are designed in a way that totally defeat the purpose of using an ORM at all.
The Doctrine PHP ORM offers lazy loading down to the property / field level. You can have everything done through proxies that will only query the database as needed. In my experience letting the ORM load the whole object once is preferable 90%+ of the time. Otherwise if you're not careful you will end up with multiple queries to the database for the same records. The extra DB chatter isn't worthwhile unless your data model is messy and your rows are very long.
Keep in mind a good ORM will also offer a built-in caching layer. Populating a whole object once and caching it is easier and more extensible then having your code keep track of which fields you need to query in various places.
So my answer is don't go nuts trying to only query the fields you need when using an ORM. If you are writing your queries by hand just in the places you need them, then only query the fields you need. But since you are talking good architectural patterns I assume you're not doing this.
Of course there are exceptions, like querying large data sets for reporting or migrations. These will require unique optimizations.
Questions
1) Don't execute queries in the controller. That is the responsibility of business or data.
How you design your application is up to you. That being said, it's always best to consider best patterns and practices. The way I design my controllers is that I pass in the data layer(IRepository) through constructor and inject that at run time.
public MyController(IRepository repo)
To query my code I simply call
repository.Where(x=> x.Prop == "whatever")
Using IQueryable creates the leaky abstraction problem. Although, it may not be a big deal but you have to be careful and mindful of how you are using your objects especially if they contain relational data. Once you query your data layer you would construct your view model in your controller action with the appropriate data required for your view.
public ActionResult MyAction(){
var data = _repository.Single(x => x.Id == 1);
var vm = new MyActionViewModel {
Name = data.Name,
Age = data.Age
};
return View();
}
If I had any queries that where complex I would create a business layer to include that logic. This would include enforcing business rules etc. In my business layer I would pass in the repository and use that.
2) Only select the columns that you need in a query.
With ORMs you usually pass back the whole object. After that you can construct your view model to include only the data you need.
My suggestion to your php problem is maybe to set up a web api for your data. It would return json data that you can then parse in whatever language you need.
Hope this helps.
The way I do it is as follows:
Have a domain object (entity, business object .. things with the same name) for Entities\Customer, that has all fields and associated logic for all of the data, that a complete instance would have. But for persistence create two separate data mappers:
Mappers\Customer for handling all of the data
Mappers\CustomerSummary for only important parts
If you only need to get customers name and phone number, you use the "summary mapper", but, when you need to examine user's profile, you have the "all data mapper". And the same separation can be really useful, when updating data too. Especially, if your "full customer" get populated from multiple tables.
// code from a method of some service layer class
$customer = new \Model\Entities\Customer;
$customer->setId($someID);
$mapper = new \Model\Mappers\CustomerSummary($this->db);
if ($needEverything) {
$mapper = new \Model\Mappers\Customer($this->db);
}
$mapper->fetch($customer);
As for, what goes where, you probably might want to read this old post.
Can someone provide me a couple clear (fact supported) reasons to use/learn DQL vs. SQL when needing a custom query while working with Doctrine Classes?
I find that if I cannot use an ORM's built-in relational functionality to achieve something I usually write a custom method in the extended Doctrine or DoctrineTable class. In this method write the needed it in straight SQL (using PDO with proper prepared statements/injection protection, etc...). DQL seems like additional language to learn/debug/maintain that doesn't appear provide enough compelling reasons to use under most common situations. DQL does not seem to be much less complex than SQL for that to warrant use--in fact I doubt you could effectively use DQL without already having solid SQL understanding. Most core SQL syntax ports fairly well across the most common DB's you'll use with PHP.
What am I missing/overlooking? I'm sure there is a reason, but I'd like to hear from people who have intentionally used it significantly and what the gain was over trying to work with plain-ole SQL.
I'm not looking for an argument supporting ORMs, just DQL when needing to do something outside the core 'get-by-relationship' type needs, in a traditional LAMP setup (using mysql, postgres, etc...)
To be honest, I learned SQL using Doctrine1.2 :) I wasn't even aware of foreign-keys, cascade operations, complex functions like group_concat and many, many other things. Indexed search is also very nice and handy thing that simply works out-of-the-box.
DQL is much simpler to write and understand the code. For example, this query:
$query = ..... // some query for Categories
->leftJoin("c.Products p")
It will do left join between Categories and Products and you don't have to write ON p.category_id=c.id.
And if in future you change relation from one-2-many to let's say many-2-many, this same query will work without any changes at all. Doctrine will take care for that. If you would do that using SQL, than all the queries would have to be changed to include that intermediary many-2-many table.
I find DQL more readable and handy. If you configure it correctly, it will be easier to join objects and queries will be easier to write.
Your code will be easy to migrate to any RDBMS.
And most important, DQL is object query language for your object model, not for your relational schema.
Using DQL helps you to deal with Objects.
in case inserting into databae , you will insert an Object
$test = new Test();
$test->attr = 'test';
$test->save();
in case of selecting from databae, you will select an array and then you can fill it in your Object
public function getTestParam($testParam)
{
$q=Doctrine_Query::create()
->select('t.test_id , t.attr')
->from('Test t ')
$p = $q->execute();
return $p;
}
you can check the Doctrine Documentation for more details
Zeljko's answer is pretty spot-on.
Most important reason to go with DQL instead of raw SQL (in my book): Doctrine separates entity from the way it is persisted in database, which means that entities should not have to change as underlying storage changes. That, in turn, means that if you ever wish to make changes on the underlying storage (i.e. renaming columns, altering relationships), you don't have to touch your DQL, because in DQL you use entity properties instead (which only happen to be translated behind the scenes to correct SQL, depending on your current mappings).
I am working on the zend project, I am referring on other zend project to create the new Zend Project.But I don't like to blindly follow that project without understanding. In the Zend Directory structure, In Model class there are mainly two type of classes I see, like as in
- models
- DbTables
- Blog.php //Extends Zend_Db_Table_Abstract
- Blog.php // Contains methods like validate() and save()
- BlogMapper.php // Also Contains methods like validate(Blog b) & save(Blog b)
Why this specific structure is followed?
Is this is to separate Object class and Database model class?
Please explain.
DataMapper is a design pattern from Patterns of Enterprise Application Architecture.
The Data Mapper is a layer of software that separates the in-memory objects from the database. Its responsibility is to transfer data between the two and also to isolate them from each other. With Data Mapper the in-memory objects needn't know even that there's a database present; they need no SQL interface code, and certainly no knowledge of the database schema.
How you store data in a relational database is usually different from how you would structure objects in memory. For instance, an object will have an array with other objects, while in a database, your table will have a foreign key to another table instead. Because of the object-relational impedance mismatch, you use a mediating layer between the domain object and the database. This way, you can evolve both without affecting the other.
Separating the Mapping responsibility in its own layer is also more closely following the Single Responsibility Principle. Your objects dont need to know about the DB logic and vice versa. This gives you greater flexibility when writing your code.
When you dont want to use a Domain Model, you usually dont need DataMapper. If your database tables are simple, you might be better off with a TableModule and TableDataGateway or even just ActiveRecord.
For various other patterns see my answer to
ORM/DAO/DataMapper/ActiveRecord/TableGateway differences? and
http://martinfowler.com/eaaCatalog/index.html
The idea of a Model is to wrap up the logical collection of data inside of your code.
The idea of a DataMapper is to relate this application-level collection of data with how you are storing it.
For a lot of ActiveRecord implementations, the framework does not provide this separation of intent and this can lead to problems. For example, a BlogPost model can wrap up the basic information of a blog post like
title
author
body
date_posted
But maybe you also want to have it contain something like:
number_of_reads
number_of_likes
Now you could store all of this data in a single MySQL table to begin with, but as your blog grows and you become super famous, you find out that your statistics data is taking an awful lot of hits and you want to move it off to a separate database server.
How would you go about migrating those fields of the BlogPost objects off to a different data store without changing your application code?
With the DataMapper, you can modify the way the object is saved to the database(s) and the way it is loaded from the database(s). This lets you tweak the storage mechanism without having to change the actual collection of information that your application relies on.
I am working on an architecture redesign at my work and we've basically settled on a loosely-basic MVC custom solution. The intentions are to have the standard CRUD operations plus additional list operations defined in each of the models in our system.
Unfortunately about 30% of the code in our system uses complex joins and otherwise advanced querying that doesn't fit this model. Which is to say it could fit the model, but the list function would be huge and certainly error prone which is something we are trying to solve with the rewrite.
Given that, where would you place complex and very specific queries in such a system? We've been toying with a few options.
Add multiple versions of list/get in addition to the basic ones
Add in custom models for these queries that reside as siblings to the model directory
Don't use models in this situation and add the work directly in the action
We have outsourced help as well so we are attempting to keep it as simple as we can in terms of implementation and maintainability. ORM solutions or other heavyweights are out of the question.
Where would you want to see such things placed as a developer?
I apparently lack the privileges necessary to comment, so I'm posting this as answer...
Could you provide an example or two of the kinds of queries you have that don't fit into a model? Generally speaking: a good ORM will get you a long way, but some queries really are just too hairy to map easily, and if your team already has strong SQL skills the ORM can also seem like it's getting in the way.
First , all you're queries should stay in you're model .
Second , most of mvc frameworks provide more than just simple crud for you're database operations like a query functionality that where you can pass the query string , in this case you can build you're queryes manualy or with a query builder like for example Zend_Db_Table_Select and that handles multiple joins prety well . Or again if we look some place else than Zend let's say Codeigniter , it still provides a query builder for the model where you can add you're joins or build any other kind of complex queries .
That being sayd , it looks like you're base model class ( the one you extend each of you're models ) needs a query builder functionality , then you should be all good as you would be able to build any query you like inside any model you like .
I have similar issues in am MVC framework I've been building from scratch.
I don't particularly like the overhead of SELECT * on complex queries so I didn't build any of that functionality in whatsoever.
It's slower to code, but I code every query by hand in the relevant class (my model calls a Class 99% of the time).
For really complex queries shared amongst various routines, I have functions that return the generic joins and then concat the additional parameters for that particular query.
Example provided as requested:
private function returnFindClientRequests(){
$query = "SELECT
SR.sign_project_name, SR.module_signregister_id_pk
,SRI.module_signregister_sign_id_pk,SRI.sign_location_address
,SRR.status, SRR.module_signregister_item_client_request_id_pk, SRR.client_comment, SRR.requested_by_user, SRR.date_created
,SRR.admin_comment, SRR.date_actioned
,CL.client_name, CL.module_client_id_pk
FROM
`module_signregister` SR, `module_signregister_item` SRI, `module_signregister_item_client_request` SRR, `module_client` CL
WHERE
SR.module_signregister_id_pk = SRR.module_signregister_id_pk
AND SRR.module_signregister_sign_id_pk = SRI.module_signregister_sign_id_pk
AND SRR.requested_by_group = CL.module_client_id_pk
AND " . Database::groupQuery('CL');
return $query;
}
This query is shared amongst some other functions but also uses a call to Database::groupQuery() that us used to return session specific variables to many of the queries.
Models are workers - if you have 100 reports you're potentially going to need 100 models. Joins have nothing to do with MVC - how your data is addressed is another pattern altogether. If you're not using ORM and you're not using active records then all that's left is sending the SQL straight to the server via a model. Probably via a dedicated database class but the model will handle the query and its results.
I am not suggesting that all models are tables.
What I am asking is whether every single table must also have its own class defined specifically for it when using Zend? Is there any way of getting away from this awkward boiler-plate coding. We're just starting to look into Zend (hoping to leave procedural PHP land!) and my colleague thinks this could end up being pretty time-consuming.
Is this the reason for people using ORM solutions? Is there any other way around this?
Thanks for your replies.
The Zend Table classes follow the Table Data Gateway pattern, which by definition
... holds all the SQL for accessing a single table or view: selects, inserts, updates, and deletes. Other code calls its methods for all interaction with the database.
In the book, Fowler is not that rigid about that, saying that
for very simple cases, you can have a single TDG that handles all methods for all tables. You can even have one for views or even for interesting queries that aren't kept in the database as views.
However, except for being able to use Views, Zend_Db_Table does not accomodate for this. You can create queries to multiple tables, but those would have to be made through the Zend_Db_Adapter directly or - when using joins - by switching off the integrity check. Otherwise, you have to use the API offered by Zend_Db_Table Relationships
So yes, one instance should correspond to one table or view. You do not need to create classes for that though if you dont plan on extending the classes. Zend_Db_Table_Definitions allow you to configure Zend_Db_Table instances on the fly.
Note that TDG is a DataSource Architectural Pattern and not an Object-Relational pattern. It's purpose is not to help with impedance-mismatch, but with separating database access code from business logic.