I am building a Laravel 5.3 app that pulls data from a number of potential sources. It's a fallback system with 3 sources:
Database
If not found, source 1
If not found, source 2
All 3 sources are quite simple and will be accessed in the same way, by using the following 2 methods:
function get($id)
function query($type, $string)
I'm aware there is various terminology around the different ways to implement this, but I'm unsure after reading the docs what the cleanest approach is. Should each data source be implemented as a Repository? A ServiceProvider wrapped in a container? I find the docs thorough but also lacking in overall/high level explanations, so any pointers are appreciated.
The Repository Pattern is the following:
Mediates between the domain and data mapping layers using a collection-like interface for accessing domain objects. Repository encapsulates the set of objects persisted in a data store and the operations performed over them, providing a more object-oriented view of the persistence layer. Repository also supports the objective of achieving a clean separation and one-way dependency between the domain and data mapping layers.
With that in mind, you can say that Eloquent itself is a much larger implementation of the Repository Pattern, but a Repository nonetheless. Since it's an ActiveRecord implementation, there isn't any real separation between the Repository and Storage mechanisms.
On to your question specifically, Laravel won't really cover the Repository Pattern itself in much the same way it doesn't cover Service classes or Singletons: It's not Laravel's responsibility to teach you these patterns, it's just giving you the means to organize these patterns more easily if you choose to implement them.
All that said, I would agree with you that each data source implement its own RepositoryInterface. From there, you can register your own ServiceProvider that in turn instantiates a custom Service Class whose purpose is to return the appropriate Repository.
If determining the appropriate Repository is light in logic, and is dependent only on the Controller responsible for the alternate data source, you can likely use Contextual Binding and skip the Service Class altogether.
Either way, there's a few ways to skin this cat, but you're on the right track.
Edit: As an aside, if you want to strictly go "by the book" on this, you would probably want to separate out different Storage classes that connect to each data store separately, which you can then query as appropriate. Then your Repository - which is likely housing the same type of data collection regardless of its storage origins - can be responsible for the returned results.
Otherwise, if you want to stick with Eloquent as much as possible, you can look into multiple data connections to house each of your data sets.
Related
there is a Repositories layer in my laravel project. I am a freshman about laravel. I can not fully understand the advantages of this layer. My colleagues tell me that it is purpose to lower the coupling of the codes. Obviously, it is true. But I also look forward to your comments.
Using repositories, we can create complicated query and isolated it behind a readable method name. The coupled used, when using eloquent, we must put the record name in eloquent query, but using repositories, we can hide the record name and put in the other place. So that is give some security reason too. Also we can use for grouped some entities for authoritative users. Repository allows abstraction from data provide engine.
This article can help you.
I need help with something I can’t get my head wrapped around regarding the Repository and Service/Use-case pattern (part of DDD design) I want to implement in my next (Laravel PHP) project.
All seems clear. Just one part of DDD that is confusing is the data structures from repositories. People seem to choose data structures the Repository should return (arrays or entities) but it all has disadvantages. One of which is performance looking at my experiences in the past. And one is which you don’t have interfaces for simple data structures (array or simple object attributes).
I’ll start with explaining the experience I have with a previous project. This project had flaws but some good strengths I learned from and like to see in my new project but with solving some design mistakes.
Previous experience
In the past I’ve build a website that was API Centric using the Kohana framework and Doctrine 2 ORM (data mapper pattern). The flow looked like this:
Website controller → API client (HMVC calls) → API controller → Custom Repository → Doctrine 2 ORM native Repository/Entity-manager
My custom Repository returned plain arrays using Doctrine2 DQL. Doctrine2 recommends array result data for read only operations. And yes, it made my site nice and light. The API controller just converted the array data to JSON. Simple as that.
In the past my company created projects relying fully on loaded Doctrine2 entities and it’s something we regretted due to performance.
My REST API supported queries like
/api/users?include_latest_adverts=2&include_location=true
on the users resource. The API controller passed include_location to the repository which directly included the location relation. The controller read latest_adverts=2 and called the adverts repository to get the latest 2 adverts of each user. Arrays were returned.
For example first user array:
[
name
avatar
adverts [
advert 1 [
name
price
]
advert 2 [
….
]
]
]
This proved to be very successful. My whole website was using the API. It would be very easy to to add a new client because the API was perfectly in production already using oauth. The whole website runs on it.
But this design had flaws too. My controller still contained A LOT of logic for validation, mailing, params or filters like has_adverts=true to get users with adverts only. It would mean that if I created a new port, like a total new CLI interface, I would have to duplicate alot of these controllers due to all the validation etc. But no duplication if I would create a new client. So at least one problem was solved :-)
My admin panels were completely coupled to the doctrine2 repository/entity-manager to speed up development (sort of). Why? Because my API had fat controllers with special functionality for the website only (special validation, mailing for registering etc). I would have to redo work or refactor a lot. So decided to use the Entities directly to still have some sort clear way of writing code instead of rewriting all my API controllers and move them to Services (for site & admin) for instance. Time was an issue in fixing my design mistakes.
For my next project I want all code to go through my own custom repositories and services. One flow for good separation.
New project (using DDD ideas) and dilemma with data structures
While I like the idea of being API centric, I don’t want my next project to be API centric in core because I think the same functionality should be available without the HTTP protocol in between. I want to design the core using DDD ideas.
But I liked the idea using a layer that just talked as a API and returns simple arrays. The perfect base for any new port, including my own frontend. My idea is to consider my Service classes as the API interface (return the array data), do the validation etc. I could have Services specially for the website (registering) and plain services used by the Admin or background processes. In some admin cases a Service would not be required anyway for simple CRUD editing, I could just use Repositories directly. Controllers would be very thin. With this creating a real REST API would just be a matter to create new controllers using the same Services my frontend controller classes do.
For internal logic like business rules it would be useful to have Entities (clear interfaces) instead of arrays from repositories. This way I could benefit from defining some methods that did some logic based on attributes. BUT If I would be using Doctrine2 and my repositories would always return Entities my application would suffer a big performance hit!!
One data structure ensures performance but no clear interfaces, the other ensures clear interfaces but bad performance when using a Data Pattern pattern like Doctrine 2 (now or in the future). Also I could end up with two data types which would be confusing.
I was thinking something similar to this flow:
Controller (thin) → UserService (incl. validation) → UserRepository (just storage) → Eloquent ORM
Why Eloquent instead of Doctrine2? Because I want to stick a bit to what’s common within the Laravel framework and community. So I could benefit from third party modules, for example to generate admin interfaces or similar based on models (bypassing my DDD rules). Other than using third party modules, I would design my core stuff so switching should always be easy and not affect data structure choices or performance.
Eloquent is an activerecord pattern. So I would be tempted to convert this data to POPO’s like Doctrine2 entities are. But nope... as said above, with doctrine2 real models would make the system very fat. So I fall back to simple arrays again. Knowing this would work for both and any other implementation in the future.
But it feels bad always rely on arrays. Especially when creating internal business rules. A developer would have to guess values on arrays, have no autocompletion in his IDE, could not have special methods like in Entity classes. But making two ways of dealing with data feels bad too. Or I am just too perfectionist ;) I want ONE clear data structure for all!
Building interfaces and POPO’s would mean a lot of duplicate work. I would need to convert an Eloquent model (just a table mapper, not entity) to an entity object implementing this interface. All is extra work. And eventually my last layer would be just like a API, thus converting it to arrays again. Which is extra work too. Arrays seem the deal again.
It seemed so easy reading up into DDD and Hexagonal. It seems so logic! But in reality I struggle with this one simple issue trying to stick to OOP principles. I want to use arrays because it’s the only way to be 100% sure I am not depended on any model choice and querying choice from my ORM regarding performance etc and don't have duplicate work in converting to arrays for views or an API. But there's no clear contract on how a user array could look. I want to speed up my project using these patterns, not slow them down :-) So not an option to have many converters.
Now I read a lot of topics. One makes POPO’s & interfaces that conform proper entities like Doctrine2 could return, but with all the extra work for Eloquent. Switching to Doctrine2 should be fairly easy, but would impact performance so bad or one would need to convert Doctrine2 array data to these own entity interfaces. Others choose to return simple arrays.
One convinces people to use Doctrine2 instead of Eloquent, but they leave out the fact that Doctrine2 is heavy and you really need to use array results for read only operations.
We design repositories to be changeable right? Not because it’s “nice” by design only. So how could we rely on full Entities if it has such big impact on performance or duplicate work? Even when using Doctrine2 only (coupled) this same issue would arise due to its performance!
All ORM implementations would be able to return arrays, thus no duplicate work there. Good performance. But we miss clear contracts. And we don’t have interfaces for arrays or class attributes (as a workaround)... Ugh ;)
Do I just miss a missing block in our programming languages? Interfaces on simple data structures??
Is it wise to make all arrays and have advanced business logic talk to these arrays? Thus no classes with clear interfaces. Any precalculated data (normally would be returned by an Entity method) would be within an array key defined the Service class. if not wise, what’s the alternative considering all of the above?
I would really appreciate if someone with great experience in this “domain” considering performance, different ORM implementations, etc could tell me how he/she has dealt with this?
Thanks in advance!
I think what you are dealing with is something similiar I'm struggling with. The solution I'm thinking works best is:
Entities/Repositories
Use and pass around Entities always when performing Write operations (Creating things, Updating things, Deleting things, and complex combinations thereof).
Sometimes you may use Entities when doing Read operations (when you anticipate the Read might need to be used for a Write soon after...ie. ->findById is soon followed by ->save).
Anytime you are working with an Entity (whether it be Write or Read), the Repositories need to be the place to go. You should be able to tell new developers that they can only persist to the database through Entities and the Repository.
The Entities will have properties that represent some Domain Object (many times they represent a database table with the fields of a table, but not always). They will also contain the domain logic/rules with them (ie. validation, calculations) so they are not anemic. You may additionally have some domain services if your Entities need help interacting with other Entities (need to trigger other events), or you just need an additional place to handle some extra domain logic (perform Repository calls to check for some unique conditions).
Your Repositories will solely be for working with Entities. The Repositories could accept Entities and do some persistence work with them. Or they could accept just some parameters, and do some reading/fetching into full Entities.
Some Repositories will know how to save some Domain Objects that are more complex than others. Perhaps an Entity that has a property which contains a list of other Entities that need to be saved along side the main entity (you can dive deeper into learning about Aggregate roots if you want).
The interfaces to Repositories rest in your Domain layer, but not the actual implementations of those Repositories. That way you can have an Eloquent version or whatever.
Other Queries (Table Data Gateway)
These queries won't work with Entities. They'll just be accepting parameters and returning things like Arrays or POPO's (Plain Old PHP Objects).
Many times you will need to perform Reads that do not return nicely into a single Entity. These Reads are typically more for reporting (not for CRUD-like operations, like Reading a user into an edit form that is eventually submitted and saved). For example, you might have a report that is 200 rows of JOINed data. If you used the Repositiory and tried to return large deep objects (with all the relationships populated, or even lazy-loaded) then you are going to have performance issues. Instead, use the Table Data Gatway pattern. You are just displaying data and not really needing OOP power here. The outputted data could however contain ID's, which through the UI could be used to initiate calls to Repository persistence methods.
As you are developing your app, when you come across the need for a new Read/Report query, create a new method in some class somewhere in your Table Data Gatway folder. You may find you have already created a similar query, so see how you can consolidate the other query. Use some parameters if necessary to make the gateway method's queries more flexible in particular ways (ie. columns to select, sort order, pagination, etc.). Don't make your queries too flexible though, this is where query builders/ORMs go wrong! You need to constrain your queries to a certain extent to where if you need to replace them (perhaps a different database engine) then you can easily perceive what the allowed variations are and aren't. It's up to you to find the right balance between flexibility (so you have more DRY code) and constraints (so you can optimize/replace queries later).
You can create services in your Domain to handle receiving parameters, then passing them to the Table Data Gateway, and then receiving back arrays to do some more mutating on. This will keep your Domain logic in the domain (and out of the infrastructure/persistence layer of the Repository & Table Data Gateway).
Again, just like the Repository, use interfaces in your domain services so that the implementation details stay out of your Domain layer, and resides in the actual Table Data Gateway folder.
What are the advantages of Repositories in Laravel? It seems to be abstracting the Model layer from the business logic of the application. Although it really just seems to make the whole request life cycle just that much more complicated for little gain.
Can someone shed light on the advantage of Laravel repositories?
Edit
After now using repositories for some time I would add the following:
Repositories enforce single responsibility
Repositories should only return one collection of entities
Although separate from dependancy injection the concepts are brothers
Storage abstraction for the actual storage implementation (e.g. MySQL)
Easier testing
Repositories, like in the provided tutorial, aren't necessary a Laravel concept. Rather, they're a form of IoC injection that is possible with Laravel. Any object that might similarly be injected doesn't mean it's a repository. See the video for a good example from Taylor Otwell, which happens to use a "repository" as well: http://vimeo.com/53029232.
In this example, the repository abstracts where the data came from that is passed to the controller. As long as the data passed implements the specified interface, the controller can "blissfully" make use of the interface's defined methods without worry about where the data initially came from. This allows switching the initial source of the data without breaking your controller. You could pull the data from a file, a database, an outside API, a mock object, or just some arbitrary array. Basically, the controller doesn't need to gather the data represented by the repository. It can just receive and use.
In addition to the other answers here, it's worth pointing out that repositories used when used in Laravel can add an extra level of expressiveness. Take for example the following:
$users = User::whereHas("role", function($q) {
$q->where('name', 'moderator');
}, '<', 1)->get();
The code is difficult to read and awkward to look at. It can be encapsulated in a repository method, and demonstrate much clearer code intent:
$users = $userRepository->getUsersWhoAreNotModerators();
This is also achievable using eloquent 'query scopes' but I think using a repository is superior, as is adheres much better to the single responsibility principal, and is doable regardless of whether you are using Eloquent.
Repositories help to keep your controller clean and make reusable code. Functions in repositories can be accessed in one or more controllers, repositories, etc.
Also, all your backend related logic (like fetching data from database or external calls) can be added in repositories to make the logical separation.
One of the main uses of repositories is to create different binding (using interfaces to define your functions and with the help of the app, bind different implementations of the function as per need). For example, two separate repositories (implementing the parent repository/interface) handling database and files for backend data.
The main reason you use repository pattern is to be able to easily change your data source. For example, in my experience the most common change is adding a caching layer, because the repository implements an interface all you need to do is build the new object implementing the same interface with the new methods handling cache and change the binding.
In my experience the repository in Laravel benefits are these:
The most important thing by using repositories is that you can change your ORM any time and for any reason that you prefer. for example, you want to migrate from MySQL eloquent to some other SQLite ORM.
Helps you to keep your controllers clean and readable.
Helps you to reuse your methods in a repository for any other controllers.
You can add BaseRepository to your repositories list, which involves all base methods like all(), get(), findOrFail(), firstOrFail(), paginate(), create(), ... and use them in other repositories.
Using Interfaces for repositories and binding them that also can be used as services in Laravel.
Looking through several tutorials and books regarding data access in Zend Framework, it seems as if most people do data access within their models (Active Record Pattern) or even controllers. I strongly disagree with that. Therefore I want to have a Data Access Layer (DAL) so that my domain layer remains portable by not having any "ZF stuff" in it. I have searched around but have not really found exactly what I wanted. Heads up: I am new to ZF.
DAL structure
So, the first problem is where to place the Data Access Layer. While it could certainly be placed within the library folder and adding a namespace to the autoloader, that does not seem logical as it is specific to my application (so the applications folder is suitable). I am using a modular structure. I am thinking of using the below structure:
/application/modules/default/dal/
However, I am not sure how include this folder so that I can access the classes within the controllers (without using includes/requires). If anyone knows how to accomplish this, that would be super! Any other ideas are of course also welcome.
The idea is to have my controllers interact with the Data Access Objects (DAO). Then the DAOs use models that can then be returned to the controllers. By doing this, I can leave my models intact.
Implementation
In other languages, I have previously implemented DAOs per model, e.g. DAL_User. This resulted in an awful lot of DAO classes. Is there a smarter way to do this (using a single class does not seem easy with foreign keys)?
I would also appreciate suggestions on how to implement my DAO classes in ZF. I have not spent an awful lot of time reading about all of the components available for database interaction, so any ideas are very welcome. I suspect that there is something smarter than standard PDO available (which probably uses PDO internally, though). Name drops would be sufficient.
Sorry for the many questions. I just need a push in the right direction.
Well, the first thing you have to take into account when dealing with the Data Access Layer, is that this layer also have sub-layers, it's unusual to find folders called "dal" in modern frameworks (I'm taking as basis both Zend Framework and Symfony).
Second, about ActiveRecord, you must be aware that by default Zend Frameworks doesn't implement it. Most of the tutorials take the easiest path to teach new concepts. With simple examples, the amount of business logic is minimal, so instead of adding another layer of complexity (mapping between database and model's objects) they compose the domain layer (model) with two basic patterns: Table Data Gateway and Row Data Gateway. Which is enough information for a beginner to start.
After analyzing it, you will see some similarity between ActiveRecord
and Row Data Gateway patterns. The main difference is that
ActiveRecord objects (persistable entities) carries business logic and
Row Data Gateway only represents a row in the database. If you add
business logic on a object representing a database row, then it will
become an ActiveRecord object.
Additionally, following the Zend Framework Quick Start, on the domain model section, you will realize that there's a third component, which uses the Data Mapper Pattern.
So, if the main purpose of your DAL is to map data between business objects (model) and your storage, the responsibility of this task is delegated to the Data Mappers as follows:
class Application_Model_GuestbookMapper
{
public function save(Application_Model_Guestbook $guestbook);
public function find($id);
public function fetchAll();
}
Those methods will interact with the Database Abstraction Layer and populate the domain objects with the data. Something along this lines:
public function find($id, Application_Model_Guestbook $guestbook)
{
$result = $this->getDbTable()->find($id);
if (0 == count($result)) {
return;
}
$row = $result->current();
$guestbook->setId($row->id)
->setEmail($row->email)
->setComment($row->comment)
->setCreated($row->created);
}
As you can see, the Data Mappers interacts with a Zend_Db_Table instance, which uses the Table Data Gateway Pattern. On the other hand, the $this->getDbTable->find() returns instances of the Zend_Db_Table_Row, which implements the Row Data Gateway Pattern (it's an object representing a database row).
Tip: The domain object itself, the guestbook
entity, was not created by the find() method on the DataMapper,
instead, the idea is that object creation is a task of factories
and you must inject the dependency in order to achieve the so called
Dependency Inversion Principle (DIP) (part of the SOLID principles). But that's
another subject, out of the scope of the question. I suggest you
to access the following link http://youtu.be/RlfLCWKxHJ0
The mapping stuff begins here:
$guestbook->setId($row->id)
->setEmail($row->email)
->setComment($row->comment)
->setCreated($row->created);
So far, I think I have answered your main question, your structure will be as following:
application/models/DbTable/Guestbook.php
application/models/Guestbook.php
application/models/GuestbookMapper.php
So, as in the ZF Quick Start:
class GuestbookController extends Zend_Controller_Action
{
public function indexAction()
{
$guestbook = new Application_Model_GuestbookMapper();
$this->view->entries = $guestbook->fetchAll();
}
}
Maybe you want to have a separated folder for the data mappers. Just change:
application/models/GuestbookMapper.php
to
application/models/DataMapper/GuestbookMapper.php
The class name will be
class Application_Model_DataMapper_GuestbookMapper
I've seen that you want to separate your domain model objects into modules. It's possible too, all you need is to follow the ZF's directory and namespace guidelines for modules.
Final tip: I've spent a lot of time coding my own data mappers for
finally realize that it's nightmare to maintain the object mapping when
your application grows with a lot of correlated entities. (i.e Account
objects that contain references to users objects, users that contain
roles, and so on) It's not so easy to write the mapping stuff at this
point. So I strongly recommend you, if you really want a true
object-relational mapper, to first study how legacy frameworks perform
such tasks and perhaps use it.
So, take some spare time with Doctrine 2, which is the
one of the best so far (IMO) using the DataMapper pattern.
That's it. You still can use your /dal directory for storing the DataMappers, just register the namespace, so that the auto loader can find it.
In my opinion you should have a gateway abstraction (not just Database access) per model. A DAO is not enough. What if you need to get the data from the cloud at some point? This is quickly coming a reality. If you abstract your gateway logic into something generic and then implement it using a database you can have the best of both worlds.
The implementation of a specific gateway interface could use a generic data mapper if you so chose. I work for a small company and have always just created my implementation using PDO. This lets me be close enough to the database to deal with any interesting bits of SQL I might need but am able to support a very abstracted interface.
I have not used the Zend Framework at all. I do not know if they have data-mapper tools that could help you implement the gateway interfaces.
What good is a repository pattern when you have an ORM?
Example. Suppose i have the following (fictional) tables:
Table: users
pk_user_id
fk_userrole_id
username
Table: userroles
fk_userrole_id
role
Now with an orm i could simply put this in a model file:
$user = ORM::load('users', $id);
Now $user is already my object, which could easily be lazy loaded:
(would be even nicer if things are automatically singular/pluralized)
foreach ( $user->userroles()->role as $role )
{
echo $role;
}
Now with the Repository pattern i'd had to create a repository for the Users and one for the Roles. The repository also needs all kinds of functions to retrieve data for me and to store it. Plus it needs to work with Entity models. So i have to create all of those too.
To me that looks like alot of stuff do... When i could simply get the data like i described above with an ORM. And i could store it just as easy:
ORM::store($user);
In this case it would not only store the user object to the database, but also any changes i made to the 'Roles' object aswell. So no need for any extra work like you need with the repository pattern...
So my question basically is, why would i want to use a repository pattern with an ORM? I've seen tutorials where to use that pattern (like with Doctrine). But it really just doesn't make any sense to me... Anyone any explanation for its use in combination with an ORM..??
The ORM is an implementation detail of the Repository. The ORM just makes it easy to access the db tables in an OOP friendly way. That's it.
The repository abstract persistence access, whatever storage it is. That is its purpose. The fact that you're using a db or xml files or an ORM doesn't matter. The Repository allows the rest of the application to ignore persistence details. This way, you can easily test the app via mocking or stubbing and you can change storages if it's needed. Today you might use MySql, tomorrow you'll want to use NoSql or Cloud Storage. Do that with an ORM!
Repositories deal with Domain/Business objects (from the app point of view), an ORM handles db objects. A business objects IS NOT a db object, first has behaviour, the second is a glorified DTO, it only holds data.
Edit
You might say that both repository and ORM abstract access to data, however the devil is in the details. The repository abstract the access to all storage concerns, while the ORM abstract access to a specific RDBMS
In a nutshell, Repository and ORM's have DIFFERENT purposes and as I've said above, the ORM is always an implementation detail of the repo.
You can also check this post about more details about the repository pattern.
ORM and repository pattern...depends on setup.
If you use your ORM entities as the domain layer, then please use no repositories.
If you have a separate domain model and you need to map from that model to ORM entities and so perform a save, then repositories are what you need.
More details you find here (but must be logged to linked-in). Also to understand the difference, check out the definition of the repository pattern.
Most people use classes that they call repositories, but aren't repositories at all, just query classes - this is how/where you should place your queries if you decided to go with the #1 option (see answer above). In this case make sure not to expose DbContext or ISession from that query class, nor to expose CUD-methods from there - remember, Query class!
The #2 option is a tough one. If you do a real repository, all the inputs and outputs on the repository interface will contain clear domain classes (and no database related object). It's forbidden to expose ORM mapped classes or ORM architecture related objects from there. There will be a Save method also. These repositories might also contain queries, but unlike query classes, these repos will do more - they will take your domain aggregate (collection and tree of entities) and save them to DB by mapping those classes to ORM classes and perform a save on ORM. This style (#2) does not needs to use ORM, the repository pattern was primarly made for ADO.NET (any kind of data access).
Anyhow these 2 options are the 2 extremes we can do. A lot of people use repositories with ORM, but they just add an extra layer of code without real function, the only real function there is the query class like behaviour.
Also I'd be careful when someone talks about UnitOfWork, especially with ORM. Almost every sample on the internet is a fail in terms of architecture. If you need UoW, why not use TransactionScope (just make sure you got a wrapper which uses other than Serializable transaction by default). In 99,9% you won't need to manage 2 sets of independent changes in data (so 2 sets of OuW), so TransactionScope will be a fine choce in .NET - for PHP i'd look for some open-session-view implementations...