You've probably heard of the Fat Model/Thin Controller vs. Thin Model/Fat Controller distinction. I recently heard that you can have something in between where some of the logic from the model goes into a service layer. How common is this? and do you know of (or can think of) any real examples that illustrate it?
Martin Fowler describes the Service Layer pattern of his great book Patterns of Enterprise Application Architecture. If you care about questions like the one you asked, you should read this book.
One use that comes to my mind is managing database transactions. Some people try to encapsulate starting and committing transactions in their domain models. But then they get confused when domain models invoke other domain models that also try to start and commit db transactions. So which model really gets to decide if a transaction is committed or rolled back? And what do you do if a given model is used in different ways by different clients?
The Service Layer is a solution for this, because this is the layer in which you can start and commit work that involves multiple domain models.
As for how common this is, I don't think it's common at all. Most people using Zend Framework (or any other PHP or Ruby framework) have just barely moved from "Active Record solves everything" to the new shiny, "Data Mapper solves everything." It seems this community learns only one new pattern every five years. They won't get to Service Layer for a while.
Re comment from #ktutnik:
No, the Service Layer pattern is different from Repository pattern. Repository is about abstracting database access so you can use a database like a Collection. Service Layer is about encapsulating complex application operations.
Another way of thinking about them is their relationship to the Domain Model. The Repository is used between the Domain Model and the database. Whereas the Service Layer uses one or more Domain Models.
Service Layer ---> Domain Model(s) ---> Repository ---> DBAL
Service layer advocacy is relatively new and still subject to a variety of interpretations. I think it means having a layer that leverages multiple domain models which the controllers call (I may be simplifying it too much though). I recently developed a website making use of this and practical advantages I've encountered are:
Features as a service helps with scalability. If you have an image service that initially uses the local service to do it's work it becomes easier to have that service point to another server or some 3rd party without having to make sweeping updates
Flexibility. Half way through the project I decided to change a core piece of functionality and was able to do so painlessly; allowing me to quickly weigh the pros and cons of the update. This flexibility is useful when rapid prototyping and instills a certain confidence because if you need to revisit something it's not going to be a nightmare.
Extensibility. I have already identified services in my application what I can forsee opening to other developers or other widgets, mobile apps in the future. Doing so in theory is just a matter of adding authentication and authorization to the service (because the features are already in it's own layer and I don't have to spend time trying to decouple what I want to expose from the rest of the code base).
Services are easy to add and drop (maybe this belongs with one of the earlier points). I have services that a specific to a specific stage in the project (e.g. invite only stage) that I can drop once that phase is over.
I think it has practical advantages and a key to success in implementation is having a good way to manage the services in the application. I use symfony's dependency injection component
See ZFEngine it's cmf on ZF with service layer realisation
Related
We have a pretty large Symfony 2 web application which has many different endpoints and features:
api for data from our legacy product
web components for use in our legacy product
api to our new iOS POS
api to loyalty end-user portal
web interface for loyalty end-user portal
web interface for (seperate) invoice end-user portal
big admin area with configuration for all of the above
The database layer (in Doctrine) on this is tightly coupled. Transactions from both the POS and our legacy product are used in the loyalty end-user portals and invoices are based on the same transactions. Obviously there's also many entities that are solely for specific parts of the application.
We originally decided on the single app+bundle approach for ease of programming, which has served us well in developing the whole platform. Unfortunately the main drawbacks are:
very bad performance (although things like further caching, minimizing assets etc can help, we think that having such a bloated bundle that needs to be able to handle everything and also included different 3rd-party libraries only used in specific parts of the application is slowing everything down.)
we use continuous integration and generating new builds and running all the functional tests is taking 20+ minutes.. and we still have many classes lacking (proper) tests.
when we change part of the application, another part breaks easily. Although more and more decoupling and functional tests help with that, it's still far from ideal.
I've done some research to splitting a Symfony project into multiple projects (each with it's own github) and using SOA to connect them. My personal experience so far with SOA is that it makes things very hard to test fully and adds lots of overhead when migrating from standard Symfony 2 forms (which I totally love).
I was also thinking on another solution by creating a shared bundle with the shared entities and repositories. This would make it much easier to test code and share common services (managers), although I've also heard argumentation against big managers. The big downside to this is that we cannot simply use doctrine:schema:update then, because sharing the database and updating the database on a project with a lower version of the shared bundle, will remove fields.. causing loss of data. Also on this approach I have been unable to find any examples or use-cases.. which leads me to wonder if it wouldn't have many more downsides.
So my question is: what are common approaches and solutions for splitting a big project like this? And: are there reasons that maybe it should not be split at all?
Although I'm answering your question, It's kinda hard to come with a magical solution for your problems. This is not an attempt to solve all of your problems, nor impose you of following it. This is not the only possible solution, actually this might not even solve your problems. That said, let's begin.
I'd split the project in 4 layers:
Presentation Layer: Desktop aplications, Web interfaces (no matter if
is php, C#, if it uses Symphony or any other framework and third
library components), Mobile Apps, everything end users can see and
interact with (also known as GUI). These guys only communicate with
Application/Service to request something, like a list of available
products, update some data somewhere, send an e-mail for customers.
The key here is they really don't know how and where is
Appication/Service layer going to do the requested actions.
Application/Service Layer: I'd treat this as controllers which can receive requests from the Presentation Layer, and external webservices as well. They look like APIs, and will decide if they have to access/manipulate data through a Repository, or send e-mails using some SMPT service. They just makes the communication between GUI or external webservices which might consume your APIs and Domain/Infra layers. Yet they don't actually know what SMPT service they are using, or where data is going to be stored and how (in a MySql through Doctrine? in Sql Server through Entity Framework? in a NoSql database? txt files?). Application layers usually have their own Models (also known as ViewModels), which are exposed to the world and returned to the requester (GUI or external Webservice), representing part of the domain models. This mapping (convert Domain classes to Application classes) can be done with patterns like Facade and Adapters (also called the Anti-corruption layer), and there are plenty of packages to resolve this (for C#, there is Automapper, for PHP there might exist something either). Why should you need this? To avoid exposing your full domain to the world. Suppose you have Invoice and Loyalty end-users, but you wanna treat them as one unique domain class "User" with their corresponding properties together. You could create a LoyaltyUser and an InvoiceUser classes in your application, each one containing only the necessary properties for that purpose, then use this Mapping technique to map the domain User class to each one of them. Therefore, the application layer usually contains authentication and authorization rules, so only the Loyalty end-user would have permission to access controller's actions which would deal with the LoyaltyUser model. Inside a single action in a controller, you shouldn't take different paths/ways depending on the requester (for mobile, do this, for website, do that). Instead, you might have different actions for each one, and tue Presentation layer knows what they want to request.
Domain Layer: This is your core, containing all business logic. This is what provide value to your business. Domain layer container
models/classes representing real entities from your world, interfaces
for services and repositories. Domain must be the most clean and
natural possible. They can't know what application is asking
something, nor how type of infra is being used. They just do business
logic. The Domain layer don't know if your are using Doctrine or Laravel as an ORM, nor if the application is a php website done with Symphony, or an Android Native App.
Infra Layer: Here you implement things like database, SMPT service, Logging, and other things your application might need.
Doctrine would reside here. Therefore, you would create Repository
classes implementing the repository interfaces of your domain. The
Repository implementation uses Doctrine to do stuff. These
implementations are providen to Application Layer (normally via
Dependency Injection). This means the Application Layer shouldn't
know if is Doctrine or Laravel, that's why the Application uses the
Repository (so logic to access database are encapsulated).
Your web interfaces would reside in Presentation. If the framework you use in your web have to use MVC and therefore have controllers, these controllers should dispatch to the Application Layer (I know it sounds redundant). Your APIs would reside in Application Layer.
This is very decoupled, if you need to change from Doctrine to Laravel, your don't need to change your Domain nor your Apps. If your need to change from Symphony to anything else, or even change your website from PHP to ASP or Java, your domain don't have to be changed.
Adding more layers, mapping objects, using DI shouldn't make requests slower, considering the hardware's price and capacity nowadays, the difference in time is almost imperceptible. You should put efforts attempting to improve your domain, which brings value for the business. Separating layers improve decoupling, chances of changing part of application breaking other parts, increase flexibility of scaling your app, and makes testing easier.
Rein, what was the solution you've finally ended up with? Have you actually split your project?
There is really a lack of information in this area, I just found one reasonable article https://ig.nore.me/presentations/2015/04/splitting-a-symfony-project-into-separate-tiers/
I'm starting to use Laravel 5.1, I got knowledge in php, js, angular, express, node, apache, mysql, sqlserver, and some other things.
In Angular I like that everything is built as singletons, so that the main concepts are: modules, controllers, services, factories, directives, routes, views, scopes, etc...
I'd like to know what concepts do I need to understand when building an app under laravel?
Basically all you need to know is very well documented at the official documentation, Taylor Otwell made a huge effort doing the documentation to make the framwork more comprenhensible.
The basic concepts are:
Routing
Controllers
Views
Blade templating
Requests and Responses
Models & Migrations
After that keep an eye to in the Middleware concept, also there's a plenty of built-in services as: Auth, Pagination, Encryption, etc... check them out.
Something you'll love is Eloquent ORM, it simplifies the interaction with the database.
Laravel community has their own site and that is laracasts you could get help there too.
One of the most important features of Laravel is the Service Container (you'll hear about it also as the IoC Container) and the way it's used to register Services, provide Dependency Injection, and be a Registry for your application.
Middleware are one of my favorite feature of laravel: they are like filters executing before and after the requests, in which you can 'prepare' some data to be handled in the request or 'fix up' something after the request has been processed
Another peculiar feature are the Facades: (non to be confused with the homonym design pattern ) you'll see them everywhere, and you'll learn how they provide easy access to the service container keeping your code succint,readable and testable in the same time
In a Symfony/Doctrine/PHP-project, a client is complaining that we've broken software development best practices. The complaint is about improper layering of the source code, and the lack of unit tests.
This is a sub $50k-project.
I believe that the client has an expert from the Java world, perhaps Spring Framework, looking at the source code.
We've been using proper MVC, as we see it.
View-logic is handled entirely by TWIG.
Database is handled entirely by Doctrine.
We're using Symfony Security for access control ($this->get('security.context')->isGranted('ROLE_ADMIN') and $this->get('security.context')->getToken()->getUser().
Beware that Symfony has changed the model a bit, since we started this project - but remains backward compatible.
In the controller the customer is specifically saying that it's wrong for the controller to handle:
Access Control (via Symfony Security)
Database Queries (via Doctrine)
"Parsing and other logic" for sending back responses (return $this->render('some_template.html.twig');)
The question
The client is saying that best practices is for the controller to simply pass on requests to another layer further down in the system.
Further he's saying that user-admin is based on a "custom model" where all users and roles are stored in the database - which makes plugging in a different access control system difficult. Specifically because role names seems to be hard coded such as via commands such as ($this->get('security.context')->isGranted('ROLE_ADMIN').
So; is there a definitive best practice on this field? What belongs in the controller, and is Doctrine, Twig, Symfony Security "sufficiently" a separate layer "below the controller".
Should there be yet another layer between the controller and Doctrine for example?
Source: http://fabien.potencier.org/what-is-symfony2.html
First, what is Symfony2?
First, Symfony2 is a reusable set of standalone, decoupled, and cohesive PHP components that solve common web development problems.
Then, based on these components, Symfony2 is also a full-stack web framework.
Depending on your project and depending on your needs, you can either pick and choose some of the Symfony2 components and start your project with them, or you can use the full-stack framework and benefit from the tight integration it provides out of the box. And choosing between the two different approaches is really up to you.
Is Symfony2 an MVC framework?
Symfony2 is really about providing the tools for the Controller part, the View part, but not the Model part. It's up to you to create your model by hand or use any other tool, like an ORM. Of course, tight integration exists for the most well known ORMs like Doctrine2 and Propel; but they are optional dependencies. The Symfony2 core features do not and will never rely on any ORM.
Symfony2 is an HTTP framework; it is a Request/Response framework. That's the big deal. The fundamental principles of Symfony2 are centered around the HTTP specification.
Symfony about best practices: https://symfony.com/doc/current/best_practices.html
You should rly read this about controller best practices:
https://symfony.com/doc/current/best_practices.html#controllers
You can read this answer https://stackoverflow.com/a/21701890/2160958
I am looking for a (small) library that helps me cleanly implement a front controller for my pet project and dispatches requests to single controller classes. The front controller/dispatcher and controller classes need to be fully unittestable without sending HTTP requests.
Requirements
PSR-0 compatible
installable via its own PEAR channel
support for unit testing:
checking if the correct HTTP headers are sent
catches output to allow inspection in unit tests
perferably PHPUnit helper methods to help inspecting the output (for different output types, i.e. HTML, XML, JSON)
allows setting of incoming HTTP headers, GET and POST parameters and cookies without actually doing HTTP requests
needs to be usable standalone - without the db abstraction, templating and so that the fat frameworks all provide
Background
SemanticScuttle, the application that is bound to get proper "C" support, is an existing, working application. The library needs to blend in it and needs to work with the existing structure and classes. I won't rewrite it to match a framework's specific required directory layout.
The application already has unittests, but based on HTTP requests which make them slow. Also, the current old way of having several dozens of .php files in the www directory isn't the most managable solution, which is why proper controller classes need to be introduced. All in all, there will be about 20-30 controllers.
Previous experience
In general, I was pretty happy with Zend Framework for some previous projects but it has several drawbacks:
not pear-installable, so I cannot use it as dependency in my pear-installble applications
only available as one fat download, so I manually need to extract the required bits from it - for each single ZF update.
while unit test support exists for ZF controllers, it's lacking some advanced utility functionality like assertions for json, HTTP status code and content type checks.
While these points seem to be nit-picking, they are important for me. If I have to implement them myself, I do not need to use an external libary but write my own.
What I don't want
StackOverflow has a million "what's the best PHP framework" questions (1, 2, 3, 4, 5), but I'm not looking for those but for a specific library that helps with controllers. If it's part of a modular framework, fine.
I also know the PHP framework comparison website, but it doesn't help answer my question since my requirements are not listed there.
And I know that I can build this all on my own and invent another microframework. But why? There are so many of them already, and one just has to have all that I need.
Related questions
What's your 'no framework' PHP framework?
How do you convert a page-based PHP application to MVC?
Knowing Symfony2 well, I can assure you it's definitely possible to use it just for the "C" in MVC. The models and templates are completely free and are typically executed from the Controllers anyway, so if you don't call Doctrine or Twig specifically, you can do what you want.
As for functional testing, which is really what you're talking about in your article, what you want to look at is the WebTestCase class, which is well complemented by the LiipFunctionalTestBundle bundle for more advanced cases.
That allows for some things like this example of testing a contact form that sends an email, where the entire HTTP request is done in process, since the framework is written to allow multiple requests per process and has no global state, this works very well and does not require a http server to be running or anything. As you can see I do assertions on the HTTP status code of the response too, and was able to capture the email without sending it since in test configuration sending of emails is disabled in the standard distro of Symfony2.
That being said, you could also just use the Request and Response classes from Symfony2's HttpFoundation component. It should allow you to test your code, but IMO you wouldn't get as many nice features as you could if you'd use the entire framework. Of course that's just my biased opinion ;)
I would recommend downloading the Symfony 2 framework Routing component: https://github.com/symfony/Routing
Documentation is found here: http://symfony.com/doc/current/book/routing.html
Perhaps it does not satisfy all you requirements, but it's the closest.
If you are familiar with symfony (which I think you are) you should check out silex From their website this is what they say about it:
A microframework provides the guts for building simple single-file apps. Silex aims to be:
Concise: Silex exposes an intuitive
and concise API that is fun to use.
Extensible: Silex has an extension
system based around the Pimple micro
service-container that makes it even
easier to tie in third party
libraries.
Testable: Silex uses
Symfony2's HttpKernel which abstracts
request and response. This makes it
very easy to test apps and the
framework itself. It also respects
the HTTP specification and encourages
its proper use.
I'd add Net_URL_Mapper, it doesn't have the assertions though. Is that why you ruled it out?
Another pretty interesting thing is silex. It also comes with controller tests. I'd use that over Symfony2. But that's my personal preference.
Quite a understandable wishlist. I think we all hate it in testing when we run into dependencies that make testing to havoc. Tests should be simple and short, having many things to solve before and after running each test can be a burden.
From the description of your question it looks like that you pretty specifically know what you're looking for.
My first reaction would be that you use PHPUnit for this. It does not qualify all your requirements, but it's a base you can build on. It's highly expendable and flexible, however it does not support PSR-0 but has an autoloader of it's own so probably that does not weight that hard.
From the information you give in your question I'm not sure if the design of your testsuite(s) or the design of your application are hindering in writing and performing the tests you would love to.
I smell sort of probably both. If your application code is not easily testable, then there is not much a testing framework like PHPUnit can do about. So for example, if your controllers do not use a request object with an interface, it's not so easy to inject some request that was not triggered by the HTTP request, but by your tests. As HTTP is most often the entry-point into a webapplication, it pays to abstract here for tests. There exist some suggestions apart from specific frameworks: Fig/Http. However this is just a pointer.
Similar is this with the database scenario you give: If your application code is depending on the database, then your tests will be as well. If you don't want to test against your database all the time, you need to have your controllers being able to work w/o the concrete database. This is comparable with the HTTP requests.
There exists numerous approaches to cope with these circumstances, but as I read you question you don't look uneducated, but it's more you're looking for a better solution than exisiting ones.
As with every own code, it's pretty hard to find something that matches the own design. The best suggestion I can give is to extend PHPUnit to add those suites and constraints you need to for your application while you use the support of automated tests to refactor your application to fit the needs of how you would like to test.
So you can start with the tests and then develop the controller like you need it. This will keep your controller light I assume and help you to find the solutions you need.
If you find something that is missing with PHPUnit, you can first extend it on your own and additionally the author is very helpful in adding missing features.
Keep in mind that if there does not exist what you need, you need to code it your own. However if you're able to share (part) of the work with others, you most often get a benefit than by doing everything alone. That's a point for an existing framework, be it for testing or the application.
So if as of yet there is no such controller / MVC that does support easy unit-testing out of the box that fits your needs, chime in and develop one TDD-wise. If done right it can exactly match your requirements. However I think you're not alone with this problem. So not a very concrete answer, but I can only say that I made very good experiences with PHPUnit and it's extendability. That includes output tests you're mentioning in your question.
And probably a little differentiation at the end: It's one thing to test code-units and another to test if they all work in concert in the application with it's various requests. The last most often requires larger test setups by nature. However, if you can separate units from each other and clearly define with which other units they interact, then you normally only need to test the interaction between those which can reduce the setup. This does not save you from infrastructure problems, but those are normally not tested with unit-tests anyway (albeit you can extend PHPUnit to perform other type of checks).
A popular framework - even with a bad design - has the big plus that components tend to be better tested by use. That normally helps to go over the first years of your application until design issues in a framework make you need to rewrite your whole code base (probably).
As controllers often are sort in the middle of everything, this can lead to the scenario that you tend to test the whole application while you only want to test the controller(s). So you should think about the design and role of the controllers and their place within the overall application, what you really want to test with your controllers, so you can really make them testable according to your needs. If you don't need to test the database, you don't need to test the models. So you could mock a model returning random data to take it to the extreme. But if you want to test if HTTP handling is right, then probably a unit that abstracts HTTP handling is needed at first. Each controller relying on this would not be needed to test (theoretically) as the HTTP processing has been tested already. It's a question of the level of abstraction as well. There is no overall solution, it's only that frameworks can offer something but you're then bound to those paradigms the framework expects. AFAIK testing in php is getting more and more popular but that doesn't mean that the existing frameworks have good support for it. I know from the zend framework that they are working on this to improve the situation since longer. So it's probably worth to look into the more recent developments in the more popular frameworks to what this leads to as well.
And for the very specifics, you need to test on your own always.
Opting to PHPUnit and own testcases however does look as a practically way to me. Code your controllers as you need them for your project in TDD and you should get what you need.
Probably the more component based approach of Symfony 2 is better fitting your needs than what you experienced with Zend Framework. However, I can not suggest you anything specific as needs highly differ within application design. What's a quick and solid solution for one application is a burden for the other. See Page Controller.
You could take a look at the http://ezcomponents.org/ witch is becoming apache zeta
There are three ways how to make eZ components available for your PHP environment, please read the whole of this article before continuing with the practical part:
Use PEAR Installer for convenient installation via command line
Download eZ components packaged in an archive
Get the latest sources from SVN
I haven't got my hands into it yet but looks like a good solution...
Seldaek: WebTestCase isn't quite the right thing - it's for testing a view directly, and a controller or model only indirectly.
A unit test case for a controller would invoke the controller, likely giving it a mock object for the templating engine (e.g. a mock Smarty object), then check the values that were assigned to that object for display: for example, if you invoked the controller for /countries/south-sudan, you could check that the template variable $continent was set to "Africa". This kind of unit testing wouldn't actually involve any template rendering in most cases.
I am a PHP developer, I read about Java EE technologies and I want to implement such technologies( n-tier, EJB, JPA...) with PHP and all what coming with (MySQL, Apache...).
Don't.
PHP is not Java. Writing PHP code like you'd write Java code is silly and counterproductive. It's very likely to make future maintainers of the code want to hurt you.
Need to persist an object? Use an ORM.
Need a multi-tier architecture? If you design your code with proper separation of concerns, you've already gotten 9/10ths of the way there.
EJBs? Every time I read the Wikipedia article, they're described differently. Reusable components? With a standardized interface for what, distributed applications and data persistence? Useful, yeah, but that's not PHP. ORMs and a a good message/work queue will get the job done.
The bottom line: For the vast majority of PHP scripts, you will not need any "enterprise technologies." If you do, you're doing something wrong: either you've having overarchitected the application, or you've chosen the wrong platform.
Start by picking a modern PHP framework, and build your application from there. If you're coming from Java, then Zend Framework will seem the least foreign. Kohana, Symfony and CodeIgniter are all worthwhile. Avoid Cake for now.
Keep it simple and you can't go wrong.
Your question is an insightful one. That's because as your enterprise becomes more successful, it will have to scale up to support the load of more traffic. So you will have to separate your PHP code into layers that run on separate tiers (either separate servers or separate Virtual Machines like with Xen.)
For example, I designed a system last year implemented in PHP on 10 Linux OpenSUSE servers running about 25 Xen Virtual Machines (VMs.) Some of the VM's were load balancers, some were front end tiers, some were middle tiers and some were back-end tiers, some contained MySQL databases, and we had a couple of dedicated servers that were RAID arrays for user file storage. We created NFS mounts as necessary to save/read files to/from the RAID array.
We grouped the tiers into three related groups, so we could have independent test sites for QA, Staging (User Acceptance) and Production.
So our PHP software was separated into loosely-coupled layers as follows:
FRONT-END TIER (VMs)
Application Layer (port 80) --
including AJAX responses, validation
code, navigation, etc.
Admin layer (port 443) --
including Admin Dashboard with
access to system metrics and Unit Test harnesses
Service Provider (port 443) -- Secure
RESTful Web Services API (with token)
to provide services to Partners and
others who use the system as a
"platform."
MIDDLE TIER (VMs)
Business Logic Layer -- calculations
specific to the system or business,
or the roles and permissions for
various use cases
Interoperability Layer --
authorizations and posts to Social
networks or Partner applications,
etc.
BACK-END TIER (VMs)
Data Access Layer -- handles SQL
queries, inserts, updates, deletes to
the database (implemented as Prepared
Statements) in a way that can be
adapted when the database changes to
a different kind...example: from
PostgreSQL to MySQL or vice versa.
Includes PHP code for backing up and
restoring databases.
The idea another respondent brought up of using a Framework for enterprise software seem pretty silly to me. If you are developing a student project or a “proof of concept” on a single server, and if you already are familiar with a framework, it may have its use for rapid prototyping.
But as you see from the above, when you are writing Production-quality code, distributed across multiple tiers, you don't need the crutch of using a Framework.
Where would you put the framework to link into all the places in your code? On every tier? Bad idea. Frameworks include many pages that you may need and you may not need. So they slow down performance, especially when multiplied by every tier on which you must install them.
Equally inefficient would be to create a “layer” just to contain a framework that every other layer would have to call. The benefit of software layers is to be loosely-coupled and independent of other layers, so that when changes occur in one layer, they do not require changes in another layer.
Besides, developers who write Production-quality code don't need to rely on a “swiss-army knife” that Frameworks represent. Such developers are quite capable of writing targeted efficient code and, if necessary, reusing classes in a library they may have developed for previous projects.