I have a enterprise webapp that is extremely ajax heavy, all data goes though some sort of ajax, so is saving, updating deleting etc. Modeled after the MVC paradigm
The site is mostly compromised of three parts
Interface page
An interface page only contains the interface and all the js that goes with it, no data processing of any kind. All data is requested after the interface has loaded. All data is transmitted and received in a common interchange format.(More below)
eg. edit_inventory.php or list_inventory.php
Backend Page
A backend page that processes data when commands are received kinda like APIs that you see on other webapps, it basically have a giant switch statement of all the commands it accepts.
eg. ajax_inventory.php
Classes Page
A page with a single class that handles a single object, eg item, invoice, account etc.
They currently handle only database connections, getting data and saving data. It is also responsible for error checking and taking care of foreign key links/cascade data.
Copying from a typical CMS, a set of classes, interface and backends are called components.
root
>comp
>inventory
>class_inventory.php
>ajax_inventory.php
>edit_inventory.php
>list_inventory.php
>ajax_inventorymerge.php
>edit_inventorymerge.php
>account
After a while, I noticed this method or idea have generated a lot of files, current at 400+ files just for interfaces and backends. The 400+ pages are quite unique in their own respects, most are complex CRUD interfaces for billing, inventory, reporting, misc data.
The questions, I am looking for answers that are commonly used.
Is this a good pattern, can it be
improved?
Should objects and classes such as "Product" responsible for formatting the data into the common interchange format as mentioned earlier?
Should objects allow unfiltered access to their data or only filtered access? Unfiltered access also meant that there must be lots of filtering to be done at the backend level. Filtering is mostly getting rid of unwanted data(waste of bandwidth) or metadata. However, sometimes it might be data that the user should not see(Permission related). Filtering is mostly done on the key-values pair(form_data)
Since there are multiple data forms outputted by an object from html(form_html), messages(msg), key-value pair(form_data) etc. How would one best request the data in one whole, rather than $a->getData(), $a->getMsg, $a->getStuff?
Another interchange format? or just use the common interchange format as mentioned earlier?
Common Interchange format(in json)
{"msg":"Sucess",
"form_data":{
"id":100,
"product_code":"BLA",
"size":"3"
},
"form_html":{
"list":"<\/li><\/ul>"
},
"script":"alert('Some stuff');"
}
Related
I'm a novice, but struggling hard to implement this interactive application I'm working on "the right way" or at least a good way in terms of scalability, maintainability, modularity, development speed and tool independence. That's why I chose the REST design guides and a framework which implements MVC.
However I can't get my head around where to put what in the following situation and any input or reading material from a more experienced developer in this techniques would be greatly appreciated :
I'm developing a single page web app which creates a resource that has several nested resources within. In the create methods and alike, I need to call the create methods from the nested resources. Right now every GET request is responded with a JSON, which the front end then parses, shows and add dynamically to the page accordingly. The question is : where should this create and store methods from nested resources be, in the controller or in the model?
Currently, my approach is : since the controller function is to handle user input, interact with model and return the view accordingly, the nested store methods are in the model since they're not created independently, their create methods are in the controller since they're requested from ajax calls, but this isn't nested, and so on. I'm worried that this is too mixed up and not general.
Am I ok ? Am I mixed up? I don't wanna make a mess for my coworkers to understand. Theory becomes tricky when applied..
I'm gonna have a go at this. I am myself still learning about this as well, so if any information is wrong, please correct me.
In terms of scalability, you should always be able to create any model independently, even though at this point it appears not strictly necessary. The REST paradigm stands for exactly this: Each model (a.k.a. resource) has its own (sub)set of CRUD endpoints, which a client application can use to perform any action, on any composition of data (compositions in which elementary entities are mostly the models you specify).
Furthermore, a model should be concerned with its own data only, and that data is typically found in a single table (in the case of relational datastores). In many cases models specify facilities to read related resources, so that this data can be included when requested. That might look like the line below, and the response is ideally fully compliant with the JSON API specification:
GET //api/my-resources/1?include=related-resource
However, a model should never create (POST), update (PUT) or delete (DELETE) these relations, not at all without explicit instructions to do so.
If you have a procedure where a model and its nested models (I assume related models) are to be created in a single go, an extra endpoint can be created for this action. You'd have to come up with a sensible name for that set of resources, and use that throughout your application's HTTP/support layer.For instance, for creation of such a set, the request might be:
POST //api/sensible-name { your: 'data' }
Keep the { your: 'data' }
part as close to a typical JSON API format as possible, preferably fully compliant. Then, in your back-end (I suppose Laravel, inn your case) you'd want to create a factory implementation that might be called <SensibleName>Factory that takes care of figuring out how to map the posted data to different models, and how their relations should be specified. Under the hood, this factory just uses the model classes' creation facilities to get you where you want to go.
When you would instead automate this process in your model it would be impossible to create the resources independently.
When you would instead automate this process in any single-resource controller that would be non-compliant with the REST paradigm.
With the factory pattern, you explicitly use that class to perform the higher level action, and none of the above concerns apply, not speaking about whether this approach is in accordance with REST at all.
The key takeaway is that the exact same result must still be achievable by performing multiple requests to single-resource endpoints, and your additional /api/sensible-name endpoint just substitutes for the need to call to those multiple endpoints, for the purpose of convenience, when you DO want to create multiple records in a single go.
Note that my statement has little to do with what endpoints to create to fetch nested resources. This SO question has some pretty good conversation as to what is acceptable, and how your specific needs might relate to that.
In the end, it's all about what works for you and your application, and REST and the like are just philosophies that propose to you an approach for similar needs in general web development as well as possible.
How far do I break down individual tasks within a typical scenario of "Web application reacts to user input"?
For example, in the case below, say a scenario is "User submits a form, causing user data to be turned into an Object (technical detail) and then saved into the database" .
I am using various services to get, filter, object-ify, and save data. Specifically, for example, in my $domainObject = ... line below solely copies data from array into an object (similar to this What is a name for a pattern where it receives data or asks for data and returns back an object?)
I am asking, if by continuing to separate individual concerns I come across, into various classes, services, and methods, am I making things harder for myself or future maintainers in the long run?
class Controller
{
//saves a domain object acquired from an HTML form & other sources
function saveAction()
{
// acquire data from GET, POST, COOKIE, SESSION, database, et
$inputData = $this->inputService->acquireData();
// clean data
$filteredData = $this->filterService->filter($inputData);
// marshall data into an object
$domainObject = $this->objectService->createObject($filteredData);
//save object into a database
$id = $this->repository->save($domainObject);
// Send $id to View
return new ViewModel(array(
'id' => $id
);
}
For Clarity
Let's call "parameter passing" as "wiring".
So, first, my wire is $inputData, which I receive from inputService.
I take that wire and feed it into filterService, which returns the
other end of the wire called $filteredData.
I take that end of wire and feed it into objectService.
In return I get $domainObject end of the wire.
I take that end and feed it into repository, and receive back ID
I take the $id end of wire and run it into my ViewModel, and that's the endpoint.
^^^^ The above is all the wiring that I do and that needs to happen when I use Separation of Concern to separate my code into various code constructs of inputService, filterService, objectService, repository, ViewModel.
The wiring connects them together.
I can "cheat" and merge some code constructs together to minimize on passing the wire. (minimize passing the parameters everywhere).
And that is what my question is about.
Question
Is wiring up of the individual code constructs (parameter passing between various services) a good thing? Should I do more? Should I do less? Is there a technique I'm not aware of that makes this a non-issue?
I would say that it depends of the business logic and the domain of the problem you're trying to solve. If you see frameworks like Ruby On Rails, or Laravel, they try to use MVC to tackle common web app problems. However, these frameworks start to get fat (for example, you can find controllers or models with more than 2000 lines of code, i.e the famous User model with a lot of business logic).
This has popularized two approachs.
The use of microservices in different technologies instead of monolithic applications.
The use of OOP concepts like concerns (traits in PHP), composition, mixins for behaviors, engines for splitting logic and more.
So I would say that if you have a simple app, with no future intentions of making it grow with hundreds of features, a simple mvc flow with helpers can be enough. If your app is planning to grow a lot, you should consider an alternative as mentioned before.
This is a very debated topic, and there are several articles of great programmers, like this one from David Heinemeier Hansson. Also this is pure gold from Sandy Metz in railsconf. And other nice resource is this question.
I am currently working on a personal project. The entire site is dynamic so I am requesting all sorts of information from the database on all pages. I noticed that I was duplicating specific code to display the data to the pages. A few examples would be applying htmlspecialchars to the data or iterating through to set image paths, etc.
What i am wondering is if this is a bad decision to make so I can course correct early on. Most internet resources have said that the controller manipulates while the model fetches, updates, inserts, etc. into db. BUT it just makes sense at least for me anyway to manipulate the output data via model since all pages will use it in the same way.
No. A model merely represents data, whether that came from a database, an XML file, a web service/API, or even just an array.
If you find yourself doing the same thing over, then it’s time to re-factor and DRY your code out. For things like sanitising data, you would normally have a presenter that takes your model, and returns it data sanitised however you wish. That way the model has its single responsibility (represent data), and your present had a single responsibility of transforming data.
I need help with something I can’t get my head wrapped around regarding the Repository and Service/Use-case pattern (part of DDD design) I want to implement in my next (Laravel PHP) project.
All seems clear. Just one part of DDD that is confusing is the data structures from repositories. People seem to choose data structures the Repository should return (arrays or entities) but it all has disadvantages. One of which is performance looking at my experiences in the past. And one is which you don’t have interfaces for simple data structures (array or simple object attributes).
I’ll start with explaining the experience I have with a previous project. This project had flaws but some good strengths I learned from and like to see in my new project but with solving some design mistakes.
Previous experience
In the past I’ve build a website that was API Centric using the Kohana framework and Doctrine 2 ORM (data mapper pattern). The flow looked like this:
Website controller → API client (HMVC calls) → API controller → Custom Repository → Doctrine 2 ORM native Repository/Entity-manager
My custom Repository returned plain arrays using Doctrine2 DQL. Doctrine2 recommends array result data for read only operations. And yes, it made my site nice and light. The API controller just converted the array data to JSON. Simple as that.
In the past my company created projects relying fully on loaded Doctrine2 entities and it’s something we regretted due to performance.
My REST API supported queries like
/api/users?include_latest_adverts=2&include_location=true
on the users resource. The API controller passed include_location to the repository which directly included the location relation. The controller read latest_adverts=2 and called the adverts repository to get the latest 2 adverts of each user. Arrays were returned.
For example first user array:
[
name
avatar
adverts [
advert 1 [
name
price
]
advert 2 [
….
]
]
]
This proved to be very successful. My whole website was using the API. It would be very easy to to add a new client because the API was perfectly in production already using oauth. The whole website runs on it.
But this design had flaws too. My controller still contained A LOT of logic for validation, mailing, params or filters like has_adverts=true to get users with adverts only. It would mean that if I created a new port, like a total new CLI interface, I would have to duplicate alot of these controllers due to all the validation etc. But no duplication if I would create a new client. So at least one problem was solved :-)
My admin panels were completely coupled to the doctrine2 repository/entity-manager to speed up development (sort of). Why? Because my API had fat controllers with special functionality for the website only (special validation, mailing for registering etc). I would have to redo work or refactor a lot. So decided to use the Entities directly to still have some sort clear way of writing code instead of rewriting all my API controllers and move them to Services (for site & admin) for instance. Time was an issue in fixing my design mistakes.
For my next project I want all code to go through my own custom repositories and services. One flow for good separation.
New project (using DDD ideas) and dilemma with data structures
While I like the idea of being API centric, I don’t want my next project to be API centric in core because I think the same functionality should be available without the HTTP protocol in between. I want to design the core using DDD ideas.
But I liked the idea using a layer that just talked as a API and returns simple arrays. The perfect base for any new port, including my own frontend. My idea is to consider my Service classes as the API interface (return the array data), do the validation etc. I could have Services specially for the website (registering) and plain services used by the Admin or background processes. In some admin cases a Service would not be required anyway for simple CRUD editing, I could just use Repositories directly. Controllers would be very thin. With this creating a real REST API would just be a matter to create new controllers using the same Services my frontend controller classes do.
For internal logic like business rules it would be useful to have Entities (clear interfaces) instead of arrays from repositories. This way I could benefit from defining some methods that did some logic based on attributes. BUT If I would be using Doctrine2 and my repositories would always return Entities my application would suffer a big performance hit!!
One data structure ensures performance but no clear interfaces, the other ensures clear interfaces but bad performance when using a Data Pattern pattern like Doctrine 2 (now or in the future). Also I could end up with two data types which would be confusing.
I was thinking something similar to this flow:
Controller (thin) → UserService (incl. validation) → UserRepository (just storage) → Eloquent ORM
Why Eloquent instead of Doctrine2? Because I want to stick a bit to what’s common within the Laravel framework and community. So I could benefit from third party modules, for example to generate admin interfaces or similar based on models (bypassing my DDD rules). Other than using third party modules, I would design my core stuff so switching should always be easy and not affect data structure choices or performance.
Eloquent is an activerecord pattern. So I would be tempted to convert this data to POPO’s like Doctrine2 entities are. But nope... as said above, with doctrine2 real models would make the system very fat. So I fall back to simple arrays again. Knowing this would work for both and any other implementation in the future.
But it feels bad always rely on arrays. Especially when creating internal business rules. A developer would have to guess values on arrays, have no autocompletion in his IDE, could not have special methods like in Entity classes. But making two ways of dealing with data feels bad too. Or I am just too perfectionist ;) I want ONE clear data structure for all!
Building interfaces and POPO’s would mean a lot of duplicate work. I would need to convert an Eloquent model (just a table mapper, not entity) to an entity object implementing this interface. All is extra work. And eventually my last layer would be just like a API, thus converting it to arrays again. Which is extra work too. Arrays seem the deal again.
It seemed so easy reading up into DDD and Hexagonal. It seems so logic! But in reality I struggle with this one simple issue trying to stick to OOP principles. I want to use arrays because it’s the only way to be 100% sure I am not depended on any model choice and querying choice from my ORM regarding performance etc and don't have duplicate work in converting to arrays for views or an API. But there's no clear contract on how a user array could look. I want to speed up my project using these patterns, not slow them down :-) So not an option to have many converters.
Now I read a lot of topics. One makes POPO’s & interfaces that conform proper entities like Doctrine2 could return, but with all the extra work for Eloquent. Switching to Doctrine2 should be fairly easy, but would impact performance so bad or one would need to convert Doctrine2 array data to these own entity interfaces. Others choose to return simple arrays.
One convinces people to use Doctrine2 instead of Eloquent, but they leave out the fact that Doctrine2 is heavy and you really need to use array results for read only operations.
We design repositories to be changeable right? Not because it’s “nice” by design only. So how could we rely on full Entities if it has such big impact on performance or duplicate work? Even when using Doctrine2 only (coupled) this same issue would arise due to its performance!
All ORM implementations would be able to return arrays, thus no duplicate work there. Good performance. But we miss clear contracts. And we don’t have interfaces for arrays or class attributes (as a workaround)... Ugh ;)
Do I just miss a missing block in our programming languages? Interfaces on simple data structures??
Is it wise to make all arrays and have advanced business logic talk to these arrays? Thus no classes with clear interfaces. Any precalculated data (normally would be returned by an Entity method) would be within an array key defined the Service class. if not wise, what’s the alternative considering all of the above?
I would really appreciate if someone with great experience in this “domain” considering performance, different ORM implementations, etc could tell me how he/she has dealt with this?
Thanks in advance!
I think what you are dealing with is something similiar I'm struggling with. The solution I'm thinking works best is:
Entities/Repositories
Use and pass around Entities always when performing Write operations (Creating things, Updating things, Deleting things, and complex combinations thereof).
Sometimes you may use Entities when doing Read operations (when you anticipate the Read might need to be used for a Write soon after...ie. ->findById is soon followed by ->save).
Anytime you are working with an Entity (whether it be Write or Read), the Repositories need to be the place to go. You should be able to tell new developers that they can only persist to the database through Entities and the Repository.
The Entities will have properties that represent some Domain Object (many times they represent a database table with the fields of a table, but not always). They will also contain the domain logic/rules with them (ie. validation, calculations) so they are not anemic. You may additionally have some domain services if your Entities need help interacting with other Entities (need to trigger other events), or you just need an additional place to handle some extra domain logic (perform Repository calls to check for some unique conditions).
Your Repositories will solely be for working with Entities. The Repositories could accept Entities and do some persistence work with them. Or they could accept just some parameters, and do some reading/fetching into full Entities.
Some Repositories will know how to save some Domain Objects that are more complex than others. Perhaps an Entity that has a property which contains a list of other Entities that need to be saved along side the main entity (you can dive deeper into learning about Aggregate roots if you want).
The interfaces to Repositories rest in your Domain layer, but not the actual implementations of those Repositories. That way you can have an Eloquent version or whatever.
Other Queries (Table Data Gateway)
These queries won't work with Entities. They'll just be accepting parameters and returning things like Arrays or POPO's (Plain Old PHP Objects).
Many times you will need to perform Reads that do not return nicely into a single Entity. These Reads are typically more for reporting (not for CRUD-like operations, like Reading a user into an edit form that is eventually submitted and saved). For example, you might have a report that is 200 rows of JOINed data. If you used the Repositiory and tried to return large deep objects (with all the relationships populated, or even lazy-loaded) then you are going to have performance issues. Instead, use the Table Data Gatway pattern. You are just displaying data and not really needing OOP power here. The outputted data could however contain ID's, which through the UI could be used to initiate calls to Repository persistence methods.
As you are developing your app, when you come across the need for a new Read/Report query, create a new method in some class somewhere in your Table Data Gatway folder. You may find you have already created a similar query, so see how you can consolidate the other query. Use some parameters if necessary to make the gateway method's queries more flexible in particular ways (ie. columns to select, sort order, pagination, etc.). Don't make your queries too flexible though, this is where query builders/ORMs go wrong! You need to constrain your queries to a certain extent to where if you need to replace them (perhaps a different database engine) then you can easily perceive what the allowed variations are and aren't. It's up to you to find the right balance between flexibility (so you have more DRY code) and constraints (so you can optimize/replace queries later).
You can create services in your Domain to handle receiving parameters, then passing them to the Table Data Gateway, and then receiving back arrays to do some more mutating on. This will keep your Domain logic in the domain (and out of the infrastructure/persistence layer of the Repository & Table Data Gateway).
Again, just like the Repository, use interfaces in your domain services so that the implementation details stay out of your Domain layer, and resides in the actual Table Data Gateway folder.
By business model, or business objects, I mean plain old objects like a "User" with all their properties name, adress, ...; in addition to all the user properties let's say each user would have an "AppointmentBook" object, each book has a set of "TimeSlot" objects, etc.
The business model has objects with references between them, at least that's how I code a business model in Java.
Here comes the question:
To intialize my business objects, in Java, I would
fetch all of the data from DB only once during application
initialization,
map data from my DB to my business objects
store in memory (maps) and they would be shared across all the requests.
PHP's Share-Nothing-Architecture is confusing me for proper OO programming:
If I use the same logic, I would have to fetch all the objects from DB, for every request (I know I could still cache, but you don't cache all of your DB, it's not a question about caching but rather about the way of programming in PHP and its architecture).
So let's say that for one HTTP request, I just need the User properties and I don't need to access his appointment book. It would be a pitty to fetch all the data from the DB for all the objects the User makes reference to, as I just need his properties. This means that I will initialize PHP objects from my model with a lot of NULL values (NULL because of the objects contained in User that I won't load) which can later on lead to errors.
I was wondering how professional PHP developers usually use their business objects?
(I'm coming from Java)
UPDATE: It was kind of stupid to say that I would load the whole database into memory during application init in Java. What I rather meant is that, if I need to fetch a specific user, I could just load all of its data and that would be accessible through all the requests.
In PHP you do not keep all the data of your domain business model in the memory. Instead you only request from DB ( though cache, if needed ), the data you want.
Model layer in php should be built from multiple domain object and data mappers ( i assume, that part is not so different from Java ). If you need User details, then you fetch only that information from database/cache. You most likely will have a separate mapper just for dealing with user(s).
You display the information about that user, and forget about the query. Next request (when and if it comes) will require different information. Maybe you will want ContactList for that User ... then you really do not need user itself, only his user_id. Again, you let you mapper to fetch data into the domain object responsible for handling contact list, and if contact list contains User instances, then just create them, but leave in "unfetched" state (object knows only own user_id). Fetch them only if you really need to, and only the parts which you will use ins that "view".
P.S. you might have notices, I told that model later should be segmented, but quite often php developers just create single class of each DB table (which implements ActiveRecord) and call it "model". This is a result caused by Ruby on Rails influence on php framework developers, which, IMHO, is one of the worst things that has happened to PHP in past 5 years.
Your Java example implies your storing your entire databases content in memory. If your doing that, what's the point of the database? Why not just create all those object and memdump them for persistence.
If I use the same logic, I would have to fetch all the objects from DB, for every request
That's simply madness, you don't need to fetch anything, you create new instances when you need them and destroy them when you no longer need them.
So let's say that for one HTTP request, I just need the User properties and I don't need to access his appointment book.
That's easy, redesign your user. Your user needs it's properties and a property called appointmentBook which is simply an array of appointment book ids.
If you actually need those appointments you can fetch them from the database later.
This means that I will initialize PHP objects from my model with a lot of NULL values (NULL because of the objects contained in User that I won't load) which can later on lead to errors.
Not really, if this is the case your User object is too big. Make it smaller, you should load the entire user. Except of course the user has to be small enough for you to sensible load it.
If you don't want that then you can always create a UserProperties class and let every User have one. When you load the User you load the properties, but you also have an option to create the properties seperately.
Even in Java you would not load all data from the database into memory. You can however - as you write - often load more compared to short Transaction Scripts you normally have in PHP.
You models should be "clever" then to only load the data from the persistence storage that is needed to perform the requested action. This requires the object to be "clever" enough to lazy-load data probably.
This can be achieved with a Domain Model that knows enough about itself and a Data Mapper that knows enough about the storage for example.
There are other patterns as well which might suit your needs depending on the type of application, however a Domain Model together with Data Mapper is quite flexible.
An exemplary data mapper in the PHP world is Doctrine.