This is less of a troubleshooting question and more of an industry standards question.
I am at a crossroads where architectural standards need to be addressed and implemented. One of these standards involves the communication route between the client application (AngularJS based, so single-page persistent with multiple views) and third party sources of information.
To me, it seems intuitive and logical to route all requests for third party libraries and data through my backend and then off to various destinations via CURL.
In this way, my server acts as a gateway between the client and the outside world (much like the relationship between a cellphone tower router and a cellphone).
I am curious as to what industry standards would say about this, and the potential pitfalls. To me, it seems like it would create more order, organization, and security long term.
Please let me know what your thoughts are regarding this, as I need outside perspectives.
Not really sure if this counts, as I am not aware of any industry standards - but I interpret it as what you're really asking for is general outside perspective. So here goes:
My short answer is that I think you're on the right track.
I think it's cleaner as it keeps the data path simple in that your client always sends requests to your server - so basically you're getting very loose coupling between your client and everything else (except towards the controller on the server, which is fine, even necessary IMO). Want to change the data source later? Client isn't affected (unless the format differs of course). It's also beneficial if you can picture yourself wanting to store the raw data in a DB for some reason some time in the future. Depending on what services you're connecting to and what you want to do with the data, going through your own server can have security benefits (like if you need to use a private key for authentication against the 3rd party service, like one has to with APIs like those MasterCard offer).
There is a performance hit though, as it gives your server more to do and requires a bit more memory, in addition to the extra requests and DNS look-ups incurred. Then again, you would be in control of caching, so you could possibly make the service slightly more robust in some cases.
So unless performance is paramount, I would go the route you were thinking of. Exactly how the routing is done on your server is a different question and might require some testing. Just make sure you use a method that lets you elegantly handle any errors that might pop up :)
Related
I'm interested in hearing about advantages and disadvantages of developing a web application using the OOP paradigm instead of Procedural.
I am NOT interested in
OOP vs Procedural
Contexts other than a typical web application, meaning a distributed application where a client makes an HTTP request to a server
My question is motivated by the fact that I've been developing a web application running on my company's Intranet, and would like this application to be "easily" maintainable by the person who will replace me, eventually, so that the application stay and is not replaced by some over abstracted over engineered paid solution a very few people will enjoy working with.
I posted some bits of my application source for review at codereview from wich I didn't get a lot of feedback. I then posted a link to my post on codereview over at reddit/r/PHP that didn't go really well.
For now, I am under the impression that the vast majority of the "web development" community have decided that OO is the way to go. I'm having a difficult time to understand why BUT I have a feeling that the next dev my company will hire will also be a strong believer in developing OO for a web application as it seems to be the norm nowadays.
I would like to agree with them and join the bandwagon, can someone try and explain to me the advantages and disadvantages of OOP in the typical context of a web application.
While doing some research I came accross this really interesting comment and this one too, wich makes it even harder for me to make the switch.
A small quote from one of the comments:
OOP is for people who can’t properly engineer software or for competent programmers who are involved in very large or complex non-web project. If I were to write an event-driven game to run on a PC, I would not choose procedural programming.
The selling point of a lot of web application frameworks these days is it is an MVC framework. You have the Controller that interfaces with the View and the Model. The model is your business and data layer which contains business objects/entities/data objects - well basically objects that communicate with each other using messages. The objects themselves are restricted to what it is unique to it and only what it can do.
These boundaries/restrictions somewhat make it easier for other people to understand your code and how to modify it. It also helps in unit testing each one of these layers separately.
I suppose if you wanted Rapidly developed application with the controller querying the database and sending information to the view, you could do that too. But the only way to test would be to run the application, check if something breaks, fix, restart it, again test.. You would not be saving a lot of time.
EDIT
You are using a single controller which has business logic in it. But your model looks very much like what an OO model should be. So the question I'd ask is, what happens if I break this controller to be domain specific? I would then have to create a UserController and a DepartmentController and UserController renders all views related to CRUDing Users. Is that worth the trouble? Absolutely. Someone else could jump in and work on AdminController or ReportingController, whatever your application needs without having to worry about changes you are making to this controller.
And if one of you colleagues wants to see some changes done to the Department section, you don't need to worry about the rest.
First of all I'm not a Php person at all but your self praising question seemed interesting so I researched a bit. My answer is general and is not related to a particular implementation of object oriented technology or web technology.
An object encapsulates the data along with the relevant functionality all in a single piece. it is not to say that you encapsulate your whole application inside a single giant object. A lot is related to how you analyze the problem, identity objects and use them.
A web application as a request-response thingy maybe true in early 90s but now the web apps are mature. It is not necessary to follow a typical request-response scenario. Have you seen a stock tracking app? it does not wait for a request to send a response. it sends whenever the data is available. I worked on the online candidate testing web app which asks questions in fixed amount of time. Once the time is up it ends the test whether the user "requests" to end it or not and results are displayed. Even if it user machine is turned off or there is a network issue, it remembers and never asked the same questions again nor extra time was ever given.
Yes there are bad programmers or "IT-guys" who are totally unaware of OO-analysis. They program the object oreinted stuf in a very procedural way. The result of their work is a request-response styled web app with 3 global variables and a 'z' namespace but that does not mean that the blame should go to the OO paradigm itself. It goes to the people who didn't understand it so much for apply it at the right place!
What I would propose would probably sound strange, but I have my reasons for it.
For a long time, we've had this Spring based API, which originated as an abstract set of REST services for CRUD functionality. With time however, we started building business and presentation layers on top until we reached a state of a dead end. Do not get me wrong, Spring/Hibernate are great frameworks, and being built on top of the JVM has its definite advantages, including performance over other web technologies, such as PHP. Compared to PHP, it gives us a much deeper way of controlling aspects such as transactioning, multithreading, dealing with byte data, control over native C++ applications, using JNI, etc.
Where the stack clearly hits a hard wall, is where the requirements get changed most often, namely the business and presentation layers. Turning the applciation into a modern, user-centered social applciation, we've experienced the toughest challenge in out careers. Java EE presentation technologies are tough to work with. Also, changing the business requirements became a tremendously long cycling because of the traditional hurdles in building testing and deploying massive Java applications.
It also felt like for a large part, we are trying to reinvent the wheel. In the PHP world, so many projects exist already, which give you a complete management system, agnostic to any kind of backend system (mapping hooks to REST/SOAP endpoints) . Many of them have all these functionalities ready, allow for admin-friendly change of scenarious, and rules, have templating, etc. Plus, being PHP based means absolutely no time wasted in building and deploying. Write the change, test, asert that it works, and switch.
Our idea now is to find a way to move the business and presentation layers in such a kind of front-server PHP-based application, and leave the pure backend stuff to Spring/Hibernate. We have a few concerns though, coming from our relative inexperience with Spring.
If we implement the business methods using PHP methods, how do we keep transactional security? I mean if a business method has to make three separate HTTP requests to the JAVA, how can we guarantee that they will all be executed in the same transaction, DB-wise?
Is there a way to use proxies or promise objects between both systems? For instance, if the PHP business method needs to call a Spring search method get a collection of objects from the database, and then pass it on to another spring method, this will mean that teh whole collection will have to be sent back and forth. Perhaps, one could store it in a session object on the JAVA side, and simply sent an empty proxy back to the frontend, which the frontend can bass back to another jav method.
A lot of our Spring based functionailities rely on a structure of plugins, using Spring events. How can we make so that our frontend server also gets notified on every ackend event that happens.
I have two ideas here: a post-processing-filter that simply makes a POST request to a controller on the frontend server, using some naming convention.
or ... using some kind of a message queue, such as JMS or RabbitMQ, or why not even something like Reddis, where one can watch data for changes
Anyone who has done that before? Is this a good idea in general? Any suggestions how to resolve the aforementioned issues?
This is not what you asked, but I think it's worth saying that some of the problems you are experiencing with Java are problems that most experienced Java developers faced some day, and most have found some solution. For instance, you can use Arquillian to do a "faster" TDD in Java. And you can use JRebel to do a "code-and-refresh" approach, like in PHP. Modern IDEs also helps in refactoring, something that helps in the "requirements change" scenario.
Granted, the presentation part is still a big problem with Java. I personally don't like JSF, and (to me), most of the other presentation technologies are either non-intuitive or they are cumbersome.
I would say that this is the reason why a lot of Java developers are becoming adopters of HTML5 and Javascript (backbone.js, underscore.js, jquery, ...) for the frontend, with REST in the backend. There's no need to have PHP in the middle.
I'm afraid I cannot answer your other questions, but perhaps a good start would be to see if PHP could be run from inside a Java EE container? I know this works for Ruby and Python apps, as JRuby and Jython would take care of the bridge between the two world.
Largely what you are trying to do is what we call: SOFEA
In this case PHP is your frontend and Java is your service layer.
Concern 1
Transactions over traditional REST or WS are a PITA.
Either consider aggregating your service calls to do more work so that the whole entire transaction is in one service calls or have Java handle your frontend. Otherwise see Concern 3 with Finagle.
Concern 2
I recommend you stay stateless and do REST or RCP.
It sounds like your concerned with speed which I think you shouldn't worry about till its a problem. However if you do want to do user based caching of objects I would use Redis and its pub/sub feature (or RabbitMQ + memcache).
Concern 3
Use RabbitMQ, or Redis, or/and Finagle.
MY final Opinion
I don't think you need PHP particularly if you know Java.
So much of the web app world today is moving towards Javascript (in the client) I don't see any benefit to using something like PHP serving pages when there are better options like Node.js or just sticking with Java.
And #jpkrohling is right... templating and frotend used to suck in Java but its gotten much better particularly since most of your UI code will be in Javascript anyway.
If you need an example of more modern web stuff in Java have a look at this project: MWA and also by the same author: Handlebars in Java.
To sum it up my opinion and go back to SOFEA: backend = Java and frontend = Javascript
Haven't done this before, not sure if it's a good idea, but here are some thoughts on your questions:
First of all, in the PHP world, each request (whether it uses keep-alive or not) is run as an isolated process, so this scenario is typically unheard of; it's possible to use persistent database connections, but any non-committed transactions are automatically rolled back when the request is done. Now, this is where my knowledge about Spring is lacking, because I'm not sure whether Java can keep a database connection in the same state between requests as long as the same HTTP connection is used; if so, you can use PHP + cURL to perform three HTTP requests while reusing the network connection.
Object (un)marshaling between PHP and Java will probably not work very well; a representation of the collection would have to be made in the form of perhaps a result identifier that can be fetched via REST by PHP and passed back to Java as an opaque string.
It should be noted that PHP requests are typically short-lived, which makes it a good candidate for horizontal scaling; long running processes, such as those for Pub/Sub systems, are therefore seen less often. Personally I prefer using the alternative of sending HTTP requests so that the frontend caching can be abstracted away from the Java service.
I'll add more when I think of anything else, do let me know if my answer gives you more questions :)
There is definitely no straightforward way out. You have to re-write your backend controller layer which exposes REST calls to PHP. You need to group some fine grained services and make them coarse grained.
A word of advise - you can make things work at the end day with tweaking here and there, introducing a queue or another framework or library etc - but you would have also introduced lot of Accidental complexity..
Things can get real nasty in future during maintenance, bug fixes, upgrades etc
If speed of development is a concern, explore other alternative like Play Framework 1.2.x framework, so that you can re-use your server side code as it is.
Currently we have a lot of web pages that either have SQL statements embedded in them or call a specific php script that does a specific job - ie getNames.php - as part of a ajax call back. Neither are particularly maintainable.
I was thinking about using a REST like API to get the necessary data to the client and then munge the data into something usable. This is attractive as this lessens the burden on maintaining highly complex sql in code and allows centralisation of data (so just one AJAX call to get the data not lots of little ones). Also allows the database to change lessening the impact on the client.
However there are two problems I can see with this strategy:
The site is a game, and so I need the RESTlike API to be protected from abuse/cheating as much as possible.
All examples of REST API's use a controller to handle the requests in root. That's not ideal for me since we are at //company/games/game/ and there already is an index.php at root (//company/).
What options and strategies do I have for the two constraints I listed?
Well, you're asking for opinion, but I'm well seasoned enough (having written many many API schemes over the years) that I'm totally willing to open myself up to Net abuse. I think the key here, and this should provide an opinion to work from on both of your questions, is that REST is simply a set of principles. Sure there are people that follow a RESTful pattern explicitly, but that isn't practical for most people.
Take the Flickr "REST" API for instance... a call may look like this: http://api.flickr.com/services/rest/?method=flickr.favorites.getContext&api_key=a114adf91150953107987e4c3dc14df8&photo_id=6033564557&format=json&nojsoncallback=1&api_sig=0d2c215992d643ef6fe4a085805f7059
Not very RESTful, from a patterning perspective... however, it contains all of the elements of REST and is a fine enough model. You can understand what it is doing at a glance, and you can easily build on top of that.
IN the end, REST is a set of principals, not a protocol, and not even a pattern in and of itself. You're free to implement it however you want. There's always an interoperability intermediate layer and the point is to just make it understandable... and many of the REST patterns actually get in the way of that, favoring form over function.
In fact, most of the patterns I've seen are insufficient for anything particularly advanced, but that's part of the point of REST... Keep It Simple (Stupid).
I run technology for a medium sized company that is about to acquire another medium sized company. Our technology is all LAMP (Linux/Apache/MySQL/PHP), the company we are acquiring is all Microsoft stack (IIS/MSSQL/ASP.NET). None of the developers on staff currently do .NET nor have ever supported Microsoft server infrastructure. I'm having a tough time deciding what to do with the situation...
Do we port all the MS stuff to LAMP (not interested in going the other way for various reasons including my team's personal inexperience with it, the cost of licensing when we are trying to slash overhead, etc)?
Do we run both technologies in parallel with separate teams to support each and write a bunch of middleware so they can talk to each other?
Neither of these choices are optimal. Has anyone ever been faced with a situation like this and how did you proceed? Keep in mind we are talking about large infrastructure in both cases with high traffic volumes and fairly extensive backend systems. Any ideas will be welcomed.
I've never done this before, so take my opinion with a grain of salt. But I would suggest NOT rewriting an existing application. I mean, if it's a 1-page application which just tells you "Hello" when you click a button, then yes, rewrite it in PHP. But business applications that make money aren't as simple as that, and you'll be starting from scratch to rewrite something that took the other company x years to develop. Not to mention you'll have to support and maintain the application you're taking over, even while you rewrite it in PHP.
If you have smart developers on your team now, and they have capacity, they'll be able to learn ASP .NET. But it might be best to hire some ASP .NET resources to help your team learn it and bear the weight (maintenance and support) of the application you're taking over. Your teams can work together to find integration points between the two applications.
Faced with the choice of writing integration points, or writing an entire business application from scratch, I'd take my chances at writing integration points.
As part of the acquisition, are your company taking on the IT support team of the acquisition?
While eventually there are likely to be 'efficiency savings' that they'll want to make from consolidating back office staff, there is a strong argument to keep both teams supporting their 'own' systems in order to keep the lights on.
Then you need to analyse the overlap - do you end up with systems on each stack doing similar things. If so, look to consolidate onto the preferred platform and remove the other. Also look at (regardless of current skills), which stack best needs the business needs in the coming years. LAMP might be perfect right now, but there may be arguments for moving to .net to meet future needs. Then again maybe not, but needs to be assessed.
Is there a business need for the 2 sets of systems to share data? If so, at what level? Creating (web)services to encapsulate shared functionality and make it available to the other system may be one way to go (SOA effectively). Alternately you may need to share a backend initially and have .NET talking to a MySQL databases or somesuch.
This is a very complicated question.
If the two applications provide similar functionality, then I would run both side by side until the one you want to keep has all of the functionality of the other one. Then I'd switch the customers over and eventually throw it away. If the customers are receptive, switch them now.
If they are radically different apps then I'd most likely just maintain both going forward. Given that these are large applications, any rewrite is going to be painful and have a high probability of failure. It's best to just get used to the idea of having different tech stacks in house.
One thing, by maintaining both apps you will be in a better position to keep the acquisition as quiet as possible as far as the client base is concerned. Clients that already use an app typically only change horses if they feel the app they are using is no longer going to be supported. At that point, you can guarantee that some will leave regardless of how good the other system is.
If the acquisition is going to result in a change in marketing (for example, the other company's logo changes etc) then I would again suggest to just maintain both. The clients are going to be nervous enough as it is.
The point of all the above is that this is more of a business problem than a tech issue and boils down to the reasons you acquired the other company in the first place and how you will present it to the existing clients. If the company was acquired for the technology or their client base, then leaving it alone is a good idea.
BTW, I've done this a couple times. The only difference was going the other route from PHP to .Net.
In one case the app was relatively small, but had a huge base of users. We ended up using some URL rewriting rules so that the user base never even knew the app changed underneath them. It was a collection of web services.
In another case, the app was large, had a big user base, and had a very public skin. Again, we heavily leveraged url rewriting to preserve google placement as well as bookmarks. The biggest problem we had was development on the original site couldn't stop while we built the replacement. This presented a lot of challenges in that every feature had to go through both teams. In the end, the project took about 3 times longer than expected but because we had some highly skilled people on it it ultimately succeeded.
Looking to develop a web service (api) in PHP to offer customers an easier way to integrate with our platform. There are workflow calls that will be validated with user/pass as well as some reporting options.
Sorry I can't post more details or code on the subject and I have never developed a web service but have had experience in using them via SOAP.
Now I would also need to offer a state or status of the workflow and I think REST would be the best choice here, but still looking for opinions on that.
For reporting I would like to offer different options such as XML,Excel/CSV any reason I would pick one over the other?
What are some of the pitfalls I should lookout for?
What are some gems anyone could offer.
Thanks in advance to any help as this is very important for me to understand.
UPDATE #1:
What would be the most secure method?
What is the most flexible method
(Platform independent)
UPDATE #2:
a little bit about the data flow.
Each user has creds to use the API and no data is shared between users. Usage is submit a request, the request is processed and a return is given. no updates. (Think Google) a search request is made and results are given, but in my case only one result is given.
Don't know if this is needed so it's an FYI.
Always handle errors and exceptions.
Problems will always make their presence felt in the application/api. Either at start or through further development. Don't leave this as an end task, and make it clear when an error occurs, with well documented response messages.
Also if your service will handle many requests, and for the same resource id (independent from user) the same resource is returned be sure to cache the information. And this not only for performance reasons, but for the cases when errors stuck up. This ways you can at least serve something to the client (possibly useful, more context required to be explicit).
The biggest and most important item I can offer is to guarantee your infrastructure behind the WS or at least what you are serving up via the WS is stateless. The moment you deviate form this it will become a nightmare and you will begin having to shoehorn code in to protect your data from getting fouled up.
An example would be a two clients making changes to data via the WS, ie...save configuration. How you deal with that on the back end makes things interesting. If your data is only heading outbound, you are in a much better situation then if you have to support pushing data into the back end.
Also, think out the API's in depth as with any public facing API. The moment you have a version out in the wild and then decide that API needs changed or deprecated begins to cause problems for the client base making use of your WS.
I am currently working on a web application that includes a web service (or 2 in ASP.NET MVC) and I highly recommend looking at the principles behind Service Oriented Architecture (SOA) as what I have found is that the most important aspect is to get the design of the web service API correct as that will effect the rest of your back end and either make your life easier or have you running into walls out of frustration.
Essentially SOA means design the web service around business processes, i.e. how the people who use your API are likely to use it.
A Good desgin is always a good idea, so I highly recommend you do a lot of reading on Web Sevice Design Principles before you start coding and save yourself or some other unlucky sod alot of grief latter on.
Also make sure that your design is agile as it will change frequently as communication between your company and your clients happens.
A little bit more related to implementation than design:
I would decide that the output of the service to be XML - this can be very easy parsed, by all decent languages. Also, if some client need other output, you could transform those XMLs through some XSLT processors and offer "other" web services, that outputs JSON, or whatever else they need.
Regarding the reporting, that depends on how the reports will be used - if the clients process them and they need only the data from the reports - then again suggest XML. In my opinion working with CSV is harder because you have to take in consideration all kind of special cases like "what happen if my data contains the separator field", "will the client be able to specify the separator", "how will I represent nested data", or all of these are straight forward with XML.
If your client needs reports to use them out of the box you could use BIRT -beautiful and free
Offering multiple outputs like JSON, CSV, YAML or XML is good - that gives the end user confort, at a very small cost! Dumping data is always easier than processing, and say that they already parse JSON for some reason - it is much easier to just hook that up for your API than implementing, say an XML-parser. Nowadays I see XML-parsers everywhere, so that should probably not be a problem, but I like the more "air-ish" nature of JSON; I have looked a little into YAML but never used it - but it looks promising, I'll definitively use that for the config files of my next project.
On the security side of anything that dynamically processes any input an user gives, one should treat such input like something you would not poke on even with a stick.
IMHO stateless REST is better than SOAP because it is less overhead, one can easily communicate with a REST-api by hand using curl or wget from the terminal. Jump-start so to say.
I would reccomend you to take a deep breath, a pencil & a paper, sit and sketch down everything that is going to be needed. Then you remove the less important stuff, and take a new paper and start to organize it. You can add the less important stuff in the next version of the API.
Try to design the API so that you don't lock yourself into a corner, make no assumptions on where it is going to head next.