I am looking for a good resource which will show me the right way for building a restful API.
Interesting topics: Authentication in particular and security in general, performance, scalability, best practices and other useful things.
I am going to build it in PHP (Slim or Silex) and before I begin I would like to think about the whole design so I can go the right way from the beginning.
There are a lot of info and posts all around the web but all of them adopt different practices and approaches.
Is there something which seems like a "standard" in the restful world?
Is there something which seems like a "standard" in the restful world?
Not beyond the level of using HTTP. There's a bunch of media types for encoding of API data (see hypermedia below), a lot of different best practices and a good amount of RFC's that covers various aspects of working with HTTP (like for instance authorization using OAuth2).
Here's a compilation of resources worth reading ... I think you will get the most out of reading through one or two of the books.
Authorative resources
Fieldings original thesis on REST: http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
Books
RESTful Web APIs: http://shop.oreilly.com/product/0636920028468.do
RESTful Web Services Cookbook: http://shop.oreilly.com/product/9780596801694.do
REST in Practice : http://shop.oreilly.com/product/9780596805838.do
RESTful Web Services: http://amzn.com/0596529260
Web API Design: https://pages.apigee.com/web-api-design-website-h-ebook-registration.html
InfoQ E-magazine on REST: http://www.infoq.com/minibooks/emag-rest
Authentication
Basic considerations: http://soabits.blogspot.dk/2014/02/api-authentication-considerations-and.html
OAuth2 spec: https://www.rfc-editor.org/rfc/rfc6749
Error handling
Best practices: http://soabits.blogspot.dk/2013/05/error-handling-considerations-and-best.html
"vnd.error" a media type for error details: https://github.com/blongden/vnd.error
Problem details for HTTP APIs: https://datatracker.ietf.org/doc/html/draft-ietf-appsawg-http-problem-00
Hypermedia
Fielding's fameous rant: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
Selling the benefits of hypermedia: http://soabits.blogspot.dk/2013/12/selling-benefits-of-hypermedia.html
HAL media type: HAL: http://stateless.co/hal_specification.html
Sirene media type: Sirene: https://github.com/kevinswiber/siren
Collection+JSON: http://amundsen.com/media-types/collection/format/
JSON API: http://jsonapi.org/
Hydra: http://www.markus-lanthaler.com/hydra/
RFC 5988 Web Linking https://www.rfc-editor.org/rfc/rfc5988
URL structures
Basic considerations: http://soabits.blogspot.dk/2013/10/url-structures-and-hyper-media-for-web.html
Partial updates
Considerations about partial updates: http://soabits.blogspot.dk/2013/01/http-put-patch-or-post-partial-updates.html
JSON-Patch: https://www.rfc-editor.org/rfc/rfc6902
Some consideration about PHP for building rest APIs
PHP is a widely used technology since many years.
But during this long period it has shown some relevant problems: it became a monstrous technology and its usage has shown some security vulnerability like SQL injection, lack of a centralized packaging registry, inconsistent API and subpar performance.
For building REST apis there are more modern technologies, like Ruby on Rails and Django, or Node.js, which is easily approachable.
Using PHP for building Rest APIS
You can of course build your apis in php also if better technologies have born in last years. Many companies still uses it in production environments.
You can choose two different approach to build your infrastructure:
building everything from scratch interfacing directly with PHP APIs.
staying on an upper level and interface with third party, open
source libraries to perform some tedious processes like routing,
authentication and so on.
The second approach lets you save time and focus more on your business logic, by delegating some common operations to trusted third party written code.
For example you can check these libraries that are commonly used in PHP applications to get the job done faster:
https://github.com/chriso/klein.php helps you performing the routing
https://github.com/PHPAuth/PHPAuth help you with authentication process (check also https://github.com/firebase/php-jwt for stateless authentication).
https://github.com/mongodb/mongo-php-driver (mongodb) or https://github.com/cagartner/sql-anywhere-client (SQL) You will need to interface with a database and this links are examples of some clients libraries that helps you with the job
Using something like node.js
Node.js is a modern technologies built to allow people do what you are going to do. It's fast, scalable (php is less), easy to use and has a very frenetic community that write code and share open source.
For example, using http://expressjs.com/ you can manage the whole routing of your application in minutes. You write your application in javascript so you will have to worry about physiological javascript's problems (closures, async calls etc.), but after have solved this quite tricky part, with node.js you will build a more efficient rest APIs, and your server will need less CPU and power to accepting and answering requests.
For best practices there is a good guideline repository on GitHub. You can go to the following page for more information:
http-api-design
You can also look at Apiary to design APIs:
Apiary
For authentication actually the best thing to do is to rely on a framework instead of reimplementing from scratch. For learning purposes however thats a different case but for putting it out to production the best bet is to use what the framework provides. Most frameworks have a good implementation which probably has been iterated over time by the community to be quite robust. Certain frameworks like laravel also provides solution for csrf (cross site request forgery) attacks.
Another useful thing that you can look into is JWT (JSON Web Token). Here is useful link which summarises why it is so awesome:
Stateless tokens with JWT
I don't know how to look into scalability from an API perspective. To me it seems more like an infrastructural issue. However with authentication (session or token based) scalability could become an issue as for example if you have different number of instances serving your users you need to maintain either sticky session or session stored in a quick access key-value store or database.
Please have a look at my opinionated presentation about REST APIs based on JSON-LD, SemWeb, Hydra Core.
Related
First of all I hope I'm allowed to ask such a broad question (first time doing so).
Ok so I'm very new to React & I need a project to work on so I thought I'd recreate my portfolio (currently created in laravel) as a react & react native app.
My questions are:
Would a Rest Api NETCORE backend be a good choice for react ?
My experience is more in ASPNET MVC than Core, should i just stick
with MVC API's for now (i've already started creating a NETCORE Api
for learning purposes)?
Would a php framework such as CodeIgniter be better for this kind of
job?
My main question is if this is a good approach? I'd like to have a basic backend (auth, news posts, portfolio items etc) and then continue building on my react apps.
I was thinking that by creating a Rest API backend it would save me a lot of trouble when trying to create a react app for PC,Android or whatever (same back-end, different client).
P.S: i'm going to host my API on Azure's free websites if its ASPNET or a shared hosting if its PHP, that is the reason for which im moving away from laravel (so laravel is a no-no).
P.S2: Firebase or other clouds (except Azure) are a no-go for me. I have access to lots of resources and i'd like to use them & not use free services like firebase or whatnot.
Just giving my 2 pence here, as this is really opinion based!
In terms of the backend, it's up to you, whatever you feel more comfortable with - but I would give a keen eye on the architecture of the backend system you build.
My choice would be to create a micro services based architecture where you create simple, atomic services which only deal within their domain. For example, you could create 'Common Services' - services which can be used as dependancies by other services (events, encryption, documents etc..) then create atomic services which deal with an aspect of your application such as User Service, Payment Service, Product Service, Basket Service etc..
The idea is simple, to create simple data driven CRUD services which are modular, atomic and reusable. I've found that learning new technology is great but understanding and learning good programming architecture is even more rewarding. You can structure data to make it the most efficient for you.
Once you have built a service, you could use services like Swagger UI to automate documentation and create testing suites for them. If you haven't used Swagger I throughly recommend it.
Implement testing for each service, and go through the whole lifecycle of software development. That will really go far in your portfolio.
Here are some articles relating to building microservices in ASP.NET Core
https://learn.microsoft.com/en-us/dotnet/standard/microservices-architecture/multi-container-microservice-net-applications/data-driven-crud-microservice
Swagger
https://swagger.io/
As a side note, I do not develop in ASP nor any other microsoft stack - but the principle is the same
UPDATE
The issue with building monolithic applications is that the code base can get more and more complicated and huge as your app grows. Some advantages of Micorservices are:
Scalability
Fault Insolation
Eliminates long-term commitment to a single technology stack
Easier for developers to understand (and document)
My type of set up would be using Spring Boot (Java) and using Eureka Server - but you are into the MS Stack, but the link I've given you above shows how to create a basic CRUD microservice with Net Core. I would give that a go, and see how it goes, then you can move to CI/CD for Azure!
Moving on from just a simple CRUD API, you can introduce WS connections with event driven updates (server to client) rather than asking for new data.
An Architect that I once worked with (a genius guy) told me never to be too reliant on a 'Framework' - they're cool when they are doing well, but a great application should be flexible to change, so I wouldn't rely too much upon a 'framework' but that was just his opinion.
Try API Platform - dockerized, but deployable to php hosting (based on Symfony), generates react-admin based admin and optional web/mobile clients (IMHO the weakest parts of this project), openAPI (swagger) docs, easily usable with graphQL ... just try.
Building portfolio with Laravel isn't a good idea. Use Gatsby - you can use graphql (WordPress, contentfull) as a source, generate static site.
I read the article on Microservices on Martin Fowler's page and find it quite interesting. Now I plan structuring an E-Commerce Web Application as proof-of-concept and am wondering if my concept is considered being a Microservice architecture.
The architecture consists of 3 components:
a javascript based single page application, which sends AJAX requests to
a server with a REST API which feeds JSON data received by calling other services (I think you call this behaviour API Gateway)
3 services: CatalogProvider, CustomersProvider, CheckoutProvider
For now the services all are API endpoints of a Magento (PHP) Shopsystem. In future I plan to then swap the providers with other systems.
So my questions are:
MS are considered to be 'independently deployable'. I understand that in the world of JAVA we are talking about one JAR- or WAR-file, but how is a PHP service 'independently deployable'?
Does my concept NOT follow the principles of a MS architecture, because the providers are all part of one big (Magento) system?
Thank you for reading. I'm happy for any suggestions.
There is nothing that says you architecture is not a MS architecture just because you're using magento and PHP. But, you have to consider a few things:
Think in terms of always being able to rewrite any of the services in any language and deploy somewhere the total system should just continue to work.
If your services are just transformation/interface very tightly linked to magento and you cannot simply rewrite them in java/C#/ruby easily, then I guess you do not have a MS architecture.
For PHP deployable artifacts, you typically have some packaging or versioning strategy around your service. Even though "deploy" in PHP is typically just swapping a folder of .php files. And you should not really share code/config between different services. You can even look at deployment tools for PHP if you want to take an extra step.
As to the microservice architecture there is SRP principle.Single Responsiplity principle.Each service has own unique responsiplity.DB schemea shouldbe decomposed also.Exporting services as rest inside a monolithic app not convert a monolithic app to micro service application.
I'm trying to develop an application server that will deliver content to a core group of websites, as well as provide third party services to other websites that also want to use this content. The app server will be hosting web services for these core + 3rd party websites. Authentication and all that comes into play. The data itself will consist of millions of records.
These records will come from a variety of sources: APIs, RSS feeds, REST services, etc. This app server will essentially collect this data on a routine basis, and update the database with this new information. This data will then be shared via some sort of web service (most likely REST) to the core websites and 3rd party websites.
FYI, I'm making a distinction between core sites and 3rd party websites because there will be different access levels, i.e. a core website will have more access than a 3rd party website.
All that said, I'm trying to make the best decision on which framework to use. At the risk of losing all credibility, I currently have a ton of this code written as a wordpress plugin. What started as a one-time site evolved into several sites, and some homebrew hacking to make my outdated infrastructure work across multiple sites.
I'm looking to migrate all of this to a new application server, with a solid framework.
Since everything is written in PHP, obviously I'm tempted to do the migration in PHP. However, I'm considering Python because of its powerful ability to manipulate data. I don't know if it's worth the hassle, though, of rewriting a lot of code in Python.
Could anyone give me some tips on what I should do? I'm really looking to clean up a big mess more than anything, and would like a framework to encourage some solid programming conventions.
All of the frameworks that you mentioned are capable. Pick one in the language that you know the best and use that.
I agree with #gpojd's opinion. All of them that you mentioned are fully capable.
However, it looks like they are too powerful for your job. Because they are all full-stack MVC-like web frameworks shipped with ORM, Template engine, URL redirections and i18n supports.
So, I suggest you to use more lighter/thinner frameworks. and if you don't mind choosing any programming languages, please check out the below frameworks.
Python
Werkzeug : http://werkzeug.pocoo.org/
Javascript
node.js : http://nodejs.org/
with Stylus framework : http://expressjs.com/
I'm writing an iPhone app as a hobby project and it will need a web service to provide it with data. It's not very different from what I do at work, but at work I only write views and controllers. Someone else is responsible for writing the model and usually the clients provide the web service.
I have done some web programming before, back when everyone were using MySQL and PHP, so my skills are a bit outdated, but I'm confident that I would be able to pull it of using the techniques I already know. However, I don't want to waste my time using obsolete tools. I've figured out that the state of the art would be to write a REST API. I was thinking that there should be some pretty good frameworks out there that pretty much just gives you a REST API with CRUD functionality as soon as you've defined a model.
I guess my question is: What would be the fastest way to get a REST API up and running? I really just want to focus on writing the iPhone app and not spend too much time on this API. It would be great if I could get web administration and revision history too. I should also add that the API isn't supposed to be public, so support for authentication would be great as well.
Just to be clear. I wouldn't mind a PHP framework. In fact it could possibly be better since I know that my current hosting supports it.
EDIT:
The links below which apparently were good for 3 years are no longer working so I went and found a couple of new tutorials that I think are going to stick around for a while. These are on the Ray Wenderlich site, a very well respected ios dev tutorial site. The first article actually references the broken links below but it is complete within itself:
How To Write A Simple PHP/MySQL Web Service for an iOS App
and the second one has a little twist to it. It used parse.com on the backend and AFNetworking. Both of which are quite excellent.
How To Synchronize Core Data with a Web Service – Part 1
I have fixed the broken links below by finding the articles in the way back machine. People seem to like the links so I will keep them. The links above should provide more food for thought.
I am doing exactly the same thing with my iphone app. I found this article on building a RESTful API in PHP:
https://web.archive.org/web/20130910164802/http://www.gen-x-design.com/archives/create-a-rest-api-with-php/
and there is also a followup article here:
https://web.archive.org/web/20130323001500/http://www.gen-x-design.com/archives/making-restful-requests-in-php/
with a link to source code at the bottom of the article.
I have programmed a REST API in ZEND Framework using the Zend_Rest_Controller, on the iPhone I used ASIHTTPRequest. My experience with both where good. At the beginning I had some trouble setting up ZEND and connecting it to mySQL, but once I figured out how to do it I was able to write the API very quickly. I can share more information with you if you have any further questions.
EDIT: There seems to be no official documentation on Zend_Rest_Controller. This link describes how to use it to create your API. You simply have to disable rendering in the init() of your subclass and implement the methods for each REST call.
Just to let you know:
I ended up using Ruby on Rails.
EDIT: Since this answer has been downvoted for not providing the reason behind choosing Ruby on Rails and also no instructions on how to write a REST API with it, I thought I would give you my motivation and some simple instructions.
I started reading a book about Ruby on Rails and realized that all I needed to do was to use scaffolding and I got a JSON REST API for free.
Here's a good guide to get you started: http://guides.rubyonrails.org/getting_started.html
When you have your Ruby on Rails environment up and running, creating your REST API isn't harder than running:
$ rails generate scaffold Post name:string title:string content:text
(Example from the above link.) I also found that Rails is very easy and free to deploy to heroku, which meant that I didn't have to pay for hosting for my very basic, low traffic, REST API. There are many other reasons why I am very happy to work with Ruby on Rails, but that's beyond the context of this question.
I followed a quite simple tutorial for creating RESTful APIs with PHP:
Corey Maynard - Creating a RESTful API with PHP
The main concept includes:
one abstract class that handles the parsing of the URI and returning the response, and
one concrete class that consists of just the endpoints for the API.
What about Python?
I'd use Python, Django and Piston.
I'd generate Django models from your
existent DB using inspectdb.
Add the Django admin to your models.
Add Django Piston to your app.
Profit.
With no experience with Python or Django probably it'll take you a day to develop this solution and all code is unit tested and proved to work.
If you want to use PHP I recommend using the CodeIgniter framework with Phil Sturgeon's REST server:
http://net.tutsplus.com/tutorials/php/working-with-restful-services-in-codeigniter-2
https://github.com/philsturgeon/codeigniter-restserver
Checkout the following PHP class that follows MVC.
http://www.phpclasses.org/package/5080-PHP-Implement-REST-Web-services-servers.html
Hope this helps.
If you already know PHP, there's nothing wrong with a PHP/MySQL backend. You can send all responses in iPhone-compatible plist xml format, and instantly turn the response into a NSDictionary/NSArray/NSNumber data structure with this short snippet of code:
NSString *response = [request responseString];
NSData* plistData = [response dataUsingEncoding:NSUTF8StringEncoding];
NSPropertyListFormat format;
NSString *errorStr;
NSDictionary* plist = [NSPropertyListSerialization propertyListFromData:plistData
mutabilityOption:NSPropertyListImmutable
format:&format
errorDescription:&errorStr];
I also use the ASIHTTP package for forming URLs, sending asynchronous requets, and receiving the responses, I highly recommend it:
http://allseeing-i.com/ASIHTTPRequest/
You should use whatever languages you are comfortable with for the web service. Any language that can formulate REST responses to requests is fine.
That said, if you want to get something running quickly, I suggest using Python on Google App Engine. It's free and you can use Java instead of Python if you so desire. App Engine supports authentication using OpenID and/or Google Accounts (not sure if they're mutually exclusive) so that should make things easier to code.
As far as making the requests on the iOS device, I suggest using ASIHTTPRequest.
Another option is restSQL, an ultra-lightweight persistence framework. See http://restsql.org. It supports MySQL and PostgreSQL and runs in a standard Java EE container, e.g. Apache Tomcat.
restSQL is a very unconventional data access layer. restSQL is not an object-oriented view of the database. It presents flat or hierarchical "views" of relational database tables. These views are query-able and updatable through a simple REST-based HTTP or Java API. The HTTP interface is based on REST principles, which use HTTP’s built-in features, rather than abstracting away from them.
You want a 'REST API with CRUD functionality' and that's exactly restSQL's sweet spot. You could do this with no code. Simply define your SQL Resources via XML files and start doing HTTP calls against them with full CRUD capability.
I know questions like this have been asked numerous times, but not quite this one. Forgive me if I overlooked an obvious duplicate.
In the core of many of my web applications is a self-written user/session management class that in its origins dates back to 2002.
I have decided that it is time for a fundamental re-write or, preferably, the introduction of a ready-made standard library.
My requirements for that library would be:
Object oriented, clean, excellent code
Full session management: Wrapper to session_start() and consorts
Would ideally provide various storage methods (PHP Standard /tmp, database based)
Would ideally be able to connect to different types of user data storage, but mySQL will do fine
Would ideally provide convenient functions for supporting OpenID, but that's a fancy thought, no requirement right now
Methods: Verify session, get user data, get session data, log in user, log out user
Settings: Session lifetime, password encryption
Must be Open Source
And if it's very generic, a user management API or a generic connector to the user management of the surrounding application would be nice:
Create/Update/delete user records
Fetch and modify data of currently logged in user
this is so basic, and so security relevant, that I would expect that there is a standard solution to this, however I don't know of any, and all the big CMSs and blogs seem to be rolling their own.
My two questions:
Do you know such a component as a generic, stand-alone library?
Could somebody with deep knowledge in Zend Framework tell me whether it is possible to use Zend_auth and/or Zend_session standalone, at the core of a big application that has otherwise nothing to do with ZF, without running in to trouble?
May I suggest the authentication library that I have written? It is a generic library (not written for or part of a framework): http://ulogin.sourceforge.net
Could somebody with deep knowledge in Zend Framework tell me whether it is possible to use Zend_auth and/or Zend_session standalone, at the core of a big application that has otherwise nothing to do with ZF, without running in to trouble?
I don't have deep knowledge of the Zend Framework, but I have used various components (e.g. Zend_Search) without creating a Zend_Application object or using the MVC framework and I am sure the rest of the library is also designed to be totally modular. Last time I dug though the Zend_Session code, I didn't find any includes outside Zend/Session/. A quick google seemed to confirm this for Zend_Auth, along with the Zend FAQ which states:
Is ZF a component library or a framework?
Simple answer: both. Zend Framework provides all the components required for most web applications in a single distribution. But Zend Framework components are also loosely coupled, making it easy to use just a few components in a web application- even alongside other frameworks! Using this use-at-will architecture, we are implementing features commonly found in more monolithic frameworks. In fact, we are currently working on a tooling component for the 1.8 release that will make it simpler to build applications using ZF components, yet will not sacrifice the use-at-will nature of existing ZF components. It's a testament to the use-at-will architecture of Zend Framework that the tooling component itself can be used standalone.
The only thing I had to do when not using Zend_Search with the MVC framework was add the directory where you installed the Zend Framework to the include path due to the includes in the Zend library. The documentation doesn't document the includes you need when not using the Zend Autoloader, but as everything uses the PEAR class naming scheme, it is easy to deduce from the class names you are using. (so the class Foo_Bar_File would require you to include Foo/Bar/File.php )
There are several OpenID libraries available.
http://wiki.openid.net/Libraries#php
For the rest you might as well roll your own, since figuring out someone else's library would probably be more trouble than it's worth.
My understanding is that there is no standard library because there is no standard definition of what a user is.
In some of my applications, users simply log in to do stuff. In others, users are part of a company and their permissions and data access are limited by the limits of that company and the subscription level paid for by the company. In other applications, some users are admins with access to everything, some users are admins with access to some data (row level), and other users are the clients of those admins, with access only to their own data. Some users are tied to firms/companies/customers, other users are not. Some users are just a username and password, others are a large object graph with clients, order histories, report preferences, comments, etc.
Maybe I'm wrong and there's some clean way of abstracting all of those requirements into a system that doesn't require five layers of subclassing and a thousand DB hits to log someone in. I haven't found it though.