REST API Design - vidyasekaran/current_learning GitHub Wiki

Software Architecture : REST API Design - The Complete Guide - Udemy - Memi Lavi

What is API?

Application programming interface -

A set of clearly defined methods of communication among various components.

Interface part - User interface and Application Programming Inteface

UI is exposed for humans and they understand it where as API's is exposed for the software programs and the programs understand it well.

Example :

a. UI - Any Browser based Portals or thin client. b. API

GET /account HTTP/1.1 Content-Type: application/json Host: apigateway.us-east1.amazonaws.com

User Interface is a visual representation of the content that you are looking at

Same is true for application

How it works

User Mgmt Service -----get users orders-----> API | orders Mgmt service

                                          (Exposing an API)

...

API Types

Operating system

My App to access file system, network device it need to ask OS

My APP need to ask Operating System API such as Win32API for filesystem and network device.

Library

My APP uses Logging Library (this library expose "Logging API" which is used by my app. both these are within same process)

Remote

                        |-------Proprietary Protocol-----|

User Component |Orders Proxy|--------------Network-----------| Order Component

User Component and order component should use same platform.

Example : DCOM, .Net Remoting, Java RMI Uses proprietary protocol.

Web API

My Web App uses Internet and access other Web app

My WebApp |-------comm. using standard protocol--------> WebApi | Weather Web App

Any platform / OS / Language

My Webapp can be .Net and Weather WebApp Java

Importance of API

Extend reach of our API Monetize

Why Do you need a Well-Designed API?

i. Your developers are customers ii. They have alternatives iii. Your API should be easy to use and consistent

Web APIs

a. WebAPI's are Platform Agnostic

.Net app can call Python

b. Standard protocols over HTTP

c. Usually request/response based

HTTP request and HTTP Response

Web API's are mainly differentiated by

  • Request Format
  • Request Contents
  • Response Format
  • Response Contents

Various Types or Web APIs

  • Soap (Simple object access protocol) -- XML based, -- RPC style request-response (client call very specific method) -- Extensible meaning it can be extended for routing,authentication,federation

<soapenv:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ser="http://www.herongyang.com/Service/"> soapenv:Header/ soapenv:Body ser:GetStockPrice ser:StockNameIBM </ser:StockName> </ser:GetStockPrice> </soapenv:Body> </soapenv:Envelope>

REST (Representational State Transfer)

Designed in 2000 by Roy Fielding

URL + JSon based - you send url and get json

Defacto standard for APIs

Message Style - we just calling URL and passing message, the URL endpoint should be able to understand what to do with this message and then pass it long and return us a response.

NOTE: We are not calling any specific method here like SOAP.

GraphQL

facebook developed in 2015, Enables very flexible querying, updating and subscribing to data changes Json in and out

Request

query{ -- query posts{ -- what we query ie post id -- fileds we want title body author{
name -- we need name alone in author | } }

Response

{ "data" :{ "posts" : [ { "id" :"1", "title":"why ....", "body":"this is published...", "author" :{ "name":"Memi" } } }

Author name is stored in someother table.

GraphQL

Very flexible, requires upfront dev effort, require perf optimization, gain groud (ie github - sees developers need only specific fields not all)

gRPC

google developed in 2015, using HTTP/2 we can implement full duplex uses Protobuf (message format we can use generators to generate clients and servers) supports bi-directional * streaming messaging

service HelloService{ rpc SayHello(HelloRequest) returns (HelloResponse); }

message HelloRequest{ string greeting=1; }

message HelloResponse{ string reply=1; }

serialization/deserialization is taken care by it quite performant, not widely used yet, requires specialized libraries at both ends

what is REST?

Authentication and Authorization

Your api is not public only authorized users need to access it Authentication - who is the user Authorization - what he is allowed to do

API itself wont perform authentication and authorization

OAuth2 does it -

Client App - Resource Server (API) - Authorization server (github,feedly etc)

  1. client app access api without security creds to resource api server

  2. resource api server redirects client app users to authorization server

  3. authorization server asks user to grant permission and returns token to client app.

  4. client app sends access token to resource server (api) and client app is allowed access.

    ----------- Redirects user to Client App | ----------------------> | Authorization Server -----------| <---------------------- | | Returns access token
    | access API | send access token
    anonymously | |

Resource Server (API)

App Registration

  • Authorization Server should be familiar with Resource server(API)
  • Resource Server must register itself with Authorization server

App Registration with github (https://github.com/settings/applications/new)

User to provide basic detail and Authorization callback URL (This url the authorization server will call after authorizing the user) Authorization sever provides - ClientId and Client Secret.

client should pass ClientId and Client Secret to authorization server so that Authorization server identifies the client app.

the JWT

json web token

Once authenticated by authorization server you can check role and user identify in table and verfiy whether he is allowed aceess or not.

this token is returned by Authorization server to resource api server which contains data the resource server needs in order to authenticate the user (the data contain "who is the user is and what he is allowed to do" )

jwt sections - header, payload, Signature

header - type of token (jwt)

{ "alg":"Hs256","typ":"JWT"}

Payload - contain data on the user and role (Have a role based access in API and you have a role in payload). Once u know how is the user then you can have users role in database table to check and allow access...

{ "sub" :"234", "name" :"john", "admin" :true

}

Signature:

3 parts , Base64, concatenated with header.payload.secret

based on secret we can make sure the data is tampered or not

the Oauth2 flow

Client App sends "access token" - the resource server api checks the signature and reads the payload.

JWT to be sent to API with Authorization : bearer header if an API requires JWT basedon which authorization is performed then

JWT should be sent with Authorization: bearer header

should be sent as header and not best practise to send it via body or request parameter

GET /resource HTTP/1.1 Host: Server.example.com Authorization: Bearer mF_9.B5f-4.1J9M

REST - Representation state transfer

REST API enables transfer of representation of a resource's state

Client request server the resources state i.e order 17 Server sends response with the resources state representation for order 17

State:

  • the current resource properties
  • result of an action on the resource

example:

Client request to delete resource "order 17" Server responds with resource's state "Resource Deleted"

rest uses http and how request and response look like

client GET /api/order/17 server

server - 200 OK, JSON (resposne with resources state representation)

How REST request/response Structure look like

Request Structure

Method --- Http verb (get,post,put,delete) URL --- location of resource + parameters Header ---metadata of the request (User Agent...) body -- contents of the request (optional)

GET /api/v1/entities/17 User-Agent: Postmanerun... Accept: / Cache-Control: no-cache Postman-Token: Host: ... Accept-Encoding: gzip Connection: keep-alive

Response Structure

Status code -- 200 (Sucess), 404 (not found) Headers --- meta data of the response (Content type...) Body ---- Contents of the response (optional)

HTTP/1.1 200 (200 sucess) status: 200 Date: Fri, 02 Aug 2019 Content-Type: text/plain Transfer-Encoding: chunked Connection: Keep-alive access-control-allow-origin: * vary: accept-encoding

{"orderid" :"17","orderDate" :"12.12.218", "items" : ["id" :"6"...]} --> body

Method - GET URL - /api/v1/entities/17

Header

User-Agent: Postmanerun... Accept: / Cache-Control: no-cache Postman-Token: Host: ... Accept-Encoding: gzip Connection: keep-alive

URL Structure

Self Explanatory Consistent across the API Predictable

  1. Domain Name

-This is the 1st part of the URL -Usually contain FQDN (full qulified domain name) or server name

GET https://api.ebay.com (FQDN)

-Protocol should be HTTPS

  1. API word (comes after domain name)

ex : https://www.myapp.com/api (tells this is an api) ex : https://language.googleapis.com/v1/documents

  1. Version (comes after API word)

only natural number and no decimal coupled with v https://www.myapp.com/api/v1

  1. Entity (url provides entity on which api act upon)

Rest deals with resource, or entities

should be one word, no more

never a verb just noun as http verb provides action get/put/post/delete . this is http verb

Entity Examples

GET /api/v1/order
/api/v2/employees /api/v2/travel

  1. ID parameter (optional)

when dealing with specific entiry

/api/v1/order/17

Relevant for GET/PUT/DELETE and not POST as this is new entity

  1. Sub Entity (Optional)

Used for accessing entities that are dependent on other entities

Example : Get all the items of order no 17

we dont want order but the items in the order

Common mistake : /api/v1/ItemsInOrder/17 (not readable and not simple)

Right one:

Sub entity should come afterr the ID parameter

Example : Get all the items of order no 17 /api/v1/order/17/items (Sub entity comes later; Sub entity Items and order main entity Order)

--> API goes to Order 17 and retrieves all items in the order

Same goes for sub sub entity etc

  1. Query Parameters
  • used to query the entities in GET method

why we have query parameter when we have id parameter?

  • Come at the end of URL, after?
  • Concatenated with & /api/v1/orders?fromDate=12/12/2018&toDate=2/2/2019

Fetch all orders that were made between fromDate and toDate

              Query Parameter                ID Parameter
              --------------------------------------------

Location At the end of URL Middle of URL

Param Count 0..N parameter 1 parameter ex fromdate - todate orderId

Example /api/v1/orders? /api/v1/order/17 fromDate=12/12/2018&toDate=2/2/2019

Return Value may return 0..N entities must return exactly 1 enity

If not found Thats fine... Error!

Singular vs Plural

the dilemma 1 : /api/v1/order/17 or /api/v1/orders/17

the dilema 2 : /api/v1/order?user=john or /api/v1/orders?user=john

No concrete answer

Best practice ; always prefer readablelity and ease of use

If u think you will get 1 result back use /api/v1/order/17 (returns single entity) if u think multiple will be returned use /api/v1/orders?user=john (return 0..N entities)

Verioning

Importance

your api will be updated you dont control your clients you can't force version upgrade

Set version support policy

API end-of-life

this api version will be supported for 3 years, so we are giving time for clients of this api to adpot new version.

Add versioning in the API

Version can come in 3 forms

(Contamination of url as per REST Principle)

--In a Header

GET /user/3 HTTP1.1 Accept: application/vnd.myname.v1+json

--In a Query Parameter

http://example.com/users/1234?version=4

which one to use??? Header is the most correct form as it adheres to REST principle.

URL is most common and easiest to implement and URL is self explanatory. DONT User Query Params

REST API - Versioning REST APIs https://www.youtube.com/watch?v=42J4KMHUJjE

URI Path Versioning

Using Path of the URL to indicate versioning Ex: http://localhost:7500/api/v1/customers/1

URI Parameter Versioning:

Version as a Query String Parameter Ex: http://localhost:7500/api/Categories?v=1.1

Versioning with Content Type in Accept Header

EX: GET http://localhost:2300/api/categories/1

Accept: application/app.v1.categories Accept: application/app.v2.categories

Use custom Header to version API calls

Ex: GET http://localhost:2300/api/categories/1 x-App-version: 1.3

We can use date as well instead of a version number. Ex. Get http://localhost:2300/api/categories/1 x-App-version: 2017-08-12

Monitoring

Down time of API cost millions so we use monitoring

  • problems before they occur
  • problems after they occur
  • whats going on with your API (how many users , from where do they come what they do etc).

Importance of Monitoring

your clients expect the API to always work without monitoring it wont happen

so we need to implement monitoring for our API

what to monitor??

How many request per/second? How many request fails? Latency - actual performance of API CPU - cpu utilization

no of users - how many use our system (no of unique users)

of sessions - total number of all user requests

Geographic Distribution - decide how to locallize and where to put resource

RAM - high may cause memory leak

Mocking Tools

-- Beeceptor (https://beeceptor.com/) --Postman

--Application Insight of Azure (signup for azur and use) We can use application insight to monitor our local service can be monitored from mulitple locations etc..

Performance of MS

Use Async operation (apply to files, db,network etc) -calling method of async can continue on other operations

Caching

store freq accessed data close to API usually in memory set expiration and invalidation (deleted) - used to void stale

Rate Limit

Limit max concurrent requests the API handles

we put a Rate Limiter in front of API - this blocks >100 request and then allow

Quota

Limit number of requests a specific client can make

a quoto can be 50,000 erquests per project per day, which can be increased. 10 queries per second per IP address

Advanced Topics in MS

netflix.github.io/genie/docs/3.0.0/rest/#_get_a_job

HATEOAS -

  • Each REST request returns related resources
  • Client shoudl not have prior knowlege about other resource

In the 1st request call's response we send additional links user may want to access so client need not just call any other request.

for genie kind of app - we send job status, contact, and esclation etc in the response

API Gateway & Discovery -

Discovery (Consol)

If 6 services to do interactions between them then we need up with SpiderWeb We use discovery server like Consol where all service registers, then each service to invoke the other service first goes to consul and get the url and invoke directly

API Gateway

Services dont know anybody other than API Gateway, so service to invoke other service use api gateway and api gateway do routing,etcetc

API Gateway Discovery

Man in middle External Service can provide Provide mainly URLs & sometimes LB

  • authz and AuthN
  • Monitoring
  • Routing
  • Policies
  • Load Balacing