Design

Overview

We intend to develop a web application to expose a variety of HGI processes to end users for self service. The entire application stack needs to be designed and built against the following requirements:

• Separation of concerns
• HTTP and REST best-practices
• Multiple, pluggable backend services

Separation of Concerns

All aspects of the system should be highly modular and, potentially, swappable with compatible components. This has the obvious potential for better maintainability and testability.

HTTP and REST

Communications should respect the HTTP protocol and follow the RESTful model. In particular, the use of hypermedia to create a navigable API (i.e., HATEOAS) with machine-readable application semantics. That is, a suitable client can access the API and understand what the data represents, so it can render a meaningful representation for the end user.

Distributed Services

The processes that need to be exposed cannot be fully known ahead of time, as such the system needs to allow additional services to be registerable and "just work", as though -- as far as the end user is concerned -- they were completely integrated into the system.

Frontend Client

The frontend will be a browser-based client to the HGI process APIs, which are provided by the backend. It can be thought of as an API browser -- not dissimilar to how Firefox, or whatever, is an HTTP browser -- although it will be set to a fixed backend (like a web browser that can only access one domain).

The frontend will be served by a frontend server, completely orthogonal to the backend. While it will have the appearance of a single-page application, the URL will be used statefully and correspond to the backend end-point. For example:

abc123 backend:
http://abc123.backend/foo/bar

Frontend URL:
http://frontend/abc123/foo/bar

Architectural goals:

• Isomorphic (desired)
• Responsive (essential)

The frontend should be able to consume at least the following media types, in order of preference:

• A JSON-based hypermedia format
• XML with XLink, etc.
• XHTML

n.b., Due to the semantics of XHTML being deficient (e.g., can't PUT, etc.), HTML will be considered "read only mode", as far as practically possible.

Security

Authentication and authorisation are ultimately the backend's problem, but the data must be delegated to the backend from the frontend.

The frontend will sit behind a Shibboleth gateway. Upon authentication, a token can be created that serialises appropriate user information (e.g., username, etc.), to be passed around for subsequent processes to consume.

A token can be passed using HTTP authentication, using the "Bearer" keyword. Tokens should be decomposable into a HTTP basic authentication username and password pair, for backwards compatibility.

Don't go to the trouble of, say, OAuth.

Backend

All APIs must use a hypermedia model. All the better if a consensus is reached (e.g., on media type, etc.), to simplify coding of frontend clients. Multiple media type support -- as described above -- is to be encouraged, largely to force API developers to separate out their serialisation logic.

Central API

The backend API acts as a central controller/gateway to service APIs. Therefore, it must have knowledge of them (their name and URL), so they can be routed, as well as allowing their administration. Likewise, the potential to be used by multiple clients (e.g., web-based, native, etc.) should be encapsulated (again, as a gateway to the client's server).

Service APIs

These can be arbitrary, just so long as they serve data using recognised hypermedia.

Initial services:

• User, group and quota administration
• Project administration
• HPC cluster metrics (e.g., Lustre usage, CPU efficiency of jobs, etc.)

Services should be able to standalone; they are only proxied through the central gateway for convenience and security.

Diagram