Cross Platform Framework - PROCEED-Labs/proceed GitHub Wiki

One main goal of PROCEED is to support as many platforms as possible. And IoT vendors should be able to easily integrate their platform if it was initially not supported.

We decided to use JavaScript as the main underlying programming language for PROCEED. It's a popular and modern language, gets continuously improved and is interpreted by a runtime environment. So the only requirement is to have a runtime available. But most modern Browsers do have one included and there is also Node.js that runs on many platforms.

The only problem with this approach is that PROCEED needs to access functionalities like data and network access, that are not standardized in the ECMAScript specifications for JavaScript. We call them the native functionalities. Node.js has its libraries for this, but the Browser has some other. That's why PROCEED included an abstraction layer to solve this problem -- the dispatcher.

The Dispatcher is an abstraction layer and library on top of the native libraries.

Technical Overview

The Dispatcher is a two-part module that exposes some native functionalities of the underlying system to the universal part of the PROCEED engine.

• The Dispatcher consists of a native part that is written in any language the system supports and that enables us to use native funtionalities (e.g. Swift on iOS). It needs to implements some predefined functions to use the required system resources.
• The other part of the module is written in pure Javascript without any native dependencies (so it can be included in the universal part of the PROCEED engine). It essentially provides well defined libraries for other modules to use, so that they don't need to use native functionalities directly. When some module calls a library function, it sends a task message to the native part of the Dispatcher, where it is executed and returns the result.
• The communication between the two parts of the Dispatcher happens via bi-directional IPC messages. This is done by first determining in which (class of) JS-Environment the universal part is operating in and then setting the IPC method (which must be provided in the JS-Runtime). For example on iOS we can use the WKScriptMessageHandler protocol or on NodeJS starting a child process automatically sets up an IPC with the new process.

In order to reach the goal of having isomorphic PROCEED engine JS files on all platforms and thus having the benefits of easy development and updates, we cannot use any native modules, whether for browser or NodeJS, that aren't available in all JS environments (e.g. http, fs, XMLHttpRequest, ...). This also means, that we can't use any npm packages that use those modules (e.g. express). Instead, the PROCEED engine can only call the functions which are provided by the Dispatcher.

Overview

As a developer or user of the PROCEED engine you will most likely only concern yourself with the functions of the JS part of the dispatcher module, as those are the ones which you will use in your code. The following is a list of all the functions of the @proceed/system module, which is the name for the JS part of the

While it is technically feasible to add other functions to this set later on, it is not recommended in practice, since it would require to update every native dispatcher part for every language / system that we or other vendors have written so far. With that in mind, the general advice is to only use the given functionality and to not propose changes to the dispatcher if they are not absolutely necessary.

Module: @proceed/system

• http()
  • get(path, options, callback)
  • put(path, options, callback)
  • post(path, options, callback)
  • delete(path, options, callback)
  • request(url, options, callback)
• data()
  • async read(key, options)
  • async write(key, value, options)
• utils()
  • makeLogger( processInstanceID )
  • makeConsoleLogger( processInstanceID )
• capability()
  • startTask(description, args, callback)

This list is meant as a general overview and first introduction into the dispatcher module and how it works. For a full and more extended up-to-date API reference, please look at the API docs.

Example Code

The following example uses both the http and data modules to answer a request with the requested process plus incrementing the request counter on the process. It showcases how to use the modules with the ES7 async/await pattern.

const system = require('@proceed/system');
const http = system.http();
const data = system.data();

http.get('/engine/process/:id', { cors: true }, async (req) => {
  const value = await data.read(req.params.id);
  const process = JSON.parse(value);
  process.requests++;
  const newProcess = JSON.stringify(process);
  await data.write(req.params.id, newProcess);
  return newProcess;
});

Native

The native part of the dispatcher is the realization of the functions for the libraries in the universal part. It will receive tasks messages over an IPC channel. This means that the native code side of the PROCEED engine can be completely can be completely different for every platform. For an example of this concept in practice, please refer to the PROCEED repo where you will find some implementations for different systems.

There are no assumptions HOW the functions are implemented, it is up to whoever will implement them. The only requirement is that the Dispatcher libraries are somehow realized in the native part. Here is the list of the required functionalities (since the Dispatcher libraries in the universal part offer some more user-friendly function, it is not the same list):

• IPC

  All calls to the native dispatcher have a unique task identifier (taskID) attached to them (usually as the first parameter). This taskID is used to identify the task and match the response to the request.

  • serve(method, path, options) Provide HTTP Server capabilities

    • method: String One of the following HTTP methods: get, put, post, delete

    • path: String Named route (e.g. /engine/:id)

    • options: Object

      • cors: Boolean Whether CORS should be enabled
      • files: Boolean If true, uploaded files should be processed and saved to the req object as the files property

      Whenever there is a request on the given path, the system is expected to delegate that request to the universal part via the IPC means. The request object has to be of the following form (for explanation of these parameters look here):

      • req: Object
        • baseUrl: String
        • hostname: String
        • ip: String
        • method: String
        • params: Object
        • path: String
        • [files: Array] The optional array of uploaded files if options.files == true

  • respond(res, reqID)

    After a serve() call all requests are delegated to the universal part which processes the request and sends the response back using this function. The response has to be sent to the initial client who started the request.

    • res: String The response that is set by the universal part
    • reqID: String The taskID of the serve() call to identify the response client

  • read(key) Read the value for the given key

    • key: String The key of the form table/id e.g. execution/1

  • write(key, value) - key: String The key of the form table/id e.g. execution/1 - value: String The value to store for the key

  • capability(description, taskArgs) Perform some kind of (possibly IoT-related) task (e.g. taking a photo). - description: String A [semantic] description of the task to perform - taskArgs: Array The arguments for the function that performs the task

As a reminder: There are no assumptions how this is implemented on the native side. So the data can be stored in a key-value database like redis, a SQL implementation or just a filesystem.