PROCEED Engine API ‐ REST, GraphQL, MQTT Interface Overview - PROCEED-Labs/proceed GitHub Wiki

Currently, the PROCEED Engine provides the following interfaces: HTTP (REST), self-implemented GraphQL, MQTT. The interfaces are mapped in a consistent way between all technologies.

REST API

GraphQL

The Engine enables an self-implemented version of an GraphQL interface on all of our REST APIs. In the path, the client can add an entries query parameter to specify the properties that you want to retrieve in the response - resulting in kind of an GraphQL API.

Example: GET /process/{definitions-Id}/instance/{instanceId}

The complete response looks like the following:

{
  "processId": "<process-id>",
  "processVersion": <number: date-time in ms since 1.1.1970>,
  "processInstanceId": "<random uuid>",
  "globalStartTime": <number: date-time in ms since 1.1.1970>,
  ...
  "tokens": [{
      "tokenId": "<random 7 char ID with numbers and letters>",
      "currentFlowElementId": "<flow-element-id>",
      "currentFlowElementStartTime": <number: date-time in ms since 1.1.1970>,
      "localStartTime": <number: date-time in ms since 1.1.1970>,
      "localExecutionTime": <number: time in ms>,
      "currentFlowNodeProgress": <number: 0-100>,
      ...
    }
  ],
  "variables": {},
  "log": [
      {
        "tokenId": "_safdss...",
        "flowElementId": "<flow-node-id>",
        "machine": { "id": "<machine-id>", "ip": "<ip>", "name": "<machine-name>"},
        "startTime": <number: date-time in ms since 1.1.1970>,
        "endTime": <number: date-time in ms since 1.1.1970>,
        ...
      },
  ],
  ...
}

Adding the query GET /process/{definitions-Id}/instance/{instanceId}?entries=processId,processInstanceId,log to the request URL will return:

{
  "processId": "<process-id>",
  "processInstanceId": "<random uuid>",
  "log": [
      {
        "tokenId": "_safdss...",
        "flowElementId": "<flow-node-id>",
        "machine": { "id": "<machine-id>", "ip": "<ip>", "name": "<machine-name>"},
        "startTime": <number: date-time in ms since 1.1.1970>,
        "endTime": <number: date-time in ms since 1.1.1970>,
        ...
      },
  ],
}

It is also possible to filter nested objects or objects inside arrays by defining the required attributes for the nested object in parentheses following the name of the nested object. For example: ?entries=processId,processInstanceId,tokens(tokenId,currentFlowElementId).

{
  "processId": "<process-id>",
  "processInstanceId": "<random uuid>",
  "tokens": [{
      "tokenId": "<random 7 char ID with numbers and letters>",
      "currentFlowElementId": "<flow-element-id>",
    }
  ],
}

MQTT

  • If configured, an Engine sends monitoring data to topics and enables the usage of the REST API via MQTT
  • The Management System receives monitoring data via MQTT. Currently it does not send data to MQTT.

Enabling the MQTT Interface in the Engine

To enable MQTT messaging in the engine a serverAddress has to be set in the Engine config file

{
...
"messaging": {
  "serverAddress": <String>, // url or ip (and port) of a mqtt broker, this will enable the base MQTT functionality on the Engine
  "username": <String>, // optional, needed if the MQTT broker requires some kind of user authentication
  "password": <String>, // optional
  "baseTopic": <String>, // optional, possible topic-name which is prepended to the usual topics
}
...
}

REST API on MQTT: Mapping of the HTTP Interface to MQTT

We support the same REST API also via MQTT. Every REST path is a topic, the message contains the request method and the response is sent on the same topic:

  • Engine needs to subscribe to every topic, i.e. Networking path => based on the configuration it is <mqtt-server>/<base-topic>/proceed-pms/engine/<engine-id>/api/<Networking-Path>
  • Engine receives in this topic the request data, e.g. {"type": "request", "id": "<short-id>", "method": "POST", "body": {...}, "query": {<query-object>}, "page": <number> }
  • Engine sends response to same topic: {"type": "response", "id": "<short-id>", "statusCode": <number>, "body": {...}, "page": <number>, "pageCount": <number>, "statusCode": <number> }
  • If there is some error the response differs: {"type": "response", "id": "<short-id>", error: <string>, "statusCode": <number>}

=> id is necessary and makes the request unique, because multiple requests at the same time are possible and without an id the response could not be mapped to a request

Hint: the REST API is only available for MQTT Engine configuration and not for process level configuration.

Monitoring data: publishing of Runtime Data

The Engine can continuously publish data about the process executions to an MQTT topic for monitoring purposes. There are two options to configure this: at the Engine configuration level (global for all processes) and inside a process definition (only for one process and all its instances).

Engine Configuration level

If MQTT is configured at Engine level, the monitoring data will be sent for all processes and all instances that are running on the Engine.

BPMN Configuration Level

MQTT can be configured inside the meta-data of a BPMN process:

image

In BPMN XML it serializes like this: image

The monitoring data will be sent for all instances of the specific BPMN process and not for any other processes. The data is only valid for the runtime of the process instance.

Transmitted Monitoring Data and their Topics

After configuring MQTT on the Engine or process level, the following monitoring data is sent to the configured <mqtt-server> or <BPMN-MQTT-Server-URL> server:

Engine Status:

  • content: the status of the PROCEED Engine that it is alive
  • has Last Will Message activated: if Engine goes down, then running is set to false
  • For Engine Level Configuration on topic: <base-topic>/proceed-pms/engine/<engine-id>/status
  • For Process Level Configuration on topic: <BPMN-configured-Topic>/proceed-pms/process/<definitions-id>/instance/<instance-id>/engine/<engine-id>/status
{
"running": <Boolean>,
"version": <String> //version number of the package.json file from the compiled Engine
}

Machine Monitoring Information:

  • content: the monitoring data of the Machine that the PROCEED Engine is running on (see Machine Module and REST API Endpoint)
  • sends the data to the MQTT topic every engine.loadInterval seconds (default 10s, see Configuration Keys)
  • For Engine Level Configuration on topic: <base-topic>/proceed-pms/engine/<engine-id>/machine/monitoring
  • For Process Level Configuration on topic: unspecified

Logging Data:

  • content: the logging data of the PROCEED Engine (see Logging REST API)
  • For Engine Level Configuration on topic
    • standard (engine) and all process logs: <base-topic>/proceed-pms/engine/<engine-id>/logging
    • only standard (engine) logs: <base-topic>/proceed-pms/engine/<engine-id>/logging/standard
    • only all process logs: <base-topic>/proceed-pms/engine/<engine-id>/logging/process
    • logs of a specific process instance: <base-topic>/proceed-pms/process/<definitions-id>/instance/<instance-id>/logging
  • For Process Level Configuration on topic: <BPMN-configured-Topic>/proceed-pms/process/<definitions-id>/instance/<instance-id>/logging

Process definition:

  • content: the process specification, its meta information and a list of included steps
  • the message is retained for new subscriptions
  • For Engine Level Configuration on two topics:
    • <base-topic>/proceed-pms/engine/<engine-id>/process/<definitions-id>/version/<version-id>
    • <base-topic>/proceed-pms/process/<definitions-id>/version/<version-id> -> last updated by any engine (decentralized approach)
  • For Process Level Configuration on topic: <BPMN-configured-Topic>/proceed-pms/process/<definitions-id>/version/<version-id>
{
processId: <String>, (aus BPMN)
processVersion: <number>, (aus BPMN)
processName: <String>, (aus BPMN)


timePlannedOccurrence: <String>, (Bsp: 2023-05-23T00:00:00+02:00, aus BPMN)
timePlannedDuration: <String>, (Bsp: P1M1DT2H2M, aus BPMN)
timePlannedEnd: <String>, (Bsp: 2023-05-25T00:00:00+02:00, aus BPMN)
costsPlanned: <String>, (aus BPMN)
orderNumber: <String>, (aus BPMN)
orderCode: <String>, (aus BPMN)
customerId: <String>, (aus BPMN)
customerName: <String>, (aus BPMN)

processSteps: [  (aus BPMN)
	{   
                stepId: <String>, 
		name: <String>, 
		taskType,  
		description, 
		milestones,  
		timePlannedOccurrence: <String>, (Bsp: 2023-05-23T00:00:00+02:00)
		timePlannedDuration: <String>, (Bsp: P1M1DT2H2M)
		timePlannedEnd: <String>, (Bsp: 2023-05-25T00:00:00+02:00)
		costsPlanned: <String>, 
		property1 .. propertyX 
        },
	{ ... step2 - stepX ...},
],

bpmnString: <String>
}

Hint: processId is actually the <definitions-id> of a BPMN file

Instance data:

  • content: the instance and token object of the running process (see respective wiki page)
  • is send every time on token change
  • For Engine Level Configuration on two topics:
    • <base-topic>/proceed-pms/engine/<engine-id>/process/<definitions-id>/instance/<instance-id>
    • <base-topic>/proceed-pms/process/<definitions-id>/instance/<instance-id> -> last updated by any engine (decentralized approach)
  • For Process Level Configuration on two topics:
    • <BPMN-configured-Topic>/proceed-pms/process/<definitions-id>/instance/<instance-id>/engine/<engine-id>/instance/<instance-id>
    • <BPMN-configured-Topic>/proceed-pms/process/<definitions-id>/instance/<instance-id> -> last updated by any engine (decentralized approach)
⚠️ **GitHub.com Fallback** ⚠️