Conveyor Service - aegisql/conveyor GitHub Wiki

Conveyor Service (REST + Dashboard)

conveyor-service is a Spring Boot application exposing conveyor operations over REST and a web dashboard.

Module: conveyor-service

Version

This page documents the current 1.7.3 behavior.

Dashboard UI Overview

The dashboard is organized around conveyor inspection and request testing.

Conveyors and Watchers

  • Conveyors tree shows hierarchy and current state (running, suspended, stopped).
  • Watchers list tracks active watches and quick switching to output tabs.
  • Selecting a conveyor in the tree drives context for all workspace tabs.

Conveyor Details

  • Runtime summary fields: Name, Running, MBean, MetaInfo, Snapshot Time, Upload Dir.
  • Meta section lists key/label/product types, labels, and supported value types.
  • Parameter table shows current values, types, and writable/read-only access.

Conveyor Operations

  • Set writable MBean parameters using Set with typed value input.
  • Invoke MBean operations with optional argument input for operations that require one.
  • Stop-related operations display an explicit warning before execution.

Conveyor Parts

  • Main controls: ID, Label, Foreach, request body, and content type.
  • Timing/priority fields: TTL, Creation Time, Expiration Time, Priority, Request TTL.
  • Supports additional custom properties and Submit / Submit and watch flows.

Conveyor Static Parts

  • Main controls: Label, Content Type, Delete Mode, request body (or file drop-in).
  • Timing fields: Priority and Request TTL.
  • Supports additional custom properties for static part placement.

Conveyor Commands

  • Main controls: Watch results, ID, Foreach, and command payload.
  • Timing fields: TTL, Creation Time, Expiration Time, Request TTL.
  • Command actions include CREATE, ADD PROPERTIES, RESCHEDULE, PEEK, PEEK ID, CANCEL, TIMEOUT, and COMPLETE EXCEPTIONALLY.

Admin Actions

  • Upload and replace conveyor JARs when upload is enabled.
  • Controlled Reload and Delete for top-level conveyors with configurable complete-and-stop timeout.
  • Reload/delete requests are scheduled asynchronously and reported in output events.

Help Tab

  • Includes direct links to Wiki, project/service READMEs, authentication docs, container guide, and testing script docs.

Output

  • Output is a shared event viewer for watcher streams, conveyor events, and admin operation status.
  • Each output source opens in its own tab, and tab settings are independent.
  • JSONPath and Cache are tab-scoped controls.
  • Timeline supports Prev, Next, See all, and Clear events.

Run

mvn -pl conveyor-service -Dspring-boot.run.profiles=demo spring-boot:run

Facebook OAuth local run (no JWT issuer required):

SPRING_PROFILES_ACTIVE=prod,facebook \
FACEBOOK_CLIENT_ID=<your-app-id> \
FACEBOOK_CLIENT_SECRET=<your-app-secret> \
mvn -pl conveyor-service spring-boot:run

Default demo credentials:

  • rest/rest - REST API role (ROLE_REST_USER)
  • viewer/viewer - dashboard viewer (ROLE_DASHBOARD_VIEWER)
  • admin/admin - REST + dashboard admin/viewer

Access

  • Dashboard: http://localhost:8080/dashboard
  • Swagger UI: http://localhost:8080/swagger-ui/index.html
  • OpenAPI JSON: http://localhost:8080/v3/api-docs

Key configuration

Main settings are under conveyor.service:

  • upload-dir (default ./upload)
  • upload-enable (default true)
  • oauth2-login-enable (default true)
  • oauth2-resource-server-enable (default true)
  • dashboard.default-watch-history-limit
  • dashboard.default-conveyor-history-limit
  • dashboard.default-ttl
  • dashboard.default-request-ttl
  • dashboard.default-admin-stop-timeout (default 1 MINUTES)

When upload-enable=false:

  • upload and delete are disabled in backend
  • upload/delete API calls return 403 with errorCode=FORBIDDEN
  • dashboard Admin tab hides upload/delete controls and shows an admin message
  • controlled reload stays available

When oauth2-login-enable=false in prod:

  • built-in OAuth2 browser login is disabled
  • JWT resource server auth and HTTP Basic remain active
  • allows replacing login flow with corporate IAM/AIM integration

When oauth2-resource-server-enable=false in prod:

  • JWT bearer-token validation is disabled
  • useful for browser OAuth-only local runs (for example prod,facebook)

In Conveyor Details, Upload Dir is shown as configured (for example ./upload), not expanded to absolute path.

Security model

  • POST /part/**, POST /static-part/**, POST /command/** require ROLE_REST_USER.
  • Dashboard pages and watch APIs require dashboard roles.
  • WebSocket watch endpoint uses /ws/watch and dashboard auth session.
  • In prod, OAuth2 login can be turned on/off with conveyor.service.oauth2-login-enable.
  • In prod, JWT resource-server validation can be turned on/off with conveyor.service.oauth2-resource-server-enable.

REST endpoints

Part loader

  • POST /part/{conveyor}/{id}/{label}
  • POST /part/{conveyor}/{label} (foreach mode)

Request body is mapped from Content-Type and conveyor meta info.

Supported query params include:

  • ttl
  • creationTime
  • expirationTime
  • priority
  • requestTTL
  • any extra params are forwarded as part properties

Static part loader

  • POST /static-part/{conveyor}/{label}

Static-part specifics:

  • no id, ttl, creationTime, expirationTime
  • optional priority
  • delete=true removes static part and does not require body
  • requestTTL supported
  • extra params are forwarded as static-part properties

Command loader

  • POST /command/{conveyor}/{id}/{command}
  • POST /command/{conveyor}/{command} (foreach mode)

Commands:

  • cancel
  • completeExceptionally
  • addProperties
  • timeout
  • reschedule
  • create (ID mode only)
  • peek
  • peekId

Rules:

  • peek and peekId require requestTTL
  • reschedule requires ttl or expirationTime
  • completeExceptionally takes plain-text message in request body
  • in foreach mode, unsupported commands are rejected

Watch APIs

  • GET /api/dashboard/watch - list active watches for user
  • POST /api/dashboard/watch/cancel - cancel by watchId or (conveyor, correlationId/foreach)
  • POST /api/dashboard/watch/history-limit - update foreach watch history limit

Part/command APIs accept watch flags:

  • watchResults=true (or watch=true)
  • watchLimit=<positive int>

Watch events are delivered via WebSocket at:

  • /ws/watch

Event types include RESULT, SCRAP, PING, and CANCELED.

Time input formats

Duration fields (ttl, requestTTL):

  • milliseconds as integer: 1000
  • value + time unit: 30 SECONDS, 1 MINUTES

Date/time fields (creationTime, expirationTime):

  • epoch millis: 1771185490587
  • ISO instant/offset/zoned/local forms
  • RFC-1123 text
  • if timezone is missing, server default timezone is used