Overview of components - michaelmarconi/oncue GitHub Wiki

This is a conceptual overview of the distributed components in the system:

images/high-level-architecture.png

Service

  • Queue Manager: When sent an EnqueueJob message, this component responds by creating a new Job and tells the Scheduler about it.

  • Scheduler: Responsible for maintaining a queue of unscheduled jobs and scheduling them across Agents when they are available. The Backing Store component is optionally used to persist the state of the Scheduler.

Agent

  • Agent: An agent will register with the Scheduler when started and emit a steady heart beat to let it know it is still alive. Agents will respond to WORK_AVAILABLE broadcasts by issuing a REQUEST_WORK message. When work arrives, a Worker is spawned for each Job.

  • Worker: A particular class of Worker that knows how to perform the specified job. Job parameters can be passed to workers as key/value pairs.

API Client

  • API: Currently exposes an enqueueJob(...)) method allowing external API clients to create and schedule new jobs.