Diagram

Scheduler object

• Scheduler object creates Execution object when it is "due"
• Scheduler "context" -- it is a template, when rendered, it becomes a JSON object
  • When rendering the scheduler context, "due" is a variable represent the execution "due" time (a datetime object)

Execution object

• Execution object has "context", it is rendered from Scheduler objects context and it is a JSON object
• Pipeline can access the execution's context.
  • Pipeline's task's parameter is template (jinja template), execution's context will be used to render the template
• Each Pipeline can declare asset dependency, execution will only invoke pipeline when dependent assets are present in the Data Manager.