Service: Python Job Runner - EyevinnOSC/community GitHub Wiki

Python Job Runner

Python Job Runner (serviceId eyevinn-python-job-runner) is an ephemeral job execution service. It clones a Python repository, installs dependencies, runs a script and then shuts down. In Eyevinn Open Source Cloud (OSC) it is a job (serviceType job), not a long-running service. It is the runtime behind the My Jobs feature but can also be created directly as a standalone instance. License: MIT. Source: Eyevinn/python-job-runner.

What it can do

  • Run a one-shot Python job from a public or private Git repository.
  • Inject the values of a Parameter Store as environment variables at job start.
  • Run a custom command instead of the default entry point.
  • Be scheduled through My Jobs, which creates a fresh runner for each execution and removes it afterwards. For recurring jobs use My Jobs rather than managing runner instances directly.

Creating an instance

Optionally create a Parameter Store first if your job needs configuration values or secrets, and note its name and API key. Store credentials as service secrets so they can be referenced as {{secrets.NAME}}:

osc create-secret myparamkey <your-api-key>

Then create the runner with create-service-instance using serviceId: "eyevinn-python-job-runner". The instance name must be alphanumeric and at most 20 characters.

{
  "name": "myjobrunner",
  "SourceUrl": "https://github.com/myorg/myrepo",
  "ConfigService": "my-param-store",
  "ConfigApiKey": "{{secrets.myparamkey}}"
}

Via CLI:

osc create eyevinn-python-job-runner myjobrunner \
  -o SourceUrl="https://github.com/myorg/myrepo" \
  -o ConfigService="my-param-store" \
  -o ConfigApiKey="{{secrets.myparamkey}}"

Via API:

curl -s -X POST \
  -H "x-pat-jwt: Bearer $PAT" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "myjobrunner",
    "SourceUrl": "https://github.com/myorg/myrepo",
    "ConfigService": "my-param-store",
    "ConfigApiKey": "{{secrets.myparamkey}}"
  }' \
  https://eyevinn-python-job-runner.svc.prod.osaas.io/eyevinn-python-job-runnerinstance

Use get-logs-for-instance to follow the run.

Options

Option Required Type Meaning
name Yes string Name of the instance. Alphanumeric/underscore only, at most 20 characters.
SourceUrl Yes string HTTPS URL of the Git repository to clone (SOURCE_URL). Supports #branch, #tag and /tree/branch; a full 40-character commit SHA as the fragment pins the run to that commit.
GitHubToken No string, sensitive Personal access token for a private repository (GITHUB_TOKEN, alternative to GIT_TOKEN). Only needed for private repositories.
OscAccessToken No string, sensitive OSC access token (OSC_ACCESS_TOKEN). Required when ConfigService is set, to read the key/value pairs from the Parameter Store; also needed if the job calls OSC services.
ConfigService No string Name of a Parameter Store instance (CONFIG_SVC). Its key/value pairs are exported as environment variables before the job runs. Requires OscAccessToken.
ConfigApiKey No string, sensitive API key for an encrypted parameter store. Pass as {{secrets.NAME}}.
WorkerCmd No string Shell command to run instead of the default entry point, for example python myjob.py --date 2026-01-01 (WORKER_CMD). Takes precedence over job.py and main.py.

Pass GitHubToken, OscAccessToken and ConfigApiKey as {{secrets.NAME}} references; a reference must be the entire field value. Overwriting a secret does not restart a running instance.

Using the instance

What the runner does

  1. Clones the repository from SourceUrl.
  2. Installs dependencies, detected in this order: pyproject.toml, requirements.txt, setup.py; then runs setup.sh from the repo root if present. These steps run without the OSC token or Git token in their environment and before the Parameter Store is loaded.
  3. Loads the Parameter Store (ConfigService) into the environment and runs the worker.
  4. Entry point resolution: WorkerCmd if set, else python job.py, else python main.py. If none exist the container exits with an error.
  5. Shuts down on completion.

Example job

A job script jobs/sync.py that reads configuration from environment variables supplied by the Parameter Store (run it with WorkerCmd set to python jobs/sync.py):

import os

db_url = os.environ["DATABASE_URL"]
api_key = os.environ["EXTERNAL_API_KEY"]

# ... job logic
print("Job complete")

Scheduling with My Jobs

# Create a job that runs every weekday at 9am
curl -s -X POST \
  -H "x-pat-jwt: Bearer $PAT" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "dailysync",
    "cronSchedule": "0 9 * * 1-5",
    "sourceUrl": "https://github.com/myorg/myrepo",
    "workerCmd": "python jobs/sync.py",
    "configService": "my-param-store"
  }' \
  https://deploy.svc.prod.osaas.io/myjobs

Resources