Enable monitorResources to record a job's total CPU time (user + system), sampled peak resident memory, and optional CPU%/RSS chart series. cronstable samples the whole process tree, including child processes, with psutil.

The run record carries the numbers to every surface that already reports a run: the web dashboard, the terminal dashboard, the HTTP API, Prometheus, statsd, failure reports, and the durable run ledger.

It is observability only. Monitoring never changes a run's success/failure verdict, never delays or crashes a job, and is off by default. A per-run sampling task is spawned only when it is on. To cap a run's CPU, memory, or process count, see resource limits.

psutil is a core dependency. If it is unavailable, monitoring reports no data and the job continues.

monitorResources is a per-job option. To enable it fleet-wide, set it once under a defaults: block:

jobs:
  - name: nightly-model-refresh
    command: python -m models.refresh
    schedule: "0 4 * * *"
    monitorResources: true

The bool form samples with the defaults: a 1s cadence and a 240-point chart series per run. DAG tasks accept the option too (see DAG tasks later).

Use a boolean or a map of optional settings:

monitorResources:
  interval: 0.5     # seconds between samples (default 1.0, minimum 0.1)
  history: 240      # chart points kept per run (default 240; 0 = summary only)

The option merges normally under defaults: and is not part of the job-set ID fingerprint. Turning monitoring on or off never changes a job's identity. The per-job option row and the numeric validation table also appear in the configuration reference.

The monitor attaches to the launched child's pid immediately after launch, takes a first reading right away (so even a short run has a chance of a sample), then polls on the configured interval. It takes one final opportunistic reading when the run ends.

• The whole tree is accounted. Each sample reads the child and all of its descendants. CPU time is tracked per tree member. A member that exits has its last reading banked before it is forgotten, so sequential children (sh -c 'a; b') accumulate instead of plateauing. For an uncapped run, the only CPU that escapes accounting entirely is a child that spawns and exits within a single sampling gap.
• A capped run's CPU total is exact. When the job also sets resourceLimits and the run is inside its own cgroup (Linux) or Job Object (Windows), each sample takes the run's members from that container instead of walking the process tree, and the CPU totals come from the kernel's accounting for the container. They include children that started and exited between samples, and processes that left the tree. max_rss_bytes is the sampled RSS sum, as for any run. On Linux, while cronstable's cpu controller is on, which takes a job with a cpu cap and a delegated cpu controller (as with Delegate=yes), every run has a cgroup of its own (see CPU caps and priority), so an uncapped run is sampled this way too. A run that never got into its container is sampled by walking its process tree. With verify, a capped run's figures end before its check starts.
• Peak RSS is a sampled high-water mark. The recorded maximum is the highest resident-set sum observed across samples, so a spike narrower than the sampling gap can be missed. A shorter interval narrows that window.
• Best-effort, never fatal. Every psutil interaction is guarded. A process that exits mid-sample, a platform that denies the read, or psutil raising anything at all yields whatever was captured so far. A run where nothing could be sampled carries no resource stats (resources is null in the API). It is never an error.
• Short runs have fewer samples. Measurements are less precise when a run finishes between samples.

A finished monitored run records a summary object: cpu_user_seconds, cpu_system_seconds, cpu_total_seconds, max_rss_bytes, and samples (how many times the tree was successfully read). While the run is still going, the live readings are cpu_seconds (cumulative), cpu_percent (usage since the previous sample; can exceed 100 across multiple cores), and rss_bytes (the tree's current resident memory).

Alongside the summary numbers, each monitored run records a [t, cpu%, rss] point per sample: the data behind the dashboard's Resources charts.

After a run exceeds its history cap, the series is downsampled in place. Adjacent buckets merge with the mean CPU% but the peak RSS (the memory spikes people monitor for are never averaged away), and the effective resolution halves, so a run of any length stays within history points with uniform bucket widths: a few KB even for a days-long run. history: 0 disables the series and keeps the summary only.

The series is embedded in the durable run record's resources.series, so charts survive restarts. It is deliberately excluded from the polled /jobs and /jobs/{name}/runs payloads. The dedicated GET /jobs/{name}/resources endpoint serves it instead, and the dashboard fetches that endpoint lazily when you open a job's Resources tab, never on the poll loop.

DAG tasks accept monitorResources with the same schema, but surface the result differently. A finished task instance's summary usage lands in the resources object of its task record inside the durable dag_run document (returned by GET /dags/{name}/runs/{run_key}). A task with a statsd sink sends the same cpu / max_rss lines.

Task instances are ephemeral and do not appear in the per-job Prometheus families on GET /metrics.

Three adjacent surfaces report the node's load rather than a job's. None of them requires monitorResources:

• GET /node samples the serving host's live CPU/memory (plus the daemon's own footprint) fresh per request and drives the dashboard header's node meter. It is container-aware: under a cgroup v2 limit the numbers describe the daemon's own slice, not the host.
• GET /node/history serves a background-sampled CPU/memory ring (every 5s, keeping the last hour, by default) charted behind the header meter. Configure or disable it with web.nodeHistory.
• cluster.observability shares each node's whole-node CPU/memory across a cluster, so the dashboard's cluster panel and fleet view show where the load is.

• Configuration Reference: the per-job option table and load-time numeric validation.
• Resource Limits: per-run CPU, memory, and process caps that the kernel enforces.
• HTTP Control API: the endpoints that carry the numbers, with response examples.
• Metrics with Prometheus: the resource metric families and their restart-durability.
• Metrics with statsd: the exact wire format of the cpu / max_rss lines.
• Reporting (Mail, Sentry, Shell, Webhook, Push, Event Log): the resource template and environment variables.
• Durable State: the run ledger the summary and series persist into.
• Orchestration and DAGs: monitorResources on DAG tasks.
• Clustering and Leader Election: the cluster.observability fleet-wide node stats.