Run

Executing backtests. Takes resolved run specs from Spec, checks them, runs each one in a process of its own and records it in Results. The only module that assembles nautilus.

Three parts:

Part	Job
preflight	Fetches missing data, runs the coverage and instrument checks and enforces the test lock, before anything starts.
execute	One backtest, in the current process (the engine). Resolved spec + output sink in. Knows nothing of processes, batches or storage.
batch	One forked process per run, its launcher (and memory cap, if it sets one), parallelism.

Tests call execute directly with a temp-folder sink; api and optimise call batch.

Layout

sbt2.core.run is the module's only public interface; everything outside it imports from there. Inside, each concern is a subpackage behind a small __init__, and the subpackages import each other only through it, by one import-linter protected contract each:

run/
  __init__.py       the public interface
  execute.py        one backtest in this process
  batching/         batch() and its orders (__init__), the forkserver and the running children, Memory, Launch, BatchSetup and BatchProgress, the study checks (studies), RunFailedError, OutOfMemoryError and the study errors
  launchers/        the Launcher base class, Uncapped, SystemdScope and the registry behind launcher_named (__init__)
  child/            the order and its JSON document, and run_child, the child's main
  preflighting/     preflight() and its checks, fetching the days the catalog lacks, DataFolders and the pre-flight errors

The subpackages that hold batch and preflight are named batching and preflighting, so that neither shadows the function sbt2.core.run exports under that name. The layering runs one way: batching uses launchers, child and preflighting; child uses execute; launchers and preflighting use nothing else in run.

preflight

1. Fetch and coverage via Data: every instrument must have every declared data type on every trading day of the asset class's calendar in [start − warm-up, end), answered by nautilus's catalog from its file names. Missing days are first downloaded and ingested from the source the venue profile names. Days on the source's known-gaps list (see Data sources) are allowed and recorded in the run's summary. Any other day still missing fails the run before it starts, with a clear message.
2. Instrument check: every instrument must be covered by the run's asset profile, i.e. its own nautilus asset_class and instrument_class match the profile's (AssetProfile.covers).
3. Snapshot check: the part's length divided by the equity interval must fit nautilus's snapshot buffer (see Equity below).
4. Liquidation check: a run with liquidation on must stream quotes, because nautilus's venue checks maintenance margin only against the top-of-book quote. A run with liquidation on and no quotes would never liquidate, so it is refused before anything is fetched (see Crypto perpetuals).
5. Test lock, for a test run only: the results store is asked for earlier test runs of the same lineage (the strategy's qualified name) whose dates overlap this one, from any version and with any parameters. If there are any, the run is refused, naming them. reuse_test (--reuse-test) overrides this, and the run's summary is then marked contaminated, naming the earlier runs. A run counts from the moment it starts, when its folder and resolved spec are written, so a crashed test run counts too. Validation isn't locked. Renaming or moving the class starts a new lineage; see Library.

There is no memory estimate. The per-run cap in batch is the only memory guard; see Memory safety below.

execute

1. Build the BacktestRunConfig from the resolved spec (see Spec): the venue, data configs for the strategy's declared inputs only over [part start − warm-up, part end), the strategy's ImportableStrategyConfig (see Strategy), the risk engine config and the portfolio config (valuation price and equity interval). execute adds only the machine settings: catalog path, chunk_size, and dispose_on_completion=False so the results can be read after the run. With nautilus's default (True) the run still succeeds, but the engine cache comes back empty: no account, positions or funding adjustments (pinned in Quality).
2. Run it with nautilus's high-level BacktestNode, streaming data from the catalog in chunks.
3. Hand the output to the sink, once, at the end. Everything comes from nautilus: the equity series from the portfolio's snapshots, the carry ledger from the positions' funding adjustments (current positions and position snapshots, see below), and nautilus's own reports (generate_fills_report, generate_positions_report, generate_account_report, and optionally generate_orders_report), and the time the strategy's drawdown guard tripped, if it did. Then finalize. No metrics are computed here. The sink interface lives in Results; execute never knows where it writes.

Reversed positions. Under netting OMS a reversal keeps the position ID, and nautilus moves the earlier position, with its funding adjustments, into cache.position_snapshots(). cache.positions() holds only the latest one. execute therefore reads funding from both, and passes the snapshots to generate_positions_report. Reading positions alone silently drops every payment before the last flip (pinned in Quality).

Rules

• Declared inputs drive fills. The venue's book type follows what the strategy declared. A bar strategy streams trades or 1-minute candles, as its bars key says, and gets its bars built by nautilus. On trades it fills on the trades from the bar's close on. On candles it fills through nautilus's bar execution, with the venue's defaults: one fill per order at a candle price (a market order at the close of the last candle), with no trade-level liquidity. A strategy that declares L2 fills against the book. L2 is never loaded for a strategy that didn't ask for it.
• Warm-up: data is streamed from before the part's start and orders are blocked until the start. The duration comes from the strategy.
• Data outside the resolved window is never streamed.

Equity

Mark-to-market equity (balance plus unrealized PnL) comes from nautilus's portfolio snapshots; sbt2 has no sampler of its own.

• PortfolioConfig.equity_curve (on by default) records a snapshot when the account opens, at every UTC midnight and at shutdown. snapshot_interval_ms, set from the run spec's equity interval (1h by default, finer for strategies like L2 market making), adds snapshots while a position is open. After the run, execute reads them with portfolio.snapshots(account_id).
• Open positions are valued by the portfolio at the asset class's valuation price. For mark price (perps) the asset class sets PortfolioConfig(use_mark_prices=True) (see Asset classes).
• While flat, equity only changes at fills, so Results forward-fills the snapshots onto a regular grid aligned to the part's start and drops warm-up.
• Nautilus keeps at most 1,000,000 snapshots per account (a ring buffer), which is over 110 years at 1h. A run that would exceed it at its interval fails pre-flight.

Memory safety

L2 data is never bulk-loaded; it's always streamed from the catalog. Streaming bounds the input and strategy state is bounded. The output (equity snapshots, fills, positions, orders, funding adjustments) is held by nautilus's portfolio and cache anyway and is taken from them at the end; it is small next to the data. The outer bound is batch's per-process cap: a run that exceeds it is killed, and the program crashes like on any other failure.

The streaming chunk size is fixed per data type in the run settings (see below), not estimated per run.

batch

1. The launcher's check (for SystemdScope, that the systemd user session is there; see Memory cap below), then the study checks and preflight for every run. A failed check refuses the batch before anything is fetched.
   • Study checks, for each run in a study (see Results): its context must be the study's (StudyContextError otherwise, naming the differing fields), its strategy's module must match the study's pinned source by normalised AST (StudyCodeError), and its spec hash must be new to the study (DuplicateStudyRunError, naming the stored run). A study the store lacks is fixed by its first run in the batch. All three share StudyError.
   • Once every run has passed, the parent keeps each new study in the store, with its context and its strategy module's source.
2. Execute each run in a process of its own (never a reused pool worker), forked from the batch's forkserver, under a hard memory cap when the launcher sets one. A process per run is needed for a cap to apply per run, and it guarantees no nautilus global state (such as its logging setup) leaks between runs. The parent picks one batch id (a uuid7) for the whole call and each run's run_id (a uuid7), and the child opens its sink with both from the ResultStore and writes straight into runs/{run_id}/.
3. Finish. The child's sink computes the headline metrics and writes the run's summary last (see Results); the run then shows in sbt2 runs list. Apart from a new study, the parent writes nothing to the store.
4. On any failure, the program crashes. If a run fails for any reason (strategy error, data error, out of memory), batch stops the other runs and exits with the error, naming the failed run and its folder. No retries, no cleanup. A failed run has no summary, so it doesn't show up in sbt2 runs list; its partial folder stays on disk for inspection until you delete it.

A batch is any list of run specs: parts, parameters, symbols, strategies or an optimiser's candidates. Runs execute in parallel, one process each, at most memory.budget ÷ memory.per_run at a time. Each child writes only its own run folder. Every run of one batch call stores the same batch id in its summary, so the batch can be listed and compared later (see Results).

Interface

batch(specs, setup, progress=None) returns the run_ids in the order of specs. BatchSetup holds:

Field	What it is
store	The ResultStore each child reopens from its locator and opens its sink from.
sources	The data source a venue profile names, by name, for pre-flight.
folders	The raw/ and catalog/ folders pre-flight fetches into.
settings	The RunSettings each child's execute gets.
launcher	Starts each child under its memory cap, or under none (see below).
memory	Memory(budget, per_run), in bytes (see Run settings).

BatchSetup.at(root, launch, log_level) sets a batch up from a data root: the store under it (store_at), the sources with its known gaps file, its raw/ and catalog/, and RunSettings reading its catalog with nautilus logging at log_level. Launch(launcher, memory) groups the launcher with the memory it caps. sbt2 run and the server set their batches up this way.

progress hears planned(runs) once and finished(run_id) as each run ends; sbt2 run shows it as a progress bar.

The child forks from a forkserver, the standard library's multiprocessing.get_context("forkserver"). The forkserver starts at the parent's first batch and lives as long as the parent; it has sbt2.core.run.child loaded, and with it nautilus, pandas and pyarrow, so a child starts in milliseconds and shares those pages copy-on-write. It never loads a strategy: each child imports its strategy afresh, so no state leaks between runs and an edited strategy module is picked up by the next batch. The child runs run_child, which waits on a one-way pipe for its order. The parent sends the order as one JSON document: the run_id and batch id, its own working directory and sys.path, the resolved spec as its hashed document plus its source and its study, the known gaps as Gap strings, the store's locator, the run settings and the error file. The child changes to the working directory first, since sys.path can hold '', and puts sys.path in place, so it imports the strategy the same way the parent did; then it rebuilds the spec with ResolvedRunSpec.from_document, the gaps with Gap.from_str and the store with open_store. The order itself is never pickled; multiprocessing pickles only the child's target and its pipe. A child sees the environment and the stdout and stderr the forkserver started with, i.e. the parent's at its first batch: its logs and nautilus's show up live, and --log-file gets the parent's records only. Nothing in a child reads the environment.

If another caller starts the standard-library forkserver before the first batch, registering the run-module preloads afterward has no effect. Runs still work, but miss the preload speedup.

Scripts guard their batches. A child re-imports the script that started the batch, as multiprocessing does, so a script calls batch (or backtest) under if __name__ == "__main__":. Without the guard, Python refuses to start the children with its own RuntimeError about the bootstrapping phase, and the batch fails. The CLI, notebooks and the server need nothing. On an error the child writes it to a file the parent reads, so the batch's error carries the child's own message.

The launcher is the abstract base class Launcher: check() before pre-flight, cap(run_id, pid, memory_max) once the child is running, and out_of_memory(run_id) after a child fails. batch sends a child its order only after cap returns, so a capped child does no work uncapped; if cap fails, the child is terminated with the rest. Each launcher has a name, and launcher_named(name) picks one from an explicit registry in sbt2.core.run.launchers, raising UnknownLauncherError with the known names otherwise. sbt2 run --launcher passes it the name (see CLI). There are two:

Launcher	Name	What it does
Uncapped	uncapped (the default)	Sets no cap: check and cap do nothing and out_of_memory is always false. The machine or the container engine limits memory, as in a container, which has no systemd user session.
SystemdScope	systemd	Caps each child in its own transient systemd scope (see Memory cap). Where there is no systemd user session, its check raises NoUserSessionError; there is no platform check.

Another launcher is a subclass with its own name, added to the registry. Tests use a launcher that leaves the child uncapped and records the cap it would get.

On a failure, the other running children get SIGTERM, then SIGKILL after 5 s, and runs not yet started never start.

Memory cap

Only --launcher systemd caps each run. Under uncapped, memory.per_run still sets how many runs go at once, but nothing enforces it: a run killed by the container engine's OOM killer fails like any other run, a RunFailedError reading that its process exited with code −9, not an OutOfMemoryError.

Under SystemdScope, each child runs in its own transient systemd scope, named after its run. SystemdScope talks to the user's systemd over its D-Bus session bus, with jeepney, and runs no subprocess:

• cap calls StartTransientUnit on org.freedesktop.systemd1.Manager for sbt2-{run_id}.scope, mode fail, with the properties PIDs=[pid], MemoryMax={memory.per_run} and MemorySwapMax=0. It returns once the unit's start job reports done in the JobRemoved signal, and raises on any other job result.
• out_of_memory reads the scope's Result and, unless it is success, resets the scope with ResetFailedUnit. A scope that is already gone reads as success.
• check opens the session bus and reads the manager's Version, and raises NoUserSessionError when either fails.

What systemd does, pinned in Quality:

• Over the cap, the kernel kills the child with SIGKILL, and the parent sees return code −9. The scope's Result is oom-kill, which tells an out-of-memory kill apart from any other SIGKILL; batch reads it and raises the out-of-memory error.
• MemorySwapMax=0 is required. With MemoryMax alone, a child over the cap spills into swap and keeps running.
• A PID is in the scope once its start job is done, so a child waiting on its pipe until cap returns does all its work under the cap. cap still takes the child's pid: the forkserver stays outside every scope.
• A forked child is charged only for the pages it touches itself. The preloaded modules stay with the forkserver, so memory.per_run need not leave room for sbt2's imports.
• Under the cap, the child runs normally and its scope disappears a few milliseconds after it exits.
• A killed scope stays loaded as a failed unit until ResetFailedUnit resets it. batch reads Result first, then resets it.
• Without a user session bus (no DBUS_SESSION_BUS_ADDRESS and XDG_RUNTIME_DIR, as in CI and most containers), no D-Bus connection opens: jeepney raises KeyError for the missing address. That is why SystemdScope checks for the session up front, before batch starts any child. An address-space rlimit can't stand in for the scope: it counts virtual memory, and an empty nautilus engine reserves 2.6 GB of it for 196 MB resident. Without systemd, use the uncapped launcher and let the container engine limit memory.

Cost of starting a run. A fresh interpreter, measured once on a 20-core machine by timing each step in a new process (median of 10):

Fresh process	Wall time	Peak RSS
Bare interpreter	0.01 s	24 MB
import nautilus_trader.backtest	0.05 s	87 MB
import sbt2.core.cli	0.36 s	204 MB
sbt2 run of the golden ma_cross spec (five days of 1-minute data)	0.43 s	325 MB

The forkserver pays the imports once per parent; a child forked from it starts in about 0 s and pays none of them, so a study or a sweep of short runs spends its time on backtests, not imports. Under the systemd cap, the small spec of the batch tests runs within 60M and not within 40M.

Run settings

Machine settings that never change results, so they're not in the run spec: memory.budget (total for a batch), memory.per_run (the memory each child is planned for, and its cgroup cap under --launcher systemd) and the chunk size per data type. They have built-in defaults, overridable in settings.toml under the root (see Architecture), and for one invocation by sbt2 run --memory-budget and --memory-per-run (see CLI).

• memory.per_run defaults to 4G. The month of real data in Quality runs under it.
• memory.budget defaults to half the machine's memory, and at least one run's. A budget below memory.per_run is refused.
• Sizes use systemd's syntax: a whole number with an optional K, M, G or T (base 1024).

Errors

Typed errors: unknown launcher (UnknownLauncherError), a run that doesn't fit its study (StudyContextError, StudyCodeError and DuplicateStudyRunError, sharing StudyError), no systemd user session under --launcher systemd (NoUserSessionError), missing data, instrument/asset-class mismatch, snapshot buffer overflow, liquidation without quotes (the pre-flight ones share PreflightError), test already seen, strategy error, data error. On error execute stops and raises. batch stops the whole program with a RunFailedError that carries the failed run's run_id and folder and the child's error; out of memory under the systemd scope is its subclass OutOfMemoryError, naming the cap. Nothing here prints; it logs.