Results

What runs produced, and how they did. Its parts: store (what was produced, stored), metrics (pure formulas), pricing (valuing a run from the catalog, and benchmarks), costs (costs and exposure), comparisons (across runs) and report (tearsheets). Run writes; api, optimise, the CLI and notebooks read. Results never runs backtests.

Layout

sbt2.core.results is the module's only public interface; everything outside it imports from there. Inside, each concern is a subpackage behind a small __init__, and the subpackages import each other only through it, by one import-linter protected contract each:

results/
  __init__.py       the public interface
  metrics/          Segment and RunTables, equity curves and returns (curves), the headline metrics (headline), the full statistics and benchmark statistics (full), feeding nautilus's analyzers (analyzers), money columns (money) and closed trades (trades)
  pricing/          PricedRun, Market and on_grid (market), the benchmarks and build_benchmark (benchmarks)
  costs/            CostsAndExposure and its records (records), the book and slices behind them (book), costs_and_exposure (activity)
  store/            ResultStore, RunIds, RunFilter, StoredRun, StoredStudy and their errors (base), OutputSink and Reports (sink), open_store, store_at and the store registry (__init__)
    parquet/        ParquetResultStore (store), its sink and the summary schema (sink), table I/O (tables)
  comparisons.py    compare_parts, degradation, batch_table, study_list, study_table
  report/           tearsheet (sheet) and sbt2's tearsheet panels (panels)

The tearsheet's module is sheet, so that it doesn't shadow the function tearsheet its package exports. The layering runs one way, by an import-linter layers contract, from the top: report, comparisons, store and costs, pricing, metrics; each imports only those below it, and store and costs don't import each other. store sits above pricing because StoredRun.priced builds a PricedRun.

RunOutput and the output sink

RunOutput: equity series (nautilus's portfolio snapshots), fills, positions, the carry ledger (nautilus's funding adjustments), optionally orders, the run status and the resolved spec. Every table comes from nautilus; sbt2 computes none of them.

The output sink is an interface, called once at the end of the run: write_equity, write_carry, write_reports with nautilus's own fills, positions, account and (optionally) orders reports, write_drawdown_trip with the time the strategy's drawdown guard tripped (only if it did), then finalize. run.execute writes to it and never knows where the data goes. The fill and position tables keep nautilus's report columns; sbt2 doesn't define its own. The store hands out a sink for each run; the default one writes parquet straight into the run's folder.

store

A ResultStore abstract base class, so the backend can be swapped without touching other code. It covers:

• a locator: locator() describes where the store lives as JSON with its kind ({"kind": "parquet", "root": "..."}), and open_store(locator) reopens it through an explicit registry of the store classes, which fails with UnknownStoreError on an unknown kind; a run child reopens its store this way (see Run);
• sinks for new runs, under the RunIds the caller gives: a run_id (batch picks a uuid7) or a fresh one, and the batch id of the batch call the run belongs to, if any; a sink's finalize computes the headline metrics and writes the run's summary last;
• a run's folder, known before the run starts, so a failed run's error can name it;
• queries: list the finished runs with runs(where), oldest first, where a RunFilter keeps those matching each field given (strategy, its import path; part; batch, the batch id; run_ids, a tuple of run ids, empty matching no run; study, the study's name), filtered in the query so only the matching summaries are read; a run's resolved spec, finished or not; load a run's equity, fills, positions, carry ledger or summary; load a finished run whole with stored_run(run_id), a StoredRun holding its spec, rebuilt from the stored document by ResolvedRunSpec.from_document, its RunTables and its known gaps parsed back into Gaps, which priced(catalog) turns into the PricedRun the tearsheet takes (a run without a summary fails with MissingTableError); and delete, which removes a failed run's partial folder too;
• studies: keep a new one with new_study(study), read one back with study(name), a StoredStudy with its name, its context and its pinned source (an unknown name fails with UnknownStudyError, naming the known ones), and list their names with studies().

Default backend:

• store_at(root) opens the store of a data root, the parquet store under results/; sbt2 and the server reach a root's runs through it.
• results/runs/{run_id}/ under the root as parquet (equity, fills, positions and carry ledger always; orders optional per run), plus a one-row summary.parquet. The carry ledger is stored in time order, and payments at the same time in instrument order, so a multi-instrument run stores the same ledger every time.
• A run's folder and its resolved spec are written when it starts; the backtest then writes directly into it. There is no staging area.
• results/studies/{name}/ holds each study: its context as context.json and its strategy module's source as strategy.py.
• The summary holds the run_id, the batch id (batch_id, null for a run started outside a batch), the study's name (study, null for a run outside a study), spec hash, strategy and its lineage, parameters, universe, split (its document as canonical JSON, as hashed) and part, the part's dates, the settlement currency, the known-gap days the run skipped (known_gaps, one "instrument data-type day" string each), the contamination mark, the headline metrics, and drawdown_tripped_at, the UTC time the drawdown guard stopped trading (null when it never tripped). It is written last, so only a finished run has one.
• The index is a DuckDB view over runs/*/summary.parquet, not a table, queried in memory and returned as a DataFrame; DuckDB stays out of the store interface. Nothing writes to it: there's no single-writer rule, no parent-side index step, and nothing to keep in sync. Deleting a run means removing its folder. Summaries are read by column name, so one written before a column existed still lists, with that column null, and one written with a column since dropped (such as alpha and beta) still lists too.
• Every run is kept, optimiser candidates included, so any result can be looked up later.

Studies. A study is a named, persistent collection of runs of one strategy, on one fixed context, that differ only in their parameters: how a parameter search is kept, extended and compared over time. A spec file names it with study (see Spec). Its first run fixes the context, the spec-file table without params, part and study, and pins the source of the strategy's module. Later batches extend it with new parameter values; batch refuses, before anything runs, a run with another context (StudyContextError, naming the differing fields), other code (StudyCodeError, compared by normalised AST, so comments, formatting and docstrings are no change) or a spec hash the study already holds (DuplicateStudyRunError, naming the stored run). See Run. A study holds many batches; batch_id keeps naming one invocation. Its runs are deleted by the same rules as any other.

Rules:

• A failed run crashes the program (see Run). It has no summary, so it isn't listed; its partial folder stays until deleted. Any train or validation run can be deleted. Test runs can't: the test lock reads them (see Run), and deleting one would unlock its dates.
• sbt2 runs list shows one row per run, oldest first. Reruns of the same spec share a spec hash.
• Cross-sectional runs record their universe and carry a survivorship warning when the data source can't provide delisted instruments.
• The store covers results only. The catalog uses nautilus's own storage layer, and raw downloads stay local files.

metrics

Formulas over a stored run. Equity, fills, positions, the carry ledger and a calendar in; numbers out. Only costs and exposure, in costs, read anything else: the valuation prices in the catalog, through pricing. Used by the sink's finalize for the headline metrics in the summary and by report for the full set.

• Built on mark-to-market equity from nautilus's portfolio snapshots, forward-filled onto the run's equity grid, never realized PnL alone.
• Computed over the part's dates only. Warm-up is excluded.
• Annualization comes from the asset class's calendar, passed explicitly as period to every nautilus statistic, which defaults to 252 days. The tearsheet takes no period: its statistics come already annualized from full_metrics, and its one annualized panel, the rolling Sharpe, is replaced by sbt2's (see report).
• Nautilus's PortfolioAnalyzer and its statistic classes are the formulas wherever nautilus has one, fed with our mark-to-market returns (add_return) instead of its defaults (252-day annualization, realized-PnL returns). sbt2 writes a formula only where nautilus has none, or has one it can't feed from a stored run: the cost waterfall, exposure, holding time, turnover, the realized-PnL profit factor, the long ratio and the probabilistic Sharpe ratio.

Group	Metrics
Performance	total and annualized return, volatility, Sharpe, Sortino, max drawdown and its duration, Calmar
Trades	count, win rate, average win vs average loss, profit factor, expectancy, holding time, turnover
Costs	gross PnL → fees → carry (from the carry ledger) → net PnL
Exposure	gross and net leverage over time, time in market
Per instrument	the same numbers, split by symbol
Overfitting	probabilistic Sharpe ratio of a run; for an optimisation, the deflated Sharpe ratio of the best candidate and the probability of backtest overfitting (below)

Full metrics

full_metrics(run, segment, curve=None) computes a run's full set from its stored tables (equity, fills, carry ledger and positions), or from its equity curve when that is already built, on demand; nothing of it is stored. It returns nautilus's three statistics dicts under nautilus's own names (e.g. Sharpe Ratio (365 days), Win Rate), so the tearsheet's create_tearsheet_from_stats takes them as they are:

• returns: a bare PortfolioAnalyzer has no statistics registered, so sbt2 registers every nautilus statistic that needs no benchmark (CAGR, Calmar, volatility, Sharpe, Sortino, max drawdown, value at risk, expected shortfall, Omega, tail ratio, Ulcer index, profit factor, average return, average win and loss, skewness, kurtosis, risk-return ratio), with period=days_per_year on every one that takes a period. It is fed the run's mark-to-market returns on the equity grid over the part's dates, warm-up excluded.
• pnls: the trade statistics (win rate, average, max and min winner and loser, expectancy), fed through add_trade with each trade's realized PnL in the settlement currency; a PnL in another currency fails. A trade is a closed position or a position snapshot (one netting-OMS cycle of a position that was reopened). A position still open at the end is left out, although the engine's own analyzer counts the realized PnL it has so far. Nautilus's PnL (total) and PnL% (total) are dropped: it computes them from the account's balances, which trades alone don't give; the headline net return is the run's total. Nautilus's ProfitFactor takes returns only, so sbt2 adds Profit Factor from the realized PnLs: gross wins over gross losses, none without a loss.
• general: Long Ratio, the share of trades entered long, rounded to two decimals as nautilus rounds it. Nautilus's LongRatio takes Position objects, which a stored run no longer has, so sbt2 reads the entry side from the positions report.

Next to them: the pnls per instrument, the same trade statistics from each instrument's own trades, keyed by instrument id; and the run's probabilistic Sharpe ratio. Without trades the pnls, general and per-instrument dicts are empty; the return statistics are still computed.

Costs and exposure

costs_and_exposure(PricedRun(spec, tables, catalog, known_gaps)) computes the rest of the metrics nautilus has no formula for, from the run's stored tables and the catalog, on demand; nothing of it is stored. It returns an Activity for the whole run (total) and one per traded instrument (by_instrument, keyed by instrument id), each with:

• costs, a CostWaterfall in the settlement currency: gross → fees → carry → net. The run's net PnL is its equity at the part end minus its equity at the part start. Fees are the commissions of the part's fills, as a negative step; carry is the carry ledger's pnl_change with its sign, so funding received raises the net and funding paid lowers it; gross PnL is what is left before both. An instrument's net PnL is its realized PnL plus the unrealized at the part end's valuation price: its gross is the cash flow of its fills plus the change in its position's value over the part. So the instruments' net PnLs add up to the run's.
• exposure: gross leverage (the sum of the absolute notionals over equity) and net leverage (the signed sum over equity) at each time of the run's equity grid, and time in market, the share of grid steps that start with an open position. An instrument's leverage is its own notional over the run's equity.
• holding time: the mean and median duration of the closed trades, the same trades the trade statistics count (closed positions and snapshots; a position still open is left out). There is none without a closed trade.
• turnover: the traded notional (quantity × fill price × multiplier) over the mean equity on the grid, per year of the asset class's calendar (days_per_year).

Positions are the cumulative signed quantities of the stored fills. A position is valued at the asset class's valuation price (mark price for perps) on the run's grid, the price nautilus marks equity with: notional = quantity × price × contract multiplier, the multiplier taken from the instrument in the catalog. Prices are read as the benchmarks read them (below): the last price at or before each grid time, carried across the run's known gaps. Both read them from the priced run's market, which reads each instrument's prices of a type from the catalog once, so a tearsheet whose benchmark holds the run's instruments reads each series once. The run's equity curve is built once too (PricedRun.equity) and feeds the full metrics, the daily returns and the costs and exposure. Missing days fail with MissingPricesError, naming the instrument and the days. Only linear instruments have that notional; an inverse one fails with InverseInstrumentError. Fills and carry count from the part start to the part end, both included, as the headline total fees and carry do. A run without trades has zero leverage, zero time in market, no holding time and zero turnover.

Overfitting statistics

An optimiser picks the best of many candidates, so the winner's Sharpe is inflated by the search itself. Three formulas from the literature measure that; sbt2 writes them itself (see Decisions).

• Probabilistic Sharpe ratio (PSR): the probability that a run's true Sharpe is above a threshold (0 by default), given its length and the skew and kurtosis of its returns (Bailey & López de Prado 2012). It is computed on the returns of the run's equity grid, not annualized: Φ((SR − SR*)·√(n − 1) / √(1 − γ₃·SR + (γ₄ − 1)/4·SR²)), with SR the grid returns' mean over their sample standard deviation, n their count, γ₃ their skewness and γ₄ their kurtosis. The skewness and excess kurtosis (γ₄ − 3) are the unbiased sample estimators nautilus's ReturnsSkewness and ReturnsKurtosis use; those classes resample to daily returns first, so sbt2 applies the same estimators to the grid returns itself. A run with no volatility has no PSR. full_metrics reports it against 0.
• Deflated Sharpe ratio (DSR): the PSR of the best candidate against the Sharpe the best of N trials would reach by luck alone (Bailey & López de Prado 2014). N and the variance of the trials' Sharpe ratios come from every validation run of the lineage in the results store, not only the current search, so earlier searches count too. A test run reports the DSR with its lineage's trial count at the time it ran.
• Probability of backtest overfitting (PBO): estimated by combinatorially symmetric cross-validation (CSCV) over the candidates' returns on train and validation, split into an even number of blocks: the share of in-sample/out-of-sample combinations in which the in-sample winner ranks below the out-of-sample median (Bailey, Borwein, López de Prado & Zhu 2017). Test is never used.

They are pure formulas like the rest: returns and trial Sharpe ratios in, numbers out. The optimisation's Report and the sbt2 run summary table show the DSR and PBO next to best_params.

Headline metrics (stored in the summary): net return, annualized return, Sharpe, max drawdown, trade count, total fees, total carry.

report

• Report: what sbt2.backtest returns: headline and full metrics, equity, fills and positions as DataFrames, and for an optimisation every candidate and best_params. It is built from the stored runs.
• Summary table: sbt2 run prints the headline metrics per part when it ends, one table per part in split order (see CLI).

Tearsheets and comparisons are generated on demand, never automatically after every run.

• Single run: tearsheet(run, path, benchmark=None) writes nautilus's own tearsheet of a PricedRun, built offline with create_tearsheet_from_stats, in the format path's extension names (HTML, or a static image through Kaleido, which needs Chrome). It is QuantStats-like, so no QuantStats dependency; plotly comes with nautilus-trader[visualization].
  • Returns: the run's equity-grid returns compounded to one per UTC day, as benchmark_statistics does, since nautilus's panels assume daily rows. A grid return covers the step up to its time, so it counts in the day that step covers: one ending at midnight belongs to the day before.
  • Stats table: full_metrics's three dicts under nautilus's names; the benchmark-relative statistics join the returns section and the probabilistic Sharpe ratio the general one.
  • Panels: nautilus's equity (with the benchmark overlaid under its class name when one is given), drawdown, monthly returns heatmap, returns distribution and yearly returns; then sbt2's own: the rolling Sharpe over 60 daily returns annualized by sqrt(days_per_year), the cost waterfall (gross → fees → carry → net) and a per-instrument breakdown table, one row per traded instrument with its net PnL, fees, carry, turnover, time in market, fill count and win rate. The grid is five rows of two.
  • Custom panels go in through register_tearsheet_chart and are placed with TearsheetCustomChart, whose args reach the renderer; register_chart registers only standalone figures, which the tearsheet rejects. The rolling Sharpe is sbt2's because create_tearsheet_from_stats takes no period and nautilus's panel hardcodes TRADING_DAYS_PER_YEAR = 252.
  • Resolving the strategy's default benchmark is sbt2 report's job; tearsheet overlays only the one it is given.
• Comparisons, in comparisons, built from the index, since nautilus has none across runs:
  • compare_parts(store, run_id): the summaries of the runs sharing the run's strategy, parameters, instruments and split, one per part, indexed by part in split order. A rerun part shows its latest run; a part never run is absent.
  • degradation(parts): one row per headline metric, one column per compared part, then the change from train to validation and from validation to test, as a difference and as a percentage of the earlier part. The percentage is left empty when that base is zero or negative, where it would mislead; a change needs both of its parts.
  • batch_table(store, batch_id): one row per run of the batch, by run_id, with only the strategy, parameter and instrument columns whose values vary across the batch, then the headline metrics. An unknown batch fails with UnknownBatchError.
  • study_list(store): one row per study, by name, with its strategy and how many runs it holds.
  • study_table(store, name): one row per run of the study, by run_id, with its part, the parameters whose values vary across the study, then the headline metrics. An unknown study fails with UnknownStudyError, naming the known ones.
• Walk-forward heatmap: candidates or parameter sets against windows, each cell the out-of-sample score, so a result that holds in one window only stands out (nieledran/backtesting-engine's wfo-batch verdict heatmap). Arrives with walk-forward in M10 (see Deferred).

Benchmarks

In pricing, next to the priced run whose market they read. Optional and chosen per strategy: the strategy declares a default in its metadata (benchmark = BuyAndHold(), see Strategy), and sbt2 report tearsheet --benchmark can override it. The benchmark is not part of the run spec, because it doesn't change the backtest. A market-neutral strategy compared against a long-only index would be misleading.

A benchmark is a Benchmark, an abstract base class whose returns(run) builds its value on the equity grid of a PricedRun and returns one return per grid step, like the run's own. The variants:

• BuyAndHold(instrument=None): one instrument, the run's first when none is named, held by the asset class's buy-and-hold convention (see Asset classes). For crypto perps: the mark-price series, no funding, the taker rate of the run's venue fee model paid once on entry.
• EqualWeight(): equal notional in each of the run's instruments at the part start, each paying the entry fee, then held without rebalancing, so the weights drift.
• External(path): a CSV or parquet file with a UTC timestamp column and a price column, forward-filled onto the grid, with no fees. It must cover the part: a price at or before the start, and one at or after the last grid time minus one equity interval.
• None: no benchmark, so no series and no benchmark-relative statistics.

build_benchmark(name, argument) builds one by name from an explicit registry: buy-and-hold (optionally an instrument id), equal-weight, external (a file) and none. An unknown name lists the known ones; external without a file, or an argument to equal-weight or none, fails.

StoredRun.benchmark(name=None, argument=None) is the benchmark a report of a stored run compares against: the strategy's own (Strategy.benchmark) without a name, otherwise build_benchmark(name, argument).

Benchmarks are computed from our own data (via Data), on the same part and the same sampling grid. Prices come from the asset class's valuation price series in the catalog (mark price for perps), since bars aren't stored. Each price is the last one at or before a grid time; entry is at the first valuation price of the part. The run's known gaps are not missing: the last price is carried across them, as the run itself skips them. A benchmark that can't cover the part fails with an error naming what is missing: an instrument without valuation prices on days that aren't known gaps names the instrument and the days; an external file that starts after the part start names the file and the first time it can't price, and one that ends early names the file and its last time.

sbt2 only builds the benchmark's return series. Benchmark-relative statistics come from nautilus: benchmark_statistics(returns, benchmark, days_per_year) feeds both series, compounded to daily returns (nautilus pairs them only on identical timestamps; a return ending at midnight belongs to the day before), to PortfolioAnalyzer.get_performance_stats_returns_vs_benchmark and returns alpha, beta, information ratio, tracking error and the up and down capture ratios under nautilus's own names (e.g. Alpha (365 days), Beta). They are computed when a report is made, never stored in the summary. The tearsheet overlays the series through benchmark_returns.