Library - sbOogway/sbt2 GitHub Wiki

Library

sbt2 is a published package. A user installs it from PyPI, writes only a strategy in their own package, and sbt2 handles the rest: data, venue config, splits, optimisation, the test lock, running and reporting. This page is the user's view; the module pages hold the details.

What the user writes

A nautilus strategy on the sbt2 base, and nothing else:

from dataclasses import dataclass
from typing import Annotated

from sbt2.core import FractionSplit, Grid, Range, Strategy, backtest


@dataclass(frozen=True)
class TrendParams:
    fast: Annotated[int, Range(5, 50, step=5)] = 10
    slow: Annotated[int, Range(20, 200, step=10)] = 30


class Trend(Strategy[TrendParams]):
    Params = TrendParams
    ...  # plain nautilus from here on


if __name__ == "__main__":
    tuned = backtest(
        Trend,
        instruments=["BTCUSDT-LINEAR.BYBIT"],
        period=("2020-01-01", "2025-01-01"),
        split=FractionSplit(validation=0.2, test=0.2),
        optimise=Grid(objective="sharpe"),
    )
    final = backtest(
        Trend,
        instruments=["BTCUSDT-LINEAR.BYBIT"],
        period=("2020-01-01", "2025-01-01"),
        split=FractionSplit(validation=0.2, test=0.2),
        params=tuned.best_params,
        part="test",
    )

A script runs its backtests under if __name__ == "__main__":: each run forks from a process that re-imports the script, and without the guard Python refuses to start the runs (see Run). sbt2 run spec.toml does the same from a spec file (see Spec); notebooks need no guard.

Front-ends

The Python API (sbt2.backtest) and the CLI with spec files are two front-ends on equal footing over one core, the api module (see Architecture). Every feature exists in both. Where Python passes an object (the strategy class, a custom objective, a third-party optimiser), a spec file names it by import path.

Strategies

Full nautilus, as described on Strategy; sbt2 puts no facade over the trading API. Strategies live in the user's own installed package, and sbt2 reaches them only through the class passed to backtest or the import path in a spec file.

Splits

A split divides a backtest's period into three parts, in time order:

Part Used for Runs
train fitting: the optimiser tunes parameters here as often as you like
validation checking what was fitted on data it wasn't fitted on: the optimiser ranks candidates here as often as you like
test the final answer once per strategy: locked

Splits are Splitter objects:

  • FractionSplit(validation=0.2, test=0.2): the last 20% of the period is test and the 20% before it validation;
  • DateSplit(validation_start="2023-01-01", test_start="2024-01-01"): the same by date.

Either way the split resolves to dates, and the dates are hashed. There is no embargo: each part starts where the one before it ends. A call without part runs train and validation; test is only run when asked for. See Spec.

Studies

A study keeps a parameter search: study = "trend-btc" in a spec file puts its runs in a named collection of runs of one strategy, on one fixed context, that differ only in their parameters. The first run fixes everything but the parameters and pins the source of the strategy's module; each later batch adds new parameter values, and one that changes the context or the code, or repeats a stored run, is refused before anything runs. sbt2 studies list lists them and sbt2 report study <name> compares a study's runs, part by part. See Results.

Optimisation

  • The search space is declared on the Params fields with Annotated[..., Range(...)], so the strategy file is self-contained. Lists in a spec file override it.
  • The optimiser is pluggable. Grid search is built in; others plug in through the same interface. The interface is ask-and-tell: the optimiser proposes a batch of candidates, run.batch runs them in parallel, and their scores are told back. That is the shape of Optuna's ask-and-tell API, so an Optuna sampler (TPE and the rest) plugs in without sbt2 depending on Optuna (Optuna ask-and-tell; Akiba et al. 2019; see References).
  • The objective is a named sbt2 metric ("sharpe") or any Callable[[RunResult], float].
  • The optimiser tunes on train and ranks on validation: every candidate runs on both, the objective is scored on validation, and train scores are shown next to it to expose overfitting. Every candidate is a stored run. The optimiser never touches test.
  • Overfitting statistics come with every optimisation: the deflated Sharpe ratio of the best candidate, and the probability of backtest overfitting of the search. The deflated Sharpe ratio discounts the best validation Sharpe for how many candidates were tried; because every run is stored, the count is the lineage's real number of validation runs, not a guess (Bailey & López de Prado 2014). The probability of backtest overfitting is estimated by combinatorially symmetric cross-validation over the candidates' train and validation returns: how often the in-sample winner ranks below the median out of sample (Bailey, Borwein, López de Prado & Zhu 2017). Details on Results.

The test lock

The test part is judged once per strategy, so it can't quietly turn into a second validation set. Validation isn't locked: it's the check you're meant to repeat.

  • Lineage: all versions of a strategy class, keyed by its qualified name (my_strats.trend:Trend).
  • A test run is refused when its dates overlap a test period its lineage already ran on, whichever version or parameters did it. Moving the boundary therefore doesn't escape the lock.
  • reuse_test=True (--reuse-test) overrides the refusal. The result is then marked contaminated for good, naming the earlier runs.
  • The lock reads the results store: a test run counts from the moment it starts, and it can't be deleted (see Results).
  • Renaming or moving the class starts a new lineage and escapes the lock. That is accepted: the lock guards against honest mistakes, not deliberate cheating.

Data

Fetched automatically. Before a run starts, missing days are downloaded and ingested from the source its venue profile names; the user never calls download or ingest. The commands stay for filling the catalog ahead of time. See Run.

Storage and config

  • One data root holds everything a user's data and runs produce: --data, else SBT2_DATA, per call root= in the API. It is required, with no default and no fallback to the working directory. Data downloaded once serves every project. Layout on Architecture.
  • Venue profiles are configuration, in the config folder's venues.toml: --config, else SBT2_CONFIG. It is required too, with no default and no fallback to the working directory, and read on every run, so an edit applies to the next one. sbt2 ships no venue profiles; the GUI holds the defaults and installs them through the server's API. Layout on Architecture.
  • The known gaps describe the data held, so they live in the data root's known_gaps.toml, edited by hand.
  • Run settings (memory budget, per-run cap, chunk sizes) have built-in defaults, overridable in settings.toml under the root.

Plug-ins

Sources are discovered through the entry point sbt2.sources. Bybit is registered the same way from inside sbt2, and other packages add more without a release of sbt2. See Data sources. Venue profiles are not plug-ins: they are configuration in the config folder.

Output

  • A Report object from backtest: metrics, equity, trades and fills as DataFrames, and for an optimisation the candidates and best_params.
  • An HTML tearsheet on demand (see Results).
  • A summary table printed by sbt2 run: headline metrics per part.
  • Every run is also kept in the results store, where sbt2 runs lists it.

Distributions

The backend component, sbt2-backend/, is a uv workspace. Its members are each their own distribution and add their subpackages to sbt2, a namespace package with no code of its own; the protocol (sbt2-protocol/) and the reserved GUI (sbt2-gui/) are the other components of the repository (see Architecture):

Distribution Import Holds
sbt2-core sbt2.core the backtester: every module on Architecture, and the sbt2 command
sbt2-strategies sbt2.strategies example strategies, which the golden runs also use
sbt2-papers sbt2.papers the Paper extraction pipeline
sbt2-server sbt2.server, sbt2.protocol.v1 the GUI's server and the sbt2-server command (see Server on Architecture), with the Python generated from the protocol in sbt2.protocol.v1 (the import path follows the .proto package; see Protocol on Architecture)
sbt2 – nothing: it installs sbt2-core and sbt2-strategies, sbt2-papers with the papers extra, sbt2-server with the server extra, and both with all

pip install sbt2 gives the backtester and the examples; pip install "sbt2[papers]" adds the paper pipeline and pip install "sbt2[server]" the server. Every distribution is released together under one version, and sbt2 pins the others to it exactly.

Publishing

sbt2 is published on PyPI and depends on a stable nautilus-trader release from PyPI. A published package can't carry the Nautech index pin: [tool.uv.sources] doesn't reach the projects that depend on sbt2. Releases are cut from tags by a Woodpecker pipeline (see Quality). For a Git install, create a Python 3.14 environment with uv venv --python 3.14, then run uv pip install "sbt2[server] @ git+https://github.com/sbOogway/sbt2.git#subdirectory=sbt2-backend". Run these commands in an empty directory. Use uv so it gets the workspace members from the same commit; pip alone needs their exact versions in a package registry. The build needs neither buf nor protoc.

For an editable source checkout, clone sbt2 and run make sync at the repository root. It installs members from sbt2-backend/packages/ into sbt2-backend/.venv. The direct backend command is uv --directory sbt2-backend sync --locked.

Milestones

The engine milestones (M2–M7) come first; the library ones follow.

Milestone Delivers
M8: Installable library the data root and the config folder, the backtest() API with Report, automatic data
M9: Optimisation and the test lock Range on Params, the ask-and-tell optimiser interface with grid search, objectives, the deflated Sharpe ratio and the probability of backtest overfitting, the lock
M10: Walk-forward and plug-ins walk-forward (its design is Deferred until then) with a window-by-window heatmap, entry-point plug-ins for sources
M11: PyPI release stable nautilus from PyPI, the tag-driven release pipeline
⚠️ **GitHub.com Fallback** ⚠️