Library - sbOogway/sbt2 GitHub Wiki
sbt2 is a published package. A user installs it from PyPI, writes only a strategy in their own package, and sbt2 handles the rest: data, venue config, splits, optimisation, the test lock, running and reporting. This page is the user's view; the module pages hold the details.
A nautilus strategy on the sbt2 base, and nothing else:
from dataclasses import dataclass
from typing import Annotated
from sbt2.core import FractionSplit, Grid, Range, Strategy, backtest
@dataclass(frozen=True)
class TrendParams:
fast: Annotated[int, Range(5, 50, step=5)] = 10
slow: Annotated[int, Range(20, 200, step=10)] = 30
class Trend(Strategy[TrendParams]):
Params = TrendParams
... # plain nautilus from here on
if __name__ == "__main__":
tuned = backtest(
Trend,
instruments=["BTCUSDT-LINEAR.BYBIT"],
period=("2020-01-01", "2025-01-01"),
split=FractionSplit(validation=0.2, test=0.2),
optimise=Grid(objective="sharpe"),
)
final = backtest(
Trend,
instruments=["BTCUSDT-LINEAR.BYBIT"],
period=("2020-01-01", "2025-01-01"),
split=FractionSplit(validation=0.2, test=0.2),
params=tuned.best_params,
part="test",
)A script runs its backtests under if __name__ == "__main__":: each run forks from a process that re-imports the script, and without the guard Python refuses to start the runs (see Run). sbt2 run spec.toml does the same from a spec file (see Spec); notebooks need no guard.
The Python API (sbt2.backtest) and the CLI with spec files are two front-ends on equal footing over one core, the api module (see Architecture). Every feature exists in both. Where Python passes an object (the strategy class, a custom objective, a third-party optimiser), a spec file names it by import path.
Full nautilus, as described on Strategy; sbt2 puts no facade over the trading API. Strategies live in the user's own installed package, and sbt2 reaches them only through the class passed to backtest or the import path in a spec file.
A split divides a backtest's period into three parts, in time order:
| Part | Used for | Runs |
|---|---|---|
| train | fitting: the optimiser tunes parameters here | as often as you like |
| validation | checking what was fitted on data it wasn't fitted on: the optimiser ranks candidates here | as often as you like |
| test | the final answer | once per strategy: locked |
Splits are Splitter objects:
-
FractionSplit(validation=0.2, test=0.2): the last 20% of the period is test and the 20% before it validation; -
DateSplit(validation_start="2023-01-01", test_start="2024-01-01"): the same by date.
Either way the split resolves to dates, and the dates are hashed. There is no embargo: each part starts where the one before it ends. A call without part runs train and validation; test is only run when asked for. See Spec.
A study keeps a parameter search: study = "trend-btc" in a spec file puts its runs in a named collection of runs of one strategy, on one fixed context, that differ only in their parameters. The first run fixes everything but the parameters and pins the source of the strategy's module; each later batch adds new parameter values, and one that changes the context or the code, or repeats a stored run, is refused before anything runs. sbt2 studies list lists them and sbt2 report study <name> compares a study's runs, part by part. See Results.
- The search space is declared on the
Paramsfields withAnnotated[..., Range(...)], so the strategy file is self-contained. Lists in a spec file override it. - The optimiser is pluggable. Grid search is built in; others plug in through the same interface. The interface is ask-and-tell: the optimiser proposes a batch of candidates,
run.batchruns them in parallel, and their scores are told back. That is the shape of Optuna's ask-and-tell API, so an Optuna sampler (TPE and the rest) plugs in without sbt2 depending on Optuna (Optuna ask-and-tell; Akiba et al. 2019; see References). - The objective is a named sbt2 metric (
"sharpe") or anyCallable[[RunResult], float]. - The optimiser tunes on train and ranks on validation: every candidate runs on both, the objective is scored on validation, and train scores are shown next to it to expose overfitting. Every candidate is a stored run. The optimiser never touches test.
- Overfitting statistics come with every optimisation: the deflated Sharpe ratio of the best candidate, and the probability of backtest overfitting of the search. The deflated Sharpe ratio discounts the best validation Sharpe for how many candidates were tried; because every run is stored, the count is the lineage's real number of validation runs, not a guess (Bailey & López de Prado 2014). The probability of backtest overfitting is estimated by combinatorially symmetric cross-validation over the candidates' train and validation returns: how often the in-sample winner ranks below the median out of sample (Bailey, Borwein, López de Prado & Zhu 2017). Details on Results.
The test part is judged once per strategy, so it can't quietly turn into a second validation set. Validation isn't locked: it's the check you're meant to repeat.
-
Lineage: all versions of a strategy class, keyed by its qualified name (
my_strats.trend:Trend). - A test run is refused when its dates overlap a test period its lineage already ran on, whichever version or parameters did it. Moving the boundary therefore doesn't escape the lock.
-
reuse_test=True(--reuse-test) overrides the refusal. The result is then marked contaminated for good, naming the earlier runs. - The lock reads the results store: a test run counts from the moment it starts, and it can't be deleted (see Results).
- Renaming or moving the class starts a new lineage and escapes the lock. That is accepted: the lock guards against honest mistakes, not deliberate cheating.
Fetched automatically. Before a run starts, missing days are downloaded and ingested from the source its venue profile names; the user never calls download or ingest. The commands stay for filling the catalog ahead of time. See Run.
- One data root holds everything a user's data and runs produce:
--data, elseSBT2_DATA, per callroot=in the API. It is required, with no default and no fallback to the working directory. Data downloaded once serves every project. Layout on Architecture. - Venue profiles are configuration, in the config folder's
venues.toml:--config, elseSBT2_CONFIG. It is required too, with no default and no fallback to the working directory, and read on every run, so an edit applies to the next one. sbt2 ships no venue profiles; the GUI holds the defaults and installs them through the server's API. Layout on Architecture. - The known gaps describe the data held, so they live in the data root's
known_gaps.toml, edited by hand. - Run settings (memory budget, per-run cap, chunk sizes) have built-in defaults, overridable in
settings.tomlunder the root.
Sources are discovered through the entry point sbt2.sources. Bybit is registered the same way from inside sbt2, and other packages add more without a release of sbt2. See Data sources. Venue profiles are not plug-ins: they are configuration in the config folder.
- A
Reportobject frombacktest: metrics, equity, trades and fills as DataFrames, and for an optimisation the candidates andbest_params. - An HTML tearsheet on demand (see Results).
- A summary table printed by
sbt2 run: headline metrics per part. - Every run is also kept in the results store, where
sbt2 runslists it.
The backend component, sbt2-backend/, is a uv workspace. Its members are each their own distribution and add their subpackages to sbt2, a namespace package with no code of its own; the protocol (sbt2-protocol/) and the reserved GUI (sbt2-gui/) are the other components of the repository (see Architecture):
| Distribution | Import | Holds |
|---|---|---|
sbt2-core |
sbt2.core |
the backtester: every module on Architecture, and the sbt2 command |
sbt2-strategies |
sbt2.strategies |
example strategies, which the golden runs also use |
sbt2-papers |
sbt2.papers |
the Paper extraction pipeline |
sbt2-server |
sbt2.server, sbt2.protocol.v1
|
the GUI's server and the sbt2-server command (see Server on Architecture), with the Python generated from the protocol in sbt2.protocol.v1 (the import path follows the .proto package; see Protocol on Architecture) |
sbt2 |
– | nothing: it installs sbt2-core and sbt2-strategies, sbt2-papers with the papers extra, sbt2-server with the server extra, and both with all
|
pip install sbt2 gives the backtester and the examples; pip install "sbt2[papers]" adds the paper pipeline and pip install "sbt2[server]" the server. Every distribution is released together under one version, and sbt2 pins the others to it exactly.
sbt2 is published on PyPI and depends on a stable nautilus-trader release from PyPI. A published package can't carry the Nautech index pin: [tool.uv.sources] doesn't reach the projects that depend on sbt2. Releases are cut from tags by a Woodpecker pipeline (see Quality). For a Git install, create a Python 3.14 environment with uv venv --python 3.14, then run uv pip install "sbt2[server] @ git+https://github.com/sbOogway/sbt2.git#subdirectory=sbt2-backend". Run these commands in an empty directory. Use uv so it gets the workspace members from the same commit; pip alone needs their exact versions in a package registry. The build needs neither buf nor protoc.
For an editable source checkout, clone sbt2 and run make sync at the repository root. It installs members from sbt2-backend/packages/ into sbt2-backend/.venv. The direct backend command is uv --directory sbt2-backend sync --locked.
The engine milestones (M2–M7) come first; the library ones follow.
| Milestone | Delivers |
|---|---|
| M8: Installable library | the data root and the config folder, the backtest() API with Report, automatic data |
| M9: Optimisation and the test lock |
Range on Params, the ask-and-tell optimiser interface with grid search, objectives, the deflated Sharpe ratio and the probability of backtest overfitting, the lock |
| M10: Walk-forward and plug-ins | walk-forward (its design is Deferred until then) with a window-by-window heatmap, entry-point plug-ins for sources |
| M11: PyPI release | stable nautilus from PyPI, the tag-driven release pipeline |