Engines Race and Hoard - Kheopsian/Hydranos GitHub Wiki
Applies to Hydranos 4.4.
This page is for anyone who wants to know what the race and hoard engines really do differently, whether to add a third engine, and which engine settings have an effect. At the end you will know how the two roles differ, how to add, remove and use an extra engine, how to move or duplicate a torrent between engines, which peer protocols are on, and what engines cannot do in 4.4. Download slots, share limits and speed caps are per engine too; they have their own page: Queue, Share and Speed Limits.
An engine is one BitTorrent session inside the hydranos process. There is no separate engine process. Each engine owns:
- its torrent table and its disk manager (an open-file pool);
- a TCP listener on its listen port, and a uTP socket on the same port number (UDP);
- its own outbound dial queue, with its own connection limits;
- its own DHT node and LSD (local peer discovery), if on;
- its own announce scheduler, and its own scrape of torrents not announced yet;
- its own peer id, drawn at every start (
-HY, the version,-, then 12 random characters), sent in its announces and in every handshake, incoming and outgoing; - its own resume state, in
<config dir>/<engine id>/state.db.
The torrent metadata (.torrent bytes) is not in the engine: it lives in the store, hydra.db, shared by all engines. See Architecture for the files on disk.
Every instance always runs two engines, race and hoard, built from the [race] and [hoard] sections of default.toml. You can add more (see Adding more engines below). Engines are built once, at boot: adding or removing an engine, and most engine settings, take a restart (the settings applied live are listed under Per-engine settings that apply).
flowchart LR
subgraph P["hydranos process"]
subgraph R["engine race, role race"]
R1["TCP + uTP listener<br/>port 16171"]
R2["dial queue, DHT, LSD"]
R3["announce scheduler"]
R4["race/state.db"]
end
subgraph H["engine hoard, role hoard"]
H1["TCP + uTP listener<br/>port 16172"]
H2["dial queue, DHT, LSD"]
H3["announce scheduler"]
H4["hoard/state.db"]
end
subgraph V["engine vpn1, example extra engine"]
V1["TCP + uTP listener<br/>port 16473"]
V2["dial queue, DHT, LSD"]
V3["announce scheduler"]
V4["vpn1/state.db"]
end
end
R2 --> DEF["default route<br/>no bind_interface"]
R3 --> DEF
H2 --> DEF
H3 --> DEF
V2 --> WG1["wg1<br/>vpn1 bind_interface"]
V3 --> WG1
DEF --> NET["peers, trackers, DHT nodes"]
WG1 --> NET
IN["incoming peers"] --> R1
IN --> H1
IN -- "through wg1" --> V1
Incoming peers reach each engine on its own port. Since 4.4, everything an engine sends (peer dials, announces, DHT, LSD, webseeds, scrapes) leaves by that engine's own way out, with its own peer id: see What an extra engine isolates below.
Each engine has a role, race or hoard. The role is a behaviour, not a name: an extra engine vpn1 with role race announces and drains like the race engine. Anything that is not race behaves as hoard for announcing.
Role race
|
Role hoard
|
|
|---|---|---|
| Purpose | churn: torrents that arrive, race in a young swarm, then leave (deleted or graduated), often on a fast, small disk | torrents that stay: a library seeded for months: built for very large catalogues (published measurement: ~293,000 torrents on 4.2, see Deployment Topologies) |
| Trackers per announce pass | every tracker of every tier: the torrent stays in every swarm | stops at the first tier that answers (BEP 12 style) |
| Re-announce | the tracker's interval, or its min interval when larger; 30 minutes without one; a wait under 60 s is raised to 60 s |
the same |
| "Unregistered torrent" retry | every 5 s for registration_retry_minutes (6 by default), while no tracker has accepted the torrent yet and it is still downloading |
never |
| Race drain (frees full disks) | yes, one per race-role engine | no |
Refusing adds when space is short (add_block_enabled) |
yes, if enabled | never |
| Section an extra engine inherits | [race] |
[hoard] |
Download slots in the shipped default.toml
|
none (active_downloads not set = no ceiling) |
active_downloads = 10 |
max_connections in the shipped default.toml
|
4000 | 12000 |
max_uploads_per_torrent in the shipped default.toml
|
100 (read only with choking = true, off in both) |
-1 |
LSD when enable_lsd is missing |
on | off |
Everything else is the same code for both roles, transfer speed included: a race torrent does not download or upload faster than a hoard torrent. Also shared: the announce scheduler, per-tracker pacing, the circuit breaker, BEP 3 events, the scrape of torrents not announced yet, DHT, PEX, LSD, UDP trackers, webseeds, the transit sweep, and the download queue, share limits and speed caps (Queue, Share and Speed Limits). The announce rules are detailed on Trackers and Announces; the drain and the add refusal on Race Drain and Graduation.
4.3 meant a downloading race torrent to re-announce every 5 s, then every 30 s, but never did; 4.4 drops that cadence on purpose: after a successful announce, race waits for the tracker's interval like hoard. Use Reannounce in the context menu (60 s cooldown per torrent) if you need an earlier announce.
The Race tab lists the torrents of every race-role engine and the Hoard tab those of every hoard-role engine; an extra engine's rows are labelled with it (local-vpn1). A torrent held by two engines is two rows, each with its own copy's figures.
A few features look at the engine id race or hoard, so an extra engine with the same role does not get them:
- The race timeline recorder (the Timeline of a race torrent) only watches the engine
race. - The Overview cards Race engine and Hoard engine address the ids
raceandhoard; every other engine has its own Engine {id} card (Monitoring and Logs)./api/hoard/torrentslists the enginehoardonly, whatever the query (for another engine:GET /api/engines/<id>/torrents). The per-torrent write routes under/api/race/…and/api/hoard/…(category, location, tags) address the id too, unless you add?engine=. Not everything under those prefixes is tied to the id:/api/race/torrents,/api/race/pageand/api/hoard/pagelist every engine of that role, and/api/hoard/torrents/<hash>/verifyfinds the torrent in any engine. Every engine is also reachable as/api/engines/<id>/…(see REST API).
If you are unsure, file your categories as Hoard. Race is for racing fresh uploads, and the race drain can delete torrents together with their data when a race disk fills. Torrents imported from qBittorrent always land in hoard.
The category picks the engine: a category with mode hoard sends a torrent to hoard, any other mode to race (rule and refusals: Categories and Routing).
- Race: torrents you grab as soon as they are posted (autobrr, Sonarr/Radarr releases you want to win on), on a disk that you want emptied automatically. Race announces to every tracker, retries a not-yet-registered torrent quickly, and the drain deletes or graduates old torrents when the disk fills.
- Hoard: everything you keep and seed for months: imports from another client, completed downloads that graduated, a library you add with existing data. Hoard announces politely (first answering tier only) and its download slots stop a big batch from downloading everything at once.
Hoard never deletes anything on its own. Choose race only for torrents you mean to let go: it puts them in every swarm from the first minutes, and its drain clears them out afterwards (Race Drain and Graduation).
An extra engine is any engine other than race and hoard. Typical reason: a separate listen port bound to its own network interface, so inbound peers of some torrents reach you through another tunnel.
From the UI: Nodes tab › This node › + Add an engine:
| Field | Meaning |
|---|---|
| Id | engine id: letters, digits, - and _ only (e.g. vpn1); up to 12 characters keeps a managed WireGuard device named wg-<id>, a longer id gets a shortened name (Managed WireGuard) |
| Role (how it behaves) |
hoard (default) or race
|
| Listen port | its own port, 1–65535. The form refuses a port held by a running engine only: an engine added but not yet started by a restart is not checked. If two engines end up with the same port, the later one is given the next free port at boot (warning listen_port missing or taken: using a free one), so pick distinct ports yourself |
| Interface (its tunnel, optional) | network interface to pin every socket of the engine to (e.g. wg1), Linux only: elsewhere the field is greyed out (Linux only). The form accepts it only if /sys/class/net/<name>/operstate reads exactly up; some tunnel interfaces (WireGuard often does) report unknown and are refused: write the block by hand instead (below). For a tunnel that Hydranos brings up itself, see Managed WireGuard
|
Add writes an engine block into default.toml and shows The config changed. Engines are built at boot, so this takes a restart. with Restart now. Nothing starts before the restart. The table under This node then shows the engine with its Socket state: listening or not listening (for example when the interface is down).
A new extra engine inherits its role's SOCKS5 proxy, announce proxy and interface, but not race's or hoard's WireGuard tunnel, gluetun port, PROXY-v2 port or direct on purpose. What the kill switch then does with it depends on the mode: in WireGuard mode it keeps the engine off the network until you give it a tunnel or Direct (default interface) in its Way out list; in SOCKS5 proxy mode the inherited proxy is its way out, so it runs; in Gluetun mode an engine with no interface runs inside gluetun's network like the others; in SOCKS5 + PROXY-v2 relay mode (and in Direct mode with kill_switch = true) it is blocked only if it has no proxy, no interface and no direct on purpose. See VPN and Egress.
By hand: the same block, written as a TOML array of tables. The [engine.session] table is a sparse override: write only what differs; every other key comes from [race] or [hoard] depending on the role.
[[engine]]
name = "vpn1"
role = "hoard"
engine_id = "vpn1"
[engine.session]
listen_port = 16473
bind_interface = "wg1"Rules: the id is engine_id, or name if engine_id is empty; a block with an addr is skipped (it describes another machine, see Nodes and Multiple Machines); a block with no role is skipped without any message; a block whose role is not race or hoard starts nothing and logs unknown role: this agent starts no engine; a block whose id is race or hoard replaces that built-in engine; a malformed override is refused as a whole and the role's section ([race] or [hoard]) is used instead. Every key: Configuration: Engines.
With Docker, publish the new port yourself (16473:16473/tcp and 16473:16473/udp); the shipped compose only publishes 16171 and 16172. Ports and port forwarding: Networking.
Removing: Remove on the engine's row (not offered for race and hoard) asks Remove engine {id}? Its data stays on disk. An engine that still holds torrents is refused (DELETE /api/engines/<id> answers 409 {id} still holds {n} torrent(s): move them to another engine or remove them first): move them with Move to engine › (below) or remove them first. An empty engine has its block deleted from default.toml and the page asks for a restart; until then the engine keeps running with its listener open, though it has no torrent left to announce or seed. Restart now closes it.
Since 4.4 an extra engine is a separate network identity. On Linux, its bind_interface (or its managed WireGuard tunnel) carries everything it sends: listener, uTP, its own outbound dial queue, HTTP and UDP announces, DHT, LSD, webseeds and scrapes, and it handshakes with its own peer id. Its own max_connections limits its own outgoing connections. A torrent on vpn1 is therefore seen by trackers and peers as vpn1's peer id and port, through wg1. What stays outside (tracker names resolved by the host for an interface Hydranos did not create, the daemon's own requests): VPN and Egress.
Other limits of extra engines:
- The Overview's Race engine / Hoard engine cards and the Benchmark comparison show only
raceandhoard; extra engines get their own cards and curves, and count in tracker health and/metrics(Monitoring and Logs). - The qBittorrent shim lists every engine, but adds only into
raceorhoard(by category); see qBittorrent Shim and Automation. - Extra engines of another node are not shown on this instance; see Nodes and Multiple Machines.
No category routes to an extra engine: a category mode only knows race and hoard. There are three ways in:
-
Add with an explicit engine (native REST API). The engine wins over the category's mode; the save path still comes from the category or the request.
curl -H "X-Api-Key: YOUR_API_KEY" \ -F "torrents=@/path/to/file.torrent" \ -F "category=tv-hoard" \ -F "engine=vpn1" \ http://localhost:8199/api/torrents/upload
Both add routes (this one and the JSON
POST /api/torrents) refuse an unknown engine with a 400 (no engine X). -
From the web UI: right-click a torrent (or a selection) › Node › Move to engine › or Duplicate to engine ›, then pick
local-<id>. The Node group appears as soon as this instance has two engines, so on every default install. The engine it already sits on is not offered. -
Move an existing torrent over the REST API between engines of this instance. Use
?engine=to say which copy moves when the torrent is in several engines.curl -X POST -H "X-Api-Key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"engine":"vpn1"}' \ "http://localhost:8199/api/torrents/0123456789abcdef0123456789abcdef01234567/engine?engine=hoard"
POST /api/torrents/<hash>/copywith the same body is Duplicate to engine.
Move and Duplicate differ in what the target engine gets:
Move to engine (…/engine) |
Duplicate to engine (…/copy) |
|
|---|---|---|
| Files | stay where they are | stay where they are; both engines read the same files |
| Source copy | released, once the target has adopted the torrent | kept, seeding as before |
| State on the target | the source's own resume record: pieces, trackers (edits included), byte counters, seeding time, stopped state; no recheck | a fresh add in seed mode: treated as complete without a recheck, upload/download counters and seeding time at 0, trackers from the .torrent, started |
| Refused | if the target refuses, nothing changes (the target engine refused it; nothing was changed); if the source refuses to let go, the target's copy is removed again |
409 {engine} already seeds it if the target holds it already; 409 … does not hold all of it yet … while the source is incomplete or being checked |
| Message in the UI | Moved {n} to "local-vpn1". The files did not move. | Seeding {n} more from "local-vpn1". Same files, second identity in the swarm. |
Only a complete torrent can be duplicated: the new copy trusts the source's pieces and does not read the files. Duplicating is for seeding one payload under two identities (two ports, two tunnels); moving is for changing which engine, and so which way out, a torrent seeds from.
Bulk actions act on each row's own copy: Set location… and Change category + move files › on a local-vpn1 row move vpn1's copy, not the hoard or race copy of the same hash.
Moves across engines and nodes as bulk tasks: Jobs, Bulk Actions and Moving Data.
Engine settings live in [race], [hoard] and each [engine.session] override (Config › Advanced settings (default.toml) › Session Race / Session Hoard; the editor covers race and hoard only, edit an extra engine's block in the file). Most are read when the engine starts: restart after changing them. The listen port has a live path (see Networking), and the queue, share-limit, speed-cap and choker keys are applied live, no restart.
The keys that matter (defaults, the value used when a key is missing, and live paths: Configuration: Engines):
| Key | Effect |
|---|---|
listen_port |
TCP listener and uTP socket; the port announced to trackers |
bind_interface |
Linux only: pins every socket of the engine (peers, announces, DHT, LSD, webseeds, scrapes) to that interface |
enable_ipv6 |
adds an IPv6 listener and IPv6 peer sources |
enable_dht / enable_pex / enable_lsd
|
DHT / PEX / local peer discovery on or off |
enable_webseed |
webseed (BEP 19) fetching |
enable_udp_trackers |
announce to udp:// trackers |
max_connections |
ceiling on this engine's outgoing dials; inbound always accepted |
aio_threads |
size of the open-file pool (not a thread count) |
registration_retry_minutes |
how long a race-role engine retries a torrent its trackers call unregistered |
start_paused |
holds the engine's announces and dials at start until released |
active_downloads, queueing, active_seeds, active_limit, dont_count_slow_torrents, slow_torrent_*
|
download and seed queue, live |
max_ratio, max_seeding_time, max_inactive_seeding_time, share_limit_action
|
share limits, live |
upload_rate_limit, download_rate_limit
|
speed caps in bytes/s, live |
choking, max_uploads_per_torrent, peer_timeout
|
choker and peer idle timeout, live |
The queue, share-limit, speed and choker keys, and start_paused: Queue, Share and Speed Limits. The network-mode keys (SOCKS5, announce proxy, PROXY-v2, gluetun, WireGuard, allow_direct) are written by the Network tab: Networking and VPN and Egress.
Default trap: when max_connections is missing, there is no cap (the shipped default.toml sets 4000 and 12000). DHT, PEX and webseeds are on when their key is missing, as in the template; LSD is on for race and off for hoard.
inactivity_timeout and [race.custom_choking] are read by nothing (see the last section). Full key reference: Configuration: Engines.
| Protocol | In 4.4 |
|---|---|
| DHT (BEP 5) | per engine, enable_dht. Each engine's DHT node is pinned to the engine's interface (bind_interface or its tunnel); behind a SOCKS5 proxy it is off whatever the key says. Private torrents are never put in the DHT. |
| PEX (BEP 11) | per engine, enable_pex. Off: PEX is not advertised and incoming PEX is ignored. |
| uTP | on, on the engine's listen port (UDP). TYPHON_DISABLE_UTP=1 turns it off (TCP only). |
| Encryption (MSE/PE) | always on, no setting. Incoming: plaintext and encrypted both accepted. Outgoing: plaintext first, encrypted as fallback. TYPHON_NO_MSE=1 disables the encrypted fallback. |
| Webseeds (BEP 19) | per engine, enable_webseed (on when the key is missing). Fetched through the engine's interface, or its proxy. |
| BitTorrent v2 | v2-only .torrent files are supported (SHA-256 piece verification). Hybrid torrents are handled as v1 and announced under their v1 hash. A magnet carrying only a v2 hash is refused: add the .torrent. |
| UDP trackers (BEP 15) | per engine, enable_udp_trackers (on when the key is missing); not announced behind a SOCKS5 proxy. See Trackers and Announces. |
| Local peer discovery (LSD, BEP 14) | per engine, enable_lsd: on for race and off for hoard when the key is missing (the template says the same). Multicast 239.192.152.143:6771 and [ff15::efc0:988f]:6771 on the engine's interface, about once a minute, each torrent at most every 5 minutes; only torrents that are downloading or have peers, never a private torrent. Off behind a SOCKS5 proxy. Read at the engine's start. |
| Scrape (BEP 48, BEP 15) | only for torrents no tracker has answered for yet; other swarm counts come from announce answers. See Trackers and Announces. |
Environment variables: Configuration: Environment Variables, CLI and Signals.
active_downloads sets an engine's download slots: how many of its torrents download at the same time. Every 30 s the queue manager starts the waiting torrents whose swarm has the most seeders, up to the ceiling, and stops the excess; a stopped-by-slots torrent shows queued (Waiting for a slot) and keeps its progress. 0, negative or missing means no ceiling. Torrents you stopped yourself are left alone.
In the shipped default.toml, hoard has 10 slots and race has no ceiling. The setting is per engine (an extra engine inherits its role's value unless its block sets one) and applied live: change it in Session Race / Session Hoard, or with the qBittorrent API's setPreferences max_active_downloads (which writes every local engine). POST/DELETE /api/hoard/download-slots no longer exist (405); the GET stays.
Slow torrents, the seeding queue (queueing), share limits, speed caps, the choker and a worked example ("keep 5 downloads running"): Queue, Share and Speed Limits.
Accepted but ignored in 4.4:
inactivity_timeout(in the template) and[race.custom_choking](from 3.x) are read by nothing — the peer idle timeout ispeer_timeout, and the choker ischoking = truewithmax_uploads_per_torrent; the daemon names[race.custom_choking]in a startup warning — leave them or delete them, nothing changes.
Also not available: an encryption mode setting, queue positions, and a client-wide speed cap outside the qBittorrent API (Queue, Share and Speed Limits). HYDRANOS_ENGINE_NET=0 takes every engine off the network, not one (what still runs: Networking); to hold one engine at start, use its start_paused.
- Concepts and Glossary — the short version of this page.
- Trackers and Announces — announce timing, events, UDP, peer id.
- Race Drain and Graduation — what happens when a race disk fills.
- Configuration: Engines — every engine key and its default.
- Queue, Share and Speed Limits — download slots, share limits, speed caps, choker, startup pause.
- VPN and Egress — what leaves through which interface.
- Managed WireGuard — one tunnel per engine, built by Hydranos.
- Networking — ports, port forwarding, the Network tab.
- Nodes and Multiple Machines — engines on other machines.
- Monitoring and Logs — what the Overview and Benchmark count.