Config Schema - ehharvey/homelab-ops GitHub Wiki
The fleet is declared in a git repo of k8s-style YAML documents. This page is
the reference for that repo: which files are read, every field each kind
accepts, and what checks it. The code is the source of truth
(internal/config parses and validates, internal/configsync reads the
repo). Where this page and the code disagree, the code is right, and this
page needs fixing.
Working examples live in examples/. A test parses and
validates every one of them the way a sync does, so they can't drift from
this schema.
Later sections:
-
What each reader does with the repo: the web app,
bootstrap render-seed, and the agent. - Known gaps: things the checks don't catch today.
- Proposed additions: a sketch of the schema changes in flight (#181, #199). None of those is implemented, and the strict parser rejects them today.
-
Every
*.yamland*.ymlfile at the repo root is read, in file-name order, and merged into one fleet.- Subdirectories are ignored, so they're a safe place for anything else (docs, scripts, drafts).
- How documents are split across files carries no meaning: one file, or one per kind, or one per node, parse the same.
-
Each file holds one or more documents, separated by
---. -
Validation errors cite positions in the merged list, e.g.
instances[2].static_ip: counted in file-name order, then document order. - A sync is all or nothing. A document that fails to parse, or a fleet that fails validation, rejects the whole commit. The previous synced state stays in place.
The web app syncs from:
| Setting | Meaning |
|---|---|
CONFIG_REPO_URL |
The repo: any go-git transport (https://, git://, a local path). Unset disables sync. |
CONFIG_REPO_REF |
The branch to read. Default main. A tag or commit SHA is not accepted. |
CONFIG_SYNC_INTERVAL |
Background poll interval (e.g. 5m). Unset means sync only on POST /sync. |
Each sync is a fresh shallow clone of that one branch. No git credentials are
supported yet, so the repo must be readable anonymously (see Known gaps). Keep
in mind that a public repo publishes everything in it. §28 of
docs/Decisions.md covers what that means for addresses (#199).
Every document must set kind:. Anything else is a parse error: a missing
kind, an unrecognised one, or an unknown field (a typo'd statc_ip fails
loudly rather than being dropped). Four kinds exist: Network, Instance,
App and Designation.
Addresses (cidr, gateway, static_ip, dns, dhcp_excluded_range) are
checked for syntax at parse time. The semantic checks below run afterwards, as
one pass that reports every issue at once.
A LAN the nodes sit on. IPAM draws static addresses from it (docs/Ipam.md).
kind: Network
name: home-lan
cidr: 192.168.1.0/24
gateway: 192.168.1.1
dhcp_excluded_range: 192.168.1.200-192.168.1.250
dns: [192.168.1.1]| Field | Type | Required | Rules and meaning |
|---|---|---|---|
name |
string | yes | Non-empty and unique among Networks. What Instance.network refers to. |
cidr |
prefix | yes | The LAN's subnet. IPv4 in practice: IPAM and the network/broadcast checks are IPv4-only. |
gateway |
address | see rule | Must be inside cidr. Needed for seed rendering whenever an instance on this network has a static_ip, since it becomes the node's default route. Validate doesn't catch its absence; rendering does. |
dhcp_excluded_range |
"<start>-<end>" |
no | The static range, meaning the addresses the LAN's own DHCP server must not hand out. Both ends must be inside cidr, with start ≤ end. When set, every static_ip on this network must fall inside it, and IPAM assigns omitted ones from it. |
dns |
list of addresses | no | Nameservers rendered into each node's network.yaml. |
One physical IncusOS machine.
kind: Instance
name: node0
mac: 02:00:00:00:00:01
network: home-lan
static_ip: 192.168.1.201
disk: single
nic: single
security:
tpm: true
secure_boot: true
applications: [incus]| Field | Type | Required | Rules and meaning |
|---|---|---|---|
name |
string | yes | Unique among Instances, and a lowercase hostname label: 1–63 of a-z, 0-9 and -, not starting or ending with -, not all digits. Lowercase only, because Node0 and node0 would be "unique" and yet name the same host. It's the node's identity everywhere downstream: its Incus cluster member name, its cert names, its web-app store key, and the agent's AGENT_NODE_NAME. #193 also makes it the node's hostname. The rule is stricter than Incus's own member-name check, so a name that passes is valid everywhere. |
mac |
string | yes, in effect | The NIC IncusOS configures as eth0. Not validated; a wrong MAC leaves the node's network unconfigured at boot. Quoting is optional. |
network |
string | yes | A declared Network's name, checked for every instance, DHCP or not. The instance must sit on that network. |
static_ip |
address | no | Must be inside the network's cidr and its dhcp_excluded_range (if set). It can't be the gateway, network or broadcast address. Omitted means something different per reader: the web app assigns one, while render-seed renders DHCP (see What each reader does). The address matters beyond networking, because it's what makes a node an Incus cluster member (§26). A DHCP node comes up unclustered. |
disk |
string | yes | Only single is implemented (checked at render). |
nic |
string | yes | Only single is implemented (checked at render). |
security.tpm |
bool | no, default false
|
Whether the machine has a TPM. false tells the IncusOS installer to proceed without one. |
security.secure_boot |
bool | no, default false
|
The same, for Secure Boot. |
applications |
list | yes | IncusOS host applications. Only [incus] is supported (checked at render). This is unrelated to kind: App. |
tunnel_ip is not a config field. The web app assigns each node's
WireGuard overlay address itself, and a tunnel_ip: in git is rejected as an
unknown field.
A workload the app-manager agent reconciles against live Incus.
docs/AppManager.md explains the design, and docs/AppClasses.md covers
which kinds of App a renderer can be.
kind: App
name: agent
type: agent
replicas: per-node
image:
server: https://ghcr.io
protocol: oci
alias: ehharvey/homelab-ops/agent:latest
params: {}| Field | Type | Required | Rules and meaning |
|---|---|---|---|
name |
string | yes | Non-empty and unique among Apps. A per-node App's instances are named <name>-<instance name>. |
type |
string | yes | The renderer-registry key, e.g. agent. Not checked against a registry at sync time: config doesn't know which renderers a binary registered. |
replicas |
count or per-node
|
yes, no default | How many, never where. A positive count, or per-node for one per declared Instance. At most one per-node App per type. |
image.server |
URL | no | Image server, e.g. https://ghcr.io. |
image.protocol |
string | no |
oci, simplestreams, incus, or empty. |
image.alias |
string | one of the two | Image alias, e.g. owner/repo:tag. |
image.fingerprint |
string | one of the two | Image fingerprint. alias or fingerprint must be set. |
params |
map of string to string | no | Opaque, renderer-specific settings. |
There's deliberately no placement, strategy or version field; see
docs/AppManager.md § kind: App schema. App reconciliation is paused under
§27 (#98) apart from the agent itself (#203).
Which node's agent leads: leaderelection.Designated, docs/Decisions.md
§25 (#101).
kind: Designation
primary: node0 # an Instance name: a node, not an agent instance| Field | Type | Required | Rules and meaning |
|---|---|---|---|
primary |
string | yes | Must name a declared Instance, exactly as written. |
-
At most one per repo. A second is a validation issue
(
designations[1]). Zero is valid and means no agent acts: the safe default, matchingMayActtreating an unknown designation as "not leader". -
There's no
epoch. A stale checkout is fenced by git itself: each agent publishes the commit it's on, and an agent behind a peer stands down. A new primary also waits for the old one to publish that it has stopped acting. So a failover is one commit changingprimary(§25). Anepoch:left in a repo is rejected as an unknown field rather than silently ignored. - It names a node, not an agent instance, so the agent's own blue-green self-upgrade needs no change here.
-
Why a kind of its own rather than
primary: trueon anInstanceor a field on the agentApp: leadership is fleet-wide, not a property of a node or of the agent's image; a separate document shows up on its own in a change report; and it can be validated, which a string inparamscan't. - The web app parses and validates it, but doesn't store it or report changes to it yet.
| Reader | Reads | Differences |
|---|---|---|
| Web app | The branch above, on POST /sync or every CONFIG_SYNC_INTERVAL
|
Runs Validate, then IPAM. An omitted static_ip gets an address from the network's dhcp_excluded_range, stable across syncs. It errors if the range is unset or exhausted. The web app stores the result, reports what changed against the previous sync, and renders each node's seed and image from it. |
bootstrap render-seed |
One local file (--file), no git |
Needs exactly one Network and one Instance; Apps are ignored. There's no IPAM, so an omitted static_ip means DHCP, and the node comes up unclustered. Use examples/single-node/. |
Agent (cmd/agent, #101) |
The same repo, from its own CONFIG_REPO_URL/CONFIG_REPO_REF
|
Runs with the web app down. It keeps a persistent full clone (AGENT_REPO_DIR) and fetches into it, rather than a fresh shallow clone, because leader election asks whether a peer's commit is in its history. It runs Validate too, and a commit that fails keeps the previous one current. Today it reads only the Designation; it never reads addressing. |
Things the checks don't catch today, recorded so nobody assumes they do:
-
Names derived from an
Instance.namecan exceed Incus's 63-character cap. A per-node App's instances are named<app>-<instance>, and the agent's also get a-g<N>generation suffix. So a name can pass the rule above and still produce an invalid derived one. CappingInstance.namealone can't fix it, because the length also depends on the App's name and on an unbounded generation. The check belongs where the derived name is built (#203, and #98 when it resumes). -
A missing
gatewaypassesValidate. It's caught at render instead, which is later than an operator would expect. macis free text.- No private repos. Sync passes no credentials. §28's git-addressed networks assume a private repo, so they need this first. So does the agent syncing one (§28 mentions a read-only deploy key).
A spike for #207: the schema changes other issues will make, sketched against the schema above so they can be judged together. Nothing here is implemented. The strict parser rejects every field and kind in this section. Each owning issue makes the final call; where there's a recommendation, it's only that.
The seed renderer and the agent need to know whether a node bootstraps the cluster or joins it (§27). The #193 spike showed the role is fixed when the node is flashed: a node seeded as a bootstrap member can never join. That makes it a property of the node.
kind: Instance
name: node1
# ...
cluster_role: join # bootstrap | joinRecommendation: a field on Instance, rather than a kind: Cluster
listing members, with exactly one bootstrap per fleet.
- No join order is needed. The agent joins missing members one at a time, each against the running cluster, so document order never becomes a contract. That was #181's worry about deriving the role from order.
-
Open: the default when omitted. Treating an omitted role as
bootstrapkeeps today's single-node repos valid. But a second node that forgets the field would then start a second cluster. Requiring the field, asreplicasis, is safer. It costs a one-line edit to existing repos, includingdev/git-fixture. - The first designated primary must be the bootstrap member, because that's where the first agent is deployed (#100). Once other members have joined, a failover can name any member.
With a public repo, a Network may carry only its name. Its addressing then lives in web app state (§28):
kind: Network
name: home-lan # addressed from web app state: no cidr, gateway, range or dns hereThe rules, from §28:
- All or nothing. A Network carries all its addressing fields or none; a partial set is an error.
- One source per network. If git and state both address the same network, the sync is rejected.
-
static_ipfollows its network's mode. In git it's allowed only on a git-addressed network.
render-seed keeps reading full Network documents from its local file. An
example repo in this mode, plus a proposed format for the web app's addressing
file, is drafted in #208 and waits on #199.
The spike showed a node that hasn't joined reports its machine UUID as its
server name unless network.yaml sets dns.hostname. The fix is for the seed
renderer to set dns.hostname from Instance.name, with no new field. That
also makes the name validation under Known gaps a prerequisite.
A placement field on App (Incus cluster groups) is anticipated, as a
separate field from replicas and invalid alongside per-node
(docs/AppManager.md). It's paused with App reconciliation (§27), so no shape
is proposed here.