Storage Seam - alexherrero/agentm GitHub Wiki

Storage seam reference

The storage seam is the contract between the memory engine and whatever holds its files (scripts/storage_seam.py). The engine never touches the filesystem directly β€” it calls seven small verbs, and a backend answers them. Every verb speaks the seam's own Locator type, so no filesystem assumption ever reaches the engine. Two backends implement the contract today: the device-local store that ships as the fresh-install default, and the synced vault, wrapped behind the same verbs. A shared conformance suite holds both to one objective contract. This page is the lookup for the verbs, the value types, the registry, and both backends; for why the seam is shaped this way, see Memory↔storage seam.

Note

All five parts of V5-1 are shipped. Part 1 laid down the abstract contract: StorageBackend, the Locator/Info/Capabilities types, the conflict_strategy slot, the BackendRegistry, and the three-tier taxonomy. Part 2 added the DeviceLocalBackend, the fresh-install default. Part 3 added the backend-agnostic conformance suite, which rides the existing cross-OS unit-test step rather than a new gate. Part 4 added the VaultBackend, wrapping the synced vault write path behind the same seam. Part 5 added backend selection: the storage.backend config key and resolver, the Capabilities-read (capability-request matching itself shipped later, in V5-7, adding select_backend(required=...) and CapabilityMismatchError), the fail-loud guard that never silently demotes to device-local, and the read-only doctor storage preview. Part 5 did not include the engine cutover β€” that landed later, in V5-3 (v5.5.0); see the Memory-storage seam design Β§ 3 for what it changed. No index or abstract-promotion ships yet: DerivedMaintenance stays an abstract class with reindex/changed_since reserved as names only, and the real derived index lands in V6. Anything this page marks (V6) is a forward reference, not current behavior.

⚑ Quick Reference

Verb Signature Returns Raises
resolve resolve(*parts) Locator β€” a backend-relative handle (resolve() = backend root) InvalidLocatorError on a .. segment
read read(locator) str β€” text content at locator FileNotFoundError if absent
write write(locator, content) Locator β€” the locator written (round-trips through read) backend's choice
list list(locator) list[Locator] β€” immediate children ([] if empty) backend's choice on a non-existent dir
exists exists(locator) bool β€” file or directory present β€”
info info(locator) Info β€” metadata (carries mtime) raises if absent
mkdir mkdir(locator) Locator β€” the directory (idempotent) β€”
capabilities property β†’ Capabilities the four-boolean descriptor the backend declares β€”

Important

No pathlib.Path crosses the seam. Every verb operates on and returns the seam's own Locator (or Info), never a pathlib.Path β€” that is what keeps a filesystem assumption from reaching the engine. A filesystem backend uses Path internally (root / key); only handing one back is the violation, enforced statically by check-storage-seam-no-path-leak. Text (str) is the v1 currency; a bytes channel is a deliberate future extension.

The Locator type

Locator is the seam's own handle β€” it names a place in a backend's namespace and does no I/O. It is not a pathlib.Path: it carries a normalized, backend-relative key and exposes only namespace operations. It is frozen and hashable, so a backend may use it as a dict key (the in-memory fixture does).

Member Type Detail
key str The normalized backend-relative key. The root locator is "". Normalized at construction via normalize_key.
parts property β†’ tuple[str, ...] The key's segments; () for the root.
name property β†’ str The final segment; "" for the root.
child(*parts) β†’ Locator Derive a sub-locator by appending parts (each normalized, no escape).
__str__ β†’ str The bare key.

Normalization + root-confinement (normalize_key)

Every Locator is canonicalized at construction β€” however built (direct, via resolve, via child) β€” by normalize_key(key) -> str:

Input segment Result
empty ("") or . dropped
leading / (absolute-looking) silently relativized β€” the empty leading segment is dropped; a locator is always backend-relative
.. rejected β€” raises InvalidLocatorError; the seam has no upward-traversal semantics
anything else kept, rejoined with /

This is the safety property: a key can never address outside the backend root.

The Info type

Info describes a locator β€” it is the frozen dataclass the info verb returns.

Field Type Detail
locator Locator The locator described.
is_dir bool Whether it's a directory.
size int Bytes (0 for a directory).
mtime float Modification time, epoch seconds. The changed_since granularity β€” the incremental-feed op reserved here keys on it, the lean v1 choice over a content-hash log.

The Capabilities type

Capabilities describes what a backend can promise β€” a frozen dataclass each backend declares. It carries four booleans, all defaulting to the conservative floor (False). The contract (part 1) only defines them and both backends already declare their values β€” device-local (kernel-built-in) all-False, vault (the crickets obsidian-vault plugin β€” see The VaultBackend) concurrent_writers/sync/conflict_files True. Part 5 adds the read side (task 2, shipped): the resolver's select_backend (backend_selection.py#L340) returns the selected backend instance, whose .capabilities property the kernel reads directly to learn what it can promise β€” proven by TestCapabilitiesRead (scripts/test_backend_selection.py#L725), which reads all four booleans off both backends through the selection surface (the vault case via a synthetic plugin fixture) and pins the four-key shape as a superset (additive growth of new keys is allowed; a rename/removal breaks the pin). This is read-and-surface only β€” capability-request matching (a consumer declaring what it needs and selection honoring it) is V5-7. V5-7 Part 1 (shipped): select_backend now accepts an optional required: Capabilities | None = None keyword parameter; when provided, it runs a subset check after resolving the backend and raises CapabilityMismatchError (a StorageSelectionError subclass) if any True requirement is unmet. required=None (default) preserves prior behavior. A dataclass so the set can grow.

Field Default Meaning
concurrent_writers False Safe under more than one writer process.
conflict_files False The backend may surface conflict copies (e.g. a sync layer's "(conflicted copy)" files) the engine must tolerate.
encryption False Content is encrypted at rest by the backend.
sync False The backend's tree is replicated by an external sync layer β€” the property that makes a SQLite index on it a corruption pattern. This is the flag the Tier.syncs policy ties to: the local-index tier is pinned never-sync precisely because a replicated database file corrupts.

The StorageBackend ABC

StorageBackend is the abstract interface a backend conforms to. A concrete backend registers under a protocol name (device-local, vault) via the BackendRegistry and implements every verb. Concrete backends are out of scope for this part; the registry they will plug into ships here.

resolve(*parts) -> Locator

Make a backend-relative locator from path parts. resolve() with no parts is the backend root. The naming verb: it produces the seam's locator type, the engine's only handle on storage.

read(locator) -> str

Return the text content at locator.

Condition Result
Content present The text as str.
Nothing there Raises FileNotFoundError β€” distinct from InvalidLocatorError (a malformed key, a caller bug).

write(locator, content) -> Locator

Write text content at locator; return the locator written. The contract: the write is durable and atomic, and the returned locator round-trips through read.

Note

A filesystem backend composes the vault write protocol here (V5-0 atomic_write + content-hash CAS + vault_mutex), building on existing write-safety instead of reinventing it. The abstract contract only declares the shape; the composition lands with the concrete backends (parts 2 / 4), not in this ABC.

list(locator) -> list[Locator]

List the immediate children of locator as locators.

Condition Result
Empty directory [].
Non-existent directory Pinned to return [], never raise by the part-3 conformance suite (the cross-backend list-on-absent contract).
Otherwise The immediate children as Locators, never paths.

exists(locator) -> bool

Whether anything (file or directory) is present at locator.

info(locator) -> Info

Return Info for locator (raises if absent). Carries mtime β€” the granularity changed_since reads.

mkdir(locator) -> Locator

Ensure a directory exists at locator; return it. Idempotent β€” calling it on an existing directory is not an error.

capabilities -> Capabilities (property)

What this backend promises β€” see Capabilities.

conflict_strategy -> str (property)

How this backend reconciles divergent concurrent writes β€” a named policy, distinct from the Capabilities booleans. Where those describe what the backend can promise, conflict_strategy names the reconciliation policy selection (part 5) reads to decide how to treat a conflict. Unlike capabilities (abstract β€” every backend must declare it), this is a concrete property (scripts/storage_seam.py#L230) defaulting to the conservative floor "none" (last write wins β€” nothing to reconcile), mirroring how the Capabilities booleans default safe.

Strategy Meaning Who declares it
"none" Last write wins; nothing to reconcile. The inherited floor β€” a backend that doesn't override it gets this. StorageBackend default; DeviceLocalBackend inherits it (single machine).
"whole-file" The whole file is the conflict unit β€” a named policy part-5 selection reads, not an executor; agentm ships no automated whole-file merger, only the detect-and-notice machinery (detect_conflict_files + the conflict-merger SessionStart hook) that surfaces each conflict/base pair for operator-by-hand resolution. The synced VaultBackend β€” now the crickets obsidian-vault plugin, not a kernel built-in (V5-3) β€” overrides the floor to this (obsidian-vault/scripts/storage_vault.py#L156, crickets repo).
(later) An iCloud numbered-suffix or a CRDT line-level strategy (design Β§6). Reserved β€” the slot is always present and always answerable.

InvalidLocatorError

InvalidLocatorError is a ValueError subclass raised by normalize_key (and therefore Locator construction / resolve) when a key escaped or malformed its backend-relative namespace β€” today, a .. segment. It signals a caller bug (an unsafe key), kept deliberately distinct from the absent-data degrade a backend reports for a missing read (FileNotFoundError).

The BackendRegistry

A hand-rolled name→backend registry (scripts/storage_seam.py#L282) — the fsspec named-protocol pattern, mirrored, importing neither fsspec nor any DB. A backend registers under a protocol name (device-local, vault); selection (part 5) resolves a configured name against the registry to choose a backend. The registry stores backend classes, not instances — selection instantiates the chosen one.

The contract that matters for part 5's fail-loud: a miss is reported as absent, never raised. get returns None and in returns False for an unregistered name. The registry reports absence; the caller turns absence into a loud failure if the configured backend doesn't exist. That split β€” absence here, fail-loud there β€” mirrors the seam's absent-vs-corrupt stance (a missing read degrades to FileNotFoundError; only a malformed key raises).

Member Signature Returns Raises
register register(protocol, backend, *, clobber=False) None TypeError if backend isn't a concrete StorageBackend subclass; ProtocolError on an empty name or a duplicate (unless clobber=True)
get get(protocol) `type[StorageBackend] Noneβ€” the registered class, orNone` if absent
__contains__ protocol in registry bool β€” whether protocol is registered β€”
protocols protocols() tuple[str, ...] β€” the registered names, sorted β€”

Important

register fails loud on a bad registration, but a miss is absence, not an error. A bad registration is a programming bug, surfaced immediately: an empty protocol name or a silent duplicate raises ProtocolError; a non-backend, the abstract StorageBackend base itself, or a backend instance (rather than a class) raises TypeError. Refusing a silent duplicate keeps one backend from shadowing another by accident. Resolving an unregistered name is the opposite case β€” it returns None, the signal part 5 reads to decide whether the absence is fatal.

BackendRegistry instances are independent β€” no shared global state β€” so a fresh BackendRegistry() can be registered into without leaking across callers (the contract tests rely on this for hermeticity).

registry (module default)

A process-wide default BackendRegistry instance (scripts/storage_seam.py#L393) the real backends register into. Since V5-3 (v5.5.0) it holds exactly one persistent entry β€” device-local, registered at import of storage_device_local. The kernel's own storage_vault.py (the part-4 built-in) was deleted in commit d95468b; the real VaultBackend now lives in the crickets obsidian-vault plugin (see The VaultBackend) and is not registered at kernel-import time at all β€” selection (_load_vault_plugin_backend) discovers the plugin's module, execs it (which self-registers vault as a side effect), reads off the class, and unconditionally pops the vault slot again in a finally β€” so the registry is back to holding only device-local immediately after any selection call, whether or not vault was chosen.

Protocol Registered class Since
device-local DeviceLocalBackend part 2 β€” permanent, registered at kernel import
vault VaultBackend (crickets obsidian-vault plugin) part 4 originally; re-homed out of the kernel in V5-3 β€” transiently registered only for the duration of a select_backend/storage_preview call

Backend selection (part 5)

Note

All five backend-selection tasks are shipped. Task 1 added the storage.backend config field and the choose_protocol / select_backend resolver. Task 2 added the capabilities-read β€” the kernel reads the selected backend's .capabilities through the returned instance; capability-request matching landed later, in V5-7, which added the required parameter to select_backend and a --requires flag on doctor (see The doctor storage check). Task 3 added the polished fail-loud guard: a named StorageSelectionError that names the missing plugin, lists the registered alternatives, and never silently falls back to device-local. Task 4 added the read-only doctor storage preview, rendering the byte-identical install-the-plugin message before any memory operation could refuse. The engine cutover was not part of this work β€” selection only resolves which backend to use and refuses a misconfigured one; the cutover that pointed the kernel at device-local shipped later, in V5-3 (v5.5.0). See the Memory-storage seam design Β§ 3 for what it changed.

Part 5 adds the read side of the seam: a config key naming the chosen backend, and a resolver that turns that name into a registered backend instance via registry.get. The resolver module is intentionally not named storage_*.py and uses no seam-verb name (choose_protocol / select_backend, never resolve/read/write/…), so the check-storage-seam-no-path-leak gate (scoped to scripts/storage_*.py) does not scan it even though it handles Path.

The storage.backend config key

A new operator-facing field on the on-host .agentm-config.json, set via the config CLI β€” see the forthcoming row there and the how-to Choose a storage backend.

Aspect Detail
Field storage.backend on .agentm-config.json (the same on-host config that holds plugins.obsidian-vault.vault_path / state_mode β€” V5-7 tasks 1+2 shipped: vault_path() reads the plugin-namespaced key first, falling back to the legacy flat key; --vault-path writes the plugin key + storage.backend=vault; the vault holds data; config is on-host only). V5-7 task 3 (shipped): choose_protocol lost its vault_root parameter; implicit vault inference removed; vault is now selected only via explicit storage.backend (step 1) or $MEMORY_ROOT env var (step 2) in the 3-step chain below.
Value A registered protocol name β€” device-local, vault, or a plugin-provided name.
Default Unset. Absent β†’ the resolver picks a backend from the existing config (see the resolution chain).
Who reads it The selection resolver (below), via _configured_backend (backend_selection.py#L262) β€” which reuses agentm_config's own prefix-resolution + reader, so the read is the same file the --storage-backend setter writes. The config CLI only stores it; it does not validate that the named backend is registered (agentm_config.py#L151 validates non-empty only) β€” that is the fail-loud guard's job at resolve time.

The selection resolver

The resolver (scripts/backend_selection.py) maps config β†’ a registered backend instance. It sits above the seam (DC-7): it imports agentm_config, harness_memory, the two concrete backend modules (so they self-register at import), and storage_seam.registry β€” but the seam itself never imports it.

Aspect Detail
Module scripts/backend_selection.py β€” deliberately not storage_*.py (keeps it out of the no-Path-leak gate's glob, even though the resolver handles Path).
choose_protocol(*, install_prefix=None) -> str Resolves the protocol name per the 3-step chain below, no instantiation (#L316). V5-7 task 3 (shipped): vault_root parameter removed; implicit vault inference from a config-supplied vault_path removed. Vault is now selected only via explicit storage.backend config (step 1) or $MEMORY_ROOT env var (step 2). vault_path() reads plugins.obsidian-vault.vault_path first, then the legacy flat key (V5-7 task 1, shipped).
select_backend(*, install_prefix=None, device_local_root=None, vault_lock_root=None, required=None) -> StorageBackend Resolves the protocol, then instantiates via registry.get (#L90). The *_root args are injection points so tests never touch the operator's real home / cache / vault. V5-7 Part 1: the optional required: Capabilities | None parameter β€” when provided, runs a subset check after resolving the backend and raises CapabilityMismatchError if any True requirement is unmet; None (default) preserves prior behavior.
How it instantiates Two different paths, since V5-3 deleted the kernel vault built-in. For device-local and any explicitly-configured third-party backend: storage_seam.registry.get(<protocol>) β†’ the registered class β†’ instantiate (#L374) β€” device-local with its optional root, a third party with the V5-1 minimal no-arg constructor. For vault: registry.get is bypassed entirely β€” _load_vault_plugin_backend (#L205) discovers + execs the crickets obsidian-vault plugin's storage_vault.py and hands back its VaultBackend class, which is then constructed directly with the vault root resolved internally from plugins.obsidian-vault.vault_path (plugin_cls(vault_root, lock_root=…), #L409). See The VaultBackend for the discovery mechanism.

The resolution chain (config β†’ backend), first hit wins:

Precedence Condition Resolved backend
1 (wins) storage.backend is set in the install config The named backend (via registry.get); the fail-loud guard fires if it isn't registered.
2 $MEMORY_ROOT env var is set (non-empty) vault β€” explicit env-based escape hatch; setting the env var IS selecting vault.
3 No storage.backend, no $MEMORY_ROOT β€” fresh install or config not yet set device-local (the bare-markdown floor under ~/.agentm/memory/).

Note

V5-7 task 3 (shipped): vault selection is now always explicit β€” choose_protocol no longer infers vault from a config-supplied vault_path. An operator migrating from a pre-V5-7 implicit setup must run agentm config --vault-path <dir> once (which writes storage.backend=vault automatically) or use the $MEMORY_ROOT env override. Both paths proven by scripts/test_backend_selection.py.

The fail-loud guard

Important

Refuse, never demote. If config names a storage.backend whose registry.get() returns None (the plugin isn't installed), the resolver raises StorageSelectionError and refuses the memory operation β€” it does not silently fall back to device-local. Silent demotion is the one failure that could orphan or mis-write an operator's configured store, so the seam forbids it. This is the loud half of the registry's absence-vs-fail-loud split: the registry reports absence (get β†’ None); the resolver decides it is fatal.

As of part-5 task 3 (and the silent-demotion hardening that followed), select_backend raises a dedicated StorageSelectionError (no longer the task-1 placeholder RuntimeError) on any of five refusals β€” a fifth was added in V5-3 when the vault built-in was re-homed to a plugin. Three are raised by select_backend itself once a protocol is chosen; the other two are raised earlier by _configured_backend (backend_selection.py#L262) β€” the reader of the explicit storage.backend value β€” which refuses rather than let a corrupt config silently drop the selection and demote to device-local:

Refusal What raises it Message shape
Unregistered backend select_backend β€” storage.backend names a backend whose registry.get() returns None storage backend '<name>' is configured (storage.backend) but no installed plugin registers it. Install the plugin that provides the '<name>' backend, or set storage.backend to an installed backend (currently registered: <sorted, comma-joined, or "(none)">). β€” built by the _install_plugin_message helper, backend_selection.py.
vault with no seed select_backend β€” vault is selected (rule 1 or 2) but no vault_path resolves to seed it The same StorageSelectionError family β€” refuse rather than guess a vault root.
vault plugin absent or unloadable select_backend β€” vault is selected and a vault_path resolves, but the crickets obsidian-vault plugin can't be discovered, or its storage_vault.py exists but doesn't export a usable VaultBackend class (_load_vault_plugin_backend returns None or raises) storage backend 'vault' is selected but the obsidian-vault plugin is not installed. Install it (crickets: \claude plugin install obsidian-vault@crickets`), or point $OBSIDIAN_VAULT_SCRIPTS at its scripts dir, or change storage.backend β€” refusing to demote to device-local, which could mis-write or orphan the vault.β€” built by the_install_obsidian_vault_message helper, [backend_selection.py#L140`](https://github.com/alexherrero/agentm/blob/main/scripts/backend_selection.py#L140).
Config present but unparseable _configured_backend β€” the config file exists but is malformed JSON (or not a JSON object) while it would otherwise carry a selection Names the unreadable config file path. A corrupt config that drops an explicit selection is the orphan-the-vault failure the never-demote invariant forbids, so the reader deliberately does not reuse agentm_config._read_config's tolerant read (which collapses missing and present-but-unreadable into the same None): file absent β†’ None (genuinely fresh, continue the chain); file present-but-unparseable β†’ raise.
storage.backend not a non-empty string _configured_backend β€” the key is present but its value is not a non-empty string (e.g. a list/number from corruption or a stray hand-edit) Refuses the malformed value. The setter only ever writes a non-empty string and --unset removes the key, so a non-string/empty value is never a legitimate write β€” treating it as "unset" would be the same silent demotion.

Warning

A module-identity gap could silently defeat this guard under direct-script invocation (R1.3 / agentmEngine#1, fixed). resolve_project()'s except (StorageBackendNotInstalledError, _bs.StorageSelectionError): raise clause (the re-raise this guard depends on) is an isinstance check, and Python class identity is per module object, not per source file. When harness_memory.py is run directly as a script (python3 harness_memory.py append-progress ... β€” the standard CLI pattern used throughout this repo; write-state was the verb at the time this gap was found and is retired under agentm-vault plan 15), Python registers the running module as __main__, not harness_memory. backend_selection.py's top-level import harness_memory then finds no harness_memory entry in sys.modules and imports a second, separate instance of the same file β€” whose StorageBackendNotInstalledError class is a different object than __main__'s class of the same name. The except clause's isinstance check no longer matches, the exception falls through to the broader except Exception: backend = None, and the guard silently demotes to device-local β€” reproducing the exact failure this section forbids, but only on the raising branch, only under direct-script invocation. The fix: the if __name__ == "__main__": block now runs sys.modules.setdefault("harness_memory", sys.modules["__main__"]) before dispatch, so any later import harness_memory reuses the already-running instance instead of re-executing the file. Caught by verify-state-routing's never-demote check (case D) β€” the first fixture to exercise this raising branch under CLI invocation.

Note

Proven by a negative test. test_no_silent_device_local_fallback (in the TestFailLoud class, scripts/test_backend_selection.py) asserts the invariant that matters: on an unregistered backend the resolver raises, does not return a DeviceLocalBackend, and never even constructs one β€” the device-local root stays uncreated. The class also pins the unregistered-backend raise, that the install message lists the registered alternatives, and the vault-without-vault_path companion guard. The guard's full bite arrives once V5-3 deletes the built-in vault backend (until then vault is always registered); the refusal machinery itself is shipped now.

The _install_plugin_message helper is deliberately factored out so the doctor preview (task 4, shipped β€” reuses it at backend_selection.py#L539) renders a byte-identical message before any memory operation could refuse.

The doctor storage check

doctor (the install-health skill) gains a storage-backend preview (task 4, shipped) β€” the operator's read-only way to see what selection would resolve before any memory operation could refuse. Invocable as python3 scripts/backend_selection.py --doctor (the _doctor_main CLI entry, backend_selection.py#L603) and read by the doctor skill's structural check 4d. It reports the selected backend, where the selection came from, whether its plugin is registered, and that the device-local root is writable β€” and it resolves all of this without constructing a backend (storage_preview, backend_selection.py#L471), because construction would mkdir the device-local root.

V5-7 Part 2 (shipped): _doctor_main gains a --requires <CAPS> flag β€” comma-separated capability names (e.g. --requires concurrent_writers,encryption). When present, the flag validates all names against dataclasses.fields(Capabilities) before touching any backend (unknown names β†’ error on stderr, exit 1, no backend constructed); then calls select_backend(required=Capabilities(**{name: True for name in requested})) and prints PASS: backend '<protocol>' satisfies required capabilities: <caps> (exit 0) or FAIL: <CapabilityMismatchError message> (exit 1). StorageSelectionError (plugin not installed, no vault path, corrupt config) also prints FAIL and exits 1. The three injection params device_local_root, vault_lock_root, and vault_plugin_scripts are forwarded to select_backend for test isolation. Proven by TestDoctorRequires (5 cases: unknown field, mismatch, satisfied cap with vault fixture, multiple unknown fields, no-flag backward compat) in scripts/test_backend_selection.py.

Field Type Detail
status str "ok" / "warn" / "fail" β€” the row's verdict; drives the --doctor exit code (1 only on "fail").
protocol Optional[str] The resolved protocol name, or None when the explicit-selection read itself failed loud.
line str The single status line doctor prints β€” storage [OK]/[WARN]/[FAIL] ….
Status / exit When Note
ok / 0 Selected backend registered and ready β€” vault seeded from a resolved vault_path, a writable device-local root, or a registered third-party protocol. β€”
warn / 0 device-local selected but its root is not writable (_root_is_writable, #L456). Preventive, never build-blocking.
fail / 1 Unregistered plugin; or vault selected with no resolvable vault_path; or vault selected but the obsidian-vault plugin is absent/unloadable; or the config file exists but is unparseable / names a non-string storage.backend. When --requires is passed: any unsatisfied capability or unknown capability name. The unregistered-plugin line reuses the same _install_plugin_message (#L539) the fail-loud guard raises (#L383), so the preview and the live refusal are byte-identical; the vault-plugin-absent line reuses _install_obsidian_vault_message the same way.

Proven by TestStoragePreview (6 cases, scripts/test_backend_selection.py#L324) for the standard preview path, and TestDoctorRequires (5 cases) for the --requires path. See the how-to Choose a storage backend for the operator-facing flow.

ProtocolError

A ValueError subclass (scripts/storage_seam.py#L272) raised by BackendRegistry.register on an empty or duplicate protocol name. Registering badly is a programming error, surfaced loudly β€” kept deliberately distinct from a registry miss, which is not an error at all (get β†’ None).

StorageSelectionError

A RuntimeError subclass (scripts/backend_selection.py) raised by the selection resolver when the configured backend cannot be produced β€” the loud half of the registry's absence-vs-fail-loud split. It lives on the resolver (above the seam, in backend_selection.__all__), not in storage_seam, because turning a registry miss into a fatal refusal is the resolver's decision, not the registry's. Raised in five cases (see the fail-loud guard) across two functions. select_backend raises it once a protocol is chosen: when storage.backend names a backend whose registry.get() returns None (the plugin isn't installed), when vault is selected with no vault_path to seed it, or when vault is selected but the crickets obsidian-vault plugin is absent or unloadable (V5-3 added this third case when the vault built-in was re-homed to a plugin). _configured_backend β€” the reader of the explicit storage.backend value β€” raises it earlier: when the config file is present but unparseable (so a corrupt config cannot silently drop the selection and demote to device-local), or when storage.backend is present but not a non-empty string. Catching this loudly β€” instead of a silent device-local fall-back β€” is how the doctor preview (task 4, shipped) surfaces the install-the-plugin message before any memory operation can refuse (the V5-3 engine cutover, Memory-storage seam design, then pointed the kernel at device-local exclusively). The preview does not raise the error β€” it converts each StorageSelectionError case into a "fail" row carrying the same message (storage_preview, backend_selection.py#L471).

CapabilityMismatchError

A StorageSelectionError subclass (scripts/backend_selection.py) raised by select_backend when a caller passes required=<Capabilities> and the selected backend's .capabilities does not satisfy every True requirement. Shipped in V5-7 Part 1.

Constructor takes (protocol: str, unsatisfied: list[str]) β€” the backend protocol name and the list of unsatisfied capability field names. str(e) produces a message naming both. Exposed in backend_selection.__all__ alongside StorageSelectionError. Never raised when required=None (the default) β€” so existing call sites that omit required are unaffected.

Aspect Detail
Superclass StorageSelectionError β†’ RuntimeError
Constructor (protocol: str, unsatisfied: list[str])
str(e) Names the backend protocol and the unsatisfied fields by name.
Proven by TestCapabilityMatching (8 new tests) in scripts/test_backend_selection.py.

The Tier taxonomy

The memory state splits across three tiers, distinguished by who owns the truth (source vs derived) and whether an external sync layer may replicate the tree (syncs). The taxonomy is reserved in this part: it designates the tiers and their sync/derived policy so the V6 vector/SQLite index lands on a contract that already exists. No index is built and no abstract is promoted here. For why the local index is pinned never-sync, see Memory↔storage seam Β§ The three tiers.

Tier

The three-tier enum (scripts/storage_seam.py#L350). Two properties carry the policy; the one hard line is that only LOCAL_INDEX is never-sync.

Member Value .syncs .derived What it holds
SOURCE "source" True False The synced, authoritative markdown the engine persists β€” the truth the others rebuild from.
SHARED_ABSTRACTS "shared-abstracts" True True Derived-but-portable summaries/abstractions; may sync (useful everywhere, rebuildable if a sync drops them).
LOCAL_INDEX "local-index" False True The V6 vector/SQLite index β€” device-local, never synced (a replicated database file is a corruption pattern).
Property Returns Detail
.syncs bool Whether an external sync layer may replicate this tier's tree. True for SOURCE and SHARED_ABSTRACTS; False only for LOCAL_INDEX β€” ties to Capabilities.sync.
.derived bool Whether the tier is rebuildable from SOURCE (so never authoritative). True for both derived tiers; False only for SOURCE.

TierLayout

A frozen dataclass (scripts/storage_seam.py#L397) designating a root Locator per tier. The defaults are tier-named placeholders a concrete backend (parts 2 / 4) overrides with its real roots β€” typically a synced-vault root for source/shared_abstracts and a device-local cache root for local_index. No tree is created β€” this is placement, not I/O.

Member Type Detail
source Locator Root for the SOURCE tier. Default Locator("source").
shared_abstracts Locator Root for the SHARED_ABSTRACTS tier. Default Locator("shared-abstracts").
local_index Locator Root for the LOCAL_INDEX tier. Default Locator("local-index").
never_sync_root property β†’ Locator Returns the local_index root β€” the one tree that must never be replicated by sync.
root_for(tier) β†’ Locator The root locator designated for a given Tier.

Important

The three roots must be distinct. __post_init__ raises ValueError if any two tier roots collide β€” so a derived tier can never overwrite the source it rebuilds from. The source/shared_abstracts placement is structurally separate from the local_index placement.

DerivedMaintenance

An abstract base (scripts/storage_seam.py#L435) naming the two derived-tier operations.

Note

Named / reserved β€” built in V6. Both methods are abstract, so the class cannot be instantiated and no implementation ships: there is deliberately no concrete subclass in the module (a scope guard asserts it). reindex/changed_since are reserved names, not working operations.

Method Signature Detail
reindex reindex(tier: Tier) -> None Full rebuild of a derived tier from SOURCE.
changed_since changed_since(mtime: float) -> list[Locator] The incremental feed β€” source locators whose Info.mtime is newer than mtime, so an incremental reindex touches only what moved.

The scope guard

The contract-only stance is enforced executably, not just by convention, in scripts/test_storage_seam.py: DerivedMaintenance cannot be instantiated (its __abstractmethods__ are {reindex, changed_since}), the module exports no concrete DerivedMaintenance subclass, and the module imports no DB / index / vector-framework library. Together these are the structural proof that the index is named, not built, in this part.

The DeviceLocalBackend

The first concrete StorageBackend (scripts/storage_device_local.py) β€” the fresh-install default: plain markdown under ~/.agentm/memory/, user-owned, no vault, no Drive, no service. It implements the seven verbs and the capabilities descriptor against the local filesystem, and registers under the device-local protocol name in the default registry at import. Bare markdown is the floor; a database is something a plugin may offer, never the kernel default. For why this is the floor, see Memory↔storage seam Β§ The first concrete backend.

Note

Engine cutover landed in V5-3 (v5.5.0). DeviceLocalBackend works as a storage backend, and V5-3 completed the cutover the seam design deferred. The Memory-storage seam design Β§ 3 is the canonical account of what it changed.

Aspect Detail
Protocol name device-local (the module's PROTOCOL constant, storage_device_local.py#L54). Registered at import β€” the class, not an instance.
Root ~/.agentm/memory/ (Path.home() + .agentm/memory). Created on construction β€” that is its "first use", so the root locator always resolves to a real directory. Injectable via root= so tests never touch the operator's real home.
capabilities The single-machine floor as a positive statement: concurrent_writers=False, conflict_files=False, encryption=False, sync=False.
conflict_strategy Inherits the seam floor "none" β€” on one machine there is nothing to reconcile.

How the verbs map to the filesystem

A locator maps to a path by joining its normalized parts under the root (root / *locator.parts). Because Locator rejects .. and a leading slash at construction (InvalidLocatorError), the join can never escape the root. Path is used internally only β€” every verb returns the seam's Locator / Info, never a Path (the check-storage-seam-no-path-leak gate enforces this statically).

Verb Device-local behavior
resolve(*parts) Builds a Locator from the joined parts.
read(locator) read_bytes().decode("utf-8") β€” byte-exact, no newline translation, so content round-trips with the atomic writer. Missing path raises FileNotFoundError natively (#L130).
write(locator, content) Composes vault_lock.atomic_write (temp + fsync + rename) β€” never an open-and-truncate, so a crash leaves prior bytes intact. Parent dirs created if absent (#L136).
list(locator) Children sorted by key; [] if the path is absent or a file (pinned by the part-3 conformance suite).
exists(locator) True if a file or directory is present.
info(locator) Info carrying mtime and size (0 for a directory); stat() raises FileNotFoundError if absent.
mkdir(locator) Idempotent (parents=True, exist_ok=True).

Important

Crash-safe by composition, not reinvention. write routes through the V5-0 atomic_write primitive rather than opening the target for truncation. Device-local needs none of the vault_mutex / content-hash CAS stack the synced VaultBackend layers on β€” it is single-machine, so the atomic file swap is sufficient. It also ships no sync, derived-index (_index), or conflict-merger machinery; an AST-based scope guard in scripts/test_storage_device_local.py asserts the module defines no such code and imports no DB / index library. Those concerns ride with the vault backend and the V6 index.

Module exports (storage_device_local)

storage_device_local.__all__ is DeviceLocalBackend, PROTOCOL. Importing the module has the side effect of registering DeviceLocalBackend under device-local in the seam's default registry. Like the seam module it ships no python -m entrypoint β€” it is consumed in-process.

The VaultBackend

Warning

Not a kernel module since V5-3. The kernel's own scripts/storage_vault.py (and its test file) were deleted for real in commit d95468b (2026-06-21, "AG Phase 2, A2.3 part 1") β€” the V5-3 engine cutover had already stopped the kernel from invoking it directly, and this commit removed the file itself, closing out the scaffolding the earlier cutover left behind. The second concrete StorageBackend now lives entirely in the crickets obsidian-vault plugin, at src/obsidian-vault/scripts/storage_vault.py β€” today's Obsidian/GDrive vault, reached through the seam. Everything below describes that plugin file, not kernel source.

The plugin's VaultBackend implements the seven verbs against the resolved per-project vault root. Unlike device-local (registered permanently at kernel import), it is discovered and loaded on demand: backend_selection._load_vault_plugin_backend (see How it instantiates) locates the plugin's scripts/ dir by convention, execs storage_vault.py (which self-registers vault into the seam's default registry as an import side effect, same as the kernel built-in used to), reads off the VaultBackend class, and then unconditionally pops the vault registry entry again in a finally β€” so nothing about the plugin lingers in kernel process state between selection calls. It is the synced, multi-writer counterweight to the device-local floor: where device-local declares the all-False floor, the vault backend declares the positive profile and overrides the conflict floor. For why the vault sits behind a seam at all, see Memory↔storage seam.

Note

Wrap, not rewrite, and now a move, not a wrap (V5-2 β†’ V5-3). V5-2 lifted the vault backend byte-faithfully out of the kernel into this plugin while both ran in parallel, proven identical by conformance + a byte-identical parallel-run on the operator's real vault. That green authorized V5-3 to delete the kernel built-in for good, leaving the plugin copy as the sole vault backend β€” a move, not a fork: the plugin's docstring itself warns against letting it "absorb new vault behavior the built-in lacks while both are live." For the full cutover account, see the Memory-storage seam design Β§ 3.

Aspect Detail
Protocol name vault (the module's PROTOCOL constant, obsidian-vault/scripts/storage_vault.py#L98, crickets repo). Registered only transiently, for the duration of a selection call β€” see above.
Root The resolved per-project vault path β€” a required ctor arg (the vault has no universal default, unlike device-local's ~/.agentm/memory/). Created on construction (idempotent), so the root locator always resolves to a real directory (storage_vault.py#L113-125, crickets repo).
lock_root Injectable ctor arg for the vault_mutex local lock base; None β†’ vault_lock's default (~/.cache/agentm/locks). Tests inject a temp dir so the real cache and the operator's live vault are never touched.
capabilities The synced, multi-writer profile β€” the positive contrast to device-local's floor: concurrent_writers=True (the mutex makes N writers safe), conflict_files=True (DriveFS surfaces "(conflicted copy)" siblings the engine must tolerate), sync=True (GDrive replicates the tree), encryption=False (no encryption at rest) (storage_vault.py#L141-153, crickets repo).
conflict_strategy Overrides the seam's "none" floor to "whole-file" (storage_vault.py#L155-169, crickets repo) β€” see the strategy note below.

Important

"whole-file" is a named policy, not an auto-merger. The string names the reconciliation unit that part-5 selection reads β€” it does not make agentm merge conflicts. The backend ships no merge machinery of its own. When GDrive sync diverges (two devices write the same file while offline), DriveFS materializes a "(conflicted copy)" sibling rather than losing a write; the existing detect-and-notice machinery β€” harness_memory.detect_conflict_files + the conflict-merger SessionStart hook β€” surfaces each conflict/base pair, and resolution stays operator-by-hand judgment. A line-level auto-merge would be a future CRDT strategy, not this. Device-local still inherits "none" (one machine, nothing to reconcile).

How the verbs map to the vault filesystem

Like device-local, a locator maps to a path by joining its normalized parts under the root (root.joinpath(*locator.parts)), and the ../leading-slash rejection at Locator construction keeps the join confined to the root. Path is used internally only; every verb returns the seam's Locator / Info (the check-storage-seam-no-path-leak gate enforces this). The difference that matters is write.

Verb Vault behavior
read(locator) read_bytes().decode("utf-8") β€” byte-exact, no newline translation, so content round-trips with the atomic writer (storage_vault.py#L176, crickets repo).
write(locator, content) Composes the full V5-0 stack: serialize fleet-local writers on the one per-vault vault_mutex, then land via a content-hash-CAS-guarded atomic_write. The mutex serializes agentm sessions against each other; the CAS catches a non-mutex writer (GDrive sync / another device) landing between the pre-write read and the rename β€” the cross-device hazard device-local does not have (storage_vault.py#L182, crickets repo).
resolve / list / exists / info / mkdir Same shape as device-local; the conformance suite holds both to the identical contract.

Important

The write composes vault_lock primitives, never the engine β€” DC-7 preserved. write imports vault_mutex, content_hash, and atomic_write from vault_lock directly, so the seam never reaches up into the memory engine module. The CAS step (_cas_atomic_write) re-reads the target under the mutex and raises ConcurrentModificationError if its content changed since the pre-write read β€” the V5-0 safe_write_replace_style discipline, recomposed from primitives.

Conformance β€” now split kernel-side / plugin-side

V5-3's deletion of the kernel built-in moved vault conformance out of this repo. Kernel-side, scripts/test_storage_conformance.py no longer runs a VaultConformance case at all (that class β€” along with VaultRunConformanceReport β€” was deleted in the same commit that removed storage_vault.py); only DeviceLocalConformance runs the full universal battery here now. A _make_scratch_vault_backend helper still exists in that file, but only to back RoutingConformanceReport's check that the V5-6 routing layer (resolve_project / repo_registry, below) works against some VaultBackend-shaped object β€” it constructs vault_backend_stub.VaultBackend (a standalone, same-contract stand-in for tests; it does not self-register into the seam registry), never the real plugin.

Plugin-side, the real vault conformance now runs in the crickets repo, against a checked-out agentm kernel (this is the obsidian-vault-conformance CI job β€” see CI gates). scripts/test_obsidian_vault_conformance.py there carries PluginVaultConformance (#L188) β€” the same universal battery, over a scratch vault β€” and PluginVaultRunConformanceReport (#L207) for the per-backend pass report.

The never-orphan invariant itself is retired β€” test_storage_vault.py (and the VaultNeverOrphanInvariant case it carried, which proved an old-way write and a seam write reached the same on-disk bytes) was deleted along with the kernel module; there is no longer an "old way" in this repo to compare against, since the seam is the only way now. What replaces it crickets-side is narrower but still load-bearing: PluginVaultBehavioralContract (#L237) closes the gap the byte-only conformance suite can't reach β€” it forces a foreign write into the CAS window (interposing on Path.read_bytes) and requires write to raise ConcurrentModificationError rather than silently clobber, and it pins capabilities/conflict_strategy to their literal vault values so a backend degraded toward device-local's floor can't pass either check.

Module exports (storage_vault, now crickets-side)

The kernel ships no storage_vault module anymore. The crickets plugin file's __all__ is VaultBackend, PROTOCOL β€” same shape the kernel built-in had. Importing it (which only ever happens inside _load_vault_plugin_backend's exec_module call, never at kernel startup) registers VaultBackend under vault in the seam's default registry, immediately popped again by the loader's finally. Like the other backend modules it ships no python -m entrypoint β€” it is consumed in-process. Kernel code that needs a same-contract VaultBackend for tests (not real plugin discovery) imports vault_backend_stub instead β€” PROTOCOL, VaultBackend; deliberately does not self-register, since a test double shouldn't compete with the real plugin for the registry slot.

Module exports

storage_seam.__all__ is the public surface: InvalidLocatorError, normalize_key, Locator, Info, Capabilities, StorageBackend, ProtocolError, BackendRegistry, registry, Tier, TierLayout, DerivedMaintenance. No DerivedMaintenance implementation and no python -m entrypoint β€” unlike the process seam, this module is consumed in-process by the engine, so it ships as an importable contract only. The two concrete backends that implement StorageBackend live in their own modules β€” see DeviceLocalBackend and VaultBackend.

backend_selection.__all__ (the resolver module above the seam) exports: StorageSelectionError, CapabilityMismatchError (V5-7 Part 1), choose_protocol, select_backend, StoragePreview, storage_preview. Capabilities is also imported (from storage_seam) so callers can construct a requirement descriptor without a separate import.

Routing layer (V5-6)

Note

Status: complete (V5-6, 2026-06-18). All five tasks shipped: resolve_project / _vault_projects_dir (task 1), repo_registry (task 2), state_mode vault→backend alias (task 3), gate extensions + conformance suite (task 4), ADR + docs (task 5). Decision: Memory-storage seam.

The third de-vaulting leg re-plumbs the kernel's routing/index mechanisms so they speak Locators to this seam instead of building vault_path() / … filesystem paths directly. Three mechanisms are in scope:

Mechanism File Status Detail
resolve_project / _vault_projects_dir harness_memory.py:395,417 Shipped (task 1) _vault_projects_dir(backend: StorageBackend) -> Locator; resolve_project returns {slug, project_locator, backend, project_root, layout} β€” project_locator is a Locator, not a Path. process_seam.py:113 uses project_locator.key.
repo_registry read/write repo_registry.py Shipped (task 2); registry home moved (agentm-vault plan 05) registry_path(vault_path) -> Path replaced by registry_locator(backend: StorageBackend) -> Locator (#L116). All five public functions take backend: StorageBackend. _mutate_registry no longer holds vault_mutex β€” VaultBackend.write() handles it; CAS retry loop handles cross-device races. On obsidian-vault resolved to identical path at task-2 ship (LC-1, proven by LC-7 parallel-run test at test_harness_memory.py#L2608) β€” the memory-root trims (plan 05) since moved the registry's home to the engine state directory (registry_store() picks the engine store whenever it holds the file, and the obsidian-vault backend's _meta/repos.json only while it is the sole copy), so the two backends no longer resolve to the same bytes at the same path. The one case that still reaches a backend, a legacy copy it holds alone, is what the routing conformance check below proves.
state_mode resolver harness_memory.py:296,669 Shipped (task 3) "vault" aliases to "backend" at read time in both _read_config_state_mode and _read_project_mode; agentm_config normalizes "vault" β†’ "backend" at write time; CLI --state-mode help surfaces the deprecation note β€” no operator action required.

Behavior-preserving invariant (LC-1): on the obsidian-vault backend every mechanism resolves to the same bytes at the same path before and after. The task 1 parallel-run assertion confirmed byte-identical resolution for resolve_project / _vault_projects_dir before vault-shaped path construction was removed.

Gate extensions (task 4 β€” shipped):

  • check-storage-seam-no-path-leak.py β€” Pass 2 (V5-6): scans harness_memory.py + repo_registry.py for seven named routing functions (resolve_project, _vault_projects_dir, registry_locator, read_registry, write_registry, register_repo, unregister_repo, list_repos) that return pathlib.Path. Driven by ROUTING_FUNCTIONS, _ROUTING_FILENAMES, _routing_files(), and an updated _scan_source(names=…) kwarg.
  • check-process-seam-import-direction.sh's LC-8 back-edge scan β€” scans the same two files for any import storage_vault / from storage_vault import and fails loud (a routing mechanism may import the seam, never a capability plugin) β€” now lives as the lc8-storage-vault rule inside scripts/check-one-way-imports.py (CONS-1 merged three separate one-way-import scripts β€” this shell check plus check-opinion-resolver-one-way.py and check-capability-resolver-one-way.py β€” into one config-driven AST checker; check-all.sh runs it as check-one-way-imports). The rule's own name for this check remains lc8-storage-vault, so the concept it enforces is unchanged even though the old standalone script no longer exists.
  • storage_conformance.py extended: check_routing_repo_registry(make_backend) proves that a registry copy held by a backend is read and written through that backend. Since the memory-root trims (agentm-vault plan 05) that legacy case is all of LC-4 left to prove. The check seeds _meta/repos.json on a fresh backend, inside an empty engine state directory of its own, and fails unless register, list and unregister read and write that copy and leave the engine state without a registry. Gated by run_conformance(include_routing=False) (default off, explicit opt-in); RoutingConformanceReport in test_storage_conformance.py runs include_routing=True on both backends, and test_storage_conformance_negative.py proves the check fails when the registry never reaches the backend.

For the intent and three-leg arc context, see the Memory-storage seam design.

Related

  • Memory↔storage seam β€” why the seam exists, why the engine never holds a path, and the graceful design choices behind the lean types.
  • Process seam β€” the other seam's reference: the read-only client a process calls. Same pattern, opposite direction.
  • Vault write protocol β€” the atomic_write primitive DeviceLocalBackend.write composes, and the vault_mutex + content-hash CAS stack VaultBackend.write adds on top.
  • CI gates β€” the check-storage-seam-no-path-leak gate that enforces the no-Path-leak rule.
  • Memory-OS Architecture (V5) β€” the design context: storage-agnostic engine, device-local default, vault-behind-the-seam.
⚠️ **GitHub.com Fallback** ⚠️