agentm memory system - alexherrero/agentm GitHub Wiki
Note
LAUNCHED (approved 2026-06-21, lifted 2026-06-24, locked 2026-06-28). Child design of agentm HLD β the Memory pillar, the largest of the four and the ground the other three stand on. See the amendment log below for build history.
Memory is the durable record agentm keeps β what it has learned, the plans and designs it works on, the standards it holds β and the single disciplined path every caller takes to reach it. It is the largest of the four pillars and the ground the other three stand on: Experience grows it, Opinions keep their learned half in it, Personas draw their state from it.
It is built to compound: entries are typed, densely linked to their neighbors, and indexed for recall β so as agentm learns and reaches outward, the record grows into an interconnected knowledge base.
The substrate serves one aim: every caller reaches storage the same way, and nothing leaks a dependency downward. A small set of generic pieces β none knowing a specific backend, host, or tool β arranged so everything points inward: a caller reaches the memory engine, the engine reaches storage through the resolution plane and the seam, and the concrete backends, personas, and crickets tools all depend up. The substrate depends on nothing below it.
The components:
-
The memory engine β the verbs the rest of the system calls (
saveΒ·recallΒ·forget, plus reflection) and the cross-cutting logic that must live exactly once: idempotency + content-hash CAS, soft-delete, token-budgeted recall, link integrity. -
The resolution plane β how a call finds its store without naming one: the config holds the backend choice; the selector (
backend_selection.py) maps a protocol name to a concreteStorageBackend, failing loud if it's missing. -
The storage seam β the one port to disk: a
StorageBackendcontract, a registry, an opaqueLocator, and the storage tiers. (The seam's full contract β the verbs, theLocatorguards, the tiers + the never-sync invariant, the reservedDerivedMaintenanceβ is the launched memory-storage-seam design; this pillar points down to it.) - Harness-state I/O β plan/progress/feature state is backend-aware: it routes through the seam to the active backend, so state and memory reach disk the same way.
The one-way rule, enforced. Routing and memory code may import the seam + selector (substrate) but never a concrete backend β the LC-8 gate (check-process-seam-import-direction.sh) fails the build on any import storage_vault. The backend is chosen at call time and injected through the abstract contract. Backends and tools point up; the substrate points at nothing below.
agentm classifies every memory on three independent axes: by kind (what sort of thing it is), by durability (does it survive the session?), and by ownership (whose space is it?). Kind drives retrieval; durability and ownership drive placement.
By kind. Every entry declares a kind β open-ended but conventional: preference Β· workflow Β· fix Β· domain-reference Β· idea Β· skill-pointer. Kind is central: it picks the storage group an entry is written to, and it scopes which entries a phase pulls into its recall budget (a /work pass recalls different kinds than a /plan pass). Kind is how a caller asks for the right sort of memory rather than a flat search over everything.
By durability + ownership. Two planes:
- Layer 0 β agent-harness memory (not durable): the host's context window β the running conversation. Short-lived; it fills and gets summarized away. Scratch space; the record lives below, on disk. (The layer the whole substrate exists to compensate for.)
- Layer 1 β the durable record (on disk): resolved at runtime, never a cached path. It splits into three ownership tiers, by ascending agent autonomy β and each tier is a place in the vault:
| Tier | What it holds | Where | The agent's hand |
|---|---|---|---|
| T1 β personal | the operator's own notes | the Obsidian vault above Agent/ (its siblings) |
reads + links; writes only when told, through a separate seam call |
| T2 β curated / collaborative | designs Β· plans Β· roadmaps, and the operator-directives the agent follows (voice, conventions, preferences) |
Agent/desk/ β our shared working surface |
writes as needed and reports each change in the digest; revertable |
| T3 β agentm-sole | the agent's own learned memory + insights | Agent/memory/ |
writes, curates, and prunes freely; no notice |
Personal (T1) sits outside agentm's vault_path (which is Agent/), so it is out of reach by default, not by policy alone. The agent writes there only through an explicit, separate storage-seam call that an operator request authorizes β keeping agent-controlled and user-controlled space cleanly apart (the memory-storage-seam design). Autonomous jobs work mostly in T3; when a job changes T2, that change lands in the digest for the operator to see and revert.
The diagram shows the nested ownership tiers (containment = autonomy) below the durability line; kind is the orthogonal third axis, carried in each entry's frontmatter.
The structure is opinionated β the agent controls most of it, so the design fixes the shape.
Paths below are written against the two resolvers rather than any machine's
literal layout β vault_path() is the Obsidian vault and git root,
memory_root() is the agent's tree one level inside it. They are not the
same directory, and addressing the wrong one silently mis-roots every
vault-relative path (see the vault-path convention in AGENTS.md).
<vault_path>/ T1 personal β the operator's vault; agentm does not touch it
βββ Projects/<project>/ T2 curated β one tree per project (vault root, sibling of Agent/): designs Β· plans (_harness) Β· roadmap Β· progress; `_global/` holds cross-project stores (e.g. wiki-style voice overlays)
βββ Calendar/ Β· Personal/ Β· β¦ the operator's own folders and loose notes
βββ Agent/ = <memory_root> the agent's tree β everything below is agentm's
βββ desk/ T2 curated β the remaining shared working surface
β βββ briefs/ Β· scratch/ short-lived working material
βββ external/ third-party sources the agent reads but never authors
βββ _meta/ device-local sidecars β health, caches, snapshots; derived, never synced as truth
βββ _dream/ dreaming output
βββ _vault-archive/ retired curated content (cold; recall skips it)
βββ memory/ T3 agentm-sole β the agent's own memory
βββ _always-load/ the heat-promoted floor (flat β kind is not a subfolder here)
βββ <kind>/<slug>.md the leaf: kind is the subfolder (insight/, feedback/, workflow/, fix/, β¦)
βββ <year>/<month>/ the dated subtree
βββ _inbox/ raw reflection candidates (recall-excluded by default)
βββ _watchlist/ Β· _opinions/ staged candidates and accumulated opinions
βββ _idea-incubator/ agent-incubated ideas
βββ _archive/ the cold zone (recall skips it)
T2 (curated) shape. Curated content lives under Projects/<project>/ at the vault root, alongside Agent/ β each project carrying designs/, plans in _harness/, a roadmap, and a progress log; desk/ keeps the shorter-lived briefs/ and scratch/. No loose notes at the root. The operator-directives the agent follows (voice, conventions, preferences, feedback) are curated: the operator owns them, the agent refines them and reports the change. Where those directives now sit is an open question β the four-space migration moved them under memory/, which this design labels T3, so either the paths or the tier boundary need an owner's ruling; the amendment log records it rather than guessing.
T3 (memory) leaves reuse the live entry convention β <kind>/<slug>.md with the locked frontmatter (below); kind is the subfolder, because kind drives recall. The reserved leading-underscore folders (_always-load, _inbox, _archive, _idea-incubator, _watchlist, _opinions) are recall-aware. _meta/ is not among them any more β it sits at the Agent/ root beside memory/, since device-local sidecars are derived state for the whole tree rather than memory leaves. crystallized/ now serves two writers β the phase-close digest and, as of V6-4 (2026-07-07), episodic-to-semantic consolidation, which reuses the same kind: crystallized + five-field digest shape for its promoted entries rather than adding a separate semantic/ folder. procedural/ (distilled how-to, the next tier up) stays designed-for β see How it grows.
Archive at every tier. Each tier carries a cold archive for retired content β _vault-archive/ at the Agent/ root, memory/_archive/ inside the memory tree, and a _archive/ beside the projects in desk/. Recall skips them by default, so the record stays cheap to load as it grows; it opens only on deep research, an explicit ask, or granted permission. Nothing is hard-deleted β "prune" means move to _archive/ (markdown stays the source of truth; the revert-log is the undo). The per-tier archive policy follows the autonomy gradient: T1 archives only on an operator-confirmed proposal; T2 archives a curated artifact when its successor supersedes it; T3 the agent archives its own cold entries on its own. Capture describes the decay pipeline that feeds the archive.
Work inside a project comes in arcs β a V5/V6/V7/V8 wave, the architecture-governance track, consolidation-review, a lettered build wave (wave-b, wave-c, β¦). An arc groups everything one stretch of work touched: designs, decisions, plans, close-out artifacts. It needed a way to show up in the vault, and the deciding question was whether that grouping should be a folder or a frontmatter fact.
In the permanent record, arc identity is a frontmatter field, never a folder. A decisions/ entry or a design outside _harness/ gains an arc: value; it does not move. Three reasons a folder-per-arc would have been worse: arcs complete β a folder tree would accumulate dead branches for every closed arc forever, where a tag just stops being queried; a move invalidates the vec-index and embedding paths already computed against a note's current location, which a metadata stamp never touches; and an arc can be cross-repo (architecture-governance spans both agentm and crickets) β a folder can only live inside one project, but a tag can be stamped in both.
arc: is its own frontmatter field, not a third segment folded into group:. group: mirrors the entry's folder path exactly, and every tool that reads it (save.py, vault_lint.py) depends on that invariant holding. Arc is an orthogonal fact about when an entry was made, carried the same way tags: carries other cross-cutting facts.
A new kind: arc-index gives each arc one note, at projects/<project>/arcs/<arc-slug>.md, linking the designs, decisions, plans, and close-out artifacts that carry its arc: stamp. A cross-repo arc has one canonical arc-index, in the project that owns it, and a thin pointer note in the sibling project pointing over. The MOC generator (moc_generator.py) emits the link-list body straight from arc: stamps found in its walk, the same way it already groups by kind: today; a hand-written header above that generated list is fine.
Arc values are a closed, checked set. arc_registry.py (mirroring kind_registry.py's shape) holds the known arc slugs; vault_lint.py flags an arc: value outside that registry the same way it already flags an unrecognized kind:. A new arc starts by joining the registry, not by an entry inventing a slug no index will ever collect.
Folder moves stay confined to _harness/ β gitignored working state, never the permanent record. Two moves are sanctioned there, both reversible and both scoped to an arc that has actually closed: (a) the flat PLAN.archive.YYYYMMDD-<slug>.md files that pile up under a project's _harness/archive/ group into an archive/<arc>/ subfolder once that arc's plans are all done; (b) a closed arc's design folder moves from _harness/designs/<arc>/ to _harness/archive/designs/<arc>/ β crickets' archive/designs/efficiency-automation/ already does exactly this, ahead of the convention being named. An arc still being worked (architecture-governance's Phase 3, v8-proving) stays put in _harness/designs/ until it closes; nothing about this convention moves a folder before its arc is actually done.
A memory is one atomic entry: a markdown note with a locked frontmatter block, stored under its kind's group. The frontmatter is what makes it addressable and curatable:
-
kindβ the classification above (routes storage, scopes recall). -
statusβactive/supersededβ recall filters superseded entries out. -
always_loadβ the boot-injection gate: an entry withalways_load: trueis assembled into context at session start (as-built, the gate is_always-load/directory membership β the authoritative source; the field mirrors it); the heat policy flips this field as the entry heats up or cools (the Experience design). -
supersedesβ a back-link to the entry this one replaces (the supersession chain β see Capture). - plus
created/updated/tags/slug.
The content rule is engagement, not encyclopedia: an entry captures why it mattered here and how to apply it, not a generic description. One note is one fragment β small, indexable on its own β and densely linked to its neighbors. Beyond the typed supersedes: back-link, a note cross-references related entries with Obsidian [[wikilinks]], the native interconnection substrate; the engine keeps those links sound (link-integrity discovery across the corpus, notes_link_discovery.py). Linking is first-class: a well-connected fragment is what lets the vector index, and later the V6 knowledge-graph, retrieve over relationships, not just text.
Everything agentm remembers reaches disk through a single storage port β the seam. The discipline is strict: there is exactly one way to storage, and every caller goes through it. Read bottom-to-top β three layers, each with one job:
-
The storage seam β the one port. The only layer that talks to a concrete store.
device-localandobsidian-vaultare interchangeable adapters; more can be added by implementing the same contract. Nothing above knows whether bytes land on the local filesystem or a synced vault β that ignorance is the point of a port. -
The memory engine β the one set of verbs.
saveΒ·recallΒ·forget+ the cross-cutting logic, every verb reaching the store through the seam. - Inbound adapters β the ways in. In-process (always present, a plain library call, zero daemon β the simple local case) and the MCP server (opt-in, a thin transport shim forwarding to the same engine).
The MCP server belongs above the seam, as a client of it β never underneath (that would force a daemon onto the simple local case and invert the layering; V5-0 put the mutex + content-hash CAS into the storage primitives precisely so the system needs no daemon). The seam, selector, device-local backend, and MCP shim are genericizable β agentm substrate; the obsidian-vault backend implements the contract one-way up β a crickets backing plugin.
The MCP server carries five key design choices:
-
Singleton streamable-HTTP β one daemon, many sessions (
Mcp-Session-Id), collapsing the host fan-out to a single writer alongside the CLI. Not stdio: stdio spawns one server per client, giving N OS processes onvault_mutexβ safe but not single-writer. -
Four snake_case tools:
memory_searchΒ·memory_appendΒ·memory_captureΒ·memory_forget. Not dot-names: OpenAI-family MCP hosts reject them. (A fifth,memory_recall, was retired β R0.9;memory_capturewas added by the capture design's front-door plan, FRIDAY ladder feature 2 β see the amendment log for both.) -
Soft-delete:
memory_forgetflipsstatus β deleted+ stampsdeleted_at; the file is never unlinked. Not hard-delete: GDrive sync resurrects a hard-deleted file from propagation cache. -
Loopback-first (
127.0.0.1/ Unix socket): the remote tier (cross-device via OAuth 2.1 tunnel) is a deferred v1.1 addition. -
Built on FastMCP, pinned
>=3,<4β with the official MCP SDK as a named fallback. Not unpinned: a FastMCP major can move the transport surface under the server.
As-built vs. target. Today harness state routes through the seam, but memory entries and the MCP server still write the vault directly β reaching around the port. Routing save/recall/forget through the engineβseam and re-platforming the MCP tools is V5-14 (see References).
The goal is concurrency-safe writes: when two sessions save to the same synced backend at once, both land and neither corrupts the other. A memory reaches disk through the V5-0 write protocol β a single atomic_write (temp β fsync β rename), guarded by an advisory per-backend mutex (a mkdir lock with an mtime heartbeat + stale-takeover, living outside the synced store) and, for replace-style files, a content-hash compare-and-swap that re-reads inside the lock before committing. Coordination lives in these primitives, not a daemon. (Honest boundary: this is grounded at the seam β the backend.write body that composes these for the vault lives in the obsidian-vault plugin.)
A session nobody was at writes no trace. The episodic trace is the record of a session someone was at. It is written only when the transcript holds an operator turn: a message you typed, or a slash command you typed. A tool result, a sidechain message, the host's own lines, a handoff prompt, a paste past the typing ceiling, a Desktop scheduled task's <scheduled-task> wrapper and a headless claude -p prompt are not turns. A scheduled run you reply to keeps its trace. The scheduled wrapper is stamped origin.kind: human like a typed turn, so it is read by its content. The test reuses the miner's reading of who said what (reflect._operator_text), so the two cannot drift apart.
Supersession is archive-then-replace. A new entry that evolves an old one flips the old to status: superseded and sets a supersedes: back-link from the new, so the history is an auditable link-chain.
Archive, decay, and prune. Supersession is one road into the archive; the other is decay β an entry that goes cold over time is retired to its tier's _archive/ rather than left to bloat recall. Three arms run today: heat curation demotes a cold always-load entry to its group root, /memory evolve archives a superseded entry, and per-note lifecycle tiering (V6-1) shipped 2026-07-07 β an optional lifecycle_tier: durable|volatile frontmatter field, defaulting to volatile, that harness/skills/memory/scripts/lifecycle.py reads alongside two hardcoded decay-exempt categories (kind: failure-incident entries and any entry under a decisions/ path segment, both always durable regardless of the tag). A volatile note decays exponentially (30-day half-life) from its last genuine recall access, tracked in a .lifecycle.json vault-root sidecar mirroring the existing .heat.json heat-policy pattern; only recall.py's prompt_submit() resets that clock, so a lint pass, an index rebuild, or a future dreaming pass touching the file never resets it by construction. recall.py's query() now returns lifecycle_tier + decay_score on every result, and the formatted recall-hit text surfaces the tier. Episodic-to-semantic consolidation (V6-4) shipped 2026-07-07 β harness/skills/memory/scripts/consolidate.py promotes a recurring target (referenced by at least 3 distinct episodic entries, using V6-2's typed-edge extractor as the recurrence signal) to a single durable, decay-exempt kind: crystallized entry, carrying a derived_from provenance field naming its sources; the episodic sources are never deleted or modified β only the new entry is added above them. Every consolidation write routes through the revert-log (RevertLog.record_and_apply), satisfying the provenance-edge prerequisite named below. A whole-corpus dreaming pass that moves cold notes to the archive shipped 2026-07-18 β see AgentM Auto-Organization, this section's own designed-for tail built out. A new tidying stage in dream.py retires a non-exempt entry past five years without a genuine recall to its tier's _archive/, with a one-cycle digest preview at 4.5 years; the stepped rank-curve retune that feeds it ships shadow-first, gated on the pinned retrieval eval never regressing before it drives live ranking. The earlier "staging gate" framing below β an autonomous job proposes to a staging inbox, the operator confirms before anything goes cold β is superseded: the auto-organization design's own review rounds concluded nothing is ever deleted, only moved, and a revert log makes every move undoable, so every tidying move auto-applies with no confirmation step, the same "retire it, reversible via the undo/revert log" basis compression's own 2026-07-11 auto-apply ruling already established. Semantic β procedural promotion and the phase-close crystallization digest's own separate trigger stay designed-for, framed in V6/V7. Prune resolves to archive; there is no hard delete.
Writing T1 (personal) takes a separate, explicit call. The normal write path reaches T2/T3 inside Agent/. Personal content sits outside vault_path, so writing it goes through a distinct seam call that the operator's request authorizes β the line between agent-controlled and user-controlled space (the memory-storage-seam design).
Memory is injected by two hooks. At session start, the always-load set (entries gated by always_load: true) is assembled under a hard ~500 ms budget. On every prompt, a five-step engine runs under ~300 ms:
- tokenize the prompt;
- embed it (a local embedder, degrading to a deterministic stub β the API embedder mode was removed) and search the local sqlite-vec index for the top-K by cosine similarity;
-
keyword-grep (filtering
status: superseded); - merge (semantic-weighted: ~0.85 similarity + ~0.05 keyword);
- dedup against always-load, and return the top few.
A stream runs whole or not at all. The 300 ms figure is a promise about interactive latency, and it is kept by admission, not by truncation: each half of the search is started only if it can finish, and a half that gets cut short has its results discarded. Cheap checks precede expensive ones β the vector index is opened before the query is embedded, because opening costs about a millisecond and embedding costs about four seconds from cold, and a host that cannot open the index would otherwise pay the whole load to produce a vector with nowhere to go. A cold embedder is declined outright under an interactive budget and attempted only where the budget can absorb it, such as the 10 s query CLI.
Truncation was the original contract and it was wrong, because a partial search here is not a partial result. The lexical scorer derives IDF and average document length from the candidate set its walk actually reached, so stopping halfway produces a different ranking over an arbitrary subset rather than the first half of the real one β confident-looking entries that no complete search would have chosen, indistinguishable downstream from real ones. Silence is the honest degradation. For the same reason a zero-result recall must say which streams were blocked and why: Loaded 0 reported identically for "searched, found nothing" and "could not search" is what let this fail silently for months (see the amendment log).
Recall is token-budgeted and scoped by kind + phase β a phase pulls the kinds its budget allows, not a flat top-K over the whole corpus (as-built today: a flat token-budgeted top-K; the --kind/--phase scoping is designed β see Risks). The vector index is device-local and never synced (the local-index tier), with mtime-vs-indexed-at drift detection that falls back to grep when an entry changed since it was embedded. Recall skips every tier's _archive/ by default β the cold store stays out of the always-load floor and the per-prompt search, so a growing record does not inflate what each call pays. An --include-archive opt-in (mirroring --include-inbox) widens the walk over the archive for deep research, an explicit ask, or a granted request; the vector index still covers archived entries, so an opened search ranks them on the same budget.
The record is built to compound. Three things turn a growing pile of entries into a navigable knowledge base:
- Learning populates it. Reflection mines durable entries from finished sessions, and the weekly field brief + deep research reach the web and the sources you favour and bring back what's worth keeping β typed and filed like anything else (the Experience design). The more agentm works and reaches outward, the larger and richer the record. This is the growth engine.
-
It's interconnected by wikilinks β built. Every entry cross-references its neighbors with
[[wikilinks]], and the engine maintains link integrity. The links are the connective tissue: the substrate everything else builds on. -
It becomes navigable by an index over the markdown β built. A device-local vector index makes the corpus semantically searchable, and the SQLite metadata table + hybrid
--filterrecall (V6-11) shipped 2026-07-06 β a query can now narrow bykind/project/status/tagin the same SQL-joined-with-vector query. The knowledge-graph extraction layer (V6-2) shipped 2026-07-07 βharness/skills/memory/scripts/graph.pywalks a note's raw markdown, finds every[[wikilink]], drops occurrences inside a fenced code block or inline backtick span, and classifies each surviving occurrence into one of nine typed edges (uses/depends-on/contradicts/caused/fixed/supersedes/decided-in/implements/references) by matching cue phrases in the surrounding prose, plus a second pass over frontmattersupersedes:/superseded_by:fields. It is deterministic β zero LLM calls, proven by a red test that AST-walks the module's own imports. The ranking itself now runs Reciprocal Rank Fusion β shipped 2026-07-07 (V6-3).recall.pyfuses ranked lists (score = sum of1/(60+rank)across sources) instead of the old weighted sum (sim * 0.85 + keyword * 0.05); the lexical stream is now real BM25 (term-frequency saturation + IDF + a deterministic stemmer) rather than a raw substring count, a MemoryOS-style 4-level fallback cascade degrades gracefully (hybrid β vector-only β lexical-only β unranked metadata-filter-only) as signals become unavailable, and a small abstraction-altitude rank boost prefers_index/_summaryanchor files at near-ties. Chunking (V6-10) and time-weighted retrieval (V6-12) shipped 2026-07-07 β the BM25 lexical scorer now scores each candidate by its single best-matching paragraph-aware chunk (harness/skills/memory/scripts/chunking.py) instead of one fixed 500-character window from the start of the file, and the per-notedecay_scorenow multiplies every candidate's fused ranking score before sorting, so a stale volatile-tier note genuinely ranks below an equally-relevant fresher one. Consolidation (V6-4) shipped 2026-07-07 βharness/skills/memory/scripts/consolidate.pypromotes episodic entries to a durable semantic entry once the same reference target recurs across at least 3 distinct episodic entries (reusing V6-2's typed-edge extractor as the recurrence signal, not a second mechanism); the consolidated entry carries aderived_fromprovenance field and reuses crystallization's five-field digest schema, and the episodic sources are never deleted or modified β only added above (the never-delete invariant). Still designed-for: folding the knowledge graph in as one more RRF stream, multi-hop traversal over its edges, and synonym expansion (an honest, undone v0 gap).
That is the trajectory toward a true knowledge base: a typed, densely-linked, indexed record that accumulates and compounds as agentm learns β the brain the other three pillars stand on. The typed entries, the wikilink substrate, the vector index, the knowledge-graph extraction, RRF-fused hybrid ranking, chunked passage scoring, time-weighted decay, and episodic-to-semantic consolidation are built today; graph fusion into retrieval, multi-hop traversal, and synonym expansion are designed-for, framed in V6 (Risks).
- The seam design β memory-storage-seam holds the deep storage contract (A2-fold, launched 2026-06-24); this pillar points down to it.
-
Experience grows the sole-owned space β reflection, heat curation, dreaming write here (Experience design); the heat policy owns the
always_loadgate. - Opinions keep their learned half here β an opinion's vault supplement is a memory entry (Opinions design).
- Personas draw their state here β Memory is the pseudo-persona beneath all (Personas design).
-
crickets backs the store β
obsidian-vaultimplements theStorageBackendcontract, one-way up. - The runner + the digest β scheduled jobs (the job-runner design, designed-for) write T2/T3 on their own and surface T2 changes to the operator through the reporting capability's digest; both route by the ownership tiers above.
-
V6-11 (the metadata table + hybrid filter) shipped 2026-07-06 β see the memory index design. V6-2 (deterministic typed-edge extraction) shipped 2026-07-07 β see the Capture section above and the amendment log below;
graph.pyextracts the 9 edge types but is not yet wired into recall as a retrieval stream. V6-3 (RRF hybrid retrieval) shipped 2026-07-07 β see the amendment log below;recall.py's ranking is now Reciprocal Rank Fusion over BM25 + vector similarity, with a 4-level fallback cascade and an abstraction-altitude boost, replacing the old weighted-sum merge and the raw-substring-count lexical signal. V6-10 (chunking) and V6-12 (time-weighted retrieval) shipped 2026-07-07 β see the amendment log below; the BM25 scorer now uses max-passage scoring over deterministic paragraph-aware chunks instead of a fixed 500-character window, anddecay_scorenow multiplies the fused ranking score before sorting instead of being a cosmetic label. V6-4 (consolidation tiers) shipped 2026-07-07 β see the amendment log below;consolidate.pypromotes episodic entries to a durable semantic entry on deterministic recurrence (β₯3 distinct episodic entries referencing the same target, via V6-2's edge extractor), never on a time-based or LLM-judged trigger; sources are never deleted. The rest of V6 indexed-recall stays designed-for, not built: the reservedDerivedMaintenanceextension point is where a future V6 slice adds multi-hop graph traversal and folds the knowledge graph in as one more RRF stream, and synonym expansion (an honest v0 gap left by V6-3). Graph and index stay layers over the markdown (the pages remain source of truth); the rest of that spec lives in the V6 plan + the seam design. -
Explicit kind-filtered recall shipped 2026-07-06 (
recall.py query --filter "kind=... AND project=...", V6-11's hybrid path) β a caller can now narrow recall by kind on demand. Still open, and distinct from the above: automatic phase-budgeted kind-scoping (a phase pulling only the kinds its budget allows, without an explicit--filter) βrecall.pystill runs a flat token-budgeted top-K by default; confirm at review whether that automatic scoping is still wanted given the explicit filter now exists. -
V6-1 (per-note lifecycle tiering) shipped 2026-07-07 β see the Capture section above and the amendment log below. V6-12 (time-weighted retrieval) shipped 2026-07-07 β see the amendment log below;
decay_scorenow multiplies the fused ranking score for every candidate before sorting, so staleness genuinely affects rank order rather than being an attached label. V6-4 (consolidation tiers) shipped 2026-07-07 β see the amendment log below; episodic-to-semantic promotion is gated on the revert-log (every consolidation write goes throughRevertLog.record_and_apply, proven by a test that reverts a run and confirms the written file disappears) and thederived_fromprovenance edge is live insave.py's frontmatter. The rest of the decay/archive lifecycle stays designed-for. Heat curation + supersession-archive + V6-1 tiering + V6-12 + V6-4 consolidation ship; crystallization's own digest schema shipped separately and is reused here; self-healing lint and the whole-corpus dreaming compaction pass stay V6/V7. Until those land, archival stays manual (/memory evolve) or operator-confirmed. The migration to the three-tier folder layout is operator-gated and updates the hardwiredpersonal/_always-loadconstants inrecall.py/heat_policy.py/save.py. -
The seam content has migrated β
memory-storage-seamis launched (A2 ADR-fold, 2026-06-24); this pillar holds the pointer down to it. -
The kernel
storage_vault.pywas deleted β removed in V5-3 (commit d95468b); the vault backend now lives only in theobsidian-vaultplugin'sstorage_vault.py. - Re-audit triggers: confirm kind-scoped recall.
-
scripts/harness_memory.pyβvault_path()resolver, config readers,resolve_project(), backend-aware*_state_file -
scripts/storage_seam.py+scripts/backend_selection.pyβ the seam contract + selector (deep detail in the seam design) -
scripts/capability_resolver.pyβ capability-availability (theenhances:runtime half) -
scripts/vault_lock.pyβatomic_writeΒ·content_hash(CAS) Β·vault_mutex(the V5-0 primitives) -
harness/skills/memory/scripts/βrecall.py(5-step engine),vec_index.py(sqlite-vec index),save.py/evolve.py(write + supersession),notes_link_discovery.py(wikilink integrity) -
harness/hooks/βmemory-recall-session-start,memory-recall-prompt-submit - memory-storage-seam design β the concurrent-write protocol, seam fail-loud selection, V5-3 cutover, routing plane, backend-aware harness state
- V5-14 β storage-convergence (memory-entry seam adoption + MCP re-platform); ROADMAP-MASTER β€
2026-10-09 β a session with no operator turn writes no trace (task 190, step 2; the operator's ruling of 2026-10-07). The write-quality audit of 2026-10-07 found that 9 of the 20 memory cards written on 10-05 and 10-06 were traces of one unattended PlayOn poll, every three hours, and that 55 of 141 traces had a bare session id as their request. Capture now says a session nobody was at writes no trace, and episodic_trace.py holds to it. Over the 293 recent sessions with output, the rule drops 39 scheduled runs and 7 sessions opened by an agent-written prompt (a chip, a handoff, a long paste) with nothing typed after. The 247 sessions that hold a typed turn are unchanged, including 2 opened by a typed /work alone. Why not key on the job: the host stamps a scheduled run as it stamps your turn, so origin cannot tell them apart, and there is no job name in an ordinary session to key on. Why not a similarity bar: consecutive polls score 0.1 to 0.4 alike, and a trace is one sighting by definition. Re-audit trigger: a scheduled job whose outcome you wanted kept in memory, which should then capture it on purpose; or a session you were at that left no trace.
2026-10-03 β the growth-engine sentence names the weekly field brief, not forward learning (task 185, step 1). Forward learning retired on the operator's ruling of 2026-10-02 and the brief replaces it; the Experience design carries the decision and its reasons. Re-audit trigger: none of its own; it follows the Experience design's.
2026-09-03 β the vault-layout diagram is reconciled again, this time to the vault-root Projects/ generation (filing-v2 part 2b). Projects/<project>/ β T2 curated content β moved from Agent/desk/projects/<project>/ to the vault root, alongside Agent/; desk/ keeps only the shorter-lived briefs/ and scratch/. Why not just substitute the new literal again: the 2026-08-15 amendment already made that mistake once and named the fix β write against the resolvers vault_path() and memory_root() β but Projects/ sits at neither; the diagram now shows it as a sibling of Agent/ explicitly, the same way the design body does. What this does not resolve: the same open question the 2026-08-15 amendment left standing β where the operator-directive spaces (preferences/, feedback/) sit relative to the T2/T3 tier boundary β is untouched by this move. Re-audit trigger: filing-v2 part 3 is expected to retire the desk/projects/ fallback rung entirely; reconcile this diagram again when that lands.
2026-08-15 β the vault-layout diagram is reconciled to the four-space vault, and rewritten against the resolvers so it cannot go stale this way again. The layout block still described the pre-migration tree: a .../Obsidian/ root, Agent/projects/, Agent/_archive/, and an Agent/Memory/ holding _meta/. None of those paths exist. The vault root moved off the cloud mount, projects/ became desk/projects/, personal/ became memory/, the curated archive became _vault-archive/, and _meta/ moved up beside memory/ rather than inside it. external/ and _dream/ were not in the diagram at all. Verified by walking the live vault on 2026-08-15. Paths are now written as <vault_path>/ and <memory_root>/ rather than a literal root β the two resolvers the code actually uses, which also makes the vault_path-vs-memory_root distinction explicit at the point of most confusion. The old block labelled Agent/ as "agentm's vault_path", which had become simply wrong: Agent/ is the memory root, one level inside the vault path, and that specific confusion mis-roots every vault-relative path when it is made in code. Why not just substitute the new literal: a literal is what went stale, and the vault has now moved twice; naming the resolver means the next move invalidates no prose. What is deliberately left unresolved: the four-space migration moved the operator-directive spaces (preferences/, feedback/) under memory/, which this design labels T3 agentm-sole, while the tier model calls those directives T2 curated. Either the paths or the tier boundary is wrong, and that is an ownership ruling rather than a documentation fix, so the body now names the open question instead of quietly re-assigning a tier. Re-audit trigger: memory/_always-load/ is empty on the operator's machine as of this amendment β expected for a heat-promoted floor with nothing yet promoted, but if it is still empty when the heat policy has had time to run, the promotion path is not working and the floor is a fiction the diagram is asserting.
2026-08-14 β a config pointer that still resolves, just not to the live vault, is now a doctor row rather than nothing at all. The vault-path convention says resolve, don't recall, and check-no-hardcoded-vault-path enforces it across tracked files. Neither reaches the place it actually broke. Both repos' gitignored .harness/project.json still named the retired Google Drive root after the vault moved and its internal layout was reorganized (projects/ β desk/projects/); the old directory had not been deleted, so every read succeeded and returned a tree frozen on the day of the move β the agentm board-items file was twelve days stale before anyone looked. Nothing raised, because nothing was broken in the way software usually breaks. The detector is machinery_doctor.py's project-json-pointers:{repo,vault} rows, which take each reachable project.json and ask whether its path-valued keys still land inside the vault this machine resolves right now β failing both on a path that is gone and on one that exists outside the live vault. Why not extend the existing gate: it walks the filesystem and prunes .harness/ by name, so gitignore is not what excludes it, and the directory legitimately holds progress.md and archived plans whose prose quotes retired paths while narrating migrations β scanning it would false-positive on exactly the write-ups that describe these fixes. It also only recognizes the retired cloud-drive mount literal, so a move between two plain local roots would pass it untouched; it protects what ships, which is a different question from whether this machine points at the right place. Why the two surfaces are checked separately: MEMORY_VAULT_PATH and items_source are validated against memory_root() while IDEAS_SURFACE_PATH is validated against vault_path(), because Ideas.md is the operator's own note one level above the memory tree and collapsing them would flag a correct install. Re-audit triggers: the path-valued key list is hand-written, so a fourth pointer key added to project.json gets no coverage until someone extends _PROJECT_JSON_PATH_KEYS β revisit whether it should derive from the consumers when a fourth appears; and discovery reaches the vault-resident config through _VAULT_PROJECTS_REL_NEW, so a further layout migration needs that constant to move with it or the vault-side row silently stops being emitted.
2026-08-02 β the hooks resolve an interpreter that can actually open the vector index, and the doctor now says so when it can't. The amendment below found that the vector index had never been reachable on the operator's machine and named the cause: the hooks ended in a bare python3, which is a PATH lookup, and on a stock macOS box that is Apple's system Python β built without --enable-loadable-sqlite-extensions, so it has no sqlite3.Connection.enable_load_extension at all. sqlite-vec is a loadable native extension, so _open_index() returned None on every call and every caller read that as the graceful "index not built yet" skip. The index itself was fine the whole time: the same script against the same vault reported {"size": 1035} under Homebrew's python3.13 and {"size": null, "note": "sqlite-vec unavailable"} under Apple's. The fix is a canonical resolver at harness/hooks/lib/resolve-python.sh (with a pwsh twin) that the four memory hooks bootstrap through: an explicit $AGENTM_PYTHON override, then the first candidate whose sqlite3 build supports extension loading, then today's bare python3 as the floor β so a box that already worked keeps its exact behavior and a partial install degrades as it always did. One hook already carried a private version of this probe, which is why the diagnosis was available at all; it listed only bare-name paths, and Homebrew installs a versioned python3.13 with no bare symlink, so on the very machine it was written for every candidate missed and it fell through to the incapable floor. Versioned names are now probed through PATH, and nothing is hardcoded to one machine's layout. Why an explicit override is honored even when it is incapable: an operator who names an interpreter gets that interpreter, and the doctor row names the override as the cause β silently substituting a better one would make that diagnosis impossible. Why the probe asks about the build and not about sqlite_vec: enable_load_extension is fixed at compile time and cannot be added in place, while a missing sqlite_vec is one pip install away, so selecting on the package would let a fixable gap veto the only viable interpreter on the box; the doctor reports the package separately, against the interpreter the resolver actually chose. The detector is the durable half. machinery_doctor.py gains a memory-hook-interpreter row that delegates to the resolver rather than re-deriving its candidate list β the same discipline as the <job>:config rows in Autonomy, and for the same reason: a check with its own copy of the logic can report a healthy interpreter the hooks never pick. check-vendored-parity's hook-config mode now gates the hooks' shared bootstrap alongside _resolve_vault_path, so one hook cannot quietly drift back to a bare python3. The index was re-drained on the resolved interpreter (1052 entries enqueued, 1182 processed, 0 errors, drift to zero across 1315 entries) and semantic search returns real matches again. What this does not fix, stated plainly: a cold sentence-transformers load is ~4.4 s and every hook invocation is a fresh process, so per-prompt recall still correctly declines the vec stream under its 300 ms budget β the admission contract below is unchanged and still doing its job. This makes semantic recall possible; a warm embedder is what would make it reachable interactively, and a persistent FTS5 lexical index is still worth building. Both stay separately sequenced, and both are now sized on measurement rather than estimate β taken the same day, on this machine, and carried onto the roadmap as #410 and #411. The amendment below asserted that FTS5 "needs no loadable extension, so it works on the very hosts where sqlite-vec cannot"; that is now checked rather than reasoned. An FTS5 virtual table with bm25() ranking creates and queries cleanly under Apple's system Python 3.9.6 β the exact interpreter that cannot load sqlite-vec β so the lexical index would lift recall even on a host where this interpreter fix has not been applied, which is a stronger property than the original claim spelled out. And the warm-embedder case is worth roughly 290Γ: a cold import plus first embed costs 4,385 ms, a warm embed ~15 ms median, so a resident model turns the blocking step from four seconds into something that fits the 300 ms budget with room to spare. Recording both here because this design's own re-audit trigger says to treat a cost figure as stale until re-measured, and an unverified capability claim is the same hazard wearing different clothes β this amendment exists because an interpreter assumption went unchecked for years. Re-audit triggers: the resolver spends ~40 ms per hook process on a box whose bare python3 is incapable β re-measure and consider caching the resolution if a warm-embedder path lands and that budget starts being spent on real retrieval; and re-check the candidate list if a packaging convention appears that ships neither a bare nor a python3.N name on PATH.
2026-08-02 β per-prompt recall enforces stream admission instead of truncation; the interactive budget is named as an index-lookup budget (GH #92). Per-prompt recall had been returning zero entries on every prompt, on any machine with sentence-transformers installed β which is embed.py's documented default mode. _vec_search embedded the query before checking whether the vector index could be opened, so a ~3900 ms cold model load ran against a 300 ms budget; it then re-checked the deadline and discarded the embedding it had just paid for; the lexical half ran next, found no budget left, and returned nothing on its first loop iteration. The hook still fired, still exited 0, and still reported the overrun honestly, so nothing went red anywhere β the same invisible-failure shape as the session-reflection gap, and found the same way, by driving the hook in a test rather than reasoning about it. Two further defects surfaced while reproducing it, both worse than the reported one. The vector index has never been reachable on the operator's own machine: the hooks run under macOS system Python, whose sqlite3 is built without enable_load_extension, so sqlite-vec cannot load and _open_index returns None on every call β the cold embed was buying a vector that had nowhere to go regardless of the budget. And the lexical half's cost estimate was stale by two orders of magnitude: its call site claimed "<50 ms typical", measured when the vault held fewer than 100 entries; a full BM25 walk now costs ~3050 ms across 1745 entries, so under a 300 ms budget the walk reached ~70 of them and returned whichever the directory traversal happened to visit first. That last one is not a partial ranking but a different one β IDF and avgdl are computed over the candidate set actually walked β and it is the state CI runs in, since the test suite's fake HOME drops the user site-packages and lands in the fast, wrong-answer path. The decision: a recall stream runs to completion or does not run, cheap checks precede expensive ones, and a zero result names the streams that were blocked. Why not simply raise the budget: a cold load per prompt is ~3900 ms and every hook invocation is a fresh process, so nothing is ever warm and every prompt would stall for seconds. Why not let the lexical half run unbounded as the documented graceful fallback: at ~3050 ms it is no cheaper than the embed it was meant to fall back from. Shipped in recall.py (admission + status reporting), embed.py (local_model_is_resident, a probe that deliberately cannot trigger the load it measures), the hook's own contract, and scripts/test_recall_stream_admission.py. Measured effect on the reported repro: 4.25 s and zero entries becomes 0.07 s and the matching entry. Consequence to accept honestly: on a vault the size of the operator's, both streams are now correctly declined and per-prompt recall returns an explicit nothing rather than junk β the failure is fixed, the capability is not. Restoring it needs the interactive path to become genuinely index-only: a persistent lexical index (SQLite FTS5 needs no loadable extension, so it works on the very hosts where sqlite-vec cannot) and a warm embedder, both sequenced separately rather than half-built here. Re-audit triggers: revisit VEC_COLD_EMBED_MIN_BUDGET_MS if a warm-embedder path lands and cold loads stop being the common case; re-measure the lexical walk's cost whenever the corpus grows materially, and treat any cost figure in a comment as stale until re-measured β this amendment exists because one was trusted for a year.
2026-07-19 β auto-organization part 3 ships: dedup, lint, and the remaining automation guards, FRIDAY ladder feature 7 (reconciling the two re-audit triggers below β this closes out the whole 3-part AgentM Auto-Organization design). All 9 tasks of PLAN-auto-org-dedup-and-lint.md shipped this same day: a write-time dedup guard reinforces an exact-duplicate note instead of writing a new _1/_2 suffix file, a cluster-aware weekly pass collapses whole suffix families (fingerprint-exact deterministically, fuzzy matches behind a cheap-model verdict seam that stays unreachable until a model tier exists), inbox triage folds into the weekly dreaming cycle instead of waiting on an operator-invoked command, ambiguous dedup/merge candidates surface on a needs-your-eye list (console + digest + morning-brief count) rather than being forced, a capped drain works down the existing suffix-family backlog, a new lint engine (/memory lint) reports orphans/broken links/contradictions/quality scores and auto-repairs the one safe case (a mis-cased wikilink with an unambiguous target), the connectivity and browse-surface meters land in the digest, and the remaining automation guards close out the design: the anomaly breaker generalizes from tidying-only to cover suffix_backlog_drain and lint too, a graph-snapshot cross-check catches drift between the persisted snapshot and a live re-extraction, and a sampled higher-tier audit narrows the ambiguous linking band automatically when disagreement runs high (deterministic-only today β no higher-tier-model primitive exists yet, a named tracked gap). See AgentM Auto-Organization for the full design and every task's own as-built note; that design's own status flips from final to launched with this landing β the archive-and-prune designed-for tail this section's own paragraph above named is now fully built out across all three parts. Re-audit trigger: none outstanding for this pillar; the child design carries its own remaining calibration-only re-audit triggers (the stepped-schedule/floor/cap defaults, the first backlog-drain cycles).
2026-07-18 β auto-organization part 2 ships: write-time linking, FRIDAY ladder feature 6 (reconciling the re-audit trigger below). The amendment below flagged parts 2 and 3 of AgentM Auto-Organization as the reconciliation points for this pillar's archive-and-prune tail; part 2 is about linking, not archival, so it doesn't change that paragraph directly, but it does build out the How it grows section's "interconnected by wikilinks β built" bullet from a manually-maintained property into an actively-maintained one: a write-time linker now adds wikilinks to every newly saved note (deferred to the next embedding-queue drain, not synchronous on save β see the child design's own as-built note for why), a weekly sweep improves links on older notes and drains the 880-note unlinked backlog at 25/cycle, and a connectivity meter now tracks organic link coverage (13.8% baseline) and the linker's own generated-link count as two independent digest numbers, so the linker can never inflate the figure it's judged against. All 7 tasks of PLAN-auto-org-write-time-linking.md shipped this same day. See AgentM Auto-Organization for the full design and Experience & Dreaming for the dreaming-pipeline-level record. Re-audit trigger: auto-organization part 3 (dedup + lint) is the only piece of that design still designed-for β reconcile here as it lands.
2026-07-18 β the archive-and-prune designed-for tail built out: auto-organization part 1 ships a real tidying stage (FRIDAY ladder feature 5). This section's own "Archive, decay, and prune" paragraph flips its "designed-for" tail to point at AgentM Auto-Organization, the new child design lifted the same day. A new dream.py tidying stage retires a non-exempt entry past five years without a genuine recall access to its tier's _archive/, with a one-cycle digest preview at 4.5 years, and a stepped rank-curve retune (full strength to 6mo, half to 1y, an eighth to 3y, a sixteenth to 5y) feeds it β shadow-first, gated on the pinned retrieval eval (scripts/health/eval_v6_retrieval.py, extended with a --decay-curve comparison mode) never regressing before it drives live ranking. The prior "staging gate" precondition this section named (an autonomous archival job proposes to a staging inbox, the operator confirms before anything goes cold) is corrected: the shipped behavior auto-applies every tidying move with no confirmation step, matching compression's own already-established "retire it, reversible via the undo/revert log" basis β that earlier text described a precondition that never actually gated the design as built, not a behavior this amendment changes. Re-audit trigger: auto-organization parts 2 (write-time linking) and 3 (dedup + lint) land as separate FRIDAY ladder features; reconcile here as each ships.
2026-07-18 β arc-as-metadata convention adopted (operator adjudication, crickets session), new Β§"Arcs β temporal grouping as metadata, not folders." Vault projects were flat at the sub-project level: an arc (a V5/V6/V7/V8 wave, architecture-governance, consolidation-review, a lettered build wave) only had folder identity inside _harness/designs/<arc-slug>/; in the permanent decisions/ record it degraded to tags, and in _harness/archive/ to filename prefixes. This amendment names arc identity as a frontmatter fact (arc:, its own field β not a third group: segment, since group: mirrors the folder path exactly) for every permanent decisions/designs entry, adds kind: arc-index (one linking note per arc at projects/<project>/arcs/<arc-slug>.md, cross-repo arcs canonical in the owning project + a thin pointer in the sibling), and validates arc: values against a new arc_registry.py (mirroring kind_registry.py). The only folder moves this convention sanctions are inside gitignored _harness/ working state, and only for an arc that has actually closed: flat PLAN.archive.* files group into archive/<arc>/, and a closed arc's design folder moves from _harness/designs/<arc>/ to _harness/archive/designs/<arc>/ (crickets' archive/designs/efficiency-automation/ is the live precedent). Why not fold folders for the permanent record too: arcs complete (a folder tree would accumulate dead branches forever), a move invalidates the vec-index/embedding paths already computed against a note's location, and a cross-repo arc can't live in one project's folder. Backfill runs through harness/skills/memory/scripts/migrate_arcs.py (mirroring the retired migrate-adr.py's dry-run-by-default shape): it infers arc: from existing tags/filenames and proposes _harness/ archive moves, printing a reviewable mapping table before anything is stamped or moved. Re-audit trigger: if the arc count or the volume of cross-repo pointer notes makes the registry unwieldy, revisit whether arc-index pages need their own index-of-indices.
2026-07-18 β memory_capture shipped as the MCP server's fourth tool (capture-front-door plan task 3, FRIDAY ladder feature 2). The design's own "Three snake_case tools" line above is corrected to four: memory_capture stages a candidate to personal/_inbox/ (never permanent memory), the second front door beside memory_append per the capture design (wiki/designs/agentm-capture.md, lifted the same day). memory_append is unchanged. See wiki/designs/agentm-capture.md's own Document History for the full build detail. Re-audit trigger: none β this is a straightforward count correction; the capture design owns the ongoing narrative for this tool's evolution.
2026-07-12 β project-archival lifecycle convention (V8 proving ledger, L3). The close-out convention already archives plans inside a project's _harness/ (PLAN.archive.YYYYMMDD-<slug>.md); nothing named what happens when a whole project is done. This amendment closes that gap: when a project's work is finished, (1) a harvest pass reads its notes and promotes still-load-bearing content elsewhere in the vault first β the mechanical move is nearly free but not lossless if a genuinely active convention or decision entry gets buried with it; (2) the project directory moves to projects/_archive/<name>/, mirroring the existing per-project _archive/ pattern one level up; (3) repo_registry unregister <slug> runs if the project's own repo is still registered; (4) Home.md's Projects section drops the entry and gains one line under a new "Archived projects" pointer. _archive/ is already excluded from recall.py and frontmatter_validator.py's walks (see the storage-tier discussion above); vault_lint.py gained the same exclusion in the same lane (it was the one walker still descending into archived entries). First run: agent-m-v4, memoryvault, small-coding-projects, presence archived 2026-07-12 (operator-named list). The harvest pass on agent-m-v4 found one genuinely load-bearing entry β silent-source-influences.md, cross-referenced from ~15 files across both repos' active design trees β and promoted it to personal/preferences/ before the project moved; the rest of that project's content archived without incident. Re-audit trigger: if a future harvest pass needs a more structured promotion mechanism than "read it, judge it," or if the per-project _archive/ and _dream-staging-style staging conventions ever need to converge into one mechanism, revisit here.
2026-07-08 β the vector-inclusive re-measurement the V6-3 entry below flagged as an honest gap (AA5 consolidation C8). The build sandbox that produced every number below couldn't load the sqlite-vec extension β this session found the fix: sqlite-vec loads fine under Homebrew Python 3.13 (enable_load_extension works, vec_version() returns v0.1.9); the stock python3 on this machine (3.9.6) lacks extension-loading support entirely, which is what actually blocked it before, not anything about sqlite-vec itself. Re-ran scripts/health/eval_v6_retrieval.py unmodified against the same pinned query-set-v0.json/edge-fixture-v0.json (hashes verified against fixtures/v6-eval/README.md before running) under python3.13, so recall._vec_search() actually loads the extension this time instead of the fallback path silently degrading past it. The result is not simply "the same numbers, confirmed" β it's a genuinely different comparison, and worth being exact about why: the old formula (combined = sim * 0.85 + keyword * 0.05) also uses _vec_search() directly, so giving it real vectors for the first time boosted it too, not just the new RRF formula. Measured against the real vault: old R@5 57.6% / new R@5 43.9% (both vector-inclusive; contrast the vector-less run below, old 31.1% / new 46.2%), old P@5 14.5% / new P@5 10.9%, compression 1.50 (old) vs. 2.09 (new), discovery-rate 6.9% (2 of 29 gold pairs found only by the new formula) β accuracy_regressed=True, so this project's own merge gate would not clear a merge today, on this comparison. This is a real, honest reversal from the vector-less headline, not a tuning suggestion acted on here (out of scope for this plan; parked as a named follow-up below) β the vector-less numbers already shipped and are not retracted, they're simply now known to be one comparison among two, and the vector-inclusive one is less favorable to the RRF formula as currently weighted. Confirmed no similar caveat applies to the sibling evals: eval_v6_graph.py never touches the vector stream (deterministic edge-extraction P/R only) and eval_v6_consolidate.py calls recall.query(mode="stub") by design, not by sandbox limitation β neither needed a re-run. Parked follow-up (not executed here, measure-only plan): if RRF's per-source weighting or the vector-similarity contribution needs retuning now that a real vector-inclusive baseline exists, that is a future plan's decision, informed by this measurement, not a conclusion this entry draws. Re-audit trigger: this result is device-local (this machine's own populated vec-index.db β not portable to CI or another machine without its own re-run); revisit when the V6-19 fusion-floor gate (AA5-REENTRY-VERDICT.md Β§2 item 9) is actually decided, since that gate's own condition (ii) is this exact vector-inclusive comparison.
2026-07-07 β V6-4 consolidation (episodic β semantic) shipped, the last task of PLAN-wave-e-v6-index (task 7). New harness/skills/memory/scripts/consolidate.py promotes recurring episodic entries to a single durable semantic entry on a deterministic recurrence gate β never time-based, never LLM-judged: when the same reference target appears across at least 3 distinct episodic entries, reusing V6-2's typed-edge extractor as the recurrence signal rather than a second detection mechanism. The consolidated entry is written as kind: crystallized (reusing the phase-close crystallization digest's own five-field schema β question/investigation/findings/lessons/open-threads β exactly as that module's docstring anticipated), is decay-exempt, and carries a new derived_from frontmatter field (save.py) naming the episodic sources it was synthesized from. The never-delete invariant holds: consolidation only adds the new entry above its sources, never deletes or edits them. Every consolidation write routes through the revert-log primitive (RevertLog.record_and_apply, from the sibling dreaming plan merged earlier this session) rather than a direct file write; scripts/test_consolidate.py's own red test reverts a consolidation run and confirms the written file disappears, proving the write was genuinely journaled. scripts/health/eval_v6_consolidate.py measured against a synthetic fixture (not the real vault β running consolidation against real content stays a deliberate, operator-invoked action, matching the dreaming pipeline's stage-and-confirm posture): entries needed for full topic coverage dropped 3 β 1, with zero recall regression (the consolidated entry is found, and all 3 sources remain independently findable). Honest caveat: because sources are never deleted, raw vault file count increases with each consolidation rather than decreasing β the "reduces corpus size" framing is satisfied as reduced retrieval redundancy, not reduced file count. This closes the "How it grows" + Risks sections' last named designed-for item from the 2026-06-21 amendment's trajectory; what remains designed-for after this task is graph fusion into retrieval ranking, multi-hop traversal, and synonym expansion. Re-audit trigger: revisit MIN_RECURRENCE (3, a named v0 default) once real query volume against the live vault exists; reconcile here when semantic β procedural promotion or the whole-corpus dreaming pass start consuming derived_from chains.
2026-07-07 β V6-10 chunking + V6-12 time-weighted retrieval shipped (PLAN-wave-e-v6-index task 6). New harness/skills/memory/scripts/chunking.py adds a deterministic, paragraph-aware chunk_text(), wired into recall.py's BM25 lexical scorer as max-passage scoring: each candidate's score is now its single best-matching chunk's score, not one fixed 500-character window from the start of the file β directly fixing a real gap task 5's own eval had traced, where long documents whose relevant content lived past the first 500 characters scored zero regardless of the ranking formula. Chunks are computed on-the-fly at query time, not persisted as new entry_meta columns; a chunk-level vector-embedding pipeline is a separate, larger future build. Separately, decay_score (task 3's lifecycle.py) now multiplies every candidate's fused ranking score before sorting, not just a label attached to already-decided results, so a stale volatile-tier note genuinely ranks below an equally-relevant fresher one. This task's own eval caught two real bugs before shipping: the chunking implementation's BM25 avgdl (length-normalization baseline) was accidentally computed over the whole vault instead of just the candidate set, silently making long documents score worse; and the decay anchor fell back to a note's created date instead of updated when no access history existed, wrongly penalizing a same-day-edited reference document for staleness it didn't have, purely because the retrieval system had never queried it before (a cold-start bug). Both were found and fixed within this task. Measured against the real vault (cumulative effect of this task plus task 5's RRF change together, vs. the original pre-task-5 baseline): recall@5 improved 31.1% β 73.5%, precision@5 improved 7.3% β 17.3%, queries finding at least one correct answer went 8 β 18 of 22, and 10 of 29 gold answer-pairs are now found only by the new retrieval stack. The compression proxy softened very slightly (1.875 β 1.83) β outweighed by the large accuracy and discovery gains, clearing this project's own guard rule against merging a ranking change on accuracy improvement alone. Re-audit trigger: revisit the chunking window size and the decay-anchor fallback once real query volume against the live vault exists; reconcile here when consolidation tiers or a chunk-level embedding pipeline start consuming chunk boundaries.
2026-07-07 β V6-3 RRF hybrid retrieval shipped, replacing the weighted-sum merge (PLAN-wave-e-v6-index task 5). recall.py's ranking formula changed from a weighted sum (combined = sim * 0.85 + keyword * 0.05) to Reciprocal Rank Fusion (RRF): score = sum of 1/(60+rank) across each source's ranked list. The lexical signal upgraded from a naive substring-count to real BM25 (term-frequency saturation + inverse-document-frequency + a deterministic stemmer). Added the MemoryOS 4-level fallback cascade β hybrid retrieval β vector-only β lexical-only β unranked metadata-filter-only β so ranking degrades gracefully as signals become unavailable, and an abstraction-altitude rank boost that prefers _index/_summary anchor files at near-ties. Knowledge-graph fusion (task 4's edges) was explicitly kept out of this task's scope β folding the graph in as one more RRF stream and multi-hop traversal are separate future items; synonym expansion is an honest, undone v0 gap. Paired with scripts/test_recall_rrf.py (12 new tests: RRF arithmetic and deterministic tie-breaking, BM25 stemming + IDF-favors-rare-terms, the altitude boost, and each reachable fallback-cascade tier) and scripts/health/eval_v6_retrieval.py (the real-vault half of the eval, comparing old vs. new formula on task 2's 22 gold queries, gracefully skipping in CI, wired into run-fast-tier.sh). Measured against the real vault: recall@5 improved 31.1% β 46.2%, precision@5 improved 7.3% β 10.9%, a compression proxy (average rank position needed to find the first correct answer β lower means fewer results need injecting for the same coverage) improved 1.875 β 1.36, and a discovery-rate of 17.2% (5 gold answers found only by the new formula, that the old one missed entirely) β all three signals improved together, clearing this project's own guard rule against merging a ranking change that only improves raw accuracy without also improving compression or discovery. One honest environmental caveat: this sandbox can't load the sqlite-vec extension, so vector similarity was unavailable for both the old and new formula in this run β the comparison measured is BM25+altitude vs. the old raw-keyword-count, not the full vector+lexical fusion; that caveat is noted in the eval script itself, not hidden. Worth recording as a real example of why this project builds the eval before declaring a ranking change done: the first draft of the abstraction-altitude boost was miscalibrated (roughly 60x too large relative to genuine RRF score gaps between adjacent ranks) and actively broke retrieval quality β recall crashed from 31% to 1.5% β before the eval caught it and it was fixed down to a tie-breaking-only magnitude. Re-audit trigger: revisit once knowledge-graph fusion (folding task 4's edges in as an RRF stream) or synonym expansion land, since both would change the source mix this formula fuses over.
2026-07-07 β V6-2 deterministic typed-edge knowledge graph shipped (PLAN-wave-e-v6-index task 4). New harness/skills/memory/scripts/graph.py adds a zero-LLM, regex/heuristic extraction cascade that reads a note's raw markdown and produces typed edges: it finds every [[wikilink]], excludes occurrences inside a fenced code block or inline backtick span (so a code-example placeholder that merely looks like a link is never extracted), classifies each surviving occurrence into one of nine edge types (uses / depends-on / contradicts / caused / fixed / supersedes / decided-in / implements / references) by matching cue phrases in the surrounding prose, and runs a second pass over frontmatter supersedes:/superseded_by: fields independent of wikilink wrapping. This is one RRF retrieval stream to be fused in a later task, not the retrieval spine itself β markdown stays source of truth, and multi-hop graph traversal is explicitly out of scope here (a separate future verb). Paired with scripts/test_graph_extract.py (19 fully-synthetic unit tests, CI-safe, including a red test that AST-walks graph.py's own imports to prove zero LLM/network calls anywhere in the extraction path) and scripts/health/eval_v6_graph.py (the real-vault half of the eval, reading the operator's private Obsidian vault via MEMORY_VAULT_PATH, gracefully skipping in CI, wired into scripts/health/run-fast-tier.sh's health-dashboard JSONL emission under the "memory persist+recall" axis). Measured against the real vault: recall 1.0 (145/145 labeled true edges found), trap-rejection 1.0 (12/12 deliberate non-edge traps correctly never extracted), type-classification precision 0.766 (111/145 edges got the exactly correct type of the 9). This is an inaugural measurement β no precision/recall floor existed before this task, since the graph didn't exist yet β and a future plan deciding whether to fuse multi-hop traversal into retrieval ranking will use this number as its reference floor. This is the extraction slice the 2026-06-21 amendment below named as the trigger for flipping the knowledge-graph from designed-for to as-built; RRF fusion (V6-3/10), consolidation tiers, chunking, and multi-hop traversal stay designed-for. Re-audit trigger: the 0.766 type-classification precision is untuned against a larger labeled set β revisit once the graph feeds live retrieval ranking; reconcile here when RRF fusion (V6-3) starts consuming these edges.
2026-07-07 β V6-1 per-note lifecycle tiering shipped (PLAN-wave-e-v6-index task 3). New harness/skills/memory/scripts/lifecycle.py adds an optional lifecycle_tier: durable|volatile frontmatter field (via save.py's locked FRONTMATTER_FIELD_ORDER, defaulting to volatile when omitted) plus two hardcoded decay-exempt categories regardless of that tag: kind: failure-incident entries and any entry under a decisions/ path segment. A volatile note decays exponentially (30-day half-life) from its last genuine recall access; only recall.py's prompt_submit() β the sole call site β resets that clock via record_recall_access(), mirroring heat_policy.record_hit()'s exact call-site discipline, so a vault-lint pass, an index rebuild, or a future dreaming/consolidation pass touching the file can never reset it. Persistence is a new .lifecycle.json vault-root sidecar (atomic writes via vault_lock, best-effort), mirroring the existing .heat.json sidecar's split (frontmatter carries the durable classification; the sidecar carries the ephemeral access clock). recall.py's query() now returns lifecycle_tier + decay_score on every result, and the formatted recall-hit text shows the tier. This is the lifecycle slice the 2026-06-28 entry below named as queued (V6-1); it ships ahead of consolidation tiers (V6-4), time-weighted retrieval (V6-12), and dream-mode compaction (V7-2), which stay designed-for. Re-audit trigger: the DECAY_HALF_LIFE_DAYS constant (30 days) is untuned against real access patterns β revisit once real usage data exists; reconcile here when consolidation tiers or the dreaming pass start consuming decay_score.
2026-07-06 β V6-11 shipped; V5-14 storage convergence shipped (AG Wave B leaders 3/5 + 5/5). The SQLite metadata table + hybrid --filter recall (V6-11) ship β see the memory index design for the column set and the _extract_meta_from_file extraction. Separately, V5-14 shipped: save/recall/forget now route through the storage seam (DeviceLocalBackend's verbs, not raw atomic_write/Path calls), and the MCP server re-platformed onto that seam-routed engine β see memory-storage-seam for the seam-routing detail. Reconciled the "How it grows" + Risks sections to distinguish what shipped (the explicit --filter, the metadata columns) from what's still designed-for (the knowledge-graph, RRF ranking, chunking, consolidation tiers β the rest of V6) and from a related-but-distinct open question (automatic phase-budgeted kind-scoping, which the explicit filter doesn't replace). Re-audit trigger: confirm at review whether automatic phase-budget scoping is still wanted now that explicit filtering exists.
2026-07-03 β retired the memory_recall MCP tool (R0.9 / agentmEngine#2). memory_recall delegated to harness_memory.phase_recall(), which has returned "" unconditionally for every call since the V5-3 vault-backend removal β dead in the sense that it always degraded, not in the sense that nothing called it. Verified no live crickets consumer actually called it (crickets' documenter sub-agent, the one consumer with a genuine live dependency on the recall machinery, calls harness_memory.py's own documenter-context CLI verb instead β a separate surface this retirement did not touch, since it still has a real caller). scripts/process_seam.py's recall_here (the seam-side wrapper around the same dead phase_recall call) was retired in the same commit for the same reason. The tool count in "How storage is served" above drops from four to three. Why not wire it to the live recall.py engine instead: that integration is R1.6's scope (the verify-mcp-surface suite, landing after the regression net); wiring a half-finished MCP tool now, before that net exists, risks a second silent-degrade bug shipping unnoticed. Re-audit trigger: if a future MCP consumer needs phase-scoped recall over the network transport, re-introduce memory_recall wired to the real engine as part of R1.6, not as a resurrection of the dead stub.
2026-07-03 β dropped scripts/harness_memory.py from governs: (R0.10 / agTrack#0 governs-overlap fix). This design and memory-storage-seam both stamped an exact-path governs: scripts/harness_memory.py, so governs_resolver.py resolved the file as {"governed": false, "reason": "overlap"} β silently treated as greenfield by grounding Hooks 1/2, the exact failure mode the resolver's fail-loud-on-tie behavior exists to surface, not paper over. memory-storage-seam is the more specific, storage-plane-scoped owner (it already lists harness_memory.py alongside backend_selection.py/storage_seam.py/etc. as one cohesive seam); this design keeps the broader harness/skills/memory/scripts/ directory pattern, which covers the engine-facing recall/save/reflect/vec_index surface harness_memory.py itself is not part of. Why not the reverse (keep it here, drop it from the seam design): the seam design's governs list is the storage-routing cohort β harness_memory.py is the seam's own resolve_project/vault_path entry point, so it belongs with its siblings, not orphaned into the broader memory-system pillar. Re-audit trigger: if harness_memory.py is ever split into a storage-facing half and an engine-facing half, re-evaluate whether the engine half should be re-added here.
2026-06-28 β lock-down sweep (operator review). Sized the three diagrams (width/height). Confirmed the aging / relevance-decay / re-weighting question is future work, not a gap β the lifecycle layer (confidence + retention decay, V6-1), consolidation tiers (V6-4), time-weighted retrieval (recency Γ relevance, V6-12), and dream-mode compaction (V7-2) are all queued; the designed-for framing here stands, and we hold modifying until V6/V7. Reordered this log newest-first (the 06-24 ADR-0007 fold had sat below the 06-21 authored entry). No content change to the pillar.
2026-06-28 β W4 as-built reconciliation (paired with the foundations gate-count clarification). Added half-sentence as-built caveats where live code differs from the designed target: the always-load gate is _always-load/ directory membership (authoritative; the always_load: field mirrors it); the embedder chain is local β stub (the API embedder mode was removed); per-prompt recall is a flat token-budgeted top-K today (the --kind/--phase scoping stays designed β see Risks). save.py's group is still an independent caller param (no kindβgroup derivation yet). The target shape is unchanged; these are honesty caveats per critique W4. Re-audit triggers: drop each caveat as recall.py / embed.py / save.py converge to the design.
2026-06-26 β codified the three ownership tiers + the vault folder layout, the per-tier archive, and the decay pipeline (operator design pass). Reworked Β§"By durability + ownership" into three explicit tiers by ascending autonomy β T1 personal (the operator's vault above Agent/, written only through a separate explicit seam call), T2 curated/collaborative (the Agent/ shared drive: designs Β· plans Β· roadmaps + the operator-directives; autonomous writes reported in the digest, revertable), T3 agentm-sole (Agent/Memory/, fully autonomous) β reconciling the prior "co-owned"/"user-owned"/"agentm-sole" band labels. Added the opinionated vault layout (the T2 projects/<project>/ shape + the prescribed Memory/ internals: <kind>/<slug>.md leaves, the reserved folders, a generated _index.md, the crystallized/procedural consolidation kinds), an _archive/ at every tier with recall skipping it by default (the --include-archive opt-in), and the archive/decay/prune lifecycle (heat curation + supersession ship; the fuller decay β consolidation β crystallization β dreaming-compaction arc is designed-for, gated on the revert-log + provenance edge + a staging inbox; prune resolves to archive, never hard delete). Why not approve-before for curated content: the operator co-owns it, so autonomous-write-with-revert + a digest report is the lighter, correct gate; a non-revertable curated change is the one case that would warrant asking, accepted as a known limitation until agent-memory is git-backed (backlogged). Grounded in the 2026-06 research (R02 lifecycle/consolidation Β· R06 token-efficiency Β· R09 content-shapes) + the live save.py/recall.py/evolve.py/heat_policy.py. Re-audit triggers: flip the decay/consolidation arms to as-built as V6/V7 land; run the operator-gated vault migration (relocate today's Agent/personal/* learned-memory β Memory/, directives β T2, updating the hardwired personal/_always-load constants); confirm the new reporting capability's digest is the T2-change report surface.
2026-06-24 β folded ADR 0017 (MCP server) into Β§βHow storage is servedβ (AG ADR-migration tail). Five DC calls, all surfaced into the body: singleton streamable-HTTP + broker property (DC-1); 4 snake_case tools memory_search Β· memory_recall Β· memory_append Β· memory_forget (DC-2, dot-names break OpenAI-family hosts); soft-delete status β deleted + deleted_at never unlinks (DC-3, not hard-delete: GDrive sync resurrects from propagation cache); loopback-first / deferred remote tier via OAuth 2.1 (DC-4); FastMCP >=3,<4 primary / official SDK named fallback (DC-5). Re-audit triggers: MCP spec deprecates streamable-HTTP; spec mandates dot-names; next FastMCP major; hard-delete obligation arises; homelab posture changes.
2026-06-24 β folded ADR 0007 into this design (AG Phase 4, move-and-retire).
0007 β Auto-context into harness phases (2026-05-22; amended 2026-05-27 / 2026-05-28 / 2026-05-31). harness_memory.py injects recalled memory into each phase at session start. Five design calls: (Q1) per-phase recall budgets to prevent context bloat; (Q2) three-tier slug detection for kind / status / tags; (Q3) graceful-skip if the vault is unavailable; (Q4) confidence-modulated ask before injecting low-confidence entries; (Q5) dual-trigger progress.md (phase-end + explicit save). Amended 2026-05-27: memory files moved to agentm/harness/skills/memory/. Amended 2026-05-28: documenter recall phase added. Amended 2026-05-31: SessionStart hook vault-path resolution order β $MEMORY_VAULT_PATH env var β .agentm-config.json::vault_path β no vault (graceful skip). Why not always-load all entries: per-phase budgets prevent context bloat; graceful skip ensures vault unavailability never breaks a phase. Why not single-trigger progress.md: dual-trigger ensures progress is written even if the phase crashes mid-run. Re-audit trigger: when V6 indexed-recall lands, re-examine whether per-phase budgets are still the right token-control knob; if dual-trigger causes duplicate entries, consolidate to a single write-and-flush.
2026-06-21 β authored, reviewed, and finalized.
Migrated from the agentm HLD, deepened against the live code, and conformed to the abbreviated-design template (Objective / Overview / Design / Dependencies / Risks) with all three diagrams. Documents the Memory pillar: a substrate where everything points inward (engine β resolution β seam), classified on three axes β kind (routes storage + scopes recall), durability, and ownership β with each memory an atomic frontmatter-keyed entry (kind / status / always_load / supersedes; engagement-not-encyclopedia; one-note-one-fragment). Capture is concurrency-safe and archive-then-replace (a supersession link-chain); recall is the kind-scoped, token-budgeted five-step loop over a device-local, never-synced index.
Operator-approved after restoring the content-first axis (the kind taxonomy + the entry contract) the storage-first reframe had dropped. Content-final; status: launched (lifted into tracked wiki/designs/ 2026-06-24, AG Phase 3). Designed-for, not built: the V6 indexed-recall work (hybrid RRF Β· knowledge-graph Β· consolidation tiers Β· chunking) under the DerivedMaintenance reservation, and V5-14 storage-convergence (memory entries + the MCP server still reach around the seam today). Re-audit triggers: flip the V5-14 as-built flags when convergence lands; migrate the seam prose into memory-storage-seam at the A2 fold; confirm recall.py enforces the kind/phase recall budget (vs flat top-K).
2026-06-21 β reopened (living-design amendment): backlinking made first-class + the interconnection/brain trajectory named (operator). The design implied the "grows into a knowledge base" direction but never stated it. Made Obsidian [[wikilinks]] first-class in the entry contract (link-integrity discovery, notes_link_discovery.py β built), added a How it grows design subsection tying the growth engine (forward learning + deep research β typed entries), the wikilink substrate (built), and the index-over-markdown (vector built β V6 knowledge-graph + metadata table + chunking + RRF designed-for) into one trajectory, and stated the trajectory in the Objective. Why not leave it implicit: the operator expects the store to grow into a true brain of knowledge, and the design should say so and name the interconnection path, not leave it to inference. The V6 graph/index stay framed designed-for (no source-attribution; markdown remains source of truth). Re-audit trigger: when V6-2 lands, flip the knowledge-graph from designed-for to as-built here.