agentm filing v2 - alexherrero/agentm GitHub Wiki
Note
LAUNCHED (2026-09-05; reconciled through 2026-09-06). This is the record of the build β six parts, v9.9.0 to v9.20.0. From 2026-09-06 the vault's shape is designed onward in AgentM Vault, the successor authored by the vault-perfection series; it inherits every decision here unchanged unless it says otherwise, and new rulings land there, not in this log.
The vault hybridizes several generations of filing systems, and the six-class memory layout that shipped in August was never populated β the corpus sits in a staging inbox whose routing step was never built. This design converges the whole vault on the layout the operator's work deployment has proven, defines who may write where, and adds the two subsystems neither deployment has solved: a memory lifecycle and a consolidation pass with authority to correct filing. It is the blueprint for AgentM as a machine-level harness rather than a repo harness.
The predecessor design, [AgentM Rescope β The Filing Contract]
(agentm-rescope-filing.md), went final on 2026-08-18 and shipped its first
two parts: standards/storage-rules.md as the runtime-read source of truth,
the six-value type taxonomy with its warrant gate, and the type-XOR-kind
contract. A census taken 2026-08-31 (INVENTORY-home-vault.md in the
research bundle) found what happened next: the six class directories each
contain only their own index file, while the real corpus β 2,652 notes β
accumulated in memory/_inbox/, 61% of it expired output from the #514
auto-mining defect. The capture side assigns types correctly; nothing ever
routed a note onward, because reflect.py's only destination is the inbox.
Around that stalled core sit the older strata: legacy kind-named directories,
a date-bucketed directory, and roughly ten underscore machinery directories.
Two things changed the picture in late August. The operator's work
deployment β AgentKV, a sibling of the same lineage, deployed fresh in
mid-August β demonstrated the target shape working without any of the
accumulated machinery: an agent/ folder with exactly three children,
first-class diagnostics with daily scorecards, an agent-maintained faceted
calendar, and no staging directories at all, their jobs carried by a
dedicated dreaming binary running beside the MCP daemon. And a six-lane
evidence review (R1βR6, desk/projects/agentm/_harness/research-filing-v2/)
plus the census established with unusual consistency what to keep, what to
change, and where the real risks are. The operator then ruled on every open
question in conversation on 2026-08-31 and 2026-09-01; those rulings are
recorded in the research bundle's synthesis and are treated as decided
throughout this design.
The environment constrains the shape. One git repository at ~/Vault is
browsed by the operator in Obsidian, synced through
Google Drive, and served to the agent by a Go daemon (agentmd) doing
hybrid lexical+dense retrieval, with hooks providing recall and reflection
at session boundaries. The live memory corpus is on the order of 1,000β1,500
notes once scratch, expired mining output, and the operator's personal
content are subtracted β so most of v2 is edits to a rules file that already
runs, a scripted one-time migration, and population of an existing
structure, not a rebuild.
The vault keeps its five top-level spaces β Agent/, Calendar/,
Projects/, Personal/, standards/ β and each space gets an explicit
write-authority level, replacing the old promotion-door doctrine. The
agent's half trims to exactly three children: desk/ (one directory per
in-flight task), diagnostics/ (per-system health with daily scorecards and
dated audits), and memory/ (the six classes, and nothing else). Memory
type stays a frontmatter field; the class directory stays the only path
axis; records keep kind: and live in the area they serve.
Three subsystems are new. Filing moves to write time: a hook classifies each memory to its final destination and decides how it relates to what already exists β add, update, supersede, or nothing β with no staging inbox; the one useful property of the old queue survives as a confidence flag in frontmatter. A graduated lifecycle status replaces archive-by-moving: demotion is a frontmatter edit read by the retrieval daemon as a ranking curve, and nothing in the memory classes ever relocates. And the calendar becomes an agent-maintained daily register with four facets and a generated day index, append-only, with corrections as linked supersessions.
Underneath, the machinery consolidates: the Python dreaming layer ports to a
second Go binary with real mutation authority over prior filings, all engine
state (caches, cursors, journals) leaves the vault for
~/.local/state/agentm/, and the vault becomes purely the knowledge surface
both audiences read.
Everything runs on the operator's Mac under launchd; no servers, no cloud components.
| component | role | status in v2 |
|---|---|---|
agentmd (Go daemon) |
index, hybrid retrieval, MCP surface, watch, rules holder | extended: lifecycle-status ranking, write-authority validation, filing endpoint |
| dreaming binary (Go) | nightly consolidation with mutation authority; rollups; audits | new β ports dream.py / crystallize.py / reflect.py and kin |
llama-server |
dense embeddings (embeddinggemma-300M) | unchanged |
| session hooks | recall (SessionStart, UserPromptSubmit), capture/reflection (Stop) | capture hook rewired from inbox-writes to the filing decision |
gate battery (check-all.sh) |
deterministic verification | extended: vocabulary, provenance, lifecycle gates |
| trigger | what runs |
|---|---|
| session start / prompt submit | recall injection from the daemon |
| session stop | capture β write-time filing decision |
| nightly (gated on elapsed time and activity, lock-coordinated, exits when done) | the dreaming binary: mutation pass, promotion, MOC regeneration, rollups, audits |
| daily | per-system scorecards into Agent/diagnostics/
|
| every push / CI | the gate battery |
Guarantees carried over and added: a storage-rules block that fails to parse
halts filing loudly (fail-closed); authoritative state lives on disk always β
the dreaming binary checkpoints its in-flight work as journaled, lock-guarded
files under ~/.local/state/agentm/, so a crash mid-consolidation is
recoverable rather than silently lossy; and no same-turn read-after-write
expectation is placed on the index.
standards/storage-rules.md remains the single runtime-read source of
truth, and most of this part is an edit to it. The block gains: a
lifecycle enum (pinned Β· active Β· dormant Β· archived Β· superseded,
retiring expired); required provenance fields (source: with a closed
transport vocabulary β operator-direct, conversation, external-fetch, email;
external sources carry lower default trust); filing_confidence on
auto-filed notes; the calendar facet registry (meetings Β· correspondence Β· docs Β· diary); and a routing change β idea routes to memory/semantic
instead of desk, with the idea-incubator's read path repointed.
The vocabularies get the gate the census showed missing: enum membership is
enforced for both type: and kind: (the XOR gate held at zero violations
while kind: drifted to 38 values against a 32 register), and a collision
check refuses any word legal in both vocabularies, renaming the record kinds
that currently shadow memory types. The warrant rule stays, paired with a
scheduled vocabulary audit in the dreaming cadence β ungoverned metadata
axes decay by default, and the 55-value history is this vault's own proof.
Write authority replaces the promotion door as one table the validator (the
daemon's door package, generalized) enforces:
| space | authority |
|---|---|
Agent/ |
agent-authored freely |
Calendar/ |
agent-maintained; operator co-writes |
Projects/ |
agent-managed under a session grant β "open the files for project <slug>" grants that project, that session; otherwise read-only |
Personal/ |
explicit per-task instruction only; never standing management |
standards/, root files |
operator-owned; agent proposes, applies only on instruction |
Validators are scoped so Personal/ is exempt from the memory contract
entirely β its notes carry no required frontmatter. Filing.md and
index.md are rewritten to say exactly this table.
Agent/ trims to desk/, diagnostics/, memory/ (plus Home.md).
Diagnostics is promoted to first-class: per-family directories (health/,
dreaming/, digests/ β as built in 2a; the drafted per-system trio
regrouped to match how the writers actually emit) holding dated
scorecard-YYYY-MM-DD.md files with a latest-scorecard.md pointer, dated
one-off audits beside them, and a generated moc-diagnostics.md. It absorbs
desk/diagnostics, desk/briefs (a daily digest is a scorecard), and the
human-facing residue of _meta/health. Desk becomes work-style: one
directory per in-flight non-project task under desk/tasks/, plus a
generated moc-tasks.md; desk/scratch retires from the vault (ruled) β
existing files sweep to a local non-synced folder, future scratch lives in
session scratchpads and task directories.
This part split at build time (ruled 2026-09-02): everything self-contained
in this repo shipped as 2a; the Projects merge became the queued 2b
(projects-merge), because moving desk/projects/ requires the crickets
development-lifecycle plugin's own repoint (resolve_plan, queue-status, the
worker-spawn flow all navigate that path) in a coordinated paired release
this repo cannot ship alone β a cross-repo dependency the sequencing never
named, logged as a plan gap. 2b shipped as v9.11.0, paired with crickets
development-lifecycle v3.37.0 (which shipped first, as the locked order
required): Projects/ at the vault root is the only projects folder β the
working trees under desk/projects/<slug>/ moved there wholesale as
git mv, basenames preserved and _harness/ state intact, desk/labelling
folded into Projects/agentm/labelling, and every path that resolves
harness state was inventoried and repointed. The root space sits above the
memory root, so the storage seam reaches it through a second backend
instance rooted one level up rather than a .. Locator β offered only under
the Obsidian witness (.obsidian/ at the parent, none at the memory root),
never on a bare parent directory (the storage-seam design carries that
decision); readers keep desk/projects/ as a documented
older probe rung β "discovered, never conjured": nothing creates the root
space, and create-when-absent defaults stay inside the memory root until
the space exists. Machine state leaves
the vault for ~/.local/state/agentm/ (ruled): corpus snapshots, learning
caches, skill-discovery cache, storage-rules-state.json, and the dreaming
binary's journals. The engine directory is itself a git repository β the
migration initializes it and the runner commits on cadence β so the
history-durability those files had from the vault's repo survives the exit
rather than being traded away for sync hygiene. _dream/
and _crystallize-staging/ retire β dreaming enriches in place, its
in-flight state journaled in the engine directory. memory/_always-load
folds into standards/ (the always-load surface is standards, as at work),
and the always-load set is size-policed with a hard budget so it cannot
become a per-session context tax.
Every area carries its own generated moc-<area>.md as an ordinary file;
memory/mocs/ holds corpus-level maps only, moc-root.md on top.
One scripted, phased migration, with three invariants: basenames are preserved so name-resolved wikilinks survive every move; the link-check gate is green before and after each phase; and every phase is a dry-run first. The map, with the operator's rulings applied:
| population | destination |
|---|---|
| inbox expired cohort (1,630) | purged, with a manifest of titles + hashes retained (ruled β the one purge in the plan) |
| inbox live (~1,015) | routed to classes via the typeβclass map, deduped, low-confidence flagged |
_archive (264) |
into their classes with lifecycle: archived
|
_opinions (470) |
into crystallized/, keeping kind: opinion-supplement (ruled); dream-writer and daemon-reader paths repointed |
| legacy kind-dirs (~560: preferences, preference, 2026, fix, idea, insight, workflow, workflow-pattern, feedback) | deprecations-map mechanical migration into classes |
external/primos (38) |
classes by nature, provenance-tagged |
desk/projects (~950) |
Projects/<slug>/ wholesale |
Shipped as part 3 (v9.12.0, 2026-09-03), with the map reconciled to the
census the migration itself took: the expired cohort was not the inbox's
1,630 but 2,638 β the same auto-miner retirements sat in preferences/,
preference/, fix/ and _archive (whose 264 turned out to be all expired,
so "archive into classes as archived" applied to nothing), and 444 of the
470 opinion supplements carried the status too; the operator ruled the purge
all-expired, and the engine offers all three scopes so the manifest carries
every figure. Opinion supplements moved into crystallized/<opinion>/ as
lanes β the accumulate loop's shape, kind kept, served file beside the
crystallized memories β not flattened. external/primos held 38 records and
no memory: a project tree, moved whole to Projects/primos/ rather than
routed. Four record kinds the corpus already carried (report, standard,
analysis, progress-log) were registered on the operator's ruling so the
membership gate could flip to strict. The "all six classes populated"
criterion does not hold for episodic, entities and mocs β derived
classes nothing routes to β and was read as "every class that has a
population". The Python capture path still writes the pre-v2 vocabulary
into _inbox/; a route pass re-files it, and part 4 retires the writer.
| desk/scratch (2,324) | out of the vault to a local non-synced folder |
| empty dirs (_watchlist, _skill-watchlist, domains, _always-load) | deleted after any contents fold into standards |
Post-migration, memory/ contains exactly six populated class directories,
episodic gains the session traces and ingested threads it was designed for,
and the class-population counts land on the daily scorecard so an
empty-shell recurrence would be visible within a day.
Capture keeps its current trigger points (the Stop-hook reflection pass and
explicit memory_capture), but every candidate now receives a complete
filing decision at write time instead of an inbox write: type, class,
destination path, and the update relationship to the existing corpus β
add (novel), update (enriches an existing note), supersede
(contradicts one; the old note is never deleted, it gains superseded_by
and flips lifecycle), or noop. Deterministic signals lead: where
extraction can produce a (subject, attribute) key, an object mismatch is an
automatic supersession and a match is a duplicate β embedding similarity is
a secondary signal only, because similarity cannot distinguish a duplicate
from a contradiction (measured AUROC 0.59; a value-flip is a smaller edit
than a paraphrase). Merge decisions bias conservative: file a probable
duplicate flagged rather than auto-merge, since a bad merge destroys
information silently and a duplicate is recoverable clutter.
Every auto-filed note carries filing_confidence and source:. Low
confidence is the soft inbox: the note is filed at its real destination
immediately and surfaces in a generated needs-review MOC for the nightly
pass and the operator. Capture volume is gated and reported independently
of filing β the predecessor inbox failed by over-capture outrunning
judgment, and removing the pile removes the old visibility, so a writes-per-
day line on the scorecard replaces it as the alarm. External-source content
is trust-tiered at write time and never treated as instructions; write-time
screening is explicitly not relied on to catch fabricated facts, because
the measured state of the art cannot.
Shipped (v9.13.0, 2026-09-04; PR #540). The engine, the class filing on
every writer, the needs-review page, the volume gate and the trust tier
landed as designed, with four calls made in the building. The soft inbox is
a status read, not a second exclusion: recall keeps serving unfiled
notes β the daemon's own captures are that population, 650-odd of them, and
it rank-penalises them rather than hiding them β and excludes only the
staging states the ingest sweep leaves in place; an early draft excluded
unfiled and the retrieval gate's canary fired (why not hide them: hiding
the queue would silently shrink the searched corpus against the indexed
one). The cap is grounded, not guessed: 200 sits above the busiest day on
record (110) and well below the last flood; the contract carries it and
zero disables it. The Go capture's source: held a provenance
reference while the contract's source: was the transport vocabulary β two
meanings of one field, left open by this part and closed in v9.19.0 (PRs
#570, #571 and #572): source: names the transport alone, a fetched page names its
address in source_url: and a mined unit its registry identity in
source_id:, readers resolve those first and still accept the legacy shape
for what nobody migrated, and the frontmatter gate refuses a value outside
the vocabulary. The corpus carried 578 transports, 124 bare URLs and 14
references; the migration moved the reference out and gave the fetched pages
the untrusted tier the contract had always specified and no reader had ever
been able to apply. And the labeled sample moved the next work upstream:
the engine's operations were judged right wherever the input was sane; the
miner had admitted a pasted handoff as the operator's speech, cut fix
candidates into report fragments, and filed tool-invocation counts as
procedures β four rulings taken (a marker the miner skips, stop the stubs
and purge the 824, a cause-and-remedy requirement, a durability cue for
HIGH), the next plan β shipped as v9.14.0 the same day (PR #546; the 830 stubs purged on the operator's confirmed count; re-mined, the sample yields 9 candidates where it yielded 20).
Calendar/YYYY/ holds per-day, per-facet notes named
YYYY-MM-DD-<facet>.md, created only when the day has content for that
facet, with four facets (ruled): meetings, correspondence, docs,
diary. Facet membership is the selection rule β a meeting happened,
correspondence produced a reply, decision, or deadline, substantive work
landed in an external artifact β with no per-item importance scoring on
top. diary is the zero-bar catch-all: operator quick capture lands there,
the agent records what it observed that mattered, and a pattern recurring
three or more times in diary entries is the promotion trigger for a new
standing facet, the same recurrence gate consolidation already uses.
The bare-date file YYYY-MM-DD.md is the generated day index: it lists
whichever facet files exist, links the day's episodic session traces, and
carries the system digest β so opening one note shows the whole day without
forcing storage back into one dilution-prone file. Facet files are
append-only while the day is open (new content as new paragraphs, cheap for
the paragraph-aware chunker to re-index); once a day has closed, a
correction is a new dated entry linked with supersedes, never a rewrite β
the register's value is the trajectory of what was believed at the time.
Weekly and monthly rollups (YYYY-Www-review.md, YYYY-MM-review.md) are
generated unconditionally by the dreaming binary, moving the periodic-review
layer from human discipline, which the practitioner record shows failing,
to schedule. The calendar and episodic/ remain distinct, cross-linked
surfaces: the calendar is the operator's agent-assisted memory of the day;
episodic is the agent's memory of its own sessions.
Shipped (v9.15.0, 2026-09-04; PR #548). The four ruled facets write
under the vault-root Calendar/YYYY/ through the same Obsidian witness the
Projects/ space uses β the vault root when the memory root is nested,
never conjured (why not create it: the vault root is the operator's, and a
conjured folder on a flat vault would land in the home directory; re-audit
if a flat vault ever needs a register β the writer then needs the operator's
instruction, not a mkdir). Facet notes are append-only while the day is
open; a closed day refuses the append and names the correction, which is a
new dated note carrying supersedes: and corrects: back to an original
that stays byte-identical, listed by both days' indexes. The day index is
generated from what exists and rewritten byte-stably after every append; it
already links the day's episodic traces, none of which exist until part 6
writes them. The rollups ride the Python dream cycle's weekly cadence for
now β every closed week in the last eight, the running month and the one
before, sparse weeks reading sparse and empty weeks saying so once (why not
only the weeks with entries: the review moved from discipline to schedule,
and an honest empty line is the record; re-audit if empty reviews pile up as
noise for a register nobody writes to β then gate on any entry in the
window); the binary takes them over in part 6. The promotion trigger is a
diary label on three or more distinct days in thirty, and its output is a
confirm-gated proposal β the contract file with one line added under
facets: β never applied by the agent (why not auto-apply at the recurrence
gate consolidation uses: the registry is the operator's, per part 1;
re-audit only if every proposal is confirmed for a quarter, and then toward
a lighter confirm surface, never toward auto-apply). Two residuals:
Calendar/ sits outside the daemon's index scope, so recall does not search
the register yet; and the operator's own flat daily notes already live in
Calendar/, untouched and unread by the register β whether Obsidian's
daily-notes template and the register become one thing is a call for part 6
or later.
One frontmatter axis, lifecycle, five values. pinned never decays.
active is the default. dormant is rank-demoted by the daemon's existing
penalty machinery (the tiers package reads the field as a continuous
demotion curve). archived is hidden from default retrieval β strongly
penalized or excluded from the recall walk, cold-indexed, on disk, in
place. superseded points at its successor. Files in the memory classes
never move for lifecycle reasons; directory-move archiving survives only
for low-link-density operational artifacts (plan close-outs keep their
archive/ convention), and the condition that makes that safe β nothing
links into them β is stated so the boundary stays principled. Who decides
is tiered by reversibility: activeβdormant runs automatic by policy, every
transition logged and summarized in a weekly what-quietly-sank digest;
entering archived is operator-confirmed or at minimum conspicuously
reviewable; purge β actual deletion β is operator-initiated only, always,
with a manifest. The demotion-curve parameters are explicitly unmeasured in
the literature for our regime and get tuned by measurement on our own
corpus.
The dreaming binary is the second Go binary beside agentmd, porting the
Python layer, and its defining property is mutation authority: it may
supersede, merge, re-flag, and re-file what write time got wrong β the
evidence is direct that write-time-only judgment has a structural ceiling a
mutation-time pass recovers. Its job list: the mutation pass over recent
filings and flagged notes; recurrence-gated promotion into crystallized/
(provenance links required β a checked invariant, since distillation that
discards sources is measurably worse); entity-file maintenance; MOC
regeneration with a context phrase per link, created at five members, split
past forty, staleness-flagged past ninety days; calendar rollups;
relative-to-absolute date conversion in aging notes; the scheduled
vocabulary audit; class-distribution and volume trend checks; and a sampled
re-classification diff whenever the filing model version changes. It runs
triggered-and-exits β never a resident daemon β dual-gated on elapsed time
and activity, lock-coordinated, journaling to the engine state directory.
Until the port reaches parity (verified by fixtures against recorded
Python-pass outputs, not mirror tests), the Python layer keeps running and
the binary lands job-by-job.
Shipped (v9.16.0, 2026-09-05; PRs #550β#555, #556). The axis ranks as
classes at the standard demotion (the daemon's own sweep found every weight
past the standard one a regression, so the "demotion-curve parameters" are
not a knob), pinned lands in durable, and archived sits behind a wall
only an explicit query lifts β recall mirrors it. The governance lanes
journal every move; archived only through the confirm surface; purge
operator-only with a manifest. The binary carries the whole job list except
entity-file maintenance, which the write path's entity rollups already own;
relative dates are glossed additively (last week (the week of 2026-08-24)),
never rewritten, since the corpus is the operator's words. Parity is a
recording of the Python layer's decisions with the clock pinned, reproduced
by the Go planners; the layers ran side by side through an overlap window
with a daily divergence review, the one review agreed on every surface, and
the operator flipped the takeover the same day β -apply on the binary, the
three Python lanes removed, the recording kept as the contract. The overlap
window's length was a gap this design left open; it closed by operator
ruling.
Amended (v9.18.0, 2026-09-05; PRs #566, #567). One vocabulary for the
superseded relation. The dream layer's stages β the copies job, the dedup
merge, the inbox collapse, the correction flow β had marked the loser with
status: superseded + supersedes: <winner>, which inverted the direction
every other writer and reader used (supersedes: on the successor; the
compactor read it as winnerβloser, so a dedup loser looked like a chain head)
and differed in effect from the write path's lifecycle: superseded +
superseded_by: (hidden from the Python walk; only demoted by the daemon).
Ruled: a superseded memory carries lifecycle: superseded + superseded_by:
naming its successor; supersedes: is only ever the successor's back-link;
both recall paths keep a superseded memory out of everyday search and answer
the explicit query with it demoted, exactly like archived; status: no
longer carries supersession for memories (readers still accept
status: superseded on a note nobody migrated); the migration walks the
whole vault, resolves the successor inside it, and revives a note whose
successor is gone rather than wall it behind nothing; the frontmatter gate
refuses a superseded memory without a successor or with a loser-side
supersedes:; the parity recording was edited on purpose. Why not keep two vocabularies: one relation with two directions
and two effects is two bugs waiting, and the inverted pointer had already
made the compactor misread lineage. Re-audit trigger: a writer that needs
the successor to be something other than a note (the sources layer's
<source id> at <version> is the one exception, kept).
Amend agentm-rescope-filing instead of a new design. Rejected (ruled):
the scope β whole vault, write authority, calendar, lifecycle, a new binary
β exceeds that design's filing-contract charter; it gets a Document History
cross-reference here and its surviving decisions are inherited unchanged.
Keep the staging inbox, fix the routing. Rejected: the inbox failed on its own terms in this exact vault (2,652 notes, 61% expired, routing never built), no surveyed production system stages, and the queue's one useful property β a place low-confidence decisions can be found β survives as frontmatter. The bet's known ceiling is priced in via the mutation pass.
Directory-move archiving. Rejected for memory classes: it fights the ID-stability invariant, Obsidian's link rewriting does not cover moves made outside the app while this vault syncs through Google Drive, and a moved note silently detaches from every MOC naming it. Retained for plan close-outs, where link density is near zero.
Entity subdirectories (the work pattern). Rejected for home: the two
most authoritative typed-entity systems (Neo4j multi-label nodes, Wikidata
multi-valued instance-of) both reject exclusive placement because real
entities carry two types; home gets the same browsing surface from
generated per-subtype MOCs over a flat entities/.
Replace the cognitive classes with a flat fact store or temporal graph (Mem0/Zep shape). Rejected: the classes' measured value is operator legibility β one of this vault's two audiences β the derived classes carry the actual retrieval evidence, and a markdown vault is a hard requirement, not an implementation detail.
Snake_case with subtype prefixes (the R6 recommendation, the work habit). Rejected on R6's own decision rule applied to this corpus: separators are retrieval-neutral, home's machine-written files are overwhelmingly kebab already, and renames are evidence-backed non-free. The retrieval-relevant half β a meaningful subtype word in the filename β is kept.
Keep engine state in the vault (_meta). Rejected (ruled): opaque
caches serve neither audience in Obsidian, cost Drive sync, and the field's
disk-not-vault distinction requires only durable disk.
The storage-rules runtime (daemon rules holder plus storage_rules.py);
the daemon's door, tiers, index, capture, enrich, and sources
packages as the seams v2 extends; the session hooks in ~/.claude/hooks/
and both launchd jobs; the crickets development-lifecycle plugin for the
part plans this design sequences into; Obsidian's name-based wikilink
resolution (load-bearing for the migration invariant) and the Google Drive
sync path; and the research bundle at
Projects/agentm/_harness/research-filing-v2/ (moved with the tree in 2b) as the evidence record β
including its evidence-versus-judgment ledger, which this design treats as
binding on how confidently each choice is stated.
The corpus migration is Detailed Design's third part; what belongs here is
the deployer's view. Order: rules and gates first (so the target state is
checkable), structural moves second, corpus third, each phase dry-run then
applied, link-check green before and after. The purge of the expired cohort
executes only after the manifest is written and the operator has seen the
count one final time. Path repoints (hooks, daemon config, scripts
referencing _meta, _inbox, desk/projects, _opinions) ship in the
same change as the move they track, inventoried by grep before any file
moves. Migration runs execute under a quiesced daemon and runner, and that
precondition sets the collision doctrine (learned in the 2a apply): when a
move finds its destination already occupied, the vault copy wins β it is by
definition the last production write, whatever sits at the destination
predates the migration, and the engine repository's history preserves what
gets replaced. A bulk move also re-keys every moved note for the dense arm,
and the daemon re-walks the tree without backfilling those vectors, so a
move's run order ends with agentmd embed (learned in the 2b apply: the
retrieval gate read 0.734 β 0.438 until the backfill restored it exactly).
Two more from the part-3 apply: a later pass over a corpus the daemon kept
writing into settles a routed note against a file already home (identical
body β filed superseded as ~dup, different body β a basename clash) rather
than refusing, so the migration stays resumable; and an instrument that pins
paths β the retrieval gate's gold set β compares them in a canonical
basename form once the corpus has moved, or its control fires on the move
itself. The purge is operator-confirmed by exact count (--confirm-count),
never defaulted.
Rollback for every phase except the purge is git revert in the
vault repository plus replaying the path repoints; the purge is
non-rollbackable by design and bounded by its manifest. The Python dreaming
layer is not retired until the Go binary's parity fixtures pass; the two
run side-by-side with the binary in report-only mode during the overlap.
- Over-capture can recur invisibly. The inbox pile was the old alarm; dispersal into six directories hides it. Mitigation: the volume gate and the writes-per-day scorecard line. Re-audit trigger: writes-per-day doubling week-over-week, or class counts growing faster than sessions.
- Write-time filing has a structural error ceiling. Priced in via the mutation pass, confidence flags, and conservative merges β but the mutation pass is load-bearing: shipping the hook without it is the one configuration the evidence argues against. Re-audit trigger: needs-review MOC exceeding ~5% of weekly filings.
- Demotion-curve parameters are unmeasured. The graduated-status shape is convergent practice, not a measured result (one citation offered as direct evidence was found on verification to address a different problem). Tune on our own corpus; re-audit trigger: stale content surfacing above better answers in recall audits.
- Capture-time poisoning is unsolved industry-wide. Fact-shaped false content passes write-time screening at measured rates approaching total. Mitigation is exposure reduction (provenance trust tiers), not detection. Re-audit trigger: email ingest landing (it widens the untrusted surface).
- The Go port can drift from the Python behavior. Parity fixtures must assert recorded outputs, not recompute expectations with the new code (the mirror-test trap). Re-audit trigger: any parity fixture rewritten during the port.
-
Session-grant enforcement is new surface. A validator gap would let
writes into
Projects/without a grant. Mitigation: deny-by-default in the authority table; grant state carried in session context, not config. - The empty-shell failure can repeat. Rescope-filing shipped structure and the corpus never followed. v2 ships structure and population in the same arc, and class-population counts sit on the daily scorecard. Re-audit trigger: any class at zero thirty days after part 3 completes.
The write-authority table is the boundary model: deny-by-default for the
operator's spaces, session-scoped grants for Projects/, purge as an
operator-only act. Capture-time prompt injection and memory poisoning are
handled by provenance trust tiers and by never treating memorized content
as instructions β explicitly not by trusting the filing model to detect
fabricated facts, which the measured evidence says it cannot. Engine state
moving to ~/.local/state/agentm/ takes cursors and caches out of the
synced, shared surface.
The rules block stays fail-closed: unparseable rules halt filing loudly. The dreaming binary is triggered-and-exits with journaled, lock-guarded state in the engine directory, so a crash mid-pass resumes instead of losing work β the named failure mode of in-process background queues. The Python layer retires only after parity, with a report-only overlap.
ID-stability is preserved by construction: lifecycle is frontmatter, memory files never move after migration, and the migration itself preserves basenames so name-resolved wikilinks survive. Nothing is deleted outside the operator-gated purge, which is manifested. Supersession retains the old note with a forward link. Crystallized provenance is a checked gate. The link-check gate brackets every migration phase.
Personal/ is exempt from the memory contract and off-limits without
per-task instruction. The calendar respects the existing no-pii gate, and
sensitive material routes to memory rather than the browsable daily
register. The purge path plus lifecycle give true deletion a defined,
operator-only lane. Provenance fields record transport, not content, and
stay inside the vault.
Filing adds LLM judgment at capture points (session stop, explicit capture), not per turn; recall paths are untouched. Append-only calendar files keep incremental re-indexing cheap under the paragraph-aware chunker. Retiring scratch and the expired cohort shrinks the dense index by roughly half its note count, which can only help the retrieval budget.
The vault sits at ~8,000 notes today β inside the range where Obsidian's graph is decorative and unscoped queries slow, but below file-explorer failure. v2 reduces indexed volume (scratch out, purge, engine state out), keeps hierarchies shallow, scopes any query surface it adds, and treats the graph as a diagnostic, not navigation. Facet-per-file keeps embedded units topically homogeneous as the calendar grows.
Every new rule lands as a deterministic gate (vocabulary membership and
collisions, provenance presence, lifecycle enum, link check, authority
validator), extending check-all.sh. The port is verified by recorded-
output parity fixtures. Quantitative claims β demotion thresholds, filing
accuracy, facet retrieval value β are measured on this corpus with local
judges; published benchmark numbers were found unusable for design choices
and are not cited as verification anywhere in this design.
| part | scope | est. |
|---|---|---|
| rules, vocabulary, and write authority | storage-rules v2 edit, validator, gates | M |
| structural moves | directory moves, engine-state exit, standards absorption | S |
| corpus migration | routing engine, purge, legacy dissolution | M |
| the write path | filing decision, update ops, confidence, volume gate | M |
| the calendar | facets, day index, rollups | M |
| lifecycle + dreaming binary | status curve, mutation pass, Go port | L |
Vault-side: Filing.md and index.md rewritten; standards/storage-rules.md
prose sections updated alongside the block. Repo wiki: this design;
reference/CI-Gates.md (new gates); reference/Memory-Daemon.md (lifecycle
ranking, filing endpoint, the second binary); a how-to for the vault layout
and write-authority model; Designs.md / _Sidebar.md entries. The
research bundle stays where it is as the evidence record.
Phased by part order; parts 1β3 land before 4β6 begin (the vault must be
true to the rules before the rules go live at write time). No dates set;
each part ships through the normal /work β /review β /release cycle
with wake-on-CI close-out.
Daily scorecards in Agent/diagnostics/ carry the new lines: writes per
day, class populations, needs-review count, lifecycle transition counts.
The weekly what-quietly-sank digest surfaces automatic demotions for
review. The health-score battery gains the new gates. The operator's alarm
surface is the scorecard plus the digest β no push alerting is added.
The dreaming binary journals every mutation (what changed, why, which rule) append-only in the engine state directory; lifecycle transitions log with their trigger; the purge writes its manifest before deleting. Journals are plain files, greppable, retained indefinitely (they are small).
Rules edits revert via git (the vault is a repository; the rules hash makes staleness visible). Structural and corpus moves revert via git plus replaying path repoints β preserved basenames make reverts link-safe. The purge alone is irreversible, which is why it is manifested and operator-gated. The dreaming binary rolls back by re-enabling the Python layer, which is not retired until parity fixtures pass.
| Date | Change | Status |
|---|---|---|
| 2026-09-01 | Initial draft created via /design author, drafted from the decided filing-v2 synthesis (six research lanes, vault census, and operator rulings β see desk/projects/agentm/_harness/research-filing-v2/); review pass ran the same session β all sections and quality attributes approved unrevised β and the operator approved as final. Translated to 6 parts via /design translate (operator-approved split): rules-and-authority, structural-moves, corpus-migration, write-path, calendar, lifecycle-dreaming β part files at the vault's _harness/designs/agentm-filing-v2/parts/. Sequenced into 6 draft plans via /design sequence: PLAN-rules-and-authority.md active (named-plan mode, beside the unrelated online-recall plan), five queued at _harness/designs/agentm-filing-v2/queued-plans/. |
final |
| 2026-09-02 | Part 2 reconciled to what shipped (v9.10.0, PRs #521/#522). Three amendments: (1) structural-moves split 2a/2b β the Projects merge needs the crickets development-lifecycle repoint in a paired release this repo cannot ship alone (why not ship it here: a one-repo move would strand every plan-resolution path; re-audit when 2b lands β fold this paragraph back to one part). (2) The engine state dir is a git repository (migration inits, runner commits on cadence) β why not plain files: the exit from the vault would silently trade away the history-durability those files had from the vault's repo; re-audit if the runner's commit cadence ever stalls (a dirty engine repo older than a week is the tell). (3) Diagnostics directories are per-family (health/dreaming/digests), not the drafted per-system trio β why: the writers emit by family, and the drafted names would have forced a router nothing needed; plus the Migrations section gains the vault-wins collision doctrine learned in the live apply (quiesce makes the vault copy the last production write; skip-on-collision stranded real state behind scratch leakage). |
final |
| 2026-09-03 | Part 2b (projects-merge) reconciled to what shipped (v9.11.0, PRs #528/#529; crickets development-lifecycle v3.37.0 first, the paired order as locked). Body: the structural-moves paragraph folds 2a/2b back to one shipped part. Two decisions recorded: (1) the vault-root Projects/ space is reached through a second seam backend instance rooted at the vault root β why not a .. Locator: Locators are root-confined by construction, and a relative escape would silently mis-root every later join onto the memory root (the storage-seam design carries the mechanism and its re-audit triggers) β the pre-tag review added the Obsidian witness: the sibling counts only when the memory root is nested inside a vault (.obsidian/ at the parent, none at the memory root), because a flat vault's parent is the operator's home, where a Projects/ is common, and the flat generation <memory-root>/Projects now reaches every walker and writer; (2) readers keep desk/projects/ as a documented older probe rung β why not drop it: "discovered, never conjured" needs the rung for a vault that has not moved, and it costs one is_dir; re-audit when part 3 retires the last desk-era path, then drop the rung. Migrations gains the embed backfill invariant (a move re-keys the dense arm; agentmd embed closes the run) β learned in the 2b apply. Dependencies now names the moved research bundle. |
final |
| 2026-09-03 | Part 3 (corpus-migration) reconciled to what shipped (v9.12.0, PRs #533β#537; applied live the same day). Body: DDΒ§3's map gains the shipped paragraph β the expired cohort was 2,638 across populations, not the inbox's 1,630 (why not purge the inbox alone: the same auto-miner retirements sat in the legacy dirs, _archive and 444 supplements, and leaving them as lifecycle: archived would have kept a rank-penalized junk corpus on disk against the design's own "maps it away"; re-audit if a future miner produces an expired cohort worth keeping); supplements kept their lane shape under crystallized/<opinion>/ (why not flatten: the accumulate loop's nine consumers walk lanes, and one resolver moved them all); external/primos moved to Projects/primos/ as a project tree (why not route: it held records, no memory); four record kinds registered (why not re-kind: 45 live records already meant those shapes). Migrations gains the resumable-pass and canonical-instrument invariants and the confirmed-count purge. Residual named for part 4: the Python capture path's pre-v2 vocabulary. |
final |
| 2026-09-04 | Part 4 (write-path) reconciled to what shipped (v9.13.0, PR #540; deployed live the same day). Body: DDΒ§4 gains the shipped paragraph β the soft inbox is a status read that keeps unfiled served (why not exclude it: the daemon's own captures are that population and the canary fired when a draft did; re-audit if the enrichment queue ever drains to zero and unfiled stops meaning "the daemon's backlog"); the cap grounded in the corpus (200 over a busiest day of 110); the two meanings of source: recorded as an open question (why not rename in this part: no writer-side plan carried it; re-audit when the trust tier is read at retrieval time); and the labeled sample's finding that the next work is the miner's, with the operator's four rulings. |
final |
| 2026-09-04 | The write path's follow-up (miner-provenance, v9.14.0, PR #546) reconciled: DDΒ§4's shipped paragraph records the four rulings as shipped. No locked call changed; the miner is upstream of this design's write path, and its rules live in reflect.py's pattern tables (why not fold the miner into the design: it predates filing v2 and serves every part; re-audit when a later labeled sample disowns a "User stated" note again). | final |
| 2026-09-04 | Part 5 (calendar) reconciled to what shipped (v9.15.0, PR #548; live the same day). Body: DDΒ§5's calendar section gains the shipped paragraph β the register discovered through the Projects witness and never conjured (why not create it: the vault root is the operator's; re-audit if a flat vault needs a register); rollups on the Python dream cadence until the binary takes them over in part 6, empty weeks saying so once (why not skip them: schedule replaced discipline; re-audit if empty reviews become noise); promotion as a confirm-gated rules proposal, never auto-applied (why not auto-apply: the registry is the operator's per part 1; re-audit only toward a lighter confirm). Residuals named: the register outside the daemon's index scope; the operator's flat daily notes coexisting in Calendar/. The write path gains one amendment this part's CI found at UTC midnight: the volume gate counts against the day the arriving note's captured stamp names, the same day the writes-per-day reading files it under (why not the wall clock: the gate and the reading disagreed about which day a write belonged to, and a flood in progress at midnight found the door open; re-audit if a writer ever backdates captured deliberately β the gate would count against the backdated day). |
final |
| 2026-09-05 | Part 6 (lifecycle-dreaming) reconciled to what shipped (v9.16.0, PRs #550β#555, #556; live the same days) and the design launched. Body: DDΒ§6 gains the shipped paragraph β the axis as classes at the standard demotion (why not a tuned curve: the daemon's 125-point sweep found every other weight a regression; re-audit when the corpus has age spread); additive date glosses (why not rewrite: the corpus is the operator's words and every other mutation is additive; re-audit if glossed notes read badly in recall excerpts); the takeover as one change to two manifests after a written disposition of the divergence reviews (why not a longer window: the one review agreed on every surface and the operator ruled; re-audit if a first applying pass ever skips an intent). Gap closed by ruling: the overlap window's length and who closes it. | launched |
| 2026-09-05 | One vocabulary for the superseded relation (v9.18.0, PRs #566, #567): the loser carries lifecycle: superseded + superseded_by:, supersedes: is the successor's back-link only, both recall paths wall a superseded memory like archived and answer the explicit query with it demoted, the gate refuses the inverted shape, the corpus migrated (9 converted, 3 revived whose successors were gone, 1 digest), the parity recording edited on purpose (why not two vocabularies: two directions and two effects for one relation; re-audit if a successor ever needs to be something other than a note). |
launched |
| 2026-09-06 | One meaning for source: (v9.19.0, PRs #570, #571, #572), closing DDΒ§4's own open question. The field names the transport alone; a fetched page's address goes in source_url: and a mined unit's identity in source_id:; readers resolve those first and still accept the legacy shape, so the source registry is never blind to the population it covers; the gate refuses a value outside the vocabulary; the capture tool's schema and the CLI advertise all three fields, so a caller can discover them. The corpus migrated β 124 fetched pages and 14 references β and the fetched pages earned the trust tier the contract had always given them. Why not renaming the contract's field, or a reading-side rule: the first migrates 578 notes to accommodate 138, the second leaves the field overloaded so no gate can ever refuse a wrong value. Re-audit when a writer needs a transport the contract does not name, or a reference neither field can hold. Same day: the vault-perfection series opens and authors the successor, AgentM Vault; this log closes here and later rulings are recorded there. |
launched |