reflection and recovery - alexherrero/agentm GitHub Wiki

Reflection sidecar + crash-recovery markers

Parent design: MemoryVault โ€” see Detailed Design ยง2 (Reflection sidecar) and ยง6 (Crash recovery โ€” session-id markers) for full architectural context. These two subsections are bundled because they share the Stop/idle hook scaffolding.

Scope

This part ships the write loop โ€” the reflection logic that mines conversation transcripts for durable entries + the crash-recovery mechanism that ensures no session's learnings are lost when the Stop hook doesn't fire.

Three trigger surfaces for reflection (all running the same logic):

  • Manual /memory reflect [--session <path>] โ€” user-initiated; runs against the current session transcript by default or a specified transcript path. Validates the reflection logic against arbitrary input.
  • Stop-event hook โ€” fires automatically on Claude Code session Stop. Scoped to the just-ended session's transcript.
  • Idle-time hook (new crickets primitive added by this part) โ€” fires when Claude Code has been idle > N minutes (N TBD, default 30); also scans .harness/session-id-*.start marker files for orphans (crashed sessions where Stop didn't fire) and runs reflection retroactively on those transcripts.

Reflection logic (shared across all three triggers):

  1. Read the session transcript at ~/.claude/projects/<repo>/<session-id>.jsonl.
  2. Two parallel mining passes:
    • 3-category mine (writes to MemoryVault): scan for Successful Workflows, User Preferences, Fixes & Workarounds. Each candidate gets a confidence rating per the heuristic in step 3.
    • Idea-candidate mine (writes to user-vault Ideas.md + deep research in _idea-incubator/): scan for follow-ups, future project ideas, research candidates. Each candidate gets a 2-sentence summary + the agent does deep research. (The actual idea-ledger writing is handled by the idea-ledger part โ€” this part just produces the candidates.)
  3. Tri-modal routing for 3-category candidates per the locked heuristic:
    • HIGH = explicit user signal during session (user said "I prefer X" / user manually corrected the agent / user locked a design call) โ†’ auto-save via /memory save. (Corrected from the original "always X" / "never Y" wording โ€” that bare pattern was demoted out of the HIGH lane by the R0.3 source-filter fix, since it fires on ordinary discussion, not just operator directives.)
    • MEDIUM = pattern-inferred (agent noticed pattern 3+ times) โ†’ interactive review prompt with approve / edit / reject / skip / supersede-existing-X options. Controlled by memory.review_mode: interactive (default) | silent.
    • LOW = single-instance inference โ†’ write to MemoryVault/personal-private/_inbox/<slug>.md for batch review later.

Crash-recovery markers:

  1. SessionStart hook writes .harness/session-id-<session-uuid>.start (one file per session, contents = session start timestamp + transcript path).
  2. Stop hook (after reflection succeeds) renames .start โ†’ .reflected. On failure, file stays as .start for retry.
  3. Idle-time hook scans for .start files older than 1 hour (idle threshold for assuming the session is dead) โ†’ runs reflection retroactively โ†’ renames to .reflected on success.
  4. GC: .reflected markers older than 30 days get deleted on next idle pass.

All markers live in .harness/ (gitignored, runtime-only).

Dependencies

  • write-primitives โ€” reflection writes via /memory save for HIGH-confidence candidates and the interactive review supersede path uses /memory evolve. Without write primitives, reflection has nowhere to land its output.
  • recall-loop โ€” interactive review's MEDIUM-confidence prompts surface "this candidate contradicts existing entry X โ€” supersede?" decisions, which requires the recall engine to find the contradicting entry. Without recall, the supersede-existing-X option degenerates to plain approve/reject.

Verification criteria

  1. Stop hook fires on session end โ€” install at the 2 host destinations; run a Claude Code session; verify Stop hook fires + reflection runs against the just-ended session's transcript.
  2. Idle hook fires after N min idle โ€” same; let session sit idle past the threshold; verify the hook fires.
  3. Reflection mines 3 categories โ€” fixture transcript with seeded "user said 'always X'" / "user corrected the agent" / "hit error Z resolved by W" patterns; verify each pattern surfaces as a candidate in the correct category.
  4. Reflection mines idea candidates โ€” fixture transcript with "we should also do X later" / "this could be its own project" patterns; verify each surfaces as an idea candidate (raw โ€” idea-ledger part handles the write).
  5. Tri-modal routing places entries correctly โ€” HIGH candidates auto-save to canonical paths; MEDIUM candidates trigger interactive review prompts; LOW candidates land in _inbox/.
  6. Interactive review prompts work โ€” memory.review_mode: interactive default; verify each MEDIUM candidate produces an approve/edit/reject/skip/supersede prompt; verify each option does the right thing.
  7. memory.review_mode: silent toggle โ€” flip the setting; verify MEDIUM candidates auto-save without prompting.
  8. Crashed-session marker recovery โ€” start a session, kill Claude Code mid-session (so Stop never fires); verify .start marker stays in place; trigger idle hook (or wait for it); verify reflection runs retroactively against the orphan session's transcript + marker gets renamed to .reflected.
  9. Marker GC at 30 days โ€” seed old .reflected markers; verify idle hook deletes them on next pass.
  10. Manual /memory reflect โ€” invoke directly against a specified transcript path; verify reflection runs + produces candidates routed per tri-modal logic.
  11. Smoke install verifies hooks land โ€” smoke-install-bash.sh + .ps1 extended for both Stop + idle hook scripts at the 2 host destinations.
  12. All 3 OS CI workflows green on the commit that lands this part.

Notes for the implementing /work session

  • The idle-time hook is the new crickets primitive. The existing Stop-event hook pattern from plan #4+#5 (commit-on-stop) is the reference shape; idle is similar but with a different trigger condition. The hook needs to know "how long has the agent been idle?" โ€” Claude Code's hook lifecycle doesn't expose this directly, so the hook likely polls a heartbeat file the SessionStart writes + updates on every UserPromptSubmit.
  • The 3-category extraction heuristic is the load-bearing-but-uncertain piece. Ship instrumented โ€” every mined candidate logs its category + confidence + rationale so the user can validate via the interactive review + manually inspect via /memory inspect.
  • The "user said 'always X'" pattern detection is straightforward (keyword scan + nearby user-turn). The "user manually corrected the agent" pattern is harder โ€” requires looking at agent turn โ†’ user reverts/corrects โ†’ agent applies correction. Defer fancy detection to a follow-up if the simple heuristic is too noisy.
  • Idea-candidate mining lives in this part but the idea-ledger writes (Ideas.md + _idea-incubator/) are the next part's scope. This part's reflection logic just emits idea candidates as in-memory objects; the idea-ledger part subscribes to those.
  • .harness/session-id-*.start marker contents: just the session start timestamp + transcript path is sufficient. Don't put PII (transcript content) in markers โ€” they're runtime metadata.
โš ๏ธ **GitHub.com Fallback** โš ๏ธ