Wiki Persistence GitOps - Chris-Cullins/wiki_bot GitHub Wiki

Wiki Persistence & GitOps

Overview

  • Automates cloning, syncing, and pushing to the GitHub wiki that houses generated documentation.
  • Orchestrates documentation generation (via Claude) and persistence, supporting fresh runs, incremental updates, and selective regeneration.
  • Shields the writing pipeline from dirty repos, authentication mishaps, and idempotency issues so that CI/CD can treat docs as a build artifact.

Key Components

  • src/github/git-repository-manager.ts: Stateful Git helper that encapsulates clone/update/commit/push flows and repository health checks.
  • src/github/github-wiki-writer.ts: High-level wiki publisher that renders pages, manages the sidebar, and delegates Git operations to GitRepositoryManager.
  • src/wiki-generator.ts: Generates page content by prompting Claude and templating results; integrates with the writer for persistence.
  • src/config.ts & src/index.ts: Surface environment-driven options, wire up the crawler, generator, and writer, and control execution modes.

How It Works

  • index.ts loads Config, resolves repo paths, and instantiates WikiGenerator and (if configured) GitHubWikiWriter.
  • GitHubWikiWriter.ensurePrepared() calls GitRepositoryManager.prepare() once to create/update the wiki checkout according to RepositoryMode (fresh, incremental, reuse-or-clone). Fresh runs optionally clear old .md files.
  • During a write, GitHubWikiWriter.writeDocumentation():
    • Ensures the repo is present and reports pre-existing uncommitted changes.
    • Writes or skips each page based on hasContentChanged, normalizing newline style to avoid churn.
    • Rebuilds _Sidebar.md in a predictable order (Home → Architecture → alphabetical remainder).
    • Uses commitAndPush() to stage, commit, and push only when changes exist.
  • GitRepositoryManager wraps git commands with consistent environment, token injection (buildAuthUrl), and sanitized logging. It refuses destructive updates if the working tree is dirty.
  • WikiGenerator streams Claude responses, normalizes headings, injects default sections (e.g., Architecture outline), applies depth instructions, and renders templates before handing content to the writer.

Important Functions/Classes

  • GitRepositoryManager.prepare() (src/github/git-repository-manager.ts): Entry point for enforcing the chosen repository mode; invokes clone, update, or reuses an existing checkout.
  • GitRepositoryManager.status(): Returns existence, cleanliness, branch, ahead/behind counts, and porcelain output—used both for guard rails and logging.
  • GitRepositoryManager.update(): Fetches, optionally checks out the target branch, and hard-resets to origin/<branch> when safe (no uncommitted files).
  • GitRepositoryManager.commit(message) / push(): Idempotent commit helper that skips when nothing changed, followed by a sanitized push.
  • GitHubWikiWriter.writeDocumentation(pages) (src/github/github-wiki-writer.ts): Main publishing workflow including preparation, page writes, sidebar maintenance, and GitOps.
  • GitHubWikiWriter.cleanupExistingPagesIfNeeded(): Recursively prunes markdown files for fresh runs when cleanupOnFresh is true.
  • WikiGenerator.generateHomePage() / generateArchitecturalOverview() / generateAreaDocumentation() (src/wiki-generator.ts): Build prompts, stream responses, enforce heading structure, and render templates for each documentation surface.
  • WikiGenerator.collectResponseText(query): Collapses Agent SDK streaming/mocked responses into deterministic strings, enabling consistent templating and logging.

Developer Notes

  • Always set WIKI_REPO_MODE to match deployment intent:
    • fresh: Deletes and reclones on every run; pair with WIKI_FRESH_CLEAN=true to purge stale markdown.
    • incremental: Updates an existing checkout but demands a clean tree; ideal for CI or cron jobs.
    • reuse-or-clone: Leaves local edits untouched (no fetch/reset); useful for manual experimentation.
  • Provide GITHUB_WIKI_URL, GITHUB_WIKI_PATH, and a token (GITHUB_TOKEN/GH_TOKEN) for wiki publishing. Without them, generation still runs but persistence is skipped.
  • GitRepositoryManager assumes Git is installed and available on PATH. Large repos may require raising maxBuffer.
  • GitHubWikiWriter normalizes newline endings to LF; copy-paste content should already include a trailing newline to avoid churn.
  • When running selectively (--target-file), the CLI forces incremental mode to keep existing pages and only pushes touched areas.
  • Enable DEBUG=true or PROMPT_LOG_ENABLED=true for verbose logging and prompt/response auditing, especially when tuning templates or diagnosing API issues.

Usage Examples

import { GitHubWikiWriter } from './github/github-wiki-writer.js';
import { GitRepositoryManager } from './github/git-repository-manager.js';

// Typically configured via environment variables and index.ts,
// but a direct invocation looks like this:

const writer = new GitHubWikiWriter({
  wikiRepoUrl: 'https://github.com/org/project.wiki.git',
  localPath: './.wiki',
  token: process.env.GITHUB_TOKEN,
  defaultBranch: 'master',
  commitMessage: 'Automated wiki update',
  mode: 'incremental',
  shallow: true,
  cleanupOnFresh: false,
});

// Pages generated elsewhere (e.g., WikiGenerator)
const pages = new Map<string, string>([
  ['Home', '# Home\n\nUp-to-date overview...'],
  ['Architecture', '# Architecture\n\nSystem layout...'],
]);

await writer.writeDocumentation(pages);
# CLI-driven end-to-end run
export GITHUB_WIKI_URL="https://github.com/org/project.wiki.git"
export GITHUB_TOKEN="ghp_..."
export WIKI_REPO_MODE="incremental"
npm run dev -- --depth standard
⚠️ **GitHub.com Fallback** ⚠️