CLI Orchestration Runtime Control - Chris-Cullins/wiki_bot GitHub Wiki

CLI Orchestration & Runtime Control

Overview

This layer wires the end-to-end wiki generation workflow. It owns CLI parsing, runtime configuration, repository discovery, LLM query orchestration, and persistence of generated pages. The goal is to provide a predictable entry point that can drive full runs, selective regenerations, or future CI integrations while abstracting away provider-specific details and GitHub wiki plumbing.

Key Components

  • src/index.ts – CLI entrypoint that parses args, loads config, orchestrates crawling, generation, and persistence.
  • src/config.ts – Environment-driven configuration loader defining documentation depth, providers, repo settings, and toggles.
  • src/query-factory.ts – Selects the appropriate query implementation (Agent SDK, Claude CLI, Codex CLI, or mock) and normalizes invocation.
  • src/logging.ts – Debug logger with optional prompt/response transcript capture per run.
  • src/wiki-generator.ts – High-level workflow for prompts, area identification, snippet gathering, and template rendering.
  • src/repo-crawler.ts – Filesystem crawler producing tree/flat views with ignore handling.
  • src/github/github-wiki-writer.ts – Writes generated pages, sidebar, and pushes commits to wiki repos.
  • src/github/git-repository-manager.ts – Git primitives (clone, update, status, commit, push) backing the wiki writer.
  • Supporting utilities: src/mock-agent-sdk.ts (test-mode responses) and src/prompt-loader.ts (template injection).

How It Works

  1. Startup (index.ts): main() parses CLI flags (--target-file, --depth), loads configuration via loadConfig, applies depth overrides, and resolves repo paths/template directories.
  2. Runtime Prep: Initializes DebugLogger, selects a query function through createQueryFunction, and prepares environment variables when using the Agent SDK.
  3. Repository Scan: RepoCrawler walks the repository respecting .gitignore and built-in ignore patterns, producing both a tree (FileNode) and a flat list of paths.
  4. Selective Targeting: resolveTargetFiles maps CLI-provided paths to repo files, enabling incremental runs that only touch affected areas.
  5. Generation Pipeline: WikiGenerator sequentially creates the Home page, Architecture overview, extracts architectural areas, identifies relevant files, and emits per-area documentation, optionally reusing existing content.
  6. Persistence: GitHubWikiWriter stages generated pages and _Sidebar.md, then commits and pushes via GitRepositoryManager. Selective runs defer writes until at least one targeted page changes.

Important Functions/Classes

main() (src/index.ts)

Central coordinator that chains configuration, logging, crawling, generation, and persistence. Handles selective runs, incremental docs, and wiki write scheduling.

parseCliArgs & resolveTargetFiles (src/index.ts)

  • parseCliArgs extracts CLI options with validation and help output.
  • resolveTargetFiles normalizes CLI paths, matches against known files, and tracks unmatched inputs to warn users.

loadConfig (src/config.ts)

Aggregates environment variables into a Config, including provider selection, authentication, repo/wiki settings, depth, and feature toggles (debug, incremental docs, test mode). Ensures required keys exist in agent-sdk mode.

createQueryFunction (src/query-factory.ts)

Returns a provider-specific query iterator:

  • agentQuery for SDK usage.
  • createClaudeCliQuery for CLI wrapper (adds repo context, system prompt).
  • createCodexCliQuery parsing JSON streaming output.
  • createMockQuery for deterministic testing.

DebugLogger (src/logging.ts)

Structured debug output and optional prompt transcript storage per run (run-specific directories, numbered prompt/response files). Handles failures gracefully.

WikiGenerator (src/wiki-generator.ts)

Encapsulates prompt preparation, LLM calls, template rendering, and area iteration. Key responsibilities:

  • Ensure heading normalization and consistent sections.
  • Support incremental updates by supplying existing docs.
  • Collect file contents per area and enforce depth instructions.
  • Provide fallbacks (mermaid diagram defaults, JSON parsing safeguards).

RepoCrawler (src/repo-crawler.ts)

Recursive filesystem crawler with ignore support. Produces FileNode trees and flat path lists for downstream prompts and targeting.

GitHubWikiWriter & GitRepositoryManager

  • GitHubWikiWriter handles page file naming, sidebar ordering, fresh mode cleanup, change detection, and push orchestration.
  • GitRepositoryManager implements prepare modes (fresh, incremental, reuse-or-clone), status inspection, and authenticated git operations.

Developer Notes

  • Provider Configuration: LLM_PROVIDER drives the query backend. When using agent-sdk, ensure ANTHROPIC_AUTH_TOKEN or ANTHROPIC_API_KEY is set; CLI providers can rely on their own auth.
  • Selective Regeneration: Use --target-file for incremental updates; the run will only regenerate areas intersecting those files and will defer wiki writes until matching pages change.
  • Incremental Docs: Enable via env (INCREMENTAL_DOCS=true) or by using repo modes (incremental / reuse-or-clone). Existing wiki content is fetched via GitHubWikiWriter.readPage.
  • Prompt Logging: Set PROMPT_LOG_ENABLED=true (and optionally PROMPT_LOG_DIR) to persist prompts/responses. Logs live under .wiki-logs/<timestamp>/.
  • Test Mode: TEST_MODE=true swaps in mock query responses, used for dry runs without API calls. Automatically injects a dummy API key.
  • Fresh Wiki Runs: Configure WIKI_REPO_MODE=fresh with WIKI_FRESH_CLEAN=true to delete old markdown before writing; ensure no valuable manual edits remain.
  • Error Handling: External command failures (claude/codex/git) log diagnostic previews and bubble explicit errors; plan retries or fallback logic in higher-level automation.

Usage Examples

  • Full Generation

    npm run build && LLM_PROVIDER=agent-sdk ANTHROPIC_API_KEY=... npm start

    Generates all pages, writes to wiki if configured.

  • Selective Regeneration

    LLM_PROVIDER=claude-cli npm run dev -- --target-file src/wiki-generator.ts

    Runs in dev mode (tsx) and only regenerates areas touching the specified file.

  • Depth Override

    DOC_DEPTH=deep npm start -- --depth summary

    Runtime flag beats environment, ensuring the run uses summary-level guidance.

  • Test Mode Dry Run

    TEST_MODE=true npm run dev

    Exercises the orchestration stack without reaching out to Anthropic or GitHub.

⚠️ **GitHub.com Fallback** ⚠️