LLM Provider Abstraction

Overview

The LLM Provider Abstraction isolates how the wiki bot talks to large language models across the Anthropic Agent SDK, Claude CLI, and Codex CLI backends. It centralizes provider selection, configuration, and command execution so the rest of the system can stream documentation tasks without knowing which LLM is behind the scenes. The abstraction also supplies a mock implementation for deterministic testing and feeds the wiki generator with consistent streaming responses regardless of provider.

Key Components

• src/query-factory.ts: Builds the concrete QueryFunction wrapper for the configured provider, including CLI adapters and shared command execution.
• src/config.ts: Parses environment variables into a typed Config, normalizing provider choices, auth keys, and documentation depth flags.
• src/index.ts: Orchestrates runtime bootstrapping—loads configuration, resolves repo paths, and injects the provider-specific query function into the WikiGenerator.
• src/wiki-generator.ts: Consumes the provider-agnostic query function, streaming responses and transforming them into Markdown wiki pages.
• src/mock-agent-sdk.ts: Supplies a fake query function for testMode, returning canned responses that mimic Agent SDK streaming.

How It Works

1. Configuration Loading (loadConfig): Environment variables are read on startup, converting string flags into booleans, enums, and defaults. Provider aliases (e.g., claude) are normalized to the internal LlmProvider union.
2. CLI Option Overrides (parseCliArgs in src/index.ts): Runtime flags like --depth or --target-file override loaded configuration before dependencies are initialized.
3. Query Function Creation (createQueryFunction): Using the resolved Config, the factory picks one of three branches:
   • Agent SDK: Returns the SDK’s native query function with minimal wrapping.
   • Claude CLI / Codex CLI: Wraps a child process invocation in an async iterator, streaming stdout back as Agent-style messages.
   • Test Mode: Short-circuits to createMockQuery for repeatable unit tests.
4. Command Execution (runCommand): Shared helper spawns subprocesses, captures stdout/stderr, handles ENOENT errors, and surfaces non-zero exits with contextual messaging. Debug logging records prompt previews and output snippets.
5. Wiki Generation (WikiGenerator): All documentation flows through this._query, which is the provider-agnostic function. The generator prepares prompts, logs them when debug is enabled, collects streamed responses, and post-processes Markdown (heading normalization, template rendering).
6. Mock Flow (createMockQuery): For test mode, prompt inspection returns deterministic Markdown or JSON, letting integration tests exercise downstream logic without LLM calls.

Important Functions/Classes

• createQueryFunction(config, repoPath, logger) (src/query-factory.ts): Switchboard that decides which provider to bind. Ensures test mode takes precedence over CLI or Agent SDK selection, so fixtures can run without external dependencies.
• createClaudeCliQuery(repoPath, logger) / createCodexCliQuery(logger) (src/query-factory.ts): Provider-specific adapters that format commands, feed prompts via stdin, and convert stdout into the streaming Query shape expected by the Agent SDK consumer.
• runCommand(command, args, input, logger) (src/query-factory.ts): Robust subprocess executor that normalizes encoding, traps launch errors, and trims stderr/stdout for error reporting. Also detects “usage limit” phrases to aid rate-limit diagnostics.
• extractCodexResponse(output) (src/query-factory.ts): Parses newline-delimited JSON from codex exec --json -, collecting only agent_message payloads and collapsing them into a Markdown string.
• loadConfig() (src/config.ts): Converts env state into a validated configuration object, enforcing required API keys when the Agent SDK is selected and deriving booleans for toggles like PROMPT_LOG_ENABLED.
• WikiGenerator (src/wiki-generator.ts): Core documentation engine; methods like generateHomePage, generateArchitecturalOverview, and generateAreaDocumentation prepare prompts, call the query function, and normalize responses. Key helpers (collectResponseText, ensureHeading, ensureArchitectureOutline) ensure consistent Markdown regardless of provider quirks.
• createMockQuery() (src/mock-agent-sdk.ts): Produces an async iterator with canned responses that imitate Agent SDK message semantics, keeping the rest of the stack oblivious to the mock.

Developer Notes

• Provider Selection Order: Test mode overrides everything. When toggled via TEST_MODE=true, the mock query is returned even if LLM_PROVIDER points to CLI adapters.
• Environment Validation: loadConfig throws if no Anthropic key is present while using the Agent SDK and test mode is off. Catch this early in automated deployments by setting LLM_PROVIDER=codex-cli or claude-cli when keys are unavailable.
• CLI Dependencies: The CLI branches assume binaries named claude and codex are on PATH. runCommand converts ENOENT into a friendly install reminder—bubble this up to users instead of swallowing it.
• Prompt Logging: When promptLoggingEnabled is true, DebugLogger persists prompts/responses. The query factory and wiki generator both rely on the same logger, so toggling DEBUG or logging paths affects provider diagnostics and content generation equally.
• Streaming Compatibility: The wiki generator expects Agent SDK-style streaming events. The CLI adapters fake this by returning a single assistant message. If you add new providers, ensure they yield messages adhering to Query’s async iterator contract.
• Error Surface: CLI failures propagate as rejected promises. Upstream callers should be ready to handle thrown errors (e.g., wrapping await wikiGenerator.generateHomePage(...) in try/catch when introducing new workflows).

Usage Examples

Selecting a Provider via Environment

# Use the Anthropic Agent SDK (requires API key)
export LLM_PROVIDER=agent-sdk
export ANTHROPIC_API_KEY=sk-ant-...

# Switch to Claude CLI without touching code
export LLM_PROVIDER=claude-cli
export CLAUDE_AUTH_TOKEN=...

# Enable deterministic test mode
export TEST_MODE=true

Injecting the Query Function at Startup (src/index.ts)

const config = loadConfig();
const repoPath = config.repoPath ? resolve(config.repoPath) : process.cwd();
const logger = new DebugLogger({ enabled: Boolean(config.debug) });

const queryFn = createQueryFunction(config, repoPath, logger);
const wikiGenerator = new WikiGenerator(queryFn, config, logger);

Consuming the Provider in Wiki Generation (src/wiki-generator.ts)

const prompt = await loadPrompt('generate-area-documentation', {
  area,
  fileContentText,
  existingDoc: existingDoc ?? '',
  depthInstruction: this.getDepthInstruction(),
});

const query = this.createQuery(prompt);
const response = await this.collectResponseText(query);
const finalized = this.ensureHeading(this.stripFenceWrappers(response), area);

These snippets demonstrate how configuration, provider abstraction, and documentation generation tie together, enabling the wiki bot to swap LLM backends without touching the core content pipeline.