recursive prompt engineering - nefarious671/sophia GitHub Wiki

Recursive Prompt Engineering Framework

  1. Core Recursive Cycle State → Prompt Generation → LLM → Parsing → State Update → Controller (Orchestrator)

Each loop updates the State, generates a new prompt, sends it to the LLM, parses the result, and updates the State for the next cycle. The Controller orchestrates this process and decides navigation, evaluation, and termination.

  1. Structured State Management Hierarchical State:

Global State governs the entire problem; Node States represent sub-problems, each with its context, task, status, history, cost tracking, and ambiguity flags.

State Update:

LLM must output structured, parsable State Update info—often using JSON, YAML, or distinct markers. The prompt must demand these formats, and robust parsing logic is required.

  1. Formalized Task Types Define a taxonomy of roles (initialization, reasoning, tool selection, evaluation, synthesis, etc.). Allows reusable, structured LLM interactions within each node.

  2. Prompt Generation Prompt Generation Module:

Dynamically builds prompts using the current Node State and context from the Global/Node State.

Each prompt should:

Clearly state the sub-task. Provide necessary history/context. Request specific, structured State Update info. Adapt to remaining cycles (expand vs. summarize).

Techniques:

Use few-shot examples, clear delimiters, specific role instructions, and iterative refinement in prompt formatting.

  1. Controller (Orchestrator) Central Decision Layer:

Navigates the State graph, manages node lifecycles, integrates evaluation metrics, tracks costs, handles stagnation/oscillation, and manages external tool/API integration. Supports advanced navigation (BFS, DFS, A*, Best-First) and learning from history.

  1. Handling Ambiguity & Uncertainty Detects ambiguity/confidence issues in LLM output; can branch, clarify, or trigger human-in-the-loop. Probabilistic states, clarification sub-tasks, and pruning uncertain branches are built-in.

  2. Inter-Node Synthesis Shares knowledge, conclusions, and context across nodes/branches to avoid redundancy and build a unified final output.

  3. Tool and External API Integration Task types in Node States allow tool selection, input/output processing. Controller can call tools/APIs as needed, process results, and update State accordingly.

  4. Learning and Adaptation Execution History:

Logs, analyzes, and learns from previous runs to refine prompts, State structure, controller logic, and tool integration. Supports both automated self-improvement and human-guided feedback.

  1. Human-in-the-Loop (HITL) Integration Explicit Points:

Controller pauses for human input at ambiguity, failure, or milestone points. UI layer for reviewing State, deciding paths, or editing JSON. Human feedback can label, adjust, or guide recursive steps and refine framework logic.

  1. Cost and Efficiency Management Tracks resource usage:

Cost and token tracking per node/task. Prunes or reprioritizes based on estimated or real-time cost/benefit calculations.

  1. Evaluation, Monitoring, and Debugging Comprehensive logging, test cases, and visualizations of State, execution flow, and bottlenecks. Metrics for process efficiency, quality, and robustness. Systematic log analysis and refinement close the loop.

  2. Termination and Completion Clear criteria for ending nodes, branches, or entire process (success, cost, stagnation, errors). Graceful shutdown and reporting on final results.

  3. Generalization and Modularity Framework can be instantiated/configured for any task by:

Using abstract interfaces for modules. Configuration-driven analysis, prompt, and control strategies. Task definition languages and reusable prompt/analysis components.

  1. Iterative Refinement Cycle The “art” is a non-linear loop:

Problem understanding → design → implement → execute → monitor → analyze → redesign → repeat. Each run improves framework structure, prompt design, and control logic.

Synthesis This framework is not just about writing individual recursive prompts, but about designing, orchestrating, and refining a whole recursive system—one that can handle complex, real-world tasks using LLMs and other tools, while balancing autonomy and human feedback, creativity and structure, efficiency and robustness.

Ready to diagram, critique, or operationalize any layer of this framework, Tim? Let me know if you want the “step-by-step methodology” or example code for one of the modules!