Chain‐of‐Thought Prompting - joehubert/ai-agent-design-patterns GitHub Wiki

Home::Overview of Patterns

Classification

Core Processing Pattern

Intent

A technique that guides Large Language Models (LLMs) to show their reasoning process step by step, leading to more reliable outcomes for complex tasks by explicitly breaking down the thought process into sequential, interconnected steps.

Also Known As

  • Step-by-Step Reasoning
  • Explicit Reasoning Prompting
  • Reasoning Trace
  • Thought Decomposition

Motivation

Complex reasoning tasks often require multiple logical steps to arrive at correct solutions. Traditional prompting approaches may lead to errors when LLMs attempt to jump directly to conclusions without properly working through the intermediate reasoning steps.

The Chain-of-Thought pattern addresses this challenge by:

  • Encouraging LLMs to decompose complex problems into manageable steps
  • Making the reasoning process explicit and inspectable
  • Reducing errors by catching flawed intermediate reasoning
  • Providing transparency into how conclusions are reached

For example, in mathematical problem-solving, direct prompting might ask "What is 15% of $67.50?", while Chain-of-Thought prompting would guide the model to show its work: "To find 15% of $67.50, I'll first convert 15% to a decimal (0.15), then multiply: $67.50 × 0.15 = $10.125, which rounds to $10.13."

Applicability

When to use this pattern:

  • Complex reasoning tasks requiring multiple logical steps
  • Mathematical or symbolic problem-solving
  • Tasks with high potential for reasoning errors
  • Situations requiring explainable AI decisions
  • Problems benefiting from methodical decomposition
  • When generating verifiable solutions is important
  • For knowledge-intensive tasks requiring fact integration

Structure

Component Diagram

graph TD
    subgraph "Chain-of-Thought Pattern"
        A[Task Prompt] --> B[Reasoning Directive]
        B --> C[LLM Agent]
        D[Few-Shot Examples] -.-> C
        C --> E[Reasoning Steps]
        E --> F[Intermediate Conclusions]
        F --> G[Final Response]
    end
    
    classDef component fill:#f9f9f9,stroke:#333,stroke-width:1px;
    classDef input fill:#d4f1f9,stroke:#333,stroke-width:1px;
    classDef output fill:#d5e8d4,stroke:#333,stroke-width:1px;
    classDef optional fill:#f8cecc,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5;
    
    class A,B input;
    class C component;
    class D optional;
    class E,F component;
    class G output;

Sequence Diagram

sequenceDiagram
    participant User
    participant System
    participant LLM
    
    User->>System: Submit complex problem
    System->>LLM: Forward task prompt + reasoning directive
    
    opt Few-Shot Mode
        System->>LLM: Include reasoning examples
    end
    
    LLM->>LLM: Generate reasoning step 1
    LLM->>System: Output reasoning step 1
    
    LLM->>LLM: Generate reasoning step 2
    LLM->>System: Output reasoning step 2
    
    LLM->>LLM: Generate reasoning step n
    LLM->>System: Output reasoning step n
    
    LLM->>LLM: Formulate intermediate conclusions
    
    LLM->>LLM: Derive final response
    LLM->>System: Output final response
    
    System->>User: Present complete reasoning chain and solution

Components

The key elements participating in the pattern:

  • Task Prompt: The initial instruction or question that defines the problem to be solved
  • Reasoning Directive: Explicit instructions for the model to "think step by step" or similar guidance
  • Reasoning Steps: Sequential thought segments that build upon each other logically
  • Intermediate Conclusions: Partial results or insights gained during the reasoning process
  • Final Response: The ultimate answer derived from the chain of reasoning
  • Few-Shot Examples (optional): Demonstrations of appropriate reasoning chains for similar problems

Interactions

How the components work together:

  1. The system presents the Task Prompt to the LLM, combined with the Reasoning Directive
  2. The LLM generates explicit Reasoning Steps, making its thought process visible
  3. Each step builds on previous steps, potentially including Intermediate Conclusions
  4. The chain concludes with a Final Response that derives from the reasoning process
  5. If Few-Shot Examples are included, they establish the pattern and level of detail expected in the reasoning process

Consequences

Benefits:

  • Improved accuracy on complex reasoning tasks
  • Enhanced explainability and transparency
  • Easier identification of reasoning errors
  • Better alignment with human problem-solving approaches
  • Self-correction opportunities during the reasoning process
  • Structured output that can be validated step-by-step
  • Improved performance on math, logic, and multi-step reasoning tasks

Limitations:

  • Increased token consumption due to verbose reasoning
  • Potential for higher latency in responses
  • May introduce unnecessary steps for simple problems
  • Reasoning quality depends on the LLM's underlying capabilities
  • Can amplify existing reasoning biases in the model
  • May produce plausible-sounding but incorrect reasoning chains

Performance implications:

  • Higher computational cost due to longer outputs
  • Increased context window requirements
  • Potential latency impacts for user-facing applications

Implementation

Guidelines for implementing the pattern:

  1. Craft clear reasoning directives:

    • Use explicit instructions like "Think step by step" or "Let's work through this systematically"
    • Specify the level of detail required in the reasoning

  2. Consider few-shot examples:

    • Provide 1-3 examples of high-quality reasoning chains
    • Ensure examples match the complexity of the target task

  3. Structure the reasoning process:

    • Request numbered or clearly delineated steps
    • Encourage the model to identify assumptions at each step

  4. Request verification:

    • Prompt the model to check its own work after completing the reasoning chain
    • Consider asking for alternative approaches to validate conclusions

  5. Common pitfalls to avoid:

    • Overly prescriptive reasoning structures that limit creativity
    • Insufficient context for complex problems
    • Failing to adjust the verbosity based on task complexity

Code Examples

Go to repo...

Variations

Zero-Shot Chain-of-Thought:

  • Simply instructs the model to "think step by step" without examples
  • Useful when example creation is difficult or when exploring novel problems
  • Generally less effective than few-shot approaches but still better than standard prompting

Few-Shot Chain-of-Thought:

  • Includes 2-8 examples of reasoning chains before the target problem
  • Demonstrates the expected reasoning style and depth
  • Often produces the best results but requires careful example curation

Self-Consistency Chain-of-Thought:

  • Generates multiple reasoning chains for the same problem
  • Selects the most common answer as the final response
  • Improves reliability through consensus across different reasoning paths

Least-to-Most Prompting:

  • Breaks complex problems into a series of increasingly complex sub-problems
  • Builds solutions progressively, using earlier answers to inform later steps
  • Particularly effective for compositional reasoning tasks

Tree-of-Thought:

  • Explores multiple reasoning branches simultaneously
  • Evaluates different paths and selects the most promising ones to continue
  • Allows for backtracking when reasoning leads to dead ends

Real-World Examples

  • Mathematical problem-solving: Chain-of-Thought significantly improves performance on arithmetic, algebra, and complex word problems by making calculation steps explicit
  • Code debugging: Developers use Chain-of-Thought to have LLMs systematically analyze code, identify bugs, and explain fixes with clear reasoning
  • Medical diagnosis assistance: Healthcare applications use this pattern to show the clinical reasoning process, increasing transparency and trust in AI-assisted diagnoses
  • Legal document analysis: Chain-of-Thought helps legal professionals track the logical steps in contract interpretation and compliance analysis
  • Scientific research: Researchers use this pattern to decompose complex hypotheses into testable components with explicit reasoning chains

Related Patterns

  • ReAct (Reasoning + Acting): Extends Chain-of-Thought by alternating between reasoning steps and external actions
  • Reflection: Complements Chain-of-Thought by adding explicit evaluation of the reasoning process
  • Tool Use: Often combined with Chain-of-Thought to show reasoning about when and how to use external tools
  • Hierarchical Task Decomposition: Uses Chain-of-Thought principles to break down complex tasks into manageable components
  • Process Transparency: Chain-of-Thought is a specific implementation of the broader goal of making agent reasoning visible