TMI Terraform Analyzer - ericfitz/tmi GitHub Wiki

TMI Terraform Analyzer

Automated Terraform infrastructure analysis tool for threat modeling using LLM providers (Claude, GPT, Grok, Gemini) selected through named profiles.

Overview

The TMI Terraform Analysis Tool automates the analysis of Terraform infrastructure code associated with threat models in the TMI platform. It uses LLM providers (Anthropic, OpenAI, x.ai, Google Gemini or OCI Generative AI) via LiteLLM to analyze infrastructure components, relationships, data flows, and security considerations, then generates comprehensive markdown reports stored as notes in TMI.

Repository: github.com/ericfitz/tmi-tf-wh

This tool can be used as a CLI for interactive analysis or deployed as a webhook-driven FastAPI service that processes analysis requests from TMI server events. The maintained deployment target is AWS (EKS, with an SQS jobs queue; see Deployment); a local Docker Compose setup is also provided. The OCI/OKE deployment was removed (see the repository's ADR 0001).

Features

  • Multi-Provider LLM Support -- Named LLM profiles (llm-profiles.yaml) select Anthropic, OpenAI, x.ai, Gemini or OCI models via LiteLLM; choose one per run or per addon invocation
  • OAuth Authentication -- Google Sign-In (CLI mode) or Client Credentials (webhook/server mode)
  • Smart Repository Discovery -- Automatically identifies GitHub repositories with Terraform code from threat models
  • Environment Detection -- Detects multiple Terraform environments (root modules) within a repository and allows selection
  • Sparse Cloning -- Efficiently clones only Terraform files (.tf, .tfvars) from repositories
  • Phased AI Analysis -- Three-phase LLM pipeline: inventory extraction, infrastructure analysis, and threat identification/analysis
  • Automatic Threat Extraction -- Extracts security vulnerabilities and creates structured threat objects using STRIDE framework, CVSS 4.0 scoring, and CWE classification
  • Data Flow Diagrams -- Generates interactive DFD diagrams showing infrastructure components, flows, and trust boundaries
  • Comprehensive Reports -- Creates separate inventory and analysis markdown reports with security observations
  • TMI Integration
    • Stores inventory and analysis results as separate notes in threat models
    • Generates and stores data flow diagrams for more precise modeling
    • Creates structured threat objects with STRIDE, CVSS, and CWE metadata
    • Attaches LLM metadata (model, token counts, cost) to all generated artifacts
  • Refusal-Aware Analysis -- Model refusals (content_filter) are never treated as a clean result; see Model Refusals
  • Webhook Server Mode -- Runs as a FastAPI service processing analysis requests from a job queue (AWS SQS in the current deployment), triggered by TMI webhook events: addon.invoked, threat_model.updated, and repository.*. threat_model.created does not start an analysis, because a new threat model has no repositories yet; an imported threat model is analyzed once, via repository.created.

Prerequisites

  • Python 3.13 or higher
  • UV package manager
  • Git
  • Access to a TMI server (https://api.tmi.dev)
  • API key for at least one LLM provider, referenced by an LLM profile:
    • Anthropic (Claude), OpenAI (GPT), x.ai (Grok) or Google (Gemini) API key
    • or OCI Generative AI (via OCI credentials)
  • Optional: GitHub personal access token (for higher API rate limits)

Installation

1. Clone the Repository

cd ~/Projects
git clone <repository-url> tmi-tf-wh
cd tmi-tf-wh

2. Configure Environment Variables

Copy the example environment file:

cp .env.example .env

Edit .env and set your API keys:

LLM_PROFILE=gpt56cyber
OPENAI_CYBER_API_KEY=your_openai_key_here  # env var name must match api_key in llm-profiles.yaml
GITHUB_TOKEN=your_github_token_here  # Optional but recommended

3. Install Dependencies

uv sync

Configuration

All configuration is managed through the .env file:

Variable Description Default
TMI_SERVER_URL TMI server URL https://api.tmi.dev
TMI_OAUTH_IDP OAuth identity provider (google or tmi) google
TMI_CLIENT_ID Client ID (required if TMI_OAUTH_IDP=tmi) None
TMI_CLIENT_SECRET Client secret (required if TMI_OAUTH_IDP=tmi) None
LLM_PROFILE Default LLM profile (see LLM Profiles) Required unless every run names a profile
LLM_PROFILES_FILE Path to the profiles file llm-profiles.yaml in the project root
key variables API keys, under the names the profiles' api_key fields reference (for example OPENAI_CYBER_API_KEY, ANTHROPIC_CYBER_API_KEY) Required for the profiles you use
OCI_COMPARTMENT_ID OCI compartment ID Required for profiles with auth: oci
GITHUB_TOKEN GitHub personal access token Optional
MAX_REPOS Maximum repositories to analyze 3
CLONE_TIMEOUT Git clone timeout in seconds 300
ANALYSIS_NOTE_NAME Base name for the generated note Terraform Analysis Report
DIAGRAM_NAME Base name for the generated diagram Infrastructure Data Flow Diagram
LATEST_COMMIT_DEPTH Commit window used by the latest environment scope and the sparse-clone pull depth 200

LLM Profiles

A profile names everything LiteLLM needs to run an analysis. Profiles live in llm-profiles.yaml (the repository ships profiles for several Anthropic and OpenAI models, for example gpt56cyber, mythos5, opus55, sonnet5):

profiles:
  gpt56cyber:
    provider: openai
    model: gpt-5.6-cyber
    api: responses
    auth: api_key
    api_key: OPENAI_CYBER_API_KEY
Field Required Meaning
provider yes anthropic, openai, xai, gemini or oci
model yes Bare model name, no LiteLLM prefix
auth yes api_key, or oci (OCI config file / resource principal and OCI_COMPARTMENT_ID)
api_key with auth: api_key Name of the env var or secret holding the key, never the key itself
api no chat (default) or responses (OpenAI Responses API; openai only)
base_url no Send calls to a proxy or compatible gateway

Select a profile with --profile on the CLI or "profile" in the addon invocation's user_data (for example {"environments": "aws", "profile": "gpt56cyber"}); otherwise LLM_PROFILE is used. An unknown profile or a missing key fails the run before any clone or LLM call. tmi-tf config-info shows which profiles are usable.

Note: The model name and timestamp are automatically appended to note and diagram names (e.g., "Terraform Inventory - prod (anthropic/claude-opus-4-6, 2026-04-04 12:00:00 UTC)").

Server/Webhook Mode Configuration

When running as a webhook-driven service (deployed on AWS EKS), additional configuration is available:

Variable Description Default
QUEUE_PROVIDER Job queue provider: aws, oci or memory Inferred: oci if QUEUE_OCID is set, aws if QUEUE_URL is set, else none
QUEUE_URL SQS jobs queue URL (AWS) None
AWS_REGION AWS region for SQS None
QUEUE_OCID OCI Queue OCID (legacy OCI provider) None
VAULT_OCID OCI Vault OCID (legacy OCI secret provider) None
WEBHOOK_SECRET HMAC shared secret for webhook validation None
WEBHOOK_SUBSCRIPTION_ID Subscription UUID filter None
MAX_CONCURRENT_JOBS Maximum parallel analysis jobs 3
JOB_TIMEOUT Job timeout in seconds 3600
MAX_MESSAGE_AGE_HOURS Maximum age for queue messages 24
SERVER_PORT FastAPI server port 8080
TMI_CLIENT_PATH Path to TMI Python client None
SECRET_PROVIDER Secret provider: oci or none oci if VAULT_OCID is set, else none
DEDUP_DEBOUNCE_SECONDS Triggers for the same threat model within this window are dropped (0 disables) 30

Configuration Details

LLM_PROFILE: Selects the default profile from llm-profiles.yaml (see LLM Profiles). All providers are accessed through LiteLLM. Profiles with auth: oci use OCI Generative AI with config-file or resource-principal authentication.

TMI_OAUTH_IDP: Set to google for browser-based PKCE OAuth (CLI mode) or tmi for client credentials flow (server/webhook mode).

Secrets in the AWS deployment: the EKS deployment runs with SECRET_PROVIDER=none; webhook secret, TMI client credentials, GitHub token and LLM keys are delivered as environment variables from a Kubernetes Secret. scripts/sync-llm-keys-aws.sh syncs rotated LLM keys from ~/.keys into that Secret (it compares hashes, updates the Terraform secrets tfvars, applies, and restarts the deployment).

GITHUB_TOKEN: Optional but highly recommended. Without a token, you are limited to 60 GitHub API requests per hour. With a token, you get 5,000 requests per hour. Create a token at GitHub Settings > Developer settings > Personal access tokens.

MAX_REPOS: Limits the number of repositories analyzed to avoid excessive API costs and processing time. Start with 1-3 repositories.

Usage

Basic Commands

View Configuration

uv run tmi-tf config-info

Shows current configuration including API endpoints, limits, LLM model, timestamp, and environment settings.

Authenticate with TMI

uv run tmi-tf auth

Opens a browser window for Google OAuth authentication. The token is cached in ~/.tmi-tf/token.json.

List Repositories

uv run tmi-tf list-repos <threat-model-id>

Lists all repositories associated with a threat model and identifies which ones are GitHub repositories.

Analyze Terraform Repositories

uv run tmi-tf analyze <threat-model-id>

The main command - analyzes Terraform code and creates inventory notes, analysis notes, data flow diagrams, and threat objects in TMI.

Analysis Options

uv run tmi-tf analyze <threat-model-id> [OPTIONS]

Options:

  • --max-repos INTEGER - Override maximum number of repositories to analyze
  • --dry-run - Analyze but do not create notes, diagrams, or threats (output to stdout)
  • --output PATH - Save markdown reports to files (generates <name>-inventory.md and <name>-analysis.md)
  • --profile TEXT - LLM profile from llm-profiles.yaml (default: LLM_PROFILE)
  • --force-auth - Force new authentication (ignore cached token)
  • --verbose - Enable verbose (DEBUG-level) logging
  • --skip-diagram - Skip generating data flow diagram
  • --skip-threats - Skip extracting and creating threat objects from security issues
  • --environment, -e TEXT - Pre-select a Terraform environment by name (skip interactive prompt)

Examples

Basic Analysis

Analyze a threat model and save results to TMI:

uv run tmi-tf analyze abc-123-def-456

Save to File

Analyze and save the reports to local files:

uv run tmi-tf analyze abc-123-def-456 --output report.md

This produces report-inventory.md and report-analysis.md.

Preview Without Saving

Dry run to preview analysis without creating artifacts in TMI:

uv run tmi-tf analyze abc-123-def-456 --dry-run

Limit Repositories

Analyze only the first repository with verbose logging:

uv run tmi-tf analyze abc-123-def-456 --max-repos 1 --verbose

Select a Specific Environment

When a repository contains multiple Terraform environments, pre-select one:

uv run tmi-tf analyze abc-123-def-456 --environment prod

Skip Diagram and Threats

Analyze infrastructure only, without generating DFD or threat objects:

uv run tmi-tf analyze abc-123-def-456 --skip-diagram --skip-threats

Full Analysis Workflow

# 1. Authenticate
uv run tmi-tf auth

# 2. List repositories to see what will be analyzed
uv run tmi-tf list-repos abc-123-def-456

# 3. Run analysis with verbose output
uv run tmi-tf analyze abc-123-def-456 --verbose --output analysis-$(date +%Y%m%d).md

How It Works

The analysis process follows these steps:

1. Authentication

Authenticates with TMI server using Google OAuth 2.0 (CLI) or client credentials (server mode). Opens browser for user consent in CLI mode, then caches the JWT token locally.

2. Discovery

Fetches the specified threat model and its associated repository references from TMI.

3. Filtering

Identifies GitHub repositories and filters to the first MAX_REPOS repositories.

4. Sparse Cloning

For each repository, performs a sparse clone to download only:

  • .tf files (Terraform configurations)
  • .tfvars files (Terraform variables)

This significantly reduces clone time and storage requirements.

5. Environment Detection

Detects multiple Terraform environments (root modules) within the cloned repository. If multiple environments are found, the CLI prompts the user for selection (or the --environment flag can pre-select one). If only one environment is found, it is auto-selected. In webhook mode, the addon invocation's user_data.environments selects the scope: latest (default; the environment with the newest commit within the last LATEST_COMMIT_DEPTH commits), all, or a comma-separated list of names/globs. Each matched environment runs as its own queue job.

6. Phased LLM Analysis

Sends Terraform code through a three-phase LLM pipeline:

Phase 1 - Inventory Extraction: Parses the Terraform statically (python-hcl2) into a pre-built inventory, then asks the LLM only for semantic details (names, purposes, services, dependencies) using security-attribute-filtered HCL. If nothing parses, it falls back to a full-LLM inventory prompt.

Phase 2 - Infrastructure Analysis: Analyzes component relationships, dependencies, data flows, trust boundaries, and architecture summary using the inventory from Phase 1.

Phase 3a - Threat Identification: Identifies potential security threats based on the inventory and infrastructure analysis.

Phase 3b - Per-Threat Analysis: For each identified threat, performs detailed STRIDE classification, CVSS 4.0 vector scoring, CWE identification, severity assessment, and mitigation recommendations. Each threat gets its own LLM call.

Script review: shell scripts and similar embedded in Terraform (such as user data and provisioners) get a separate check: static script rules, an isolated LLM script review, and a metadata prompt-injection scan. Strings flagged as suspected injection are redacted from all LLM input. Findings feed Phase 3b as threats (see Threat Provenance).

Retries and refusals: an empty LLM response that stopped normally is retried once. A response with finish_reason=content_filter is a refusal and is never treated as a clean result; see Model Refusals.

7. Report Generation

Generates two separate markdown reports:

  • Inventory Report -- Detailed listing of discovered infrastructure components and services
  • Analysis Report -- Infrastructure relationships, data flows, security observations, and threat modeling recommendations

8. Diagram Generation

Creates a data flow diagram (DFD) using LLM-generated structured component and flow data, then builds TMI-native diagram cells. The diagram is stored in the threat model.

9. Threat Creation

Creates structured threat objects in TMI from the security findings, including:

  • STRIDE classification
  • CVSS 4.0 vector and computed score
  • CWE identifiers
  • Severity (derived from CVSS score when available)
  • Mitigation recommendations
  • Links to the generated diagram (when available)
  • Provenance metadata for findings that came from script review, static rules, the injection scan or a model refusal (see Threat Provenance)

10. Metadata Attachment

Attaches LLM provenance metadata to all generated artifacts (notes, diagrams, threats): creation-agent, creation-timestamp, llm-provider, llm-model, llm-profile, input-tokens, output-tokens and cost-estimate-usd.

Analysis Output

The tool generates the following artifacts in your TMI threat model:

1. Inventory Report (Note)

Detailed listing of infrastructure components discovered from Terraform code:

  • Compute resources (EC2, Lambda, ECS, etc.)
  • Storage resources (S3, RDS, DynamoDB, etc.)
  • Network resources (VPC, subnets, security groups, etc.)
  • Security resources (IAM, KMS, secrets, etc.)
  • Services and managed offerings

2. Analysis Report (Note)

A comprehensive markdown report including:

  • Infrastructure relationships and dependencies
  • Data flow mapping
  • Trust boundaries
  • Security observations and concerns
  • Architecture summary
  • Threat modeling recommendations

3. Data Flow Diagram (DFD)

An interactive diagram showing:

  • Infrastructure components (processes, data stores, external entities)
  • Data flows between components
  • Trust boundaries and security zones

4. Threat Objects

Structured threat objects automatically extracted from security analysis, including:

  • Name: Clear, concise threat description
  • Type: STRIDE classification (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of Privilege); can be multi-valued
  • Description: Detailed threat description and risk assessment
  • Severity: Critical, High, Medium, or Low (derived from CVSS score when available)
  • Score: CVSS 4.0 base score (0.0-10.0)
  • CVSS: CVSS 4.0 vector string and computed score
  • CWE: Associated CWE identifiers (e.g., CWE-284)
  • Mitigation: Recommended security controls and remediation strategies
  • Status: Open (default for new threats)

Model Refusals

A model refusal (finish_reason=content_filter, even when partial text came back) is treated as "unreviewed", never as "clean":

  • Script review: a refused chunk of scripts is re-reviewed one script at a time. A script the model refuses on its own becomes a High-severity threat named "Model refused to review script in ..." whose description asks for manual review (a refusal usually means the script looks malicious). Static rules still apply to that script.
  • Phase 3b (per-threat analysis): a refused threat is skipped and counted, and the run continues.
  • Phases 1 and 2 (inventory, infrastructure analysis): a refusal fails the phase explicitly instead of producing an empty or clean-looking result.
  • DFD generation: a refusal produces no diagram, even when partial output came back. An empty response that stopped normally is retried once, like phases 1-3; if it is still empty, the diagram is skipped.
  • Reporting: refusal counts appear in the job status line and in the analysis note as "Model refusals (content_filter)" (for example script review: 1 script(s); phase 3b: 1 threat(s)). A DFD refusal adds the status line "Model refusals (content_filter): DFD generation", and the analysis note's job information lists it.

Optional: Jev Shadow Comparison

The service can compare its script-review findings against the Jev detector in shadow mode. Enable it with JEV_SHADOW=1 and JEV_API_KEY, and install the optional dependency with uv sync --extra jev. Secrets are masked before scripts are sent. The comparison never changes findings; its result appears as a jev_summary line in the job status note.

Threat Provenance (finding-source)

Threats that originate from the script and metadata checks carry extra threat metadata keys, in addition to the LLM provenance keys above (only non-empty values are written):

Metadata key Meaning
finding-source Where the finding came from (values below)
rule-id Rule or category that fired
script-digest Digest of the script or flagged string, so scripts are identified without embedding their text

finding-source values:

Value Origin rule-id
static-rule A static script rule matched The rule ID
script-review The isolated LLM script review The review category
llm-refusal The model refused to review the script (finding-source=llm-refusal) content_filter
injection-scan Metadata scan flagged a suspected prompt-injection string The detector name

Threats from the ordinary phase 3a/3b pipeline carry no finding-source key. To find everything that needs manual review because of a refusal, filter threat metadata for finding-source=llm-refusal.

Deployment

The maintained deployment target is AWS: an EKS deployment in the tmi-eks cluster (namespace tmi-tf) behind an ALB at https://webhook.tmi.dev/tf/webhook, with an SQS jobs queue, ECR image, and Terraform under infra/aws/ (see the repository's infra/aws/README.md). Images are built from deploy/docker/Dockerfile.aws (Amazon Linux 2023) and pushed with scripts/push-aws.sh. deploy/docker/Dockerfile.local (Wolfi) with docker-compose.yml supports local development.

The GCP, Azure, Heroku and OCI (OKE) Dockerfiles, infra/oci and the OCI push script were removed (ADR 0001, 2026-09-28). The OCI LLM provider and the OCI Vault/Queue provider code remain in tmi_tf/providers/, but OCI is not a deployment target.

Project Structure

tmi-tf-wh/
├── tmi_tf/
│   ├── __init__.py
│   ├── cli.py                  # CLI interface (Click commands)
│   ├── config.py               # Configuration management
│   ├── auth.py                 # OAuth authentication (Google PKCE, client credentials)
│   ├── analyzer.py             # Shared analysis pipeline (CLI + webhook)
│   ├── tmi_client_wrapper.py   # TMI API client
│   ├── github_client.py        # GitHub API integration
│   ├── repo_analyzer.py        # Repository cloning, sparse checkout, environment detection
│   ├── llm_profiles.py         # llm-profiles.yaml loading and validation
│   ├── tf_parser.py            # Static HCL parsing (python-hcl2)
│   ├── tf_filter.py            # Registry-driven filtering and inventory merge
│   ├── script_scan.py          # Static script rules
│   ├── script_review.py        # Isolated LLM script review, finding merge, refusals
│   ├── metadata_scan.py        # Prompt-injection scan of metadata strings
│   ├── llm_analyzer.py         # Phased LLM analysis pipeline (via LiteLLM)
│   ├── markdown_generator.py   # Report generation (inventory + analysis)
│   ├── dfd_llm_generator.py    # LLM-based DFD component/flow generation
│   ├── diagram_builder.py      # DFD cell builder (TMI-native format)
│   ├── threat_processor.py     # Threat extraction, STRIDE/CVSS/CWE classification
│   ├── cvss_scorer.py          # CVSS 4.0 vector validation and scoring
│   ├── tf_validator.py         # Terraform file validation and sanitization
│   ├── artifact_metadata.py    # LLM provenance metadata for artifacts
│   ├── retry.py                # Transient LLM error retry logic
│   ├── server.py               # FastAPI webhook server
│   ├── webhook_handler.py      # Webhook payload parsing and HMAC validation
│   ├── worker.py               # Async worker pool for analysis jobs
│   ├── job.py                  # Job tracking and lifecycle
│   ├── addon_callback.py       # Addon status callback
│   └── providers/
│       ├── __init__.py          # Secret, queue and LLM provider registry
│       ├── aws.py               # AWS SQS queue provider
│       ├── api_key.py           # API-key LLM provider
│       ├── oci.py               # OCI Vault/Queue/LLM providers (legacy, not deployed)
│       ├── memory.py            # In-memory queue (tests, local)
│       └── none.py              # No-op secret provider
├── prompts/
│   ├── inventory_system.txt                # Phase 1 system prompt
│   ├── inventory_user.txt                  # Phase 1 user prompt template
│   ├── infrastructure_analysis_system.txt  # Phase 2 system prompt
│   ├── infrastructure_analysis_user.txt    # Phase 2 user prompt template
│   ├── threat_identification_system.txt    # Phase 3a system prompt
│   ├── threat_identification_user.txt      # Phase 3a user prompt template
│   ├── threat_analysis_system.txt          # Phase 3b system prompt
│   ├── threat_analysis_user.txt            # Phase 3b user prompt template
│   ├── dfd_generation_system.txt           # DFD generation system prompt
│   ├── dfd_generation_user.txt             # DFD generation user prompt template
│   ├── terraform_analysis_system.txt       # Legacy single-pass system prompt
│   └── terraform_analysis_user.txt         # Legacy single-pass user prompt template
├── scripts/
│   ├── push-aws.sh             # Build and push the image to ECR
│   └── sync-llm-keys-aws.sh    # Sync rotated LLM keys to the EKS Secret
├── tests/                      # Test suite
├── infra/aws/                  # Terraform for the EKS deployment
├── docs/                       # Additional documentation
├── deploy/docker/              # Dockerfile.aws (EKS) and Dockerfile.local
├── llm-profiles.yaml           # Named LLM profiles
├── .env                        # Environment configuration (not in git)
├── .env.example                # Example environment file
├── pyproject.toml              # Project dependencies
├── uv.lock                     # UV lock file
├── LICENSE                     # Apache License 2.0
├── CLAUDE.md                   # Claude Code instructions
└── README.md

Customization

Modifying Analysis Prompts

Edit the prompt templates in the prompts/ directory to customize what the LLM analyzes. The analysis uses separate prompt pairs for each phase:

Phase 1 - Inventory (prompts/inventory_system.txt, prompts/inventory_user.txt):

  • Defines how infrastructure components and services are extracted from Terraform code

Phase 2 - Infrastructure (prompts/infrastructure_analysis_system.txt, prompts/infrastructure_analysis_user.txt):

  • Defines how relationships, data flows, and trust boundaries are identified

Phase 3a - Threat Identification (prompts/threat_identification_system.txt, prompts/threat_identification_user.txt):

  • Defines how security threats are identified from the inventory and infrastructure analysis

Phase 3b - Threat Analysis (prompts/threat_analysis_system.txt, prompts/threat_analysis_user.txt):

  • Defines how each threat is analyzed for STRIDE classification, CVSS 4.0 scoring, CWE mapping, and mitigation

DFD Generation (prompts/dfd_generation_system.txt, prompts/dfd_generation_user.txt):

  • Defines how data flow diagram components and flows are generated from structured analysis data

Adjusting Analysis Scope

Modify sparse checkout patterns in repo_analyzer.py to include/exclude file types:

patterns = [
    "*.tf",           # Terraform files
    "*.tfvars",       # Terraform variables
    # Add more patterns as needed:
    # "*.yaml",       # Kubernetes manifests
    # "*.yml",        # CI/CD configs
    # "Dockerfile*",  # Docker files
]

Switching LLM Providers

Change the default profile in .env:

LLM_PROFILE=mythos5
ANTHROPIC_CYBER_API_KEY=your_key_here   # env var named by the profile's api_key

Or pass --profile <name> per run, or add a profile to llm-profiles.yaml.

Troubleshooting

Authentication Issues

Problem: OAuth flow fails or token is rejected

Solution: Clear cached token and re-authenticate:

uv run tmi-tf clear-auth
uv run tmi-tf auth

Problem: "Token expired" error

Solution: Tokens expire after a set period. Force re-authentication:

uv run tmi-tf analyze <tm-id> --force-auth

Rate Limits

GitHub API Rate Limits:

  • Unauthenticated: 60 requests/hour
  • Authenticated: 5,000 requests/hour

Solution: Set GITHUB_TOKEN in .env with a personal access token.

LLM API Rate Limits:

  • Varies by provider and account tier (Anthropic, OpenAI, x.ai, Google)
  • Tool implements exponential backoff retry logic for transient errors

Solution: Check your provider's account limits. Upgrade if needed or reduce MAX_REPOS.

Clone Timeouts

Problem: Large repositories timeout during clone

Solution: Increase timeout in .env:

CLONE_TIMEOUT=600  # 10 minutes

Or exclude problematic repositories from the threat model.

Large Repositories

Problem: Analysis fails or is truncated for very large Terraform codebases

Cause: LLMs have varying context windows. When the LLM response is truncated, the tool logs a warning with finish_reason=length.

Solutions:

  1. Reduce MAX_REPOS to analyze fewer repositories
  2. Use --environment to analyze one environment at a time
  3. Split large Terraform files into smaller modules
  4. Use --max-repos 1 to analyze one repository at a time

Analysis Quality Issues

Problem: Analysis is too generic or misses important details

Solution: Customize prompts to be more specific:

  • Edit the phase-specific prompt files in prompts/ to add domain expertise
  • Add examples of good analysis to the prompts
  • Try a different LLM profile (LLM_PROFILE or --profile)

Environment Detection Issues

Problem: Tool does not detect environments or selects the wrong one

Solution: Use the --environment flag to explicitly select an environment by name. Run with --verbose to see which environments were detected and how they were resolved.

Limitations & Considerations

Technical Limitations

  • Proof of Concept -- This is a PoC tool, not production-ready
  • Token Limits -- LLMs have varying context windows (varies by model); very large files may be truncated
  • GitHub Only -- Currently supports only GitHub repositories (not GitLab, Bitbucket, etc.)
  • Public Repos -- Best suited for public repositories; private repos require GitHub authentication
  • Sequential Processing -- Repositories are analyzed sequentially (not parallelized)
  • No State Management -- No resume capability if analysis fails mid-way

Security Considerations

  • API Keys -- Never commit the .env file; it contains sensitive credentials
  • Token Cache -- OAuth tokens are cached in ~/.tmi-tf/token.json
  • Temporary Files -- Cloned repositories are stored in temp directories and cleaned up automatically
  • LLM Response Files -- Raw LLM responses are saved to a session temp directory for debugging
  • Network Security -- All API calls use HTTPS
  • Webhook Validation -- Server mode validates HMAC signatures on incoming webhook payloads
  • AI Limitations -- LLM analysis should complement, not replace, human security review

Cost Considerations

  • LLM API -- Each analysis incurs API costs based on tokens processed. The phased pipeline makes multiple LLM calls per repository (inventory, infrastructure, threat identification, plus one call per identified threat). Cost estimates are logged and attached as metadata to generated artifacts
  • GitHub API -- Free with authentication token (5,000 requests/hour)
  • Storage -- Sparse cloning minimizes storage, but multiple analyses accumulate temporary files

Operational Considerations

  • Analysis Time -- Can take several minutes per repository depending on size and number of threats identified
  • Network Dependency -- Requires a stable internet connection for API calls
  • Authentication -- OAuth tokens expire and require periodic re-authentication
  • Phased Pipeline -- If a phase fails, subsequent phases are skipped for that repository

Best Practices

For Accurate Analysis

  1. Add Repository Context -- Include README files and architecture docs in repositories
  2. Use Terraform Modules -- Well-structured modules improve analysis quality
  3. Document Decisions -- Add comments in Terraform explaining security decisions
  4. Review Analysis -- Always manually review LLM findings for accuracy

For Efficient Analysis

  1. Start Small -- Begin with MAX_REPOS=1 to test and refine prompts
  2. Use Dry Run -- Test with --dry-run before creating artifacts in TMI
  3. Save to Files -- Use --output to keep historical analysis records
  4. Limit Scope -- Only add relevant repositories to threat models
  5. Select Environment -- Use --environment to analyze one environment at a time

For Security

  1. Protect API Keys -- Never commit the .env file
  2. Use Read-Only Tokens -- GitHub token only needs repo read access
  3. Review Before Sharing -- Analysis reports may contain sensitive infrastructure details
  4. Regular Updates -- Re-run analysis when infrastructure changes

Future Enhancements

Potential improvements for future versions:

  • Support for other Git providers (GitLab, Bitbucket)
  • Parallel repository processing
  • Resume capability for long-running analyses
  • Terraform state file analysis
  • Integration with terraform security scanners (tfsec, checkov)
  • Custom analysis rules and filters
  • Incremental analysis (only changed files)
  • Multi-cloud support (AWS, Azure, GCP specific analysis)
  • Cost estimation integration
  • Compliance framework mapping (PCI-DSS, HIPAA, SOC 2)

Contributing

This is a proof-of-concept tool. Contributions welcome for:

  • Additional cloud provider support
  • Enhanced analysis prompts
  • Performance improvements
  • Additional output formats
  • Integration with other security tools

Related Pages

Support

For issues and questions:

  1. Check logs with --verbose flag
  2. Review configuration with config-info command
  3. Ensure all prerequisites are installed
  4. Verify TMI server accessibility
  5. Check LLM API key validity for your selected provider
  6. See Getting-Help for support channels

Version: 0.1.0 Status: Proof of Concept License: Apache License 2.0

⚠️ **GitHub.com Fallback** ⚠️