GraphRAG How It Works User Guide - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki

How the MCP-RAG System Traverses Code: A User-Level Guide

Date: July 22, 2026 Audience: Developers, researchers, and operators new to the MDC MCP-RAG platform Context: Explains item #6 from the Platform-Parity-Check-AWS-vs-COTS-2026-07-22 — why setuprad returns 55 direct neighbors identically on both AWS and COTS, and what "GraphRAG" actually means in this system.

The Big Picture

When you ask "what calls setuprad?" or "trace the execution chain from JGLOBAL_FORECAST", the system doesn't just search text — it walks a knowledge graph. Think of it like asking for directions in a city: the graph is the street map (functions connected by "calls" relationships), and the traversal is the act of following those streets to find every path that leads to or from your destination.

The result: 55 direct neighbors of setuprad on both platforms — qc_amsua, init_crtm, call_crtm, nc_diag_write, setup_sst_retrieval, etc. These are the actual call-graph relationships extracted from the Fortran source code, not keyword matches. The system knows setuprad calls init_crtm because it parsed the CALL init_crtm(...) statement in the source.

Three Layers Make This "GraphRAG"

The system combines three distinct capabilities:

┌─────────────────────────────────────────────────────────────────┐
│  Layer 1: GRAPH CONSTRUCTION (offline, at ingest time)          │
│  "Building the map from source code"                            │
│                                                                 │
│  Fortran parser  → CALLS / USES / CONTAINS edges                │
│  Shell parser    → SOURCES / INVOKES / EXPORTS / DEPENDS_ON_ENV │
│  Bridge builder  → Shell-to-Fortran EXECUTES edges              │
│  Rocoto parser   → Workflow DAG (task dependencies)             │
│  Python parser   → IMPORTS / DEFINES edges                      │
└────────────────────────────────────┬────────────────────────────┘
                                     │ writes to
                                     ▼
┌─────────────────────────────────────────────────────────────────┐
│  Layer 2: GRAPH STORAGE + TRAVERSAL (runtime queries)           │
│  "The map, and how you walk it"                                 │
│                                                                 │
│  COTS: Neo4j Community + Graph Data Science (GDS 2.13.7)        │
│  AWS:  Amazon Neptune (managed, openCypher)                     │
└────────────────────────────────────┬────────────────────────────┘
                                     │ queried by
                                     ▼
┌─────────────────────────────────────────────────────────────────┐
│  Layer 3: GRAPH-GUIDED RETRIEVAL (the "RAG" part)               │
│  "Using the map to answer your question"                        │
│                                                                 │
│  MCP tools combine graph walks with vector search               │
│  to give you context-aware, relationship-aware answers          │
└─────────────────────────────────────────────────────────────────┘

Layer 1: How the Graph Gets Built

Before you can traverse code, someone has to read every source file and extract the relationships. This happens offline via ingestion scripts:

Parser	What it reads	Edges it creates	Example
Fortran (`ingest_fortran_graph_v8.py`)	`.f90`, `.F90` files under `sorc/`	`CALLS`, `USES`, `CONTAINS`, `DEFINES`	`setuprad` CALLS `init_crtm`
Shell (`ingest_shell_graph_v8.py`)	`.sh`, `.ksh` under `ush/`, `scripts/`, `jobs/`	`SOURCES`, `INVOKES`, `EXPORTS`, `DEPENDS_ON_ENV`	`exgfs_atmos_analysis.sh` SOURCES `parsing_namelists_FV3.sh`
Bridge (`create_shell_fortran_bridge.py`)	Cross-references shell→Fortran	`EXECUTES`	`JGLOBAL_FORECAST` EXECUTES `ufs_model`
Python (`ingest_python_graph.py`)	`.py` under `ush/`, `workflow/`	`IMPORTS`, `DEFINES`	`wxflow.task` IMPORTS `datetime`
Rocoto (`ingest_rocoto_xml_v8.py`)	`*.xml` workflow definitions	`DEPENDS_ON`, `SETS_ENV`	`gdas_analysis` DEPENDS_ON `gdas_prep`
Config (`ingest_config_files_v8.py`)	`parm/config/config.*`	`SETS_ENV`, `READS_CONFIG`	`config.fcst` SETS_ENV `FHMAX`

The result for the gw (develop) tenant today:

Platform	Nodes	Relationships	Fortran subroutines	Shell scripts
COTS (Neo4j)	225,836	4,051,374	80,745	589
AWS (Neptune)	148,976	4,555,408	27,941	315

Why the difference? COTS ran the regex-fallback parser (catches 99.9% of Fortran files including ones that trip fparser2's strict AST). AWS ran fparser2 only (fewer nodes but richer AST-level DEFINES edges — hence more relationships). Both are correct; they're different parser depths on the same source code.

Layer 2: How the Graph Gets Traversed

Once the graph exists, the MCP tools walk it at runtime. This is where the Neo4j vs Neptune gap matters.

What both do identically (openCypher)

Both Neo4j and Neptune speak openCypher — a graph query language. When you call find_callers_callees("setuprad"), the system runs a query like:

MATCH (target:FortranSubroutine {name: "setuprad"})
MATCH (target)-[r]-(neighbor)
RETURN neighbor.name, type(r)

This is why setuprad returns the same 55 neighbors on both — the query is identical, the data is equivalent, and the traversal logic is the same.

Where they diverge: algorithms

Capability	Neo4j (COTS)	Neptune (AWS)
Basic traversal (MATCH paths, shortest path)	Yes	Yes
Variable-length paths (`-[:CALLS*1..5]->`)	Yes	Yes
Fan-out protection (degree > 100 → one-hop only)	Yes (our code)	Yes (our code)
Depth-bounded traversal (max_depth parameter)	Yes	Yes
Graph Data Science (GDS) — community detection (Leiden), PageRank, centrality, similarity	Yes (GDS 2.13.7)	No
Statement timeout backstop (30s kill)	N/A (local, fast)	Yes (prevents Neptune OOMs)

The GDS gap is the one we discovered a few weeks ago (documented as Gap J in the multi-tenant gap tracker). It means:

On COTS, we can run Leiden community detection to group related functions/modules into "subsystems" (e.g. "the GSI EnKF subsystem", "the atmospheric forecast model subsystem") — these power search_architecture and get_code_context(..., include_community=True).
On AWS, community detection doesn't exist as a server-side algorithm. Community summaries were migrated from a prior Neo4j run (2,113 entries) but can't be regenerated natively. The workaround (planned for Q3) is to export the graph, run Leiden in Python (leidenalg + igraph), and write the results back.

The traversal tools you can use

MCP Tool	What it does	Graph operation
`find_callers_callees`	Who calls this function? What does it call?	1-hop neighbor walk
`find_dependencies`	What files/modules does this file import? Who imports it?	Multi-hop IMPORTS/USES walk
`trace_execution_path`	Follow the call chain from a function N levels deep	Recursive depth-bounded walk
`trace_full_execution_chain`	Cross Shell → Fortran → Python boundaries	Multi-label path walk (EXECUTES + CALLS + SOURCES)
`find_env_dependencies`	What scripts depend on env var `$X`?	DEPENDS_ON_ENV / EXPORTS edge walk
`get_change_impact`	If I change this function, what breaks?	Reverse-direction fan-out (blast radius)
`search_architecture`	What subsystem does this belong to?	Community membership lookup (GDS-derived, COTS only for new ones)

Layer 3: Graph-Guided Retrieval (the "RAG" fusion)

The "RAG" in GraphRAG means the system combines graph traversal with vector (semantic) search to answer questions. Here's the flow when you ask something:

Your question: "How does setuprad handle quality control for AMSU-A?"
                          │
          ┌───────────────┼───────────────────┐
          │               │                   │
          ▼               ▼                   ▼
   Vector Search     Graph Walk         Code Lookup
   (embeddings)      (relationships)    (file content)
          │               │                   │
   "docs about       "setuprad calls    "the actual
    AMSU-A QC"        qc_amsua"          source lines"
          │               │                   │
          └───────────────┼───────────────────┘
                          │
                          ▼
              Combined, context-rich answer
              with documentation + call graph + code

This is more powerful than either approach alone:

Vector-only (plain RAG) would find docs mentioning "AMSU-A" but wouldn't know the call relationship.
Graph-only would know setuprad calls qc_amsua but couldn't explain what the QC does.
GraphRAG gives you both: the structural relationship AND the semantic context, so the answer includes "setuprad calls qc_amsua (line 847), which performs channel-by-channel quality control for AMSU-A brightness temperatures using a gross-check + cloud-detection algorithm."

The `setuprad` Example — What the Graph Reveals

When you asked find_callers_callees("setuprad"), here's what the graph showed:

                    ┌─── qc_amsua (AMSU-A QC)
                    ├─── qc_mhs (MHS QC)
                    ├─── qc_atms (ATMS QC)
                    ├─── qc_irsnd (IR sounder QC)
                    ├─── qc_avhrr (AVHRR QC)
                    ├─── qc_ssmi (SSM/I QC)
                    ├─── qc_gmi (GMI QC)
     setuprad ──────├─── init_crtm (radiative transfer init)
     (degree 174    ├─── call_crtm (forward model)
      on COTS,      ├─── destroy_crtm (cleanup)
      193 on AWS)   ├─── nc_diag_init (diagnostic output)
                    ├─── nc_diag_write (write diagnostics)
                    ├─── setup_sst_retrieval (SST)
                    ├─── radiance_obstype_search (obs lookup)
                    ├─── comp_fact10 (10m wind factor)
                    ├─── dtime_setup / dtime_check (timing)
                    └─── ... (55 total direct neighbors)

This tells you that setuprad is the central radiance observation processing routine in GSI — it orchestrates quality control for every satellite instrument type, manages the CRTM (Community Radiative Transfer Model) lifecycle, and writes observation diagnostics. A change to setuprad has a blast radius of 55+ direct dependents across QC, radiative transfer, and I/O.

The degree difference (174 on COTS vs 193 on AWS) reflects the parser depth: AWS Neptune has additional DEFINES edges from fparser2's AST containment analysis (functions defined within modules), which adds to the degree count without changing the set of called/calling functions.

The Neo4j vs Neptune Gap — What It Means for You

As a user, you mostly won't notice the difference — the same tools, the same queries, the same setuprad neighbors. The gap matters in two specific scenarios:

Scenario	COTS (Neo4j + GDS)	AWS (Neptune)
"What subsystem does this function belong to?"	Full answer — Leiden community detection groups functions into subsystems	Partial — uses migrated community summaries (2,113 from Q1); can't regenerate
"Show me the most central functions in the codebase"	Full answer — PageRank / betweenness centrality via GDS	Not available — no server-side algorithm catalog
"Find functions similar to X by graph structure"	Full answer — node similarity via GDS	Not available
"What calls setuprad?"	Identical	Identical
"Trace the full execution chain from JGLOBAL_FORECAST"	Identical	Identical (with 30s timeout backstop)
"What depends on $FHMAX?"	Identical	Identical

Bottom line: For day-to-day code analysis (call graphs, dependencies, env vars, execution chains, change impact), both platforms are equivalent. For advanced graph analytics (community detection, centrality, structural similarity), COTS has capabilities that AWS currently lacks — planned for Q3 via an external Leiden runner.

Quick Reference: Tools to Try

What you want to know	Tool to use	Example
What calls this function?	`find_callers_callees`	`find_callers_callees("setuprad")`
What does this file import?	`find_dependencies`	`find_dependencies("ush/forecast_predet.sh")`
Full execution chain across languages	`trace_full_execution_chain`	`trace_full_execution_chain("JGLOBAL_FORECAST")`
What uses this env variable?	`find_env_dependencies`	`find_env_dependencies("FHMAX")`
What breaks if I change this?	`get_change_impact`	`get_change_impact("setuprad")`
What subsystem is this part of?	`search_architecture`	`search_architecture("radiance processing")`
Explain this component with full context	`get_code_context`	`get_code_context("setuprad")`

All tools accept tenant_id to query specific branches: tenant_id="gw_v17" for the GFS v17 development branch, or omit it for the develop baseline.