GraphRAG Gaps Detected Jul 2026 - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki
GraphRAG Gaps Detected (Jul 8β22, 2026)
Date: July 22, 2026 Scope: All GraphRAG-specific gaps surfaced in the last two weeks, pulled from parity checks, SDD phases, and the multi-tenant gap tracker. Status: Working document β not checked in.
Gap Inventory
| # | Gap | When Found | Platform | Status |
|---|---|---|---|---|
| G1 | Neo4j GDS vs Neptune: no community detection on AWS β Neptune has no gds.leiden, gds.louvain, gds.pageRank. Community summaries (2,113 on AWS) were migrated from a prior Neo4j run and cannot be regenerated natively. |
Jun 11 (documented); re-confirmed Jul 14β22 | AWS | OPEN |
| G2 | Neptune deep-traversal OOM / timeout β highly-connected nodes (like setuprad, degree 193) can blow up unbounded path queries. |
Jun 10 (resolved); re-verified Jul 22 | AWS | RESOLVED |
| G3 | COTS gw_v17 community summaries empty β gw_v17_mdc-community-summaries-titan1024 = 0 docs on AWS; COTS has the gw baseline (2,113) but nothing v17-specific. |
Jul 14 | Both | OPEN |
| G4 | Coverage Gap check couldn't see Fortran graph β integrity check [SKIP]ped due to hard-coded stale path, masking whether the graph covered the source tree. |
Jul 14 (AWS); Jul 20 (COTS) | Both | CLOSED on COTS; OPEN on AWS |
| G5 | Graph node-count scope confusion β three tools reported 344K / 226K / 108K with no explanation. | Jul 20 | Both | CLOSED on COTS; OPEN on AWS |
| G6 | COTS +76K nodes but -504K rels vs AWS β parser depth divergence (regex-fallback vs fparser2 AST). Not a bug but undocumented. | Jul 14β15 | Both | DOCUMENTED |
| G7 | ShellβFortran bridge sparsity on AWS β only 97 EXECUTES edges on Neptune vs 129 on COTS (16 attempts, 36 unmatched refs). |
Jul 14 | AWS | KNOWN |
| G8 | gw_v17 graph slightly richer on COTS (+18 nodes, +1,591 rels) β COTS shell-graph ran more recently. |
Jul 14β22 | COTS leads | EXPECTED |
| G9 | Stale graph (no re-ingest since ~May) β both platforms' graph edges reflect a point-in-time parse of develop. New commits not represented. |
Jul 22 | Both | OPEN |
| G10 | search_architecture returns nothing for gw_v17 β community-summaries index empty for non-default tenants. |
Jul 14 | Both | OPEN |
Detail: Each Gap
G1 β Neo4j GDS vs Neptune (Gap J)
What: Amazon Neptune speaks openCypher but does not ship the Graph Data
Science (GDS) plugin. Neo4j Community on COTS does have GDS 2.13.7. The
pipeline that produces community summaries (mcp_server_node/scripts/ run_community_detection.js) calls gds.graph.project, gds.leiden.write, and
gds.louvain.stream β procedures that simply don't exist on Neptune.
Impact: search_architecture("radiance processing") works on COTS (returns
community summaries like "GSI EnKF subsystem") but would return stale/migrated
data on AWS and nothing at all for non-default tenants.
Resolution: Q3 β Option A (recommended): export the graph, run Leiden in
Python (leidenalg + igraph), write communityId properties + Community
nodes back to Neptune. ~10-30 min for an 80K-node graph.
Where documented: .kiro/steering/12-multi-tenant-gap-tracker.md Gap J.
G2 β Neptune Deep-Traversal OOM (RESOLVED)
What: trace_full_execution_chain("JGLOBAL_FORECAST") on a 500+ degree hub
could timeout or OOM on Neptune.
Fix deployed: Pre-flight degree probe + depth cap + 30s statement-timeout
backstop in NeptuneAdapter.query. setuprad (degree 193 on Neptune) now
short-circuits cleanly with a one-hop Degraded_Result when it exceeds the
fan-out threshold (100).
Where documented: Gap G in gap tracker (RESOLVED, [8.36.0]).
G3 β v17 Community Summaries Empty
What: gw_v17_mdc-community-summaries-titan1024 = 0 on AWS.
mdc-community-summaries-mpnet768 on COTS has 2,113 entries but only for the
gw baseline β no v17-specific communities exist on either platform.
Impact: get_code_context("setuprad", tenant_id="gw_v17", include_community=True) returns no subsystem context for v17.
Resolution: Same as G1 β the community pipeline needs to run per-tenant after the Leiden port to Neptune/Python.
G4 β Coverage Gap Check Couldn't See the Graph (CLOSED on COTS)
What: check_knowledge_integrity Coverage Gap sub-check hard-coded
/app/supported_repos/global-workflow (a stale pre-Phase-61 path) and always
[SKIP]ped despite 107K+ Fortran nodes in the graph.
Fix (Phase 72): Resolves via tenant_ctx.workflow_root; graph-based fallback
when filesystem unavailable; extends to Fortran + Python + Shell.
Result on COTS (live):
Coverage Gap (Fortran) | [OK] 107794 Fortran nodes for 7242 files under sorc/
Coverage Gap (Python) | [OK] 8607 Python nodes for 41 files under ush/workflow/
Coverage Gap (Shell) | [OK] 589 Shell nodes for 114 files under ush/scripts/jobs/
AWS status: Still [SKIP] β needs the agentcore-image-rebuild-v3 deploy.
G5 β Node-Count Scope Confusion (CLOSED on COTS)
What: Three tools reported three numbers (344,604 / 225,836 / 108,280) with no labeling. First-time users couldn't tell which to trust.
Fix (Phase 73): Scope annotations on every count + all_tenants=True flag +
docs/development/graph_node_count_scopes.md.
Result on COTS (live):
Graph Database: 108,280 nodes (health-check scope)
Total Nodes (tenant gw): 225,836
Total Nodes (all tenants, all labels): 344,604
AWS status: Still unlabeled β needs image rebuild.
G6 β Parser Depth Divergence (DOCUMENTED, not a bug)
What: COTS has 225,836 nodes (deeper Fortran regex-fallback parse catches 99.9% of files). AWS has 148,976 nodes (fparser2 strict AST, ~85% parse rate). AWS has 504K more relationships (DEFINES edges from AST containment that the regex fallback doesn't produce).
Why both are correct: Different parser depths on the same source code. The regex fallback captures CALL/USE relationships (the important ones for code analysis) from files that trip fparser2's strict parser (preprocessor macros, non-standard extensions).
Resolution: Post-framework re-ingest, both will run the same hybrid pipeline (fparser2 + regex fallback) and converge.
G7 β ShellβFortran Bridge Sparsity
What: AWS Neptune has 97 EXECUTES edges; COTS Neo4j has 129. The bridge
builder (create_shell_fortran_bridge.py) found 16 matchable pairs on the AWS
run (out of 36 shell references to Fortran executables β 20 pointed at
executables not in the graph at that time).
Why: The bridge runs after both the shell and Fortran graph ingesters. If the Fortran graph is smaller (AWS: 27,941 subroutines vs COTS: 80,745), fewer bridge targets are findable.
Resolution: Re-run the bridge after the next full Fortran ingest on AWS
(cots-reingest-ralph-framework bridge stage).
G8 β v17 Graph COTS Slightly Richer (EXPECTED)
What: COTS gw_v17 has +18 nodes and +1,591 relationships vs AWS. The delta
is concentrated in SOURCES (+551), DEPENDS_ON_ENV (+571), INVOKES (+464),
and ShellScript (+18 nodes).
Why: The COTS shell-graph ingester ran more recently (Jul 9 PoC) than the AWS v17 ingest (May timeframe). A few scripts were added to the branch between runs.
Resolution: Self-resolves when both platforms re-ingest against the same branch HEAD.
G9 β Stale Graph (No Re-Ingest Since ~May)
What: Both platforms' graph relationships reflect a point-in-time parse of the
develop branch from ~May 2026. Any commits since then (new functions, renamed
files, deleted modules) aren't represented in the graph.
Impact: A developer asking "what calls my new function added last week?" gets no results β the function isn't in the graph yet.
Resolution: The cots-reingest-ralph-framework (COTS, 44 pending units) and
a future AWS graph re-ingest. The doc refresh spec (aws-doc-refresh-ingest)
addresses vector staleness but not graph staleness on AWS.
G10 β search_architecture Empty for Non-Default Tenants
What: search_architecture(query, tenant_id="gw_v17") returns no results
because the community-summaries vector index for v17 is empty (0 docs). Same for
gw_sfs, gw_jedi_gfs, gw_gefs_v12.
Impact: The "high-level subsystem understanding" capability works for the
develop baseline but not for any other tenant/branch.
Resolution: Same dependency chain as G1/G3 β the community pipeline needs:
- Leiden β tenant-scoped community detection
- LLM summarization β per-community prose
- Embed + write β tenant-prefixed community-summaries index
All three steps are documented in Gap J (the Q3 pipeline port).
Summary by Status
| Status | Count | Gaps |
|---|---|---|
| CLOSED (COTS) | 2 | G4, G5 |
| RESOLVED (both) | 1 | G2 |
| DOCUMENTED (not a bug) | 2 | G6, G8 |
| OPEN (needs action) | 5 | G1, G3, G7, G9, G10 |
What Closes the Open Gaps
| Gap | Resolution | Spec / Tracker |
|---|---|---|
| G1, G3, G10 | External Leiden community detection (Python leidenalg) |
Gap J, Q3 plan |
| G4 (AWS) | AgentCore image rebuild β python-tenants-v3 |
agentcore-image-rebuild-v3 |
| G5 (AWS) | Same image rebuild | Same spec |
| G7 | Re-run bridge after full Fortran re-ingest on AWS | cots-reingest-ralph-framework bridge stage |
| G9 | Full re-ingest on both platforms | Framework spec (COTS) + future AWS graph re-ingest |
The Biggest Remaining Gap
G1/G3/G10 β the GDS community-detection absence on Neptune (and by extension for non-default tenants on COTS). This is the only gap that removes an entire class of capability (subsystem-level architecture queries) rather than just degrading precision or freshness. Everything else is closed, documented, or has a concrete spec ready to execute.
Working document β not checked in. Generated July 22, 2026.