NIH AWS MCP RAG Extension Proposal - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki

NIH AWS Sandbox — MDC MCP-RAG Extension Proposal

Date: June 25, 2026 Author: Terry McGuinness, NOAA NWS POCAI Software Engineering (OMD/CAT Unit) Audience: NIH AWS Sandbox sponsors, NOAA Office of Mission Delivery (OMD) management Status: Prototype complete — requesting extension to operationalize external access

Executive Summary

Under the NIH AWS Sandbox funding envelope, the OMD CAT Unit has delivered a fully operational AWS-native AI assistance platform for the NOAA Global Workflow — the MDC MCP-RAG Server. The platform exposes 51 tools across 9 modules over the Model Context Protocol (MCP), backed by a 148K-node code graph and 206K-document vector store. The initial prototype is now production-quality on AWS and serves developers from their IDEs via a private VPC path.

We are requesting an extension of the NIH AWS Sandbox engagement to complete the one major piece of unfinished work: secure exposure of the MCP endpoint to two additional consumer classes that were always part of the strategic vision but out of scope for the prototype — GitHub Actions CI/CD pipelines and RDHPCS researcher sessions (Hera, Orion, Hercules, Gaea, Ursa). The design is already authored (MCP-External-Access-Design-Path-B); this proposal funds the build-out, hardening, and rollout.

What We Built (Prototype Accomplishments)

A complete description of the deployed system is in MDC-MCP-RAG-AWS-Architecture-v3. Headline accomplishments:

  • 51 MCP tools across Workflow Info, Code Analysis, Semantic Search, EE2 Compliance, Operational Guidance, GraphRAG, GitHub Integration, SDD Workflow, and Utility modules — all running on AWS-native backends with zero tool-code changes courtesy of the adapter pattern.
  • Hybrid Graph-Guided Semantic Retrieval (GGSR) — combines Amazon Neptune structural traversal with Amazon OpenSearch hybrid (BM25 + k-NN) ranking, giving answers no grep can produce.
  • Knowledge base coverage — Neptune: 148,723 nodes / 2,820,440 relationships across Shell, Python, and Fortran; OpenSearch: 206,341 documents across 17 indices, three embedding families (MPNet 768-dim, Titan 1024-dim, Nova 1024-dim).
  • Multi-tenant catalog — gw (develop), gw_sfs, gw_jedi_gfs, gw_v17, gw_gefs_v12 with per-tenant Neptune label and OpenSearch index isolation.
  • Infrastructure as Code — three CDK stacks (MdcVpcStack, MdcSecurityStack, MdcDataStack) with RemovalPolicy: RETAIN data-safety guardrails after the April 22 post-mortem; no manual console changes.
  • Quality metrics — RAG P@5 = 0.71, MRR = 0.93, 93% coverage, P95 latency 135 ms. All 9 module health checks and 1,234 unit tests green.

AWS Services in Use (Prototype)

Tier AWS Service Role
Compute (MCP host) Amazon Bedrock AgentCore Runtime (Firecracker microVM, MCP/Streamable-HTTP) Hosts the 51-tool server with per-session isolation
Embeddings Amazon Bedrock (amazon.titan-embed-text-v2:0, Nova) Production embedding generation
Graph database Amazon Neptune (db.r8g.xlarge, openCypher, SigV4 IAM) Code structure graph
Vector database Amazon OpenSearch (2× r6g.large.search, zone-aware, k-NN + BM25) Documentation + code semantic search
Compute (dev/ingest) Amazon EC2 (c6g.xlarge ARM64) Developer workstation, ingestion driver
Container registry Amazon ECR (mdc-mcp-rag repo, ARM64) AgentCore container image
File system Amazon EFS (encrypted, 30-day lifecycle) Workflow worktrees, persistent state
Object storage Amazon S3 (versioned, encrypted) Snapshots, ingestion staging, backups
Secrets / config AWS Secrets Manager + SSM Parameter Store Neptune creds, GitHub token, endpoints, backend mode
Identity AWS IAM (mdc-mcp-rag-ecs-task-role, SigV4) Inter-service auth, developer access
Networking Amazon VPC with 10 VPC Endpoints (S3, Bedrock, ECR, Secrets, SSM, Logs, SageMaker, API Gateway) Fully private data plane — no IGW, no NAT
Observability Amazon CloudWatch (logs, metrics, alarms) Audit logs, health monitoring
IaC AWS CloudFormation via AWS CDK (TypeScript) Reviewable, reproducible deploys
Future (built, deferred) Amazon SageMaker (drift detection, fine-tuning) Self-healing RAG loop, activated when needed

What's Unfinished — The Extension Request

The prototype validated the platform on the developer workstation path (IAM SigV4 from EC2/Kiro). The two remaining consumer classes that unlock the return on the NIH investment are out of scope of the prototype:

  1. GitHub Actions CI/CD — automated EE2 compliance scans and root-cause analysis on every failed Rocoto pipeline, so reviewers and CMs get an AI-assisted diagnosis in the PR thread before a human looks at the log.
  2. RDHPCS researcher sessions — meteorologists on Hera, Orion, Hercules, Gaea, and Ursa accessing the MCP from their HPC login node without running the heavyweight server locally and without storing long-lived tokens on shared filesystems.

The full design is captured in the wiki page MCP-External-Access-Design-Path-B. The extension funds its execution.

Proposed Work

Workstream New AWS Services Outcome
Amazon Cognito User Pool + CI/HPC app clients Cognito JWT-based inbound auth on the existing AgentCore Runtime
GitHub OIDC → Token Broker Lambda AWS Lambda, AWS STS (GitHub OIDC federation) Zero long-lived secrets in GitHub repos; per-workflow-run attribution
HPC CLI helper (Client-side, calls Cognito) One-line eval $(mcp-token) on HPC login nodes; ephemeral tokens only
Server-side scope enforcement (In MCP server) CI scope = 40 read-only tools; HPC scope = 48 tools; Developer SigV4 unchanged with all 51
Audit logging CloudWatch Logs (JSON Lines) Every invocation attributable to a GitHub run ID or HPC user sub
Runbooks — One onboarding doc per consumer class, ready to hand to new teams

All new infrastructure remains under CDK; the developer SigV4 path stays byte-identical and unaffected (preserving daily productivity during rollout).

Cost Posture

The extension adds only Cognito (free tier covers expected MAU) and a low-traffic Lambda (Token Broker, sub-second invocations). The NIH Sandbox Cost-Control System (operator-driven sleep/wake for the entire compute footprint) is already specified and will land in the same window, extending the operational lifetime of every NIH dollar by hibernating Neptune, OpenSearch, EC2, and AgentCore during nights, weekends, and inter-burst gaps without losing any ingested data.

Why This Matters

The prototype proved the technology. The extension converts a single-user demonstration into a shared NOAA capability — meteorologists across five HPC systems and every CI pipeline in the Global Workflow organization get the same AI assistance that has accelerated OMD CAT Unit development for the last six months. The architecture is intentionally layered (the Two-Layer Consumer Model) so each new consumer class plugs in without touching the 51-tool core.

References