MDC MCP RAG AWS Architecture v3 - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki
Design Document: MDC MCP RAG
AWS Architecture (Operational)
Overview
This document describes the deployed and operational AWS architecture for the MDC MCP RAG Platform as of May 2026. The platform provides 51 MCP tools across 9 modules for NOAA Global Workflow AI assistance, backed by Amazon Neptune (graph: 148,723 nodes, 2,820,440 relationships) and Amazon OpenSearch (vector: 206,341 documents across 17 indices).
The architecture evolved from the original Phase 46 plan (ECS Fargate + CloudFront + Cognito) to a simpler, more cost-effective deployment using AWS Bedrock AgentCore Runtime for managed MCP server hosting. AgentCore provides serverless Firecracker microVMs with built-in session management, eliminating the need for ECS clusters, load balancers, API Gateway, and Cognito. The platform runs in a fully private VPC with no internet gateway β all AWS service access is via VPC endpoints.
A parallel legacy system on NOAA RDHPCS infrastructure (Neo4j + ChromaDB, Docker-based) remains operational as a reference implementation, accessible via dev tunnel.
Architecture
Deployed AWS Service Topology
graph TD
subgraph "AWS Cloud β us-east-1 (<ACCOUNT_ID>)"
subgraph "Bedrock AgentCore"
AC["AgentCore Runtime<br/>mdc_mcp_rag_server (v4)<br/>Protocol: MCP | Network: VPC<br/>Firecracker microVM"]
end
subgraph "VPC: <VPC_NAME> (10.40.132.0/22)"
subgraph "Private Subnet: us-east-1a (10.40.137.0/24)"
OS1["OpenSearch Node 1<br/>r6g.large.search"]
end
subgraph "Private Subnet: us-east-1b (10.40.138.0/24)"
OS2["OpenSearch Node 2<br/>r6g.large.search"]
Neptune["Neptune Writer<br/>db.r8g.xlarge<br/>Engine 1.4.6.0<br/>Port 8182 (Bolt+WSS)"]
end
subgraph "Private Subnet: us-east-1d (10.40.136.0/24)"
EC2["EC2: c6g.xlarge (ARM64)<br/>10.40.136.39<br/>Kiro IDE (SSH Remote)<br/>Node.js 20 | Python 3.9 | Docker"]
end
subgraph "VPC Endpoints (10)"
VPCE["S3 (Gateway) | Secrets Manager | SSM<br/>CloudWatch Logs | ECR API | ECR DKR<br/>Bedrock Runtime | SageMaker API/Runtime<br/>API Gateway (execute-api)"]
end
end
subgraph "Supporting Services"
ECR["ECR: mdc-mcp-rag<br/>Tag: agentcore (302MB)"]
S3["S3: mdc-mcp-rag-migration<br/>Versioned, encrypted"]
EFS["EFS: mdc-mcp-rag-efs<br/>Encrypted, lifecycle 30d"]
IAM["IAM: mdc-mcp-rag-ecs-task-role<br/>Trust: ECS + AgentCore"]
SecMgr["Secrets Manager<br/>Neptune creds | GitHub token"]
SSMParam["SSM Parameter Store<br/>Neptune endpoint | OpenSearch endpoint"]
end
subgraph "CDK Stacks (CloudFormation)"
CDK1["MdcVpcStack<br/>(2026-04-06)"]
CDK2["MdcSecurityStack<br/>(2026-04-07)"]
CDK3["MdcDataStack<br/>(2026-04-07)"]
end
end
AC -->|"Bolt+WSS (SigV4)"| Neptune
AC -->|"HTTPS (SigV4)"| OS1
AC -->|"HTTPS (SigV4)"| OS2
EC2 -->|"invoke_agent_runtime<br/>(boto3 SigV4)"| AC
EC2 -->|"SSH Remote"| Kiro["Kiro IDE<br/>(Developer Laptop)"]
ECR -->|"Container image"| AC
IAM -->|"Execution role"| AC
SecMgr -->|"Credentials"| AC
SSMParam -->|"Endpoints"| EC2
EFS -->|"/mdc-mcp-rag"| EC2
sequenceDiagram
participant Dev as Developer Laptop
participant Kiro as Kiro IDE
participant EC2 as EC2 (10.40.136.39)
participant Proxy as agentcore-kiro-proxy.py
participant AC as AgentCore Runtime (v4)
participant MCP as MCP Server (Node.js)
participant Neptune as Neptune (Bolt+WSS)
participant OS as OpenSearch (HTTPS)
Note over Dev,OS: Production Path (AgentCore)
Dev->>Kiro: Open workspace
Kiro->>EC2: SSH Remote connection
EC2->>Proxy: Spawn stdio process
Proxy->>AC: invoke_agent_runtime (SigV4)
AC->>MCP: JSON-RPC (tools/call)
alt Graph query
MCP->>Neptune: openCypher (SigV4 Bolt)
Neptune-->>MCP: Graph results
end
alt Vector search
MCP->>OS: k-NN search (SigV4 HTTPS)
OS-->>MCP: Ranked documents
end
alt Hybrid (GGSR)
MCP->>Neptune: Graph traversal
Neptune-->>MCP: Structural context
MCP->>OS: Semantic search
OS-->>MCP: Vector results
Note over MCP: Merge & re-rank (GGSR)
end
MCP-->>AC: MCP response (SSE)
AC-->>Proxy: SSE stream
Proxy-->>EC2: JSON-RPC stdout
EC2-->>Kiro: MCP result
Loading
Original Plan vs Actual Deployment
Planned (Phase 46)
Actual (Deployed)
Reason
ECS Fargate cluster
Bedrock AgentCore Runtime
Simpler, managed microVMs, no cluster ops
Application Load Balancer
AgentCore DEFAULT endpoint
Built-in load balancing
API Gateway (REST)
AgentCore MCP protocol
Native MCP support, no HTTP wrapper needed
CloudFront + WAF
Not needed
Private VPC, no public access
Amazon Cognito (OAuth 2.0)
IAM SigV4 (via proxy)
VPC-internal, IAM sufficient
NAT Gateway
No NAT Gateway
VPC endpoints for all services
Neptune Serverless
Neptune db.r8g.xlarge (provisioned)
Consistent latency, admin-created
OpenSearch Serverless
OpenSearch 2.11 (provisioned, 2Γ r6g.large)
Zone-aware, admin-created
~$545-745/month estimated
AgentCore pay-per-session + provisioned DBs
Different cost model
Components and Interfaces
Component 1: AgentCore Runtime (MCP Server Host)
Purpose: Host the Node.js MCP server in a managed Firecracker microVM with native MCP protocol support, replacing the planned ECS Fargate + API Gateway + CloudFront stack.
Note: Neptune and OpenSearch are admin-created resources imported into CDK β CDK does not manage their lifecycle. All stateful resources have removalPolicy: RETAIN per the April 22, 2026 post-mortem guardrails.
Component 5: Configuration & Secrets (Deployed)
Config Item
AWS Service
Key
Neptune endpoint
SSM Parameter Store
/mdc-mcp-rag/neptune/endpoint
OpenSearch endpoint
SSM Parameter Store
/mdc-mcp-rag/opensearch/endpoint
DB backend mode
SSM Parameter Store
/mdc-mcp-rag/db-backend
Neptune credentials
Secrets Manager
mdc-mcp-rag/neptune/credentials
GitHub token
Secrets Manager
mdc-mcp-rag/github/token
Resolution: aws-config.js fetches from SSM/Secrets Manager with process-lifetime cache, falls back to environment variables.
Neptune IAM policy fix, NeptuneAdapter stats optimization, Runtime v4
2026-05-01
Full parity test: 20/22 AgentCore tools passing, 9/9 healthy
All data in this document reflects live AWS API queries performed on 2026-05-01.
Resource IDs, endpoints, counts, and configurations are from the deployed system.