Per User RAG Isolation Developer Guide - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki
Audience: SME developers (Anna, Anton, Brian, Georgios) with a personal scratch-clone of the MCP repo on the shared Parallel Works host.
TL;DR: You can iterate on EE2 standards, ingestion chunking, and search tuning inside your own personal tenant without disturbing the shared knowledge base. Your changes are namespaced β only you see them β until you merge to
develop.
The shared MCP/RAG knowledge base (gw tenant) serves everyone:
- 17 ChromaDB collections (220 K+ embedded documents)
- 344,604 Neo4j nodes (graph of every Fortran/Python/Shell/CMake entity)
-
53 MCP tools exposed via the Docker MCP Gateway on
:18888
When you're on a feature branch and want to change how code gets embedded or how EE2 standards are chunked, you need a safe space to re-ingest and test β without breaking the baseline that everyone else queries.
Personal tenants give you that space on the same shared databases.
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Shared Infrastructure (always running, serves everyone) β
β β
β ββββββββββββββββ ββββββββββββββββ ββββββββββββββββββββββββββ β
β β ChromaDB β β Neo4j β β Docker MCP Gateway β β
β β :8080 β β :7474/:7687 β β :18888 β β
β β β β β β (shared 53-tool server)β β
β β Collections: β β Labels: β β Default tenant: gw β β
β β gw_* β β Function β ββββββββββββββββββββββββββ β
β β p_anna_* β β P_ANNA_* β β
β β p_anton_* β β P_ANTON_* β β
β β p_brian_* β β P_BRIAN_* β β
β ββββββββββββββββ ββββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β² write (your tenant only) β² read-only (shared gw)
β β
ββββββββββΌβββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββ
β YOUR workspace β β
β /mcp_rag_eib/SCRATCH_SPACE/<you>/eib-mcp-rag-server β
β Branch: feature/my-ee2-improvement β β
β β β
β ββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββ β
β β Your local MCP stdio server (runs from YOUR branch) β β
β β β β
β β β’ Queries gw tenant (read-only) β see baseline results β β
β β β’ Queries personal-<you> tenant β see YOUR changes β β
β β β’ --diff-tenant gw,personal-<you> β side-by-side compareβ β
β β β β
β β Connects to: bolt://localhost:7687, http://localhost:8080β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β VS Code Remote Tunnel β .vscode/mcp.json β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Your personal tenant (personal-anna, personal-anton, etc.) writes to
prefixed collections and labels on the same physical ChromaDB and Neo4j
instances. The shared gw data is never touched.
cd /mcp_rag_eib/SCRATCH_SPACE/$USER/eib-mcp-rag-server
git checkout -b feature/ee2-chunk-reformpython -m mcp_server_python.scripts.create_personal_tenant $USERThis appends a new entry to tenants.yaml:
- tenant_id: personal-anna
label_prefix: "P_ANNA_"
index_prefix: "p_anna_"
lifecycle: personal
owner: Anna.SmootEdit the ingestion logic β for example, split EE2 standards by subsection instead of whole-document:
$EDITOR scripts/ingest_ee2_standards.py
# ... change chunking strategy ...python scripts/ingest_ee2_standards.py --tenant personal-annaThis writes to p_anna_ee2-standards-v5-0-0-enhanced (ChromaDB) and any
EE2-related graph nodes with P_ANNA_ prefix (Neo4j). The shared gw
collections are untouched.
For the 34-document EE2 corpus, this takes seconds.
Query against your personal tenant:
# Using the MCP tool directly (via your local stdio server)
search_ee2_standards --query "module header comment format" --tenant personal-annaSide-by-side comparison of your changes vs the shared baseline:
search_ee2_standards --diff-tenant gw,personal-anna \
--query "module header comment format"Output (example):
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Query: "module header comment format" β
ββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββ€
β Baseline (gw) β Personal (personal-anna) β
ββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββ€
β 1. EE2-STD-003 (score 0.72) β 1. EE2-STD-003Β§2.1 (score 0.91) β
β 2. EE2-STD-017 (score 0.65) β 2. EE2-STD-003Β§2.3 (score 0.88) β
β 3. EE2-STD-022 (score 0.61) β 3. EE2-STD-017Β§1.2 (score 0.85) β
β ... β ... β
β Relevant: 3/8 β Relevant: 6/8 β improvement β
ββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββ
git add -A
git commit -m "feat(ee2): split EE2 standards by subsection for better retrieval"
git push origin feature/ee2-chunk-reform
# β Open Merge Request on GitLab β review β merge to developAfter merge, the shared CI pipeline will re-ingest into the gw tenant with
your improved chunking β benefiting all users.
Your personal tenant data is automatically cleaned up after 30 days of inactivity. You can also manually remove it:
python -m mcp_server_python.scripts.delete_personal_tenant $USER| Change Type | Example | Personal Tenant Needed? |
|---|---|---|
| Tool query logic | Edit ee2_compliance.py scoring |
No β just run your local MCP server against shared DBs (read-only) |
| Chunking / embedding rules | Edit ingest_ee2_standards.py chunk size |
Yes β re-ingest into your personal tenant |
| New document sources | Add a new standards PDF to the EE2 corpus | Yes β ingest the new docs into your personal tenant |
| Search ranking / filtering | Change max_results, add category filter |
No β tool code only, no data change |
| Compliance report templates | Edit report formatting in generate_compliance_report
|
No β tool code only |
-
Never ingest into the
gwtenant from your feature branch. Always use--tenant personal-<you>. -
The shared gateway on
:18888always serves thegwbaseline. Your VS Code MCP connection to the gateway gives you the production view. Your local stdio server (from your branch) gives you the personal view. -
Embeddings are deterministic. The
mpnet768model loads from a shared read-only cache. Your personal embeddings and the eventual shared re-ingest produce identical vectors for the same input text. -
Personal tenants are ephemeral. They exist to support your iteration loop. Once merged, the feature branch's logic runs on the shared tenant.
/mcp_rag_eib/SCRATCH_SPACE/<you>/
βββ eib-mcp-rag-server/ β YOUR clone (feature branches here)
βββ .vscode/mcp.json β points at shared DBs + your local server
βββ mcp_server_python/ β tool code you're editing
β βββ src/config/tenants.yaml β your personal tenant entry
βββ scripts/ β ingestion scripts you're modifying
βββ supported_repos/ β read-only submodules (don't modify)
The shared operator checkout at /mcp_rag_eib/eib-mcp-rag-server is
read-only to you. You work exclusively in your scratch-space clone.
Q: Can I accidentally break the shared knowledge base?
A: No. Your local server runs with MCP_READ_ONLY=true for the gw tenant.
Write operations are only permitted against your personal-* tenant. Even if
you forget the flag, the adapter refuses writes to non-personal tenants from a
user-owned process.
Q: How much disk does my personal tenant use? A: The EE2 corpus is 34 documents β trivial (< 1 MB). A full code-context personal ingest would use ~180 MB. There's a 500 MB soft limit per personal tenant.
Q: Do I need to restart anything after creating my personal tenant? A: Restart your local stdio server (not the shared gateway). The shared gateway doesn't need to know about personal tenants β they're for your local iteration only.
Q: What if my branch gets very old relative to develop?
A: The system tracks the commit SHA at ingest time. If your personal tenant's
data diverges > 100 commits from develop, check_knowledge_integrity will
surface a WARN. Rebase your branch and re-ingest.
Q: Can I compare my personal tenant against another user's?
A: Yes β --diff-tenant personal-anna,personal-anton works. Useful for
collaborative iteration on the same feature.
- Verify your scratch clone exists:
ls ~/eib-mcp-rag-server(or/mcp_rag_eib/SCRATCH_SPACE/$USER/eib-mcp-rag-server) - Create a feature branch:
git checkout -b feature/<your-change> - Create your personal tenant:
python -m mcp_server_python.scripts.create_personal_tenant $USER - Make your changes to ingestion/tool code
- Re-ingest:
python scripts/ingest_ee2_standards.py --tenant personal-$USER - Compare:
search_ee2_standards --diff-tenant gw,personal-$USER --query "β¦" - Push branch β open MR β merge β shared re-ingest happens automatically
Document created 2026-07-20 as part of Phase 74 (Per-User RAG Isolation). See the full technical spec: phase74_per_user_rag_isolation.md (linked from the gap analysis). For the shared-infrastructure health status, see Docker-MCP-Gateway-COTS-Gap-Analysis-2026-07-20.