Dynamic_MCP_Server_Self_Provisioning - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki
Dynamic MCP Server Self-Provisioning: LLM Agents That Expand Their Own Capabilities
Date: January 13, 2026
Status: β
Demonstrated Working
Significance: Paradigm Shift in Agentic AI Architecture
Executive Summary
We have achieved a major breakthrough in LLM agent capabilities: an AI assistant that can discover, configure, and activate new tool servers autonomously through the Docker MCP Gateway's dynamic management toolsβwithout requiring CLI commands, config file edits, or service restarts.
This transforms the agent from a static tool user to a dynamic capability builder.
The Breakthrough
Traditional Approach vs. What We Achieved
| Step | Traditional Approach | Dynamic Self-Provisioning |
|---|---|---|
| 1. Need new capability | Ask user to configure server | mcp-find discovers it in catalog |
| 2. Configure the server | Edit config files manually | mcp-config-set sets parameters |
| 3. Add the server | Restart gateway, reload VS Code | mcp-add activates it live |
| 4. Use new tools | Hope the tools work | mcp-exec calls the new tools |
Live Demonstration: Research Paper Search
When asked about coupled atmosphere-ocean modeling papers, the LLM:
User asks about coupled modeling papers
β
LLM recognizes it needs arxiv search capability
β
mcp-find β discovers arxiv-mcp-server in catalog
β
mcp-config-set β configures storage path: {"storage_path": "/tmp/arxiv-papers"}
β
mcp-add β activates 4 new tools dynamically
β
mcp-exec β searches papers with query "coupled atmosphere ocean data assimilation"
β
Returns relevant research papers to user
No CLI. No config files. No restarts. Pure MCP tool orchestration.
Papers Retrieved in Live Demo
The search_papers tool returned these relevant results on coupled atmosphere-ocean data assimilation:
| Paper | Authors | Year | Key Topics |
|---|---|---|---|
| Coupled Seasonal DA of Sea Ice, Ocean, and Atmospheric Dynamics | Meng, Hakim, Steig | 2025 | Online DA, coupled linear inverse model, El NiΓ±o reconstruction |
| Simulation and DA in Idealized Coupled Atmosphere-Ocean-Sea Ice Model | Mou, Stechmann, Chen | 2024 | Discrete element sea ice, QG ocean/atmos, cloud effects |
| Deep Learning of Systematic Sea Ice Model Errors from DA Increments | Gregory, Bushuk, et al. | 2023 | CNN to predict DA increments, SPEAR/GFDL model |
| A Local Ensemble Kalman Filter for Atmospheric DA | Ott, Hunt, Szunyogh, Kalnay, et al. | 2002 | LETKF - foundational paper for ensemble DA |
| EnKF with Divided State-Space for Coupled DA | Luo, Hoteit | 2014 | Coupled sub-systems, multi-scale Lorenz 96 |
| Ensemble DA for Wildland Fires (WRF Coupled) | Mandel, Beezley, Coen, Kim | 2007 | WRF coupling, morphing EnKF |
| Assimilation of Semi-Qualitative Sea Ice Thickness with EnKF-SQ | Shah, Bertino, et al. | 2019 | TOPAZ4 Arctic model, thin ice measurements |
| Impact of Atmospheric and Model Physics Perturbations on Red Sea DA | Sanikommu, Toye, et al. | 2020 | EAKF/DART, MITgcm, ensemble generation strategies |
All papers discovered and retrieved autonomously by the LLM using dynamically-added arxiv-mcp-server tools.
The 7 Gateway Management Tools
These tools are injected by the Docker MCP Gateway when the dynamic-tools feature is enabled:
| Tool | Purpose |
|---|---|
mcp-find |
Search the MCP catalog for available servers by name/description |
mcp-add |
Dynamically add and activate a server to the current session |
mcp-remove |
Remove a server from the active session |
mcp-config-set |
Configure server parameters (storage paths, credentials, etc.) |
mcp-exec |
Execute any tool from any active server |
mcp-create-profile |
Save current gateway state as a reusable profile |
code-mode |
Create JavaScript-enabled tools combining multiple MCP servers |
Configuration Required
Enabling Dynamic Tools
The gateway must be started without the --servers flag to enable dynamic tools:
# β This DISABLES dynamic tools:
docker mcp gateway run --servers eib-mcp-rag --transport streaming --port 18888
# β
This ENABLES dynamic tools (uses registry.yaml instead):
docker mcp gateway run --transport streaming --port 18888 --long-lived --verbose
Systemd Service Configuration
Updated /etc/systemd/system/mcp-gateway.service:
ExecStart=/home/USER/.docker/cli-plugins/docker-mcp gateway run \
--transport streaming \
--port 18888 \
--long-lived \
--verbose
Key change: Removed --servers eib-mcp-rag and --static flags.
Feature Flag
Verify the dynamic-tools feature is enabled:
docker mcp feature ls
# Should show: dynamic-tools enabled
Available MCP Servers in Catalog
The LLM can discover and add any of these servers on-demand:
Research & Productivity
- arxiv-mcp-server - arXiv paper search, download, analysis
- simplechecklist - Task management with SQLite
GitHub & Source Control
- github / github-official - GitHub API operations
- gitlab - GitLab API integration
- deepwiki - Query GitHub repos
Databases
- database-server - PostgreSQL, MySQL, SQLite
- neo4j-memory - Neo4j graph database
- mongodb - MongoDB/Atlas clusters
- oracle - Oracle database access
Filesystem
- filesystem - Local file access
- rust-mcp-filesystem - High-performance Rust implementation
- desktop-commander - File management + terminal
Implications for Agentic AI
What This Enables
- Self-Expanding Toolset - Agent recognizes capability gaps and fills them
- Context-Aware Tool Selection - Discovers the right tool for the task
- Zero-Configuration Experience - Users don't need to pre-configure every possible tool
- Composable Capabilities - Multiple servers can be combined dynamically
Example Workflow
User: "Find papers on JEDI data assimilation and summarize the key algorithms"
Agent thinks: "I need arxiv search capability"
β mcp-find("arxiv") β found arxiv-mcp-server
β mcp-config-set(storage_path)
β mcp-add("arxiv-mcp-server")
β mcp-exec("search_papers", query="JEDI data assimilation")
β Returns formatted research summary
Technical Details
Tool Count Progression
| Configuration | Tools Available |
|---|---|
| EIB MCP RAG Server only | 35 tools |
| + Gateway Management Tools | 42 tools |
| + Dynamically Added Servers | 42 + N tools |
Session Persistence
Dynamically added servers persist for the gateway session. Use mcp-create-profile to save configurations for future sessions.
References
- Docker MCP Gateway Documentation
- Docker_MCP_Gateway_MultiUser_Architecture - Multi-user gateway architecture
- MCP_TOOL_ARCHITECTURE - EIB MCP tool organization
- ADVANCED_FUTURE_WORK - Strategic roadmap for MCP/RAG evolution
Conclusion
This capability represents a fundamental shift from static AI assistants (limited to pre-configured tools) to dynamic AI agents (self-provisioning capabilities as needed). The Docker MCP Gateway's management tools, when properly configured, enable truly autonomous tool discovery and activation.
The LLM can now recognize what it doesn't know how to do, find a tool that can do it, configure that tool, and use itβall without human intervention.
This is the next level of agentic AI.