FMS Analysis — MCP Tool Effectiveness Report

Date: May 4, 2026
Query: "Where in the global-workflow infrastructure is FMS used?"
Platform: AgentCore Runtime v5 (Neptune 148K nodes, OpenSearch 206K docs)
Companion: FMS-Usage-in-Global-Workflow-Infrastructure

---

Tool Calls Summary

#	Tool	Args (key)	Latency	Rating	Contribution
1	search_documentation	"FMS Flexible Modeling System usage"	Fast	⭐⭐⭐⭐	Found the key UFS Weather Model docs on FMS diag_manager, diag_table, field_table
2	find_env_dependencies	variable: "FMS"	Fast	⭐⭐	No shell-level FMS env var found — correct (FMS is Fortran-level, not shell)
3	get_code_context	symbol: "FMS"	Fast	⭐⭐⭐	Symbol not found directly, but suggested _add_fms_cube_sphere_increments and cice_hist2fms — valuable leads
4	search_architecture	"FMS diagnostic manager coupler"	Fast	⭐⭐	Returned community nodes but not directly FMS-related — architecture search less effective for library-level queries
5	find_similar_code	"fms_mod diag_manager fms2_io"	Fast	⭐	No results above 0.7 threshold — similarity search not effective for module names
6	get_code_context	symbol: "_add_fms_cube_sphere_increments"	104ms	⭐⭐⭐⭐	Found the Python→Fortran DA bridge: finalize → to_fv3time → add_increments chain
7	search_documentation	"FMS diag_table diag_manager output"	Fast	⭐⭐⭐⭐⭐	Excellent — returned detailed diag_table format, field_table format, diag_manager_nml, output file specs
8	find_similar_code	"fms_init fms_end mpp_init" (threshold 0.5)	Fast	⭐⭐⭐⭐	Found 15 matches: MOM_io_infra.F90, MOM_diag_manager_infra.F90, mpas_dmpar.F, fort.F90 — confirmed MOM6 FMS integration
9	find_callers_callees	function: "fms_init"	Fast	⭐⭐⭐⭐⭐	Key finding: 188 callers — quantified FMS pervasiveness across the codebase
10	find_callers_callees	function: "diag_manager_init"	Fast	⭐⭐	0 callers found — likely not ingested as a standalone symbol (called via FMS init chain)
11	find_callers_callees	function: "fms_end"	Fast	⭐⭐⭐⭐⭐	Key finding: 316 callers — even more widely used than fms_init, confirmed MPI/domain/halo usage
12	get_code_context	symbol: "cice_hist2fms"	618ms	⭐⭐⭐	Found CICE→FMS format bridge in SOCA background utilities
13	describe_component	"sorc/ufs_model.fd/FMS"	Fast	⭐	Component not found — describe_component searches workflow dirs, not deep submodule paths
14	get_code_context	symbol: "fms_mod"	89ms	⭐⭐⭐⭐	Confirmed FV3 climate nudging, remapping, variable state management as primary consumers
15	get_code_context	symbol: "fms2_io_mod"	147ms	⭐⭐⭐⭐	Found FV3 control init, namelist readers, JEDI FV3 interface — confirmed FMS2 I/O usage
16	get_code_context	symbol: "diag_manager_mod"	83ms	⭐⭐⭐⭐	Same consumer set as fms_mod — confirms tight coupling between FMS core and diagnostics
17	get_code_context	symbol: "mpp_mod"	86ms	⭐⭐⭐⭐	Confirmed message passing module used by same FV3 routines
18	get_code_context	symbol: "constants_mod"	53ms	⭐⭐⭐⭐⭐	Found UFS mediator usage: med_kind_mod, ufs_const_mod, all diagnostic phases — confirmed cross-component reach
19	explain_with_context	"FMS role in UFS Weather Model"	Fast	⭐	Returned minimal content — RAG explanation thin for this topic

Total tool calls: 19
Unique tools used: 7 (out of 51 available)

---

Effectiveness by Tool Category

Graph Tools (Neptune) — ⭐⭐⭐⭐⭐ Most Effective

Tool	Calls	Avg Rating	Notes
find_callers_callees	3	4.0	Star performer — quantified fms_init (188) and fms_end (316) callers
get_code_context	7	3.6	Excellent for tracing module dependencies and GGSR relationships

The Neptune graph database was the primary value driver for this analysis. The caller/callee counts (188 and 316) are the most impactful data points in the entire report — they quantify FMS pervasiveness in a way that no text search could.

Semantic Search Tools (OpenSearch) — ⭐⭐⭐⭐ Very Effective

Tool	Calls	Avg Rating	Notes
search_documentation	2	4.5	Found detailed FMS diag_table docs, field_table format, output file specs
find_similar_code	2	2.5	Mixed — failed at 0.7 threshold, succeeded at 0.5 for MOM6 FMS files
search_architecture	1	2.0	Community-level search not ideal for library-level queries
explain_with_context	1	1.0	Returned minimal content — needs tuning for infrastructure topics

Info/Operational Tools — ⭐⭐ Limited

Tool	Calls	Avg Rating	Notes
find_env_dependencies	1	2.0	Correctly found no shell-level FMS env var (FMS is Fortran-level)
describe_component	1	1.0	Can't reach deep submodule paths — limited to workflow-level dirs

---

Supplementary Investigation (Non-MCP)

The following information was gathered via direct filesystem queries (find, grep) to supplement the MCP tool results:

Method	Finding	MCP Gap
find -iname "*fms*" -type d	Found FMS1/, FMS2/, FMS_cap/ in MOM6	describe_component can't reach submodule paths
find -iname "*fms*" -type f	Found 21 FMS-related files across gdas.cd, gfs_utils.fd, MOM6	No MCP tool for filename-pattern search in the repo
grep -ri "FMS" .gitmodules	FMS not a direct submodule (transitive via ufs_model.fd)	No submodule-aware MCP tool

---

Recommendations for MCP Tool Improvement

1. find_similar_code: Lower the default similarity threshold from 0.7 to 0.5 for Fortran module names — module names are short strings that don't embed well at high thresholds
2. describe_component: Extend path search to include deep submodule paths (sorc/*/) not just top-level workflow directories
3. explain_with_context: Needs richer context assembly for infrastructure/library topics — currently returns thin results
4. New tool needed: search_files_by_name — pattern-based filename search across the repo tree (the find command equivalent)
5. get_code_context: The [object Object] entries in GGSR output for fms_init/fms_end callers suggest a serialization bug in the Neptune record-to-object conversion for nodes with complex properties

---

Rating Scale

Rating	Meaning
⭐⭐⭐⭐⭐	Excellent — provided key insights not available elsewhere
⭐⭐⭐⭐	Very good — directly contributed to the analysis
⭐⭐⭐	Good — provided useful supporting information
⭐⭐	Fair — returned results but limited direct value
⭐	Poor — returned no useful results or failed

---

Report generated from 19 MCP tool calls across 7 unique tools on AgentCore Runtime v5.