ICSDIR_ROOT Removal Impact Analysis - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki
Date: March 16, 2026
Analysis Method: MCP GraphRAG tools (find_env_dependencies, get_code_context, trace_data_flow, search_documentation) + codebase grep
Scope: All CI platform configs, CI case YAMLs, CTest infrastructure, and downstream config.stage_ic.j2 templates
ICSDIR_ROOT is an environment variable exported by all 6 CI platform config scripts under dev/ci/platforms/. It provides the root path to pre-staged initial condition data used by CI test experiments. Analysis reveals that ICSDIR_ROOT is redundant with BASE_IC (defined per-platform in dev/workflow/hosts/<platform>.yaml), and the config.stage_ic.j2 template already has a built-in fallback that constructs ICSDIR from BASE_IC when the user-provided value is blank.
Verdict: Safe to remove from platform configs and most CI case YAMLs, with documented exceptions for 3 non-standard cases and a required CTest infrastructure update.
| Platform | File | Path |
|---|---|---|
| Gaea C6 | dev/ci/platforms/config.gaeac6 |
/gpfs/f6/drsa-precip3/world-shared/role.glopara/data/ICSDIR |
| Hera | dev/ci/platforms/config.hera |
/scratch3/NCEPDEV/global/role.glopara/data/ICSDIR |
| Hercules | dev/ci/platforms/config.hercules |
/work/noaa/global/glopara/data/ICSDIR |
| Orion | dev/ci/platforms/config.orion |
/work/noaa/global/glopara/data/ICSDIR |
| WCOSS2 | dev/ci/platforms/config.wcoss2 |
/lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR |
| Ursa | dev/ci/platforms/config.ursa |
/scratch3/NCEPDEV/global/role.glopara/data/ICSDIR |
Both dev/parm/config/gfs/config.stage_ic.j2 and dev/parm/config/gefs/config.stage_ic.j2 contain:
export ICSDIR="{{ ICSDIR }}" # User provided ICSDIR; blank if not provided
# ...
# Set ICSDIR (if not defined)
if [[ -z "${ICSDIR}" ]] ; then
dir_name="${CASE}${ensic:-}${ocnic:-}"
ic_ver="${ic_versions[${dir_name}]}"
export ICSDIR="${BASE_IC}/${dir_name}/${ic_ver}"
fiWhen ICSDIR is blank (i.e., --icsdir not provided or empty), the template constructs the path automatically using:
-
BASE_IC— platform-specific root fromdev/workflow/hosts/<platform>.yaml -
Resolution — e.g.,
C96C48,C48mx500 -
IC version — from
versions/ic.ver
| Platform |
ICSDIR_ROOT (CI config) |
BASE_IC (host yaml) |
Match? |
|---|---|---|---|
| gaeac6 | /gpfs/f6/drsa-precip3/world-shared/role.glopara/data/ICSDIR |
/gpfs/f6/drsa-precip3/world-shared/role.glopara/data/ICSDIR |
Exact |
| hera | /scratch3/NCEPDEV/global/role.glopara/data/ICSDIR |
/scratch3/NCEPDEV/global/role.glopara/data/ICSDIR |
Exact |
| wcoss2 | /lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR |
/lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR |
Exact |
| ursa | /scratch3/NCEPDEV/global/role.glopara/data/ICSDIR |
/scratch3/NCEPDEV/global/role.glopara/data/ICSDIR |
Exact |
| hercules | /work/noaa/global/glopara/data/ICSDIR |
/work2/noaa/global/role-global/data/ICSDIR |
Differs |
| orion | /work/noaa/global/glopara/data/ICSDIR |
/work2/noaa/global/role-global/data/ICSDIR |
Differs |
Note: Hercules and Orion have divergent paths (
/work/vs/work2/,gloparavsrole-global). This should be verified to determine which is the current authoritative path before removingICSDIR_ROOT.
All CI case YAMLs use Jinja templating to resolve the path:
icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C96C48/20250808| YAML Suffix |
ic_versions Default |
Match? |
|---|---|---|
C96C48/20250808 |
20250808 |
Yes |
C48mx500/20250808 |
20250808 |
Yes |
C96mx100/20250808 |
20250808 |
Yes |
C384C192/20240610 |
20240610 |
Yes |
C96mx100/20240610 |
20250808 |
No — sfsv1 case uses older IC date |
C96C48/20250327 |
20250808 |
No — gcafsv1 cases use older IC date |
C96mx025/20251217 |
(not in ic.ver) | No — no entry exists |
All files under dev/ci/cases/pr/ and dev/ci/cases/weekly/ whose resolution+date matches ic.ver:
-
pr/C96C48_hybatmDA.yaml,pr/C96_atm3DVar.yaml,pr/C96_atm3DVar_extended.yaml -
pr/C96C48_hybatmsnowDA.yaml,pr/C96C48_hybatmsoilDA.yaml,pr/C96C48_ufsgsi_hybatmDA.yaml -
pr/C96C48_ufs_hybatmDA.yaml,pr/C96C48mx500_S2SW_cyc_gfs.yaml -
pr/C48mx500_3DVarAOWCDA.yaml,pr/C48mx500_hybAOWCDA.yaml -
pr/C48_gsienkf_atmDA.yaml,pr/C48_ufsenkf_atmDA.yaml -
pr/C96_gcafs_cycled.yaml,pr/C96_gcafs_cycled_noDA.yaml pr/C96mx100_S2S.yaml-
weekly/C384C192_hybatmda.yaml,weekly/C384_atm3DVar.yaml
These use IC versions or paths that differ from ic.ver defaults and cannot rely on the fallback:
| File | Reason |
|---|---|
sfsv1/C96mx100_S2S.yaml |
Uses 20240610, ic.ver has 20250808
|
sfsv1/C96mx025_S2S.yaml |
C96mx025 not in ic.ver at all |
gcafsv1/C96_gcafs_cycled.yaml |
Uses C96C48/20250327, ic.ver has 20250808
|
gcafsv1/C96_gcafs_cycled_noDA.yaml |
Same mismatch |
gcafsv1/C96_gcafs_cycled_noDA_dev.yaml |
Same mismatch |
gcafsv1/C384_gcafs_cycled.yaml |
Uses C96C48/20250327
|
gcafsv1/C384_gcafs_cycled_noDA.yaml |
Same mismatch |
gcafsv1/C384_gcafs_cycled_noDA_dev.yaml |
Same mismatch |
These 8 files could either keep explicit
icsdirvalues (hardcoded paths instead ofICSDIR_ROOT+ getenv) or have their IC versions added toversions/ic.ver.
Files under dev/ci/cases/gfsv17/ and pr/C48_S2SWA_gefs_RT.yaml already use platform-specific hardcoded paths and are unaffected by ICSDIR_ROOT removal.
CI platform config (exports ICSDIR_ROOT)
→ sourced by ci_utils.sh / driver_weekly.sh / launch_gitlab_runner.sh
→ YAML cases resolve {{ 'ICSDIR_ROOT' | getenv }} → icsdir value
→ create_experiment.py --yaml <case.yaml>
→ setup_expt.py maps icsdir → ICSDIR in experiment config
→ config.stage_ic.j2 (fallback: BASE_IC + resolution + ic.ver)
→ parm/stage/*.yaml.j2 templates use {{ ICSDIR }} for staging files
dev/ctests/CMakeLists.txt treats ICSDIR_ROOT as required:
set_from_env_or_default(ICSDIR_ROOT ICSDIR_ROOT "")
if (NOT DEFINED ICSDIR_ROOT)
message(WARNING "ICSDIR_ROOT must be set. CTests will not be created.")
return()
endif()This would need to either:
- Remove the
ICSDIR_ROOTrequirement (sinceconfig.stage_ic.j2handles the fallback) - Or replace it with
BASE_ICawareness
Also dev/ctests/scripts/setup.sh.in passes ICSDIR_ROOT to create_experiment.py directly:
ICSDIR_ROOT="${ICSDIR_ROOT}" \
./create_experiment.py --yaml "${YAML_FILE}" --overwritedev/ci/scripts/unittests/test_create_experiment.py explicitly sets ICSDIR_ROOT:
env['ICSDIR_ROOT'] = ICSDIR_FAKEThis would need updating to either remove the env var or set ICSDIR directly.
The MCP find_env_dependencies tool confirmed:
-
ICSDIR_ROOT: 0 scripts found in the Neo4j graph — it exists only in CI infrastructure, not the operational workflow -
ICSDIR: 0 direct script dependencies, but GGSR weighted analysis revealed 15config.stage_icnodes withSETS_ENVrelationships across experiment directories (C48_ATM, C96C48_hybatmDA, etc.)
This confirms ICSDIR is a config-level variable resolved at experiment creation time, not a runtime variable sourced by operational scripts.
-
Remove
export ICSDIR_ROOT=...from all 6 platform configs -
Remove
icsdir:lines from the 19 standard PR/weekly YAML case files -
Convert 8 non-standard cases to use hardcoded paths or update
versions/ic.ver -
Verify Hercules/Orion paths — determine if
/work/or/work2/is current -
Update CTest CMakeLists.txt — remove
ICSDIR_ROOTrequirement -
Update unit test — remove
ICSDIR_ROOTfrom env setup -
Update
versions/ic.ver— addC96mx025entry if GCAFS/SFS cases should use the fallback