ICSDIR_ROOT Removal Impact Analysis - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki

ICSDIR_ROOT Removal Impact Analysis

Date: March 16, 2026 Analysis Method: MCP GraphRAG tools (find_env_dependencies, get_code_context, trace_data_flow, search_documentation) + codebase grep Scope: All CI platform configs, CI case YAMLs, CTest infrastructure, and downstream config.stage_ic.j2 templates


Summary

ICSDIR_ROOT is an environment variable exported by all 6 CI platform config scripts under dev/ci/platforms/. It provides the root path to pre-staged initial condition data used by CI test experiments. Analysis reveals that ICSDIR_ROOT is redundant with BASE_IC (defined per-platform in dev/workflow/hosts/<platform>.yaml), and the config.stage_ic.j2 template already has a built-in fallback that constructs ICSDIR from BASE_IC when the user-provided value is blank.

Verdict: Safe to remove from platform configs and most CI case YAMLs, with documented exceptions for 3 non-standard cases and a required CTest infrastructure update.


Where ICSDIR_ROOT Is Exported (6 Platform Configs)

Platform File Path
Gaea C6 dev/ci/platforms/config.gaeac6 /gpfs/f6/drsa-precip3/world-shared/role.glopara/data/ICSDIR
Hera dev/ci/platforms/config.hera /scratch3/NCEPDEV/global/role.glopara/data/ICSDIR
Hercules dev/ci/platforms/config.hercules /work/noaa/global/glopara/data/ICSDIR
Orion dev/ci/platforms/config.orion /work/noaa/global/glopara/data/ICSDIR
WCOSS2 dev/ci/platforms/config.wcoss2 /lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR
Ursa dev/ci/platforms/config.ursa /scratch3/NCEPDEV/global/role.glopara/data/ICSDIR

The Existing Fallback in config.stage_ic.j2

Both dev/parm/config/gfs/config.stage_ic.j2 and dev/parm/config/gefs/config.stage_ic.j2 contain:

export ICSDIR="{{ ICSDIR }}" # User provided ICSDIR; blank if not provided

# ...

# Set ICSDIR (if not defined)
if [[ -z "${ICSDIR}" ]] ; then
  dir_name="${CASE}${ensic:-}${ocnic:-}"
  ic_ver="${ic_versions[${dir_name}]}"
  export ICSDIR="${BASE_IC}/${dir_name}/${ic_ver}"
fi

When ICSDIR is blank (i.e., --icsdir not provided or empty), the template constructs the path automatically using:

  • BASE_IC — platform-specific root from dev/workflow/hosts/<platform>.yaml
  • Resolution — e.g., C96C48, C48mx500
  • IC version — from versions/ic.ver

BASE_IC vs ICSDIR_ROOT — Per-Platform Comparison

Platform ICSDIR_ROOT (CI config) BASE_IC (host yaml) Match?
gaeac6 /gpfs/f6/drsa-precip3/world-shared/role.glopara/data/ICSDIR /gpfs/f6/drsa-precip3/world-shared/role.glopara/data/ICSDIR Exact
hera /scratch3/NCEPDEV/global/role.glopara/data/ICSDIR /scratch3/NCEPDEV/global/role.glopara/data/ICSDIR Exact
wcoss2 /lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR /lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR Exact
ursa /scratch3/NCEPDEV/global/role.glopara/data/ICSDIR /scratch3/NCEPDEV/global/role.glopara/data/ICSDIR Exact
hercules /work/noaa/global/glopara/data/ICSDIR /work2/noaa/global/role-global/data/ICSDIR Differs
orion /work/noaa/global/glopara/data/ICSDIR /work2/noaa/global/role-global/data/ICSDIR Differs

Note: Hercules and Orion have divergent paths (/work/ vs /work2/, glopara vs role-global). This should be verified to determine which is the current authoritative path before removing ICSDIR_ROOT.


CI Case YAML Files Using ICSDIR_ROOT (25 files)

All CI case YAMLs use Jinja templating to resolve the path:

icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C96C48/20250808

Subdirectory Suffixes vs ic.ver Defaults

YAML Suffix ic_versions Default Match?
C96C48/20250808 20250808 Yes
C48mx500/20250808 20250808 Yes
C96mx100/20250808 20250808 Yes
C384C192/20240610 20240610 Yes
C96mx100/20240610 20250808 No — sfsv1 case uses older IC date
C96C48/20250327 20250808 No — gcafsv1 cases use older IC date
C96mx025/20251217 (not in ic.ver) No — no entry exists

Files Safe to Remove icsdir (standard PR/weekly cases — 19 files)

All files under dev/ci/cases/pr/ and dev/ci/cases/weekly/ whose resolution+date matches ic.ver:

  • pr/C96C48_hybatmDA.yaml, pr/C96_atm3DVar.yaml, pr/C96_atm3DVar_extended.yaml
  • pr/C96C48_hybatmsnowDA.yaml, pr/C96C48_hybatmsoilDA.yaml, pr/C96C48_ufsgsi_hybatmDA.yaml
  • pr/C96C48_ufs_hybatmDA.yaml, pr/C96C48mx500_S2SW_cyc_gfs.yaml
  • pr/C48mx500_3DVarAOWCDA.yaml, pr/C48mx500_hybAOWCDA.yaml
  • pr/C48_gsienkf_atmDA.yaml, pr/C48_ufsenkf_atmDA.yaml
  • pr/C96_gcafs_cycled.yaml, pr/C96_gcafs_cycled_noDA.yaml
  • pr/C96mx100_S2S.yaml
  • weekly/C384C192_hybatmda.yaml, weekly/C384_atm3DVar.yaml

Files Requiring Explicit icsdir (non-standard — 8 files)

These use IC versions or paths that differ from ic.ver defaults and cannot rely on the fallback:

File Reason
sfsv1/C96mx100_S2S.yaml Uses 20240610, ic.ver has 20250808
sfsv1/C96mx025_S2S.yaml C96mx025 not in ic.ver at all
gcafsv1/C96_gcafs_cycled.yaml Uses C96C48/20250327, ic.ver has 20250808
gcafsv1/C96_gcafs_cycled_noDA.yaml Same mismatch
gcafsv1/C96_gcafs_cycled_noDA_dev.yaml Same mismatch
gcafsv1/C384_gcafs_cycled.yaml Uses C96C48/20250327
gcafsv1/C384_gcafs_cycled_noDA.yaml Same mismatch
gcafsv1/C384_gcafs_cycled_noDA_dev.yaml Same mismatch

These 8 files could either keep explicit icsdir values (hardcoded paths instead of ICSDIR_ROOT + getenv) or have their IC versions added to versions/ic.ver.

Files Already Using Hardcoded Paths (no ICSDIR_ROOT — 13 files)

Files under dev/ci/cases/gfsv17/ and pr/C48_S2SWA_gefs_RT.yaml already use platform-specific hardcoded paths and are unaffected by ICSDIR_ROOT removal.


Dependency Chain

CI platform config (exports ICSDIR_ROOT)
  → sourced by ci_utils.sh / driver_weekly.sh / launch_gitlab_runner.sh
    → YAML cases resolve {{ 'ICSDIR_ROOT' | getenv }} → icsdir value
      → create_experiment.py --yaml <case.yaml>
        → setup_expt.py maps icsdir → ICSDIR in experiment config
          → config.stage_ic.j2 (fallback: BASE_IC + resolution + ic.ver)
            → parm/stage/*.yaml.j2 templates use {{ ICSDIR }} for staging files

Additional Changes Required

1. CTest CMakeLists.txt

dev/ctests/CMakeLists.txt treats ICSDIR_ROOT as required:

set_from_env_or_default(ICSDIR_ROOT ICSDIR_ROOT "")
if (NOT DEFINED ICSDIR_ROOT)
  message(WARNING "ICSDIR_ROOT must be set. CTests will not be created.")
  return()
endif()

This would need to either:

  • Remove the ICSDIR_ROOT requirement (since config.stage_ic.j2 handles the fallback)
  • Or replace it with BASE_IC awareness

Also dev/ctests/scripts/setup.sh.in passes ICSDIR_ROOT to create_experiment.py directly:

ICSDIR_ROOT="${ICSDIR_ROOT}" \
./create_experiment.py --yaml "${YAML_FILE}" --overwrite

2. Unit Test

dev/ci/scripts/unittests/test_create_experiment.py explicitly sets ICSDIR_ROOT:

env['ICSDIR_ROOT'] = ICSDIR_FAKE

This would need updating to either remove the env var or set ICSDIR directly.


GraphRAG Tool Findings

The MCP find_env_dependencies tool confirmed:

  • ICSDIR_ROOT: 0 scripts found in the Neo4j graph — it exists only in CI infrastructure, not the operational workflow
  • ICSDIR: 0 direct script dependencies, but GGSR weighted analysis revealed 15 config.stage_ic nodes with SETS_ENV relationships across experiment directories (C48_ATM, C96C48_hybatmDA, etc.)

This confirms ICSDIR is a config-level variable resolved at experiment creation time, not a runtime variable sourced by operational scripts.


Recommended Actions

  1. Remove export ICSDIR_ROOT=... from all 6 platform configs
  2. Remove icsdir: lines from the 19 standard PR/weekly YAML case files
  3. Convert 8 non-standard cases to use hardcoded paths or update versions/ic.ver
  4. Verify Hercules/Orion paths — determine if /work/ or /work2/ is current
  5. Update CTest CMakeLists.txt — remove ICSDIR_ROOT requirement
  6. Update unit test — remove ICSDIR_ROOT from env setup
  7. Update versions/ic.ver — add C96mx025 entry if GCAFS/SFS cases should use the fallback
⚠️ **GitHub.com Fallback** ⚠️