jgfs_fcst_fsm ecf (v17) Analysis - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki

jgfs_fcst_fsm.ecf (GFS v17)

Branch / tenant: dev/gfs.v17 (gw_v17) Path: ecf/scripts/gfs/forecast/jgfs_fcst_fsm.ecf Analysis date: 2026-06-11 Method: AWS Bedrock AgentCore agentcore-mcp-rag MCP cross-checked against the on-disk v17 worktree.


Summary

jgfs_fcst_fsm.ecf is the Forecast Segmentation Manager (FSM) task for the GFS forecast. It is a lightweight coordination job that monitors and manages the multi-segment GFS forecast execution. When the forecast is split into segments (for checkpointing and parallel post-processing), the FSM orchestrates the segment transitions, resubmitting forecast segments and coordinating with dependent tasks. It calls JGLOBAL_FSM, which drives exglobal_fsm.sh.


1. The script itself

#PBS -S /bin/bash
#PBS -N gfs_fcst_fsm_%CYC%
#PBS -j oe
#PBS -q %QUEUE%
#PBS -A %PROJ%-%PROJENVIR%
#PBS -l walltime=06:00:00
#PBS -l select=1:ncpus=1:ompthreads=1:mem=20GB
#PBS -l place=vscatter:shared
#PBS -l debug=true

model=gfs
export cyc="%CYC%"
%include <head.h>
%include <envir-p1.h>

############################################################
# Load modules
############################################################
module load intel/${intel_ver}
module load craype/${craype_ver}

module list
############################################################
# CALL executable job script here
############################################################
export RJN="%RJN%"

export HOMEglobal=${HOMEgfs}
${HOMEgfs}/jobs/JGLOBAL_FSM
if [ $? -ne 0 ]; then
   ecflow_client --msg="***JOB ${ECF_NAME} ERROR RUNNING J-SCRIPT ***"
   ecflow_client --abort
   exit
fi

%include <tail.h>
%manual

%end

What each part does

Section Role
#PBS directives PBS Pro: 1 node, 1 cpu, 20 GB, 6-hour wall — lightweight manager process that runs for the duration of the forecast.
module load intel/... craype/... Direct module loads (not load_modules.sh) — minimal modules for the coordination script.
RJN="%RJN%" ecFlow variable: Related Job Name — identifies the forecast job this FSM manages.
${HOMEgfs}/jobs/JGLOBAL_FSM The FSM J-Job.

2. Single-instance pattern

This is a single task per cycle (gfs_fcst_fsm_%CYC%). It runs alongside the main forecast task and manages segment transitions.


3. Execution chain

ecf/defs/gfs_prod.def : task jgfs_fcst_fsm
  +-- jgfs_fcst_fsm.ecf                       (PBS submit)
       +-- ${HOMEgfs}/jobs/JGLOBAL_FSM          (J-Job)
            +-- ${HOMEglobal}/scripts/exglobal_fsm.sh   (ex-script)
                 +-- monitors forecast segments
                 +-- resubmits/triggers next segment
                 +-- coordinates with ecFlow task dependencies

4. What the J-Job actually does

The FSM does not produce data products. Instead it:

  1. Monitors the forecast job's progress through its segments.
  2. Detects when a segment completes (by checking for restart files or sentinel files).
  3. Triggers the next forecast segment submission.
  4. Coordinates with ecFlow to release dependent tasks that can run on partial forecast output (e.g., post-processing for early hours while later hours are still integrating).

This enables overlap between forecast and post-processing, reducing the critical path wall-clock time.

Error handling

Uses err_exit on non-zero return (EE2-correct pattern). The set -xue in the J-Job provides strict mode natively.


5. Key environment variables

Variable Origin Meaning
RJN ECF %RJN% Related Job Name — the forecast task being managed.
CDATE J-Job (${PDY}${cyc}) Current cycle datetime.
EXPDIR J-Job Experiment configuration directory.
HOMEglobal ECF script Package home directory.
DATAROOT / DATA J-Job Working directory paths.
KEEPDATA env Controls temp directory cleanup.

Configuration

The J-Job does not use jjob_header.sh — it has a simplified setup:

  • Directly sets EXPDIR, DATA, cycle, then sources PDY.
  • No config files loaded (the FSM reads the forecast's config indirectly via exglobal_fsm.sh).

6. v17-specific notes

  • No load_modules.sh: unlike all other v17 ECF scripts, this one loads modules directly (intel, craype) — minimal footprint for a coordination process.
  • Long walltime: 6 hours matches the forecast walltime — the FSM must remain alive for the entire forecast duration.
  • Low resources: 1 cpu, 20 GB — just enough to run the coordination logic.
  • RJN ecFlow variable: links the FSM to its associated forecast task for ecFlow coordination.
  • set -xue in J-Job: uses bash strict mode directly rather than the standard set_strict.sh sourcing pattern.
  • Uses scripts/ path: calls ${HOMEglobal}/scripts/exglobal_fsm.sh rather than ${SCRglobal}/..., suggesting this is in the top-level scripts directory rather than dev/scripts.

7. Change-impact snapshot

Symbol Change type Risk Direct dependents
JGLOBAL_FSM behavior MEDIUM Coordinates the forecast — failure stops segment transitions
exglobal_fsm.sh signature MEDIUM Single caller, but critical path for forecast overlap

The FSM is critical for the segmented forecast execution model. If it fails, the forecast may not advance beyond segment 0, stalling all downstream processing.


Sources

  • On-disk v17 worktree: ecf/scripts/gfs/forecast/jgfs_fcst_fsm.ecf, dev/jobs/JGLOBAL_FSM.
  • agentcore-mcp-rag MCP (tenant gw_v17): on-disk analysis only (FSM not in graph).
⚠️ **GitHub.com Fallback** ⚠️