jgfs_atmos_gempak_master ecf (v17) Analysis - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki
Branch / tenant: dev/gfs.v17 (gw_v17)
Path: ecf/scripts/gfs/product/atmos/gempak/gempak/jgfs_atmos_gempak_master.ecf
Analysis date: 2026-06-10
Method: AWS Bedrock AgentCore agentcore-mcp-rag MCP (graph traversal, GraphRAG,
code-context) cross-checked against the on-disk v17 worktree.
A companion page, agentcore-mcp-rag Tool Usage Analysis, records which MCP tools were used, how useful each was, and a separation-of-concerns assessment of the toolset.
jgfs_atmos_gempak_master.ecf is the NCO production ecFlow task template that turns
"GFS produced GRIB2 for forecast hour NNN" into "GEMPAK/NAWIPS files for hour NNN, on
every output grid that hour qualifies for." It is the single PBS-batch entry point shared
by every per-forecast-hour GEMPAK task in the operational GFS suite. At runtime it loads
modules and immediately hands off to the J-Job ${HOMEgfs}/jobs/JGFS_ATMOS_GEMPAK, which
drives exgfs_atmos_nawips.sh once per grid/forecast-hour combination.
The _master suffix is the NCO/global-workflow convention for one ecf template that is
copied and renamed to every task instance in a family — here jgfs_atmos_gempak_f000
through jgfs_atmos_gempak_f384.
#PBS -S /bin/bash
#PBS -N gfs_atmos_gempak_%CYC%_f%FHR%
#PBS -j oe
#PBS -q %QUEUE%
#PBS -A %PROJ%-%PROJENVIR%
#PBS -l walltime=00:20:00
#PBS -l select=1:mpiprocs=28:ompthreads=1:ncpus=28:mem=6GB
#PBS -l place=vscatter:shared
#PBS -l debug=true
model=gfs
export cyc="%CYC%"
%include <head.h>
%include <envir-p1.h>
############################################################
# Load modules
############################################################
#TODO: load only the modules needed for each job
source ${HOMEgfs}/dev/ush/load_modules.sh run
module list
export WGF="%WGF%"
export FHR3="%FHR%"
export FHR_LIST="%FHR_LIST%"
export COMPONENT=${WGF}
############################################################
# CALL executable job script here
############################################################
export CDATE=${PDY}${cyc}
export HOMEglobal=${HOMEgfs}
#### ${HOMEgfs}/dev/job_cards/rocoto/gempak.sh
${HOMEgfs}/jobs/JGFS_ATMOS_GEMPAK
if [ $? -ne 0 ]; then
ecflow_client --msg="***JOB ${ECF_NAME} ERROR RUNNING J-SCRIPT ***"
ecflow_client --abort
exit
fi
%include <tail.h>
%manual
%end| Section | Role |
|---|---|
#PBS directives |
PBS Pro batch resources: 1 node, 28 cpus/mpiprocs, 6 GB, 20-min wall, shared placement. Job name gfs_atmos_gempak_%CYC%_f%FHR%. |
%include <head.h> / <envir-p1.h>
|
NCO boilerplate: environment bootstrap, ecflow_client init, production environment (envir-p1). |
load_modules.sh run |
v17 centralizes module loading in one ush script instead of per-task module load lines. |
WGF, FHR3, FHR_LIST, COMPONENT
|
ecFlow %VAR% placeholders injected per task by the suite definition. WGF = Weather-Generating-Forecast component (atmos). |
HOMEglobal=${HOMEgfs} |
Bridges the v17 HOMEglobal naming used inside the J-Job to the NCO HOMEgfs. |
${HOMEgfs}/jobs/JGFS_ATMOS_GEMPAK |
The actual work — the J-Job. Non-zero exit triggers ecflow_client --abort. |
%include <tail.h> / %manual
|
NCO completion handling and the operator manual block. |
A single .ecf file backs hundreds of ecFlow tasks. The global-workflow ecFlow generator
(dev/workflow/ecflow/ecflow_definitions.py, add_task(..., template=...)) looks up the
named template and copies it to the destination, renaming it to the task name:
"If template is set, it will look for that template and then copy and change the name of the template at the destination to the name of the task." —
ecflow_definitions.py,add_taskdocstring (rephrased for compliance)
So jgfs_atmos_gempak_master.ecf is deployed as jgfs_atmos_gempak_f000.ecf,
jgfs_atmos_gempak_f001.ecf, … by the build step. The generated suite definition
(ecf/defs/gfs_prod.def) shows the expansion:
family gempak
family gempak
task jgfs_atmos_gempak_f000
edit FHR '000'
edit FHR_LIST '0'
trigger ../../product/jgfs_atmos_product_f000==complete
task jgfs_atmos_gempak_f001
edit FHR '001'
edit FHR_LIST '1'
trigger ../../product/jgfs_atmos_product_f001==complete
...
task jgfs_atmos_gempak_f384
- The GFS atmos GEMPAK family runs f000 → f384.
- Each task overrides
FHR,FHR_LIST, and inheritsWGF 'atmos'from the product family. -
edit ECF_FILES '%PACKAGEHOME%/ecf/scripts'sets the template search root. - The trigger gate is the matching upstream
product/jgfs_atmos_product_fNNN==complete, so a given hour's GEMPAK conversion only fires once that hour's GRIB2 products exist.
The higher-level NCO suite intent is declared in dev/workflow/prod.yml under the
gempak: family (sibling tasks jgfs_atmos_gempak, jgfs_atmos_gempak_meta,
jgfs_atmos_gempak_ncdc_upapgif, jgfs_atmos_pgrb2_spec_gempak).
ecf/defs/gfs_prod.def : task jgfs_atmos_gempak_fNNN (gated on product_fNNN == complete)
└─ jgfs_atmos_gempak_master.ecf (PBS submit; deployed as *_fNNN.ecf)
└─ ${HOMEgfs}/jobs/JGFS_ATMOS_GEMPAK (J-Job)
└─ ${SCRglobal}/exgfs_atmos_nawips.sh (ex-script, per grid)
└─ ${USHglobal}/product_functions.sh (shared helper, sourced)
This chain was confirmed via the MCP graph (trace_data_flow, find_callers_callees,
find_dependencies) against the gw_v17 tenant:
-
JGFS_ATMOS_GEMPAKINVOKESexgfs_atmos_nawips.shand SOURCESjjob_header.sh,jjob_standard_vars.sh,jjob_shell_setup.sh,set_strict.sh. -
exgfs_atmos_nawips.shcallers ={JGFS_ATMOS_GEMPAK}(fan-in 1); callee =product_functions.sh(fan-out 3, low complexity). -
product_functions.shis a leaf helper shared by 5 NAWIPS/AWIPS ex-scripts (exgfs_atmos_nawips.sh,exgfs_atmos_goes_nawips.sh,exgfs_atmos_awips_20km_1p0deg.sh,exgdas_atmos_nawips.sh,interp_atmos_master.sh).
JGFS_ATMOS_GEMPAK upstream imports (10): jjob_header.sh, jjob_standard_vars.sh,
set_strict.sh, jjob_shell_setup.sh, exgfs_atmos_gempak_gif_ncdc_skew_t.sh,
exgfs_atmos_nawips.sh, exgfs_atmos_goes_nawips.sh, exgfs_atmos_gempak_meta.sh,
plus the ush/-prefixed header/vars. No circular dependencies; no downstream importers
(it is a suite leaf).
JGFS_ATMOS_GEMPAK defines COM directories and runs exgfs_atmos_nawips.sh once per grid,
gated by forecast hour. Output lands under
${ROTDIR}/${RUN}.${PDY}/${cyc}/products/atmos/gempak/<grid>/.
| Output grid | Hour range | Cadence | DBN alert |
|---|---|---|---|
| 1p00 | 0–240 | every 3 h | GFS_GEMPAK |
| 1p00 | 252–384 | every 12 h | GFS_GEMPAK |
| 0p50 | 0–240 | every 3 h | GFS_GEMPAK |
| 0p50 | 246–276 | every 6 h | GFS_GEMPAK |
| 0p50 | 288–384 | every 12 h | GFS_GEMPAK |
| 0p25 | 0–120 | every 1 h | GFS_GEMPAK |
| 0p25 | 123–240 | every 3 h | GFS_GEMPAK |
| 0p25 | 252–384 | every 12 h | GFS_GEMPAK |
| 35km_atl / 35km_pac / 40km | 0 – min(180, FHMAX_GFS) |
every 3 h | GFS_GEMPAK_WWB |
Error handling uses the EE2-correct err_exit pattern (not set -e) on each grid block.
| Variable | Origin | Meaning |
|---|---|---|
WGF |
suite edit WGF 'atmos'
|
Weather-Generating-Forecast component; also exported as COMPONENT. |
FHR / FHR3
|
per-task edit FHR
|
Forecast hour (FHR3 zero-padded), drives the grid/cadence gates. |
FHR_LIST |
per-task edit FHR_LIST
|
Forecast-hour list. Degenerate (single hour) in the NCO def; the Rocoto counterpart dev/job_cards/rocoto/gempak.sh loops over it to bundle hours. |
COMOUT_ATMOS_GEMPAK_<grid> |
J-Job (declare -rx) |
Output COM dirs per grid; consumers per graph: JGFS_ATMOS_GEMPAK, JGDAS_ATMOS_GEMPAK, JGFS_ATMOS_GEMPAK_PGRB2_SPEC, exgfs_atmos_goes_nawips.sh. |
COMIN_ATMOS_GRIB_<grid> |
J-Job | Input GRIB2 COM dirs (0p25/0p50/1p00). |
DBN_ALERT_TYPE |
J-Job (:-GFS_GEMPAK) |
DBNet alert type for product distribution. |
NAGRIB, GEMGRD, GRIBIN, WGRIB2, SENDDBN, DBNROOT
|
nawips ex-script | GEMPAK conversion + distribution controls (from find_callers_callees GGSR env edges). |
########## config.gempak ##########
# GFS gempak step specific
export MAX_TASKS=42
source "${EXPDIR}/config.resources" gempakMinimal — sets the per-member task cap and pulls task resources from config.resources.
-
Centralized module loading via
load_modules.sh run(vs develop's per-taskmodule loadblock) — single source of truth for module versions across the suite. -
product/family layer: the v17 path insertsproduct/(ecf/scripts/gfs/product/atmos/gempak/gempak/…) compared with develop'secf/scripts/gfs/atmos/gempak/…, reflecting the reorganized product family tree. -
HOMEglobalshim:export HOMEglobal=${HOMEgfs}is present in the v17 master.ecf (not in the develop analog), bridging the v17 J-Job'sHOMEglobal/USHglobal/SCRglobalnaming to the NCOHOMEgfspackage layout. -
FHR_LISTplumbed but single-valued in the NCO def — the interface is identical between Rocoto (where bundling can happen) and NCO ecFlow (one task = one hour), so a future change to bundle hours only needsedit FHR_LIST '0,3,6'on a task. - The commented
#### ${HOMEgfs}/dev/job_cards/rocoto/gempak.shis a breadcrumb to the Rocoto wrapper; in NCO the master.ecf calls the J-Job directly.
| Symbol | Change type | Risk | Direct dependents |
|---|---|---|---|
JGFS_ATMOS_GEMPAK |
behavior | LOW (0.10) | 0 (suite leaf) |
exgfs_atmos_nawips.sh |
signature | LOW (0.30) | 1 (JGFS_ATMOS_GEMPAK) |
Editing exgfs_atmos_nawips.sh's argument contract (grid fhr3 alert comout) requires
updating only its single J-Job caller, but note product_functions.sh is shared by five
ex-scripts — changes there have wider blast radius.
- On-disk v17 worktree:
ecf/scripts/gfs/product/atmos/gempak/gempak/jgfs_atmos_gempak_master.ecf,dev/jobs/JGFS_ATMOS_GEMPAK,dev/parm/config/gfs/config.gempak,ecf/defs/gfs_prod.def,dev/workflow/prod.yml,dev/workflow/ecflow/ecflow_definitions.py. -
agentcore-mcp-ragMCP (tenantgw_v17):trace_data_flow,find_dependencies,find_callers_callees,trace_execution_path,find_env_dependencies,get_change_impact,get_code_context.
Content was rephrased for compliance with licensing restrictions where source docstrings were quoted.