C48_ATM gfs_fcst_seg0 - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki

C48_ATM GFS Forecast Segment 0 - Test Case Documentation

Test Case: C48_ATM-gfs_fcst_seg0.yaml
Configuration: C48_ATM (Atmosphere-Only)
Job: JGLOBAL_FORECAST
Duration: 120-hour forecast (f000-f120, 3-hourly output)
Status: ✅ VERIFIED CORRECT Last Updated: October 1, 2025

Overview

This test validates the complete atmospheric forecast capability of the global-workflow system, executing a 5-day deterministic forecast with the UFS Weather Model in atmosphere-only mode. This is the baseline test for forecast model stability and output generation.

Total Files:

  • Input: 13 files (atmospheric initial conditions)
  • Output: 209 files (4 + 41×5 categories)

File Breakdown

Input Files: 13

All initial condition files located in gdas.{PDY}/{cyc}/model/atmos/input/:

Category Count Files Purpose
Control File 1 gfs_ctrl.nc Model configuration and metadata
Atmospheric Data 6 gfs_data.tile[1-6].nc 3D atmospheric state (cubed-sphere tiles)
Surface Data 6 sfc_data.tile[1-6].nc 2D surface fields (cubed-sphere tiles)

Cubed-Sphere Structure: C48 resolution uses 6 tiles covering the globe, each tile is 48×48 grid points.

Output Files: 209

All output files located in gfs.{PDY}/{cyc}/model/atmos/ subdirectories:

Category Count Frequency Location Purpose
Configuration 4 One-time history/ UFS model settings
Atmosphere Logs 41 Every 3hr history/ Diagnostic output
Atmosphere History 41 Every 3hr history/ atmf*.nc - 3D atmospheric state
Surface History 41 Every 3hr history/ sfcf*.nc - 2D surface fields
Master GRIB2 41 Every 3hr master/ master.grb2f* - Full fields for products
Surface Flux GRIB2 41 Every 3hr master/ sfluxgrbf*.grib2 - Surface flux fields

Why 41 files per category?
Formula: (120 hours - 0) / 3 hours + 1 = 41 time steps

  • Starts at f000 (initial time)
  • Ends at f120 (5 days)
  • Output interval: 3 hours (configured via FHOUT)
  • Includes both endpoints

Key Insights

Directory Structure Pattern

gfs.{PDY}/{cyc}/model/atmos/
├── input/          # Initial conditions (13 files)
├── history/        # NetCDF history files (82 files + 4 config)
└── master/         # GRIB2 files for products (82 files)

Critical: Uses model/ directory for raw forecast output, NOT products/ directory. This is correct - products are generated in a separate downstream job.

Master GRIB2 Generation

Configuration Setting: WRITE_DOPOST=.true. (default)

  • Location: parm/config/gfs/config.base.j2 line 345
  • Effect: Enables inline POST processing during forecast
  • Output: Creates master.grb2f* files automatically
  • Processing: Done by ush/forecast_postdet.sh lines 310-350

Why inline POST?

  • Efficiency: POST runs in parallel with forecast continuation
  • Memory: Data processed while still in memory
  • Storage: Avoids storing intermediate unprocessed files

Critical Downstream Dependencies

These forecast outputs are required inputs for downstream jobs:

  1. JGLOBAL_ATMOS_PRODUCTS (atmospheric product generation)

    • Requires: master.grb2f* files (mandatory input)
    • Requires: sfluxgrbf*.grib2 files (for flux products)
    • Reference: scripts/exglobal_atmos_products.sh lines 40 & 176

  2. JGLOBAL_ATM_ANALYSIS (data assimilation cycling)

    • Requires: atmf*.nc and sfcf*.nc files
    • Used for first guess in analysis

Test Case Dependency Chain:

C48_ATM-gfs_fcst_seg0.yaml (this test)
    ↓ Provides: master.grb2f000, master.grb2f003
    ↓
C48_ATM-gfs_atmos_prod_f000-f002.yaml
    ↓ Generates: products/atmos/grib2/ files

Data Flow

Initial Conditions (13 files)
    └─> Model Input: gfs_ctrl.nc, gfs_data/sfc_data tiles
    ↓
UFS Weather Model Execution (120-hour run)
    ├─> FV3 Dynamical Core (cubed-sphere dynamics)
    ├─> Physics Package (parameterizations)
    └─> Inline POST Processing (WRITE_DOPOST=.true.)
    ↓
Output Generation (every 3 hours)
    ├─> NetCDF History Files
    │   ├─> 41 × atmf*.nc (3D atmospheric state)
    │   └─> 41 × sfcf*.nc (2D surface fields)
    └─> GRIB2 Master Files
        ├─> 41 × master.grb2f* (full fields for regridding)
        └─> 41 × sfluxgrbf*.grib2 (surface flux fields)
    ↓
Total Output: 209 files in model/ directory

Processing Timeline:

  • Hours 0-24: Critical forecast period (weather events)
  • Hours 24-72: Medium-range forecast (3-day outlook)
  • Hours 72-120: Extended forecast (days 4-5)

Comparison with Products Test

Aspect Forecast Test Products Test
Purpose Generate model output Post-process into user products
Duration 120 hours (5 days) 2 hours (f000, f001, f002)
Input Files 13 (IC only) 17 (IC + forecast outputs)
Output Files 209 14
Output Directory model/ products/
File Count Ratio 11.6× more files Minimal for efficiency
Test Focus Model stability & physics Product generation logic
Run Time ~15-30 minutes ~1-2 minutes
File Size ~15-20 GB ~1-2 GB

Why such different file counts?

  • Forecast test: Must validate 5-day model stability at all timesteps
  • Products test: Only needs to prove regridding/formatting logic works

Configuration Details

Key Parameters from config.base.j2

# Forecast Configuration
CASE="C48"                    # ~200 km resolution
FHMAX=120                     # Maximum forecast hour
FHOUT=3                       # Output frequency (hours)
WRITE_DOPOST=".true."         # Enable inline POST

# Output Control
FHMAX_GFS=120                 # GFS-specific max hour
FHOUT_GFS=3                   # GFS output frequency

Resolution Characteristics

C48 Grid:

  • Global coverage: 6 cubed-sphere tiles
  • Horizontal resolution: ~200 km
  • Purpose: Fast testing (operational is C384 = ~25 km)
  • Grid points per tile: 48 × 48 = 2,304
  • Total atmosphere columns: 6 × 2,304 = 13,824

Operational Significance

What This Test Validates

✅ Model Stability: 120-hour integration without crashes
✅ Physics Packages: All parameterizations execute correctly
✅ Output Generation: Files created at all forecast hours
✅ Inline POST: GRIB2 files generated during forecast
✅ File Naming: Operational naming conventions followed
✅ Downstream Readiness: Products job has required inputs

Critical for Production

This test ensures:

  • 5-day GFS forecasts will run to completion
  • All output files needed for downstream processing exist
  • File formats match operational standards
  • No memory leaks or numerical instabilities over 120 hours

Operational Context

Production GFS:

  • Resolution: C384 (~25 km) vs C48 test (~200 km)
  • Duration: 384 hours (16 days) vs 120 hours test
  • Output: Every 3 hours early, every 6 hours late
  • Ensemble: 31 members (GEFS) vs single deterministic

This test provides:

  • Rapid validation (30 min vs hours for full production run)
  • Confidence in code changes before operational deployment
  • Verification of forecast-products job chain

Verification Commands

Run This Test

# Execute forecast test
ctest -R "C48_ATM.*fcst.*validate" --verbose

# Check specific outputs
ls -lh /path/to/gfs.{PDY}/{cyc}/model/atmos/history/atmf*.nc
ls -lh /path/to/gfs.{PDY}/{cyc}/model/atmos/master/master.grb2f*

Verify Output Count

# Should find 41 files each
find . -name "atmf*.nc" | wc -l     # 41
find . -name "sfcf*.nc" | wc -l     # 41
find . -name "master.grb2f*" | wc -l # 41

Recent Updates & Insights

October 1, 2025 - Validation Success ✅

Status: All tests passing after products/ path fixes in other test cases

Verification: This test case required no changes because:

  1. ✅ Correctly uses model/ paths (not products/)
  2. ✅ Generates forecast output (doesn't consume products)
  3. ✅ All 209 expected files present
  4. ✅ Provides correct inputs for downstream products test

Key Learning: The directory structure distinction is critical:

  • Forecast jobs → Write to model/ (raw output)
  • Product jobs → Read from model/, write to products/ (distribution)

Related Test Cases

  1. C48_ATM-gfs_atmos_prod_f000-f002.yaml - Consumes this test's outputs
  2. C48_S2SW-gfs_fcst_seg0.yaml - Coupled version (ATM+OCN+ICE+WAV)
  3. C48_S2SW-gefs_fcst_mem001.yaml - Ensemble member version

Technical Notes

Why 3-Hour Output?

Balance between:

  • Storage: 3-hourly saves ~60% disk space vs hourly
  • Detail: Sufficient for NWP applications
  • Products: Matches operational product generation frequency

Operational Strategy:

  • High-frequency period: Hourly f000-f048 (for rapid weather)
  • Standard period: 3-hourly f051-f120 (for extended outlook)

File Size Estimates

File Type Size per File Total (41 files)
atmf*.nc ~200 MB ~8 GB
sfcf*.nc ~50 MB ~2 GB
master.grb2f* ~100 MB ~4 GB
sfluxgrbf*.grib2 ~20 MB ~800 MB
Total ~15 GB

References

Source Files

  • Test Definition: dev/ctests/cases/C48_ATM-gfs_fcst_seg0.yaml
  • Job Script: jobs/JGLOBAL_FORECAST
  • Execution Script: scripts/exglobal_forecast.py
  • Forecast Logic: ush/forecast_det.sh
  • POST Processing: ush/forecast_postdet.sh

Configuration Files

  • Base Config: parm/config/gfs/config.base.j2
  • Forecast Config: parm/config/gfs/config.fcst
  • UFS Templates: parm/ufs/fv3/

Documentation

  • Repository: TerrenceMcGuinness-NOAA/global-workflow
  • Branch: ctest_case_updates
  • Changelog: CTEST_UPDATES_CHANGELOG.md

Created: January 16, 2025
Updated: October 1, 2025
Status: Production-ready test case, verified correct