Adding a New Mesh - StormSurgeLive/asgs GitHub Wiki

Introduction

Adding support for a new mesh is not really hard, but over there years varying levels of support have been introduced.

Adding support for a mesh consists first of all, adding a new entry in the configs/mesh_defaults.sh. this file is where all of the "officially" supported mesh defaults go. It is also from where the command list meshes gets its information.

Quick Start

Adding a new mesh to the ASGS consists of the following steps:

  1. Create an issue for the new mesh in the ASGS issue tracker
  2. Make a branch from master with the issue name and number and check out the branch
  3. Add a directory to hold the files related to the mesh in $SCRIPTDIR/input/meshes/<MESHNAME> (hereafter referred to as the $MESHNAMEDIR in these instructions)
  4. Place a sample fort.15 that has been used successfully with this mesh into $MESHNAMEDIR along with the mesh file (fort.14), the nodal attributes file (fort.13) and any other static input files
  5. Make a copy of the fort.15 file with the name <MESHNAME>_fort.15.template
  6. Edit the <MESHNAME>_fort.15.template file to turn it into a template
  7. The file $SCRIPTDIR/input/meshes/HSOFS/hsofs_explicit.15.template can be used as an example of how to make a fort.15 template file
  8. Copy the file $SCRIPTDIR/input/meshes/common/swan/swaninit.template into $MESHNAMEDIR
  9. Copy the file $SCRIPTDIR/input/meshes/common/swan/adcirc_swan_v53_fort.26.template into $MESHNAMEDIR
  10. Copy the station lists (for water surface elevation, water current velocity, and meteorology) from the original fort.15 file into their own files (3 files total), putting them into standard form if necessary
  11. Standard form for stations is lon lat ! stationID ! agency ! description
  12. Create properties files for the nodal attributes (fort.13), mesh (fort.14), and control (fort.15) files to hold the values of the sea_surface_height_above_geoid, general geographic location (asgs property), and the wet/dry threshold (H0); the nodal attribute value can be discovered by looking at the header for the nodal attributes file, and the wet/dry threshold can be easily determined from the fort.15 file with the command grep -i H0 fort.15

Note: These unstructions are useful for adding a locally supported mesh, please see the information on adding local assets.

Configuration Sensitive Input Files

From the start, ASGS has maintained the ability to update critical input files, such as the fort.15, from run-to-run. This ability is required for any forecasting system to be automated. As such, all of the officially supported meshes have had their input files templatized, i.e., certain values in the fort.15 (e.g.) have been replaced with placeholders. When ASGS prepares a run, there is a stage where it fills in these values.

Adding a mesh and input files requires that some of the fort.15 and other files be turned into templates. This process is tedious, but once completed, the mesh may be used operationally in the the automated forecasting mode.

The best place to see how this is done is to inspect the official mesh_defaults.sh file itself. For example, here is an entry for the EC2001v2e mesh:

   "ec2001_v2e"|"EC2001v2e")
      #
      nodes=254565
      elements=492179
      INPUTDIR=$SCRIPTDIR/input/meshes/EC2001
      GRIDFILE=ec2001_v2e.grd   # mesh (fort.14) file
      MESHPROPERTIES=${GRIDFILE}.properties
      CONTROLTEMPLATE=ec2001_v2e_fort.15.template   # fort.15 template (designed for 1s timestep)
      # wind at 10m fort.15 template
      CONTROLTEMPLATENOROUGH=$CONTROLTEMPLATE  # same b/c no inundation coverage
      ELEVSTATIONS=hsofs_stations_20180907.txt
      VELSTATIONS=$ELEVSTATIONS
      METSTATIONS=$ELEVSTATIONS
      NAFILE=ec2001_v2e.13
      # intersection between mesh, models, hpc platform, and number of compute cores:
      HINDCASTWALLTIME="06:00:00" # hindcast wall clock time
      ADCPREPWALLTIME="01:00:00"  # adcprep wall clock time, including partmesh
      NOWCASTWALLTIME="01:00:00"  # longest nowcast wall clock time
      FORECASTWALLTIME="01:00:00" # forecast wall clock time
      UNITOFFSETFILE=null
      ;;
Variable Value Comment
nodes 254565
elements 492179
INPUTDIR $SCRIPTDIR/input/meshes/EC2001 this is the local directory that contains the input files
GRIDFILE ec2001_v2e.grd mesh (fort.14) file
MESHPROPERTIES ${GRIDFILE}.properties primary fort.15 template
CONTROLTEMPLATE ec2001_v2e_fort.15.template fort.15 template (designed for 1s timestep)
CONTROLTEMPLATENOROUGH $CONTROLTEMPLATE same b/c no inundation coverage
ELEVSTATIONS hsofs_stations_20180907.txt
VELSTATIONS $ELEVSTATIONS
METSTATIONS $ELEVSTATIONS
NAFILE ec2001_v2e.13 nodal attributes template (fort.13)
HINDCASTWALLTIME "06:00:00" hindcast wall clock time
ADCPREPWALLTIME "01:00:00" adcprep wall clock time, including partmesh
NOWCASTWALLTIME "01:00:00" longest nowcast wall clock time
FORECASTWALLTIME "01:00:00" forecast wall clock time
UNITOFFSETFILE null

Since one may put essentially any environmental variable in these sections to affect the runs, the best place to get a sense of what can go in there is to look at the officially supported mesh_defaults.sh file. It has evolved over nearly 20 years, and contains a wealth of operationally relevant details about what can be done inside of this file.

Configuration Sensitive Input Files with multiple parameter packages

ASGS also supports Parameter Packages; this allows us to use the same grid, but to specify different configurations based on the use case.

This is an advanced set-up, and may be best to reach out to us if you wish to go this route. Below is an example from the mesh_defaults.sh file:

   "CPRA23v05b"|"CPRA23v05c")
      #
      nodes=1577268
      elements=3072131
      INPUTDIR=$SCRIPTDIR/input/meshes/CPRA23
      GRIDFILE=cpra_2023_hurricane_v05b_chk.grd     # mesh (fort.14) file
      if [[ $GRIDNAME == "CPRA23v05c" ]]; then
         GRIDFILE=cpra_2023_hurricane_v05c_chk.grd  # mesh (fort.14) file
      fi
      MESHPROPERTIES=${GRIDFILE}.properties
      ELEVSTATIONS=combined_stations_20230824.txt
      VELSTATIONS=$ELEVSTATIONS
      METSTATIONS=$ELEVSTATIONS
      # intersection between mesh, models, hpc platform, and number of compute cores:
      HINDCASTWALLTIME="18:00:00" # hindcast wall clock time
      ADCPREPWALLTIME="02:00:00"  # adcprep wall clock time, including partmesh
      NOWCASTWALLTIME="07:00:00"  # longest nowcast wall clock time
      FORECASTWALLTIME="07:00:00" # forecast wall clock time
      case $parameterPackage in
      "hardcoded")
         CONTROLTEMPLATE=CPRA23v01c_5kcms_fort.15.template
         # wind at 10m fort.15 template
         CONTROLTEMPLATENOROUGH=CPRA23v01c_nowindreduction_fort.15.template
         nodal_attribute_default_values["sea_surface_height_above_geoid"]="0.362102"
         NAFILE=cpra_2023_hurricane_v05b.13
         SWANTEMPLATE=CPRA23v01c_fort.26.template
         if [[ $GRIDNAME == "CPRA23v05c" ]]; then
            NAFILE=cpra_2023_hurricane_v05c.13
         fi
         ;;
      "default")
         CONTROLTEMPLATE=CPRA23.15.ASGS2024.1.template
         # numerics/physics (fort.15)
         advection="off"                       # on|off for advection (NOLICA=1|0/NOLICAT=1|0)
         solver_time_integration="implicit"    # implicit|explicit|full-gravity-wave-implicit
         time_weighting_coefficients="0.35 0.3 0.35" # A00 B00 C00 in fort.15
         lateral_turbulence="eddy_viscosity"   # "smagorinsky" or "eddy_viscosity"

To activate a specific section, the following set up should be available in the $ASGS_CONFIG file:

GRIDNAME=CPRA23v05b
parameterPackage=default
...

Now if one wishes to create a new set of parameters for the same mesh, then they would just need to add a case like default or hardcode above. The names are not special, we tend to use the same jargon to name the common cases.

Note, any variable may be redefined in each of these sections, including INPUTDIR, CONTROLTEMPLATE, etc.

Controlling Download and File Sources

ASGS supports defining remote URIs fromw which the specified files may be downloaded.

Variable Value Comment
INPUTDIR $ASGS_LOCAL_DIR/input/meshes/atl-parameterPackage-default differs from initialization
MESHEXT xz supports 3 different compressinos: xz (default), gz, null (no compression)
MESHURL ssh://tds//data/archive/meshes/atl/parameterPackage-default supported schemes: https://, http://, scp:// ssh://
NODALATTRIBUTESURL $MESHURL
OFFSETURL $MESHURL
LOADTIDEURL $MESHURL
INITINPUT get-lfs.sh optional script that can be run if it exists in $INPUTDIR; it is just a name, not a path; it is assumed to exist inside of $INPUTDIR and run from within there. This is for advanced users; use with care. Ask for help if you are unsure.

For example, these are described in hypothetical entry below:

case $GRIDNAME in
      #   
   "atl")
      #   
      nodes=6056968
      elements=11969815
      INPUTDIR=$ASGS_LOCAL_DIR/input/meshes/atl
      INITINPUT=asgs-lfs.sh # name of file, not path; it is always run inside of $INPUTDIR, `if [ -x "$INPUTDIR/$INITINPUT" ]` is true
      GRIDFILE=atl.14  # mesh (fort.14) file
      MESHPROPERTIES=${GRIDFILE}.nc.properties
      # wind at 10m fort.15 template
      ELEVSTATIONS=atl_stations.txt
      VELSTATIONS=$ELEVSTATIONS
      METSTATIONS=$ELEVSTATIONS
      RIVERINIT=null
      RIVERFLUX=null              # this mesh has no rivers ...RIVERFLUX=null
      HINDCASTRIVERFLUX=null
      # interaction between mesh and models:
      TIMESTEPSIZE=2.0            # adcirc time step size (seconds)
      SWANDT=1800                 # swan timestep / coupling interval (seconds)
      #zNorth="112.8516 40.3289  ! China-Argentina"  #only needed for negative ICS, like -ICS22 (rotated)    
      selfAttractionEarthLoadTide=atl.24
      # intersection between mesh, models, hpc platform, and number of compute cores:
      HINDCASTWALLTIME="36:00:00" # hindcast wall clock time
      ADCPREPWALLTIME="02:00:00"  # adcprep wall clock time, including partmesh
      NOWCASTWALLTIME="24:00:00"  # longest nowcast wall clock time
      FORECASTWALLTIME="24:00:00" # forecast wall clock time
      # unit offset url https://asgs-static-assets.sfo2.digitaloceanspaces.com/offsets/unit_offset_hsofs.dat.xz
      UNITOFFSETFILE=null
      case $parameterPackage in
        "hardcoded")
           CONTROLTEMPLATENOROUGH=atl.nowindreduction.15.template
           CONTROLTEMPLATE=atl_fort.15.template
           CONTROLPROPERTIES=${CONTROLTEMPLATE}.properties
           NAFILE=atl.13
           NAPROPERTIES=${NAFILE}.properties
           ;;  
        "default")
           #INITINPUT=get-lfs.sh
           MESHEXT=xz
           MESHURL=ssh://tds//data/archive/meshes/atl/parameterPackage-default
           NODALATTRIBUTESURL=$MESHURL
           OFFSETURL=$MESHURL
           LOADTIDEURL=$MESHURL
           INPUTDIR=$ASGS_LOCAL_DIR/input/meshes/atl-parameterPackage-default      # differs from initialization
           CONTROLTEMPLATE=atl-default-fort.15.template

Example $INITINPUT Script

This script must implement checking itself, since it is generic. In the example below, $INITINPUT is set up to be always run. It is always run in $INPUTDIR, and checks for each required file individually. If the file doesn't exist, it scps it from the remote location to thw current working directory, which will always be $INPUTDIR.

#!/bin/sh

# fetches additional large files from a last know good source

if [ ! -e ./atl.13 ]; then
  scp tds:/data/archive/meshes/atl/atl.13 atl.13
fi

if [ ! -e ./atl.14 ]; then
  scp tds:/data/archive/meshes/atl/atl.14 atl.14
fi

if [ ! -e ./atl.24 ]; then
  scp tds:/data/archive/meshes/atl/atl.24 atl.24
fi

Make an Upstream Request

If you wish for a mesh to be added, create a new issue; and provide the following information:

  • name of the mesh, where we can find the mesh file itself
  • where we can find all relavent input files
  • justification for including it in the official repo

Note: if there is no justification for adding it to the official repo, you may still request help in creating your own local assets. ASGS supports the inclusing of locally maintained assets, such as meshes. More information on adding and maintaining and including your own local assets is available here.

Table of Contents | Need Help?

⚠️ **GitHub.com Fallback** ⚠️