APS3 ACCESS-C Experiment: BoM Operational City Model

Wenming LU [email protected]

Please see APS1 ACCESS City (aps1_access_c) and APS2 ACCESS City (aps2_access_c) for the introduction of this ACCESS City NWP forecasting suite.

APS3 ACCESS City suite is an upgrade from APS2 ACCESS-C, in which the UM version is updated from 8.2 to 10.6,

and the spatial resolution is kept same at 1.5km.

Checkout The suite

On accessdev, please

rosie checkout u-as385

The suite should be checked out at the default location at accessdev:$HOME/roses/u-as385.

Update the settings

On accessdev, please

cd  $HOME/roses/u-as385

Although it looks that the suite is complicated, in most cases, we only need to update rose-suite.conf for our runs,

[jinja2:suite.rc]
# Build UM in suite, or link in
BUILD_UM=true
# Domain to run; valids are "ADEPT", "DN", "AD", "PH", "BN", "SY", "VT" 
DOMAIN="ADEPT"
# End cycle
END_CYCLE="20181231T0000Z"
# HPC host
HPC_HOST="$(rose host-select hpc --timeout=20)"
# start dump name
IC_FILENAME="iau-start-dump.%Y%m%d%H%M"
# Location of start dump
IC_LOCATION="/g/data/lb4/ops_aps2/access-r/1/%Y%m%d/%H00/restart"
# LBC filename template
LBC_FILENAME="qwxbnka_piXXX_%Y%m%d%H_utc_fc.um"
# Location of LBC files
LBC_LOCATION="/g/data/lb4/ops_aps2/access-r/1/%Y%m%d/%H00/restart/pi_%Y%m%d%H"
# 2 step build with mirror
MIRROR_BUILD=true
# Forecast run length
RUN_END=48
# Start cycle
START_CYCLE="20180108T0000Z"
# Start hour(s) in format THH[,THH]
START_HR="T00"
# UMDIR. needed?
UMDIR="/projects/access/umdir"
# Location of ancillaries
UM_ANCIL_DIR="/g/data/access/ANCIL/APS3/"
# Location of UM build, if BUILD_UM=False
UM_BUILD_LOC=""
# Variable grid?
VARIABLE_GRID=true

This setting file is quite self-explanatory, you may check each comment for how to set variables.

Rarely, users may need to change the number of core for the runs. In this case, you need to modify the following section of suite.rc,

...
    [lam_um_fcst](/ACCESS-NRI/accessdev-Trac-archive/wiki/lam_um_fcst)
        inherit = HPC, TASK_ENV, GRID
        [[environment](/ACCESS-NRI/accessdev-Trac-archive/wiki/[environment)]
            ROSE_APP_OPT_CONF_KEYS = baseSTASHBOM_red {% if VARIABLE_GRID %} vargrid {% endif %} ([SITE])
            UM_ATM_NPROCX = 16
            UM_ATM_NPROCY = 47
            FLUME_IOS_NPROC = 16
            TOTAL_MPI_TASKS = 768 
            OMP_NUM_THREADS =2
            HYPERTHREADS = 1
        [[job](/ACCESS-NRI/accessdev-Trac-archive/wiki/[job)]
            execution time limit = PT80M
...

Note that certain decomposition may not be legally permitted by PBS on raijin. You need to test/experiment the optimal/legal decomposition for your own needs.

Run the suite

On accessdev, please

#The following two lines set the Rose/Cylc env; if the default env is different, please export the env for the suite
#export CYLC_VERSION=7.4.0
#export ROSE_VERSION=2017.05.0
cd  $HOME/roses/u-as385
rose suite-run --opt-conf-key=nci

The suite will run and start to generate boundary and initial conditions and the start forecasting.

Check the forecast output

• Check the built execs

  if you choose in rose-suite.conf,

  MIRROR_BUILD=true

  the suite will build the binary execs first and use your own built binaries in tasks. Please go to raijin:/short/$PROJECT/$USER/cylc-run/u-as385/share/fcm_make_um for checking those execs.

• Check the data

  If all tasks run successfully, please go to raijin:/short/$PROJECT/$USER/cylc-run/u-as385/share/cycle/{YYYYMMDD}T{HH}00 for the output,

  lam.rcf                  #initial condition
  lam_alabc_{DOMAIN}       #lateral boundary condition; {DOMAIN} will be replaced by the value of the domain
  lam_um                   #folder that contains forecast data
     umqvaa_20180115T03_cld_000      #clound fields
     ...
     umqvaa_20180115T03_mdl_000      #model level fields
     ...
     umqvaa_20180115T03_prs_000      #pressure level fields
     ...
     umqvaa_20180115T03_slv_000      #single level surface fields
     ...
     umqvaa_20180115T03_spec_000     #special sub-hour fields
     ...