access_re_access_x_vn7.8_1.5km_detail - ACCESS-NRI/accessdev-Trac-archive GitHub Wiki

Model in Details

access_x_vn7.8_1.5km shares the same executable with access_c_vn7.8_1.5km. Please go to

https://trac.nci.org.au/trac/access/wiki/access_c_vn7.8_1.5km_detail

for the instructions on compiling the source code. We will focus on detailing ACCESS-X

system, suite and design in this section.

The standard experiment access_x_vn7.8_1.5km consists of four parts,

  • System files: located on raijin:$HOME/roses/access_x_vn7.8_1.5km, copied by the installation script.
  • Suite files: located on accessdev:$HOME/roses/access_x_vn7.8_1.5km, copied by the installation script.
  • Data output: located on raijin:/short/$PROJECT/$USER/access_x_vn7.8_1.5km_S and

raijin:/short/$PROJECT/$USER/ncdata/access_x_vn7.8_1.5km_S;

those are the forecast data produced by the UM run jobs.

  • Source code: located on raijin:/short/$PROJECT/$USER/access_x_vn7.8_1.5km_src, copied by the installation script

we will explain the components and the structure of the system files in details; we also describe

the tasks/steps in the suite definition in details for the better understanding of the experiment.

access_x_vn7.8_1.5km in Details

The structure of access_x_vn7.8_1.5km system files is detailed here for deeper understanding of the system.

After grabbing those knowledge, users may be able to engage in advanced research activities.

  • System files:raijin:$HOME/roses/access_x_vn7.8_1.5km
  • beans:
  • ancil: executable/scripts for producing ancillary files
  • makebc: executable/scripts for making lateral boundary condition
  • mars: executable/scripts for converting/archiving data
  • um: executable/scripts for um forecasting
  • um_components: advanced users may modify the configuration files inside the two following components

for research * xbcjx: um component called by recon and um_run jobs; shared by 6 domains. * xbdmx: um component called by recon_lores; shared by 6 domains.

  • umr: executable/scripts for making initial condition
  • util: utility Python scripts
  • bin:
  • env.access_x_vn7.8_1.5km: settings for the system, such as data input and output directory, log directory,

working directory, archiving options, locations of dump and pi files etc;

you may configure settings as you wish

  • produceLinks.ksh: script to create soft links to Cylc scripts
  • Cylc scripts: Cylc job script; you may check what is really done by investigating scripts closely
  • conf (only advanced users may need to modify some of the configuration files if users understand

what they are doing):

  • ANCILmaster:

  • PS25:

  • STASHmaster:

  • ancil: ancillay files, including land-sea-mask, landfrac and orog produced on fly

  • spectral:

  • log.S: log files from all tasks of the suite when running

  • Cylc Suite:accessdev:$HOME/roses/access_x_vn7.8_1.5km

  • suite.rc: suite definition file which defines jobs, job flow, cycle time, resources for jobs etc;

the command rose suite-run picks up this file suite.rc to run the model

  • other files but not important in this case

  • Output data: raijin:/short/$PROJECT/$USER/

  • raijin:/short/$PROJECT/$USER/access_x_vn7.8_1.5km_S conatins all input and output data running the model;

in this folder each run is given an individual directory as YYYYMMDDHH, such as 2014011800 to

accommodate inputs and outputs. In this directory of YYYYMMDDHH,

  • ic: dump file for producing IC for each domain
  • lbc: pi files for producing LBC for each domain; usually soft links to pi files to save disk usage
  • SW1: IC file astart.BN, LBC file lbc.BN and forecast pp files
  • SW2: similar to SW1
  • Note: The directory raijin:/short/$PROJECT/$USER/access_x_vn7.8_1.5km_S/YYYYMMDDHH will be moved to

raijin:/short/$PROJECT/$USER/access_x_vn7.8_1.5km_S/previous_dates/YYYYMMDDHH.$$ after suite completes.

$$ is the run ID for the specific job does the move

  • raijin:/short/$PROJECT/$USER/ncdata/access_x_vn7.8_1.5km_S contains all forcast data in nectCDF format;

go to YYYYMMDDHH/DOMAIN, e.g., 2014011800/SW1 for SW1 forecasting data at 2014011800Z run.

  • Source codes: raijin:/short/$PROJECT/$USER/access_x_vn7.8_1.5km_src;

see https://trac.nci.org.au/trac/access/wiki/access_c_vn7.8_1.5km_detail

for compilation instruction

  • umbase: Modify source codes in here for testing new science
  • umrecon: Contains configuration file to compile recon executable
  • ummodel: Contains configuration file to compile um executable

access_x_vn7.8_1.5km Cylc Suite in Details

The suite defintion file suite.rc of access_x_vn7.8_1.5km is located at accessdev:$HOME/roses/access_x_vn7.8_1.5km/.

The suite definition file defines all the jobs of access_x_vn7.8_1.5km standard experiment.

You may modify the suite definition file for the following purposes,

  • add or remove tasks from the suite.rc
  • update the cycle time by modifying initial cycle time and final cycle time;

in the standard experiment, initial cycle time and final cycle time are both set at

2014011800: it only does one run at 2014011800Z

  • tune the resources, such as number of cores and memory, for each task in the directive item

We use a tree-like structure to display all tasks in suite.rc and their descriptions.

Note that tasks are listed in logically running order, which may not be the case in some Cylc views which may

use alphabetical order.

  • -->setup
  • AC_setup : setup the folders for all jobs; Please go to

DATADIR=/short/$PROJECT$USER/access_x_vn7.8_1.5km_S to check the output

  • wait_for: triggered by completion of AC_setup; check the availability of

the input data for creating initial and LB conditions

  • AC_get_initial_conditions: triggered by wait_for; get the dump file
  • AC_get_LB_conditions: triggered by wait_for; get the pi files

(in most cases produce soft links to pi files to save disk usage)

  • --> SW1
  • ancil_nml: read raijin:$HOME/roses/access_x_vn7.8_1.5km/SW1 or SW2 and

produce the ancil namelist to raijin:$HOME/roses/access_x_vn7.8_1.5km/beans/ancil;

the output is grid.SW1.nml or grid.SW2.nml.

  • ancil: read ancil namelist and create the ancillary files onto

raijin:$HOME/roses/access_x_vn7.8_1.5km/conf/ancil/SW1 or SW2;

ancillary files include qrparm.landfrac, qrparm.mask and qrparm.orog.

  • makebc_nml: read grid.SW1.nml or grid.SW2.nml and produce the makebc namelist;

output is makebc.SW1.nml or makebc.SW2.nml in raijin:$HOME/roses/access_x_vn7.8_1.5km/beans/makebc

  • um_nml: read ancillary files and produce um namelist; output is SW1.um.nml or SW2.um.nml in

raijin:$HOME/roses/access_x_vn7.8_1.5km/beans/um

  • make_lbc: producing the lateral boundary condition
  • recon: reconfiguration step to produce the initial condition at ACCESS-X resolution
  • recon_lores: reconfiguration step to produce the initial condition at

the original resolution of the nested NWP system, i.e., ACCESS-R (12km) in this case

  • finalise_recon: copy wind fields from the IC of step recon_lores to that of recon

  • um_run: UM forecasting job

  • um2grb: convert forecast data from PP format to grib as requested

  • ncdf: convert forecast data from PP format to netCDF as requested

  • --> SW2

  • -->finalise

  • AC_cleanup: clean up unnecessary files after forecasts finish

Note:

  • Jobs under each domain are same, i.e. as shown for SW1 domain.
  • Please refer to the Cylc scripts located in raijin:$HOME/roses/access_x_vn7.8_1.5km/bin for more information.

In every script, usually the purpose of the script is described at the header section. In addition,

the input data required by the job and the output produced by the job are also recorded at the header

section for reference.