Documentation on building APS1 Global model executable (re UMUI job xblnf)

---

Ilia Bermous, 6 August 2013

Here is an updated version of the description provided below on 1 August 2013.

There are several important components in relation to build a UM executable using UMUI:

• UM sources
• GCOM library and its location
• software stack (Intel compiler versions and OpenMPI library version) to build an executable
• building procedure including Fortran and C compiler options
• fcm stuff

Let’s address the above mentioned information:

1. UM sources are specified in a Model Selection => FCM Configuration => FCM Options for Atmosphere and Reconfiguration panel. In case when a UM executable is built from a single repository “location” which can be a version from the trunk or a version from a user branch which includes all required changes then only a single top section of the above mentioned panel should be used with a revision number specified in the Specify revision number or keyword of code base to use. Note that a word “HEAD” can be specified instead of an explicit revision number if the head of the corresponding repository pointing out to the right source version.

2. GCOM library version and its location is specified by the gcom_path term setting in the

`configs/machines/linux-mpich2-meto/ext_libs/gcom_mpp.cfg` 

file for a parallel version of the corresponding UM executable.

3. Software stack to build an executable is defined by an Intel compiler revision and an OpenMPI library version with the corresponding module settings. Currently the required module settings are hard coded and defined in

`/access/umdir/vn7.5/linux/scripts/prg_env_vn7.5_01` 

file. The corresponding hard coded reference to this file is included in a UMUI

`umui2.0/vn7.5/processing/nds_bld_gencmd`

script. Note that the software stack used to create the GCOM library mentioned in item 2 should be the same as the software stack to build a UM executable.

I would like to note that another way of setting up the building environment is done via script

`.umsetvars_$VN (where $VN is UM version, in our case it is 7.5)`

If this file exists in the $USER home directory then the related environment is sourced by UMUI

`umui2.0/vn7.5/processing/fcm_script` 

script otherwise a general version

`$UMDIR/vn$VN/$TARGET_MC/scripts/.umsetvars_$VN`

is used.

4. Building procedure including the names of Fortran and C compilers as well as compilation and some pre-processor options for a parallel version are set via

`configs/machines/linux-mpich2-meto/machine.cfg`

file for our Solar system.

5. A UK Met Office fcm utility is used to build an executable. fcm1.5 version is used to produce APS1 Global executable, so this version of fcm is hard coded in the related UMUI scripts.

6. Compilation output location is set by default to the “$HOME/output” location. A different location can be specified via an MY_OUTPUT environment variable setting in the Defined Environment Variables for Directories table of the Model Selection => Input/Output Control and Resources => Time Convention and SCRIPT Environment Variables. panel.

7. The location of the extraction directory to build an executable is defined via the following things

• UM_ROUTDIR variable setting in the Model Selection => FCM Configuration => FCM Extract directories and Output levels panel

• then a UMUI

   `vn7.5/processing/nds_extract` 
  

script adds “automatically” a couple of subdirectory levels $USERID/$RUNID to the UM_ROUTDIR setting mentioned above

• the building is done under

   `$UM_ROUTDIR/$USERID/$RUNID/ummodel`  and 
  

• a model binary file is produced in

   `$UM_ROUTDIR/$USERID/$RUNID/ummodel/bin`
  

directory in case of success, also

• if the settings for the

               **specify the directory for the executable** option `        `(1)
  

in 2 panels (Model Selection => Compilation and Modifications => Compile options for the UM model and Model Selection => Compilation and Modifications => Compile options for the UM reconfiguration) are the same (where DATAW environment variable is defined via Model Selection => Input/Output Control and Resources => Time Convention and SCRIPT Environment Variables. panel) then executables for UM and reconfiguration as well as other UM small utilities are placed in the directory specified by option (1).

8. From my experience the netcdf library to produce a UM executable is not used so references to the netcdf library should be removed from the corresponding compilation settings and the linking procedure. The related references are in

`machines/linux-mpich2-meto/ext_libs/netcdf.cfg`.

9. UMUI building job should not include any “explicit” references to $USER locations for input/output.

10. A reconfiguration executable “qxreconf “ should be built as well and it can be found under 2 directories:

`$UM_ROUTDIR/$USERID/$RUNID/umrecon/bin`


`DATAW/bin`

---

Michael Naughton, 2 August 2013

Update:

• Ilia has updated this job and notes below; notes on updated version will replace those below in due course.
• I have done a "peer-review" check of this job, consisting of running xblnf job with process-submit to check that compilation succeeds to the point of creating executable for me. So far so good.
• This build job still has further steps ahead before it's completed:
  • update to use branch with Zhian's mods (as noted below).
  • change from v11 to v12 software stack.
  • porting to do the build on ngamai.
• On the testing side, Ilia has done some testing of the UM exec; more standalone UM forecast testing may be done, or else the ACCESS-G exec will just be tested in ngamai ACCESS-G NWP suite (item 26).

---

Michael Naughton, 1 August 2013

In principle, all that is needed to build the APS1 ACCESS-G UM executable is to process and run the xblnf UMUI job. Ilia's notes here document the details about the build procedure that are important for developers to understand what is going on in the build job.

Note that this is not the final version, Zhian's changes to clouds settings still need to be included, which will only require change to the SVN source version used for the build.

---

Ilia Bermous, 1 August 2013

There are several important components in relation to build a UM executable using UMUI:

• UM sources
• GCOM library and its location
• software stack (Intel compiler versions and OpenMPI library version) to build an executable
• building procedure including Fortran and C compiler options
• fcm stuff

Let’s address the above mentioned information:

1. UM sources are specified in a Model Selection => FCM Configuration => FCM Options for Atmosphere and Reconfiguration panel. As usually the sources are taken from a branch so a corresponding branch reference should be included in the “User Modification” table of the above mentioned panel using an explicit revision number specified in the “Revision” column. It is recommended that an “Include modifications from user working copy” option should not be selected.

2. GCOM library version and location is specified in the gcom_path variable in the configs/machines/linux-mpich2-meto/ext_libs/gcom_mpp.cfg file.

3. Software stack to build an executable is defined by Intel compiler revision and OpenMPI library version with the corresponding module settings. Currently the required module settings are hard coded and defined via /access/umdir/vn7.5/linux/scripts/prg_env_vn7.5_01 file. The corresponding hard coded references to this file is included in a UMUI umui2.0/vn7.5/processing/nds_bld_gencmd script. Note that the software stack used to create the GCOM library mentioned in item 2 should be the same as the software stack to build UM executable.

I would like to note that another way of setting up the building environment is done via .umsetvars_$VN (where $VN is UM version, in our case it is 7.5) script. If this file exists in the $USER home directory then the related enviornemnt is sourced by UMUI umui2.0/vn7.5/processing/fcm_script script otherwise a general version $UMDIR/vn$VN/$TARGET_MC/scripts/.umsetvars_$VN is used.

4. Building procedure including the names of Fortran and C compilers as well as compilation and some pre-processor options for a parallel version are set via configs/machines/linux-mpich2-meto/machine.cfg file for our system.

5. UK Met Office fcm utility is used to build an executable. **fcm1.**5 version is used to produce APS1 Global executable, so this version of fcm is hard coded in the related UMUI scripts. At the same time there is a possibility that some building jobs may include references to fcm related names for example such as nci_um_br which should be correctly defined in the user $USER/.fcm file. For example,

set::repos::nci_um_br !https://access-svn.nci.org.au/svn/um/branches

I would like to add some other comments which maybe are useful for a person not very familiar with the UMUI building procedure:

6. Compilation output location is set by default to the $HOME/output location. A different location can be specified via MY_OUTPUT environment variable setting in the Defined Environment Variables for Directories table of the Model Selection => Input/Output Control and Resources => Time Convention and SCRIPT Environment Variables panel.

7. The location of the extraction directory to build an executable is defined via the following things

• UM_ROUTDIR variable setting in the Model Selection => FCM Configuration => FCM Extract directories and Output levels panel.
• Then a UMUI vn7.5/processing/nds_extract script adds “automatically” a couple of subdirectory levels $USERID/$RUNID. The building is done under $UM_ROUTDIR/$USERID/$RUNID/ummodel and a model binary file in the case of success is produced in $UM_ROUTDIR/$USERID/$RUNID/ummodel/bin directory.

./nds_target_vars:export UM_ROUTDIR=%UMFCM_ROUTDIR/$USERID/$RUNID

8. From my experience the netcdf library to produce a UM executable is not used so references to the netcdf library should be removed from the corresponding compilation settings and the linking procedure in the above mentioned it can be removed from the building procedure to compile the sources and to link. The related references are in machines/linux-mpich2-meto/ext_libs/netcdf.cfg.

9. UMUI building job should not include any references for input/output which includes explicitly a $USER location.

10. UMUI building job should built a reconfiguration executable as well.

11. The purpose for UM_SVN_BASE environment variable setting in the Defined Environment Variables table of the Model Selection => Input/Output Control and Resources => Script Inserts and Modifications is not clear to me.

P.s. As I have not worked “closely” with UMUI for the past 1-2 years it took some time to prepare the above mentioned document which may not be so precise and requires further improvement.