access_UMStandAloneExecs - ACCESS-NRI/accessdev-Trac-archive GitHub Wiki
Building the UM within Stand Alone Rose Suites
Rose has been designed not only to run the model but also build the model. Stand alone build suites are suites that contain all the FCM information required to build the UM and do not reference external make files as found in the access-svn repository. au-aa059 is a simple stand alone suite that can be used for building the UM model (v8.5). To run the suite, login onto accessdev and run the following commands.
rosie checkout au-aa059
rose suite-run -C ~/roses/au-aa059
This will be build the executable for UM 8.5. After running the suite, the default exectuable is located on raijin in;
$HOME/cylc-run/au-aa059/share/fcm_make/build-atmos/bin/um-atmos.exe
If you want to build the reconfiguration, there is a suite au-aa063 that builds the reconfiguration executable. To build the reconfiguration for vn8.5
rosie checkout au-aa063
rose suite-run -C ~/roses/au-aa063
This will be build the executables for UM 8.5. After running the suite, the default exectuables are located on raijin in;
$HOME/cylc-run/au-aa063/share/fcm_make/build-atmos/bin/um-recon.exe
How FCM works within Rose
On NCI machines the model is built in two steps. The first step fcm_make extracts the source code. The second step fcm_make2 builds the model on the compute nodes on raijin. It's possible to do the extraction on accessdev as an alternative location and then mirror it across to raijin if required.
FCM make files can read environment variables such as $ROSE_SUITE_NAME or $HOME in the following example. These can be to set alternative locations, compiler flags if required.
The make file is defined in a suite's FCM application. Eg, au-aa059/app/fcm_make/file/fcm-make.cfg
In the case of the au-aa059. The fcm make file contains the following lines.
################################################################################
#Compiliation Steps
step.class[preprocess-atmos] = preprocess
step.class[build-atmos] = build
steps = extract mirror
$MIRROR_TARGET{?} = $HOME/cylc-run/$ROSE_SUITE_NAME/share/fcm_make
mirror.target = $MIRROR_TARGET
mirror.prop{config-file.steps} = preprocess-atmos build-atmos
#Build Target
$atmos_exec{?} = um-atmos.exe
build-atmos.target = $atmos_exec
build-atmos.target-rename = um_main.exe:$atmos_exec
################################################################################
#Extraction
extract.ns = um jules
extract.location[um] = fcm:[email protected]
extract.location[jules] = fcm:[email protected]
extract.path-excl[jules] = /
extract.path-incl[jules] = includes/shared \
src/control/shared \
src/control/um \
src/initailisation/shared \
src/initialisation/um \
src/science
# Branches/working copies:
extract.location{diff}[um] = \
fcm:um_dev/vn8.5/GlobalAtmos6p0@7678 \
fcm:um_dev/vn8.5/local_changes@HEAD
extract.location{diff}[jules] = \
branches/dev/mrd599/um8.5_GlobalLand6p0@387
################################################################################
# Pre Process definitions
$keys_atmos = A03_9C=a03_9c \
A13_2A=a13_2a \
A19_1B=a19_1b \
C84_1A=c84_1a \
C95_2A=c95_2a \
C96_1C=c96_1c \
C97_3A=c97_3a \
C98_0A=c98_0a \
UM_JULES=um_jules \
L19_1A=l19_1a \
C_LONG_LONG_INT=c_long_long_int \
C_LOW_U=c_low_u \
FRL8=frl8 \
LINUX=linux \
BUFRD_IO=bufrd_io \
LITTLE_END=little_end \
LFS=lfs \
_LARGEFILE_SOURCE=_largefile_source \
_FILE_OFFSET_BITS=64
preprocess-atmos.prop{fpp.defs} = $keys_atmos
preprocess-atmos.prop{cpp.defs} = $keys_atmos
################################################################################
# Special dependancies
build-atmos.prop{dep.o.special}[um/src/control/top_level/um_main.F90] = blkdata.o
build-atmos.prop{no-dep.f.module} = mpl \
gcom_mod \
netcdf \
mod_prism_proto \
mod_prism_put_proto \
mod_prism_grids_writing \
mod_prism_def_partition_proto \
mod_prism_get_proto
################################################################################
# Compiler flags
$fcflags_common = -g -nofpp -i8 -8e3262e565652ac69b4b02b09b064c4f88b8c8e2 -xHost -traceback
$fcflags_level = -O2 -fp-model precise
$fcflags_omp{?} =
$ldflags_omp{?} =
$ldgcom_omp{?} =
$ldflags{?} = -openmp -lgcom -lnetcdff -lnetcdf \
\ -lgrib_api_f90 -lgrib_api $ldflags_omp
$flags_coupling{?} =
$fcflags_coupling{?} = $flags_coupling
$fppflags_coupling{?} = $flags_coupling
$ccflags_coupling{?} = $flags_coupling
$cppflags_coupling{?} = $flags_coupling
$ldflags_coupling{?} = -lanaisg -lanaism -lpsmile.MPI1 -lfscint -lmpp_io -lscrip
$fcflags_overrides{?} # $fcflags_all $fcflags_common $fcflags_level $fcflags_omp $fcflags_overrides
preprocess-atmos.prop{cpp} = cpp
preprocess-atmos.prop{cpp.flags} =
build-atmos.prop{cc} = mpicc
build-atmos.prop{fc} = mpif90
build-atmos.prop{cc.flags} = -g -O3 -std=gnu99
build-atmos.prop{fc.flags} = $fcflags_all
build-atmos.prop{fc.flags-ld} = $ldflags
################################################################################
# Additional flags for coupled models:
$fc_cpl = $fcflags_coupling $fcflags_all
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_atmos_init_mod.F90] = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_grid.F90] = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_split_comm_mod.F90] = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_geto2a.F90] = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_puta2o.F90] = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_get.F90] = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_put.F90] = $fc_cpl
# Attempt to keep compile times down (excludes $fcflag_omp)
$fc_noomp = $fcflags_common $fcflags_level $fcflags_overrides
build-atmos.prop{fc.flags}[um/src/control/top_level/u_model.F90] = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/top_level/initial.F90] = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/top_level/atm_step.F90] = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/top_level/u_model_4A.F90] = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/top_level/initial_4A.F90] = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/top_level/atm_step_4A.F90] = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/stash/st_diag3.F90] = $fc_noomp
# Required to compile with OpenMP
build-atmos.prop{fc.flags}[um/src/atmosphere/dynamics_advection/idl_surface_setup.F90] = \
\ $fcflags_common $fcflags_level $fcflags_overrides
# Required to prevent OpenMP multi-thread crash
build-atmos.prop{fc.flags}[um/src/atmosphere/boundary_layer/excf_nl_9c.F90] = \
\ $fcflags_common -O2 -fp-model strict $fcflags_omp $fcflags_overrides
Definition of variables
extract.location[um] => defines where the um source code and what version of the branch
extract.location[jules] => defines where the jules source code and what version of the branch
$keys_atmos => defines the ifdefs for the UM model.
steps => defines what steps do in the first fcm make step. "extract" extracts the code. "mirror" copies the source code the location specified in $MIRROR_TARGET.
$MIRROR_TARGET => Location of where the extracted source code should be sent to.
extract.location{diff}[um] => Apply any branches over the um base.
extract.location{diff}[jules] => Apply any branches over the jules base.
atmos_exec => Name of the UM executable file.
Log files
There are three types of log files that can be used for solving problems when compiling the UM.
Suite Logs
Suite log files contain top level information about what jobs failed and job submission. They are located on Accessdev in
$HOME/cylc-run/SUITENAME/suite/out
$HOME/cylc-run/SUITENAME/suite/log
$HOME/cylc-run/SUITENAME/suite/err
Job Logs
Job log files contain more specific infortion about running a task. They are located on both Accesdev and Raijin in
$HOME/cylc-run/SUITENAME/log/job/fcm-make.1.1.err
$HOME/cylc-run/SUITENAME/log/job/fcm-make.1.1.out
FCM Logs
FCM log file is a low level log that can information about about how specific files are compiled. They are located on Raijin
$HOME/cylc-run/SUITE/share/fcm_make/fcm-make.log