This pages shows how to set up the SRIL34-FABM configuration. This configuration couples the SRIL34 model to one module of ERSEM (!not the full ERSEM!) though FABM, allowing simulation of a passive tracer. The configuration is very similar to that of the SRIL34 physics only, but there are some differences in the setup and keys used for the coupling. These instructions are specific to the SRIL34 model but more information on configurations coupling NEMO to FABM can be found here.

First of all, go to you workspace, eg. /work/n01/n01/<USER>/ and create the directory you want to work in eg. mkdir SRIL_FABM. You caa choose whichever file structure you prefer, in the examples below we created a separate code directory to store all that relates to the codes.

To get started, set the environment and working paths running the set_paths.sh. In this example the configuration is called SANH_FABM, as it can be used for both the SANH and SRIL34 model. Change the paths to the directories you want to work in. Always run this before running any scripts.

export CONFIG=SANH_FABM
export WORK=/work/n01/n01/<USER>/SRIL_FABM
export CODE_DIR=$WORK/code

You now need to clone the correct XIOS, NEMO, FABM and ERSEM repositories. This will get you the files you need to be able to install the programs in the next step. Again, you can close the repositories wherever you prefer, here we created four subdirectories inside $WORK/code:

mkdir $CODE_DIR/xios
mkdir $CODE_DIR/ersem
mkdir $CODE_DIR/fabm
mkdir $CODE_DIR/nemo

Once you directories are set, run the clone_repositories.sh script.

NOTE: In this example I clone the ERSEM-yuti repository rather than the standard ERSEM, from a private page as the public version was not available at the time. The current public version of this ERSEM repository is now available here.

XIOS_DIR=$CODE_DIR/xios
svn checkout http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/branchs/xios-2.5 $XIOS_DIR

ERSEM_DIR=$CODE_DIR/ersem
git clone https://github.com/yutiPML/ersem-yuti.git $ERSEM_DIR

FABM_DIR=$CODE_DIR/fabm
git clone https://github.com/fabm-model/fabm.git $FABM_DIR

NEMO_DIR=$CODE_DIR/nemo
git clone https://github.com/pmlmodelling/NEMO4.0-FABM.git $NEMO_DIR

You now have everything you need to compile.

IMPORTANT: XIOS and FABM must be compiled before NEMO

To compile the XIOS, you need to run the make_xios_cray.sh. This will load the modules than you need to compile on archer2

module swap craype-network-ofi craype-network-ucx
module swap cray-mpich cray-mpich-ucx
module load cray-hdf5-parallel/1.12.0.7
module load cray-netcdf-hdf5parallel/4.7.4.7

export the directories for the installation of XIOS

XIOS_DIR=$CODE_DIR/xios
XIOS_INSTALL=$CODE_DIR/xios-cray
cd $XIOS_DIR

add the xios architecture files and the Config.pm file.

# Copy arch files
cp $CODE_DIR/extra-files/xios/arch-X86_ARCHER2-Cray* $XIOS_DIR/arch/
cp $CODE_DIR/extra-files/xios/Config_cray.pm $XIOS_DIR/tools/FCM/lib/Fcm/Config.pm

NOTE: in the example these files were cloned from the AMM7_NEMO_FABM git extra-files. You will need to change the $CODE_DIR/extra-files/xios/ depending on where you get the files from. In this git the xios arch files are here, while the Config.pm file is here.

Then, Compile the XIOS:

export CC=cc export CXX=CC export FC=ftn export F77=ftn export F90=ftn
cd $XIOS_DIR && ./make_xios --prod --arch X86_ARCHER2-Cray --netcdf_lib netcdf4_par --job 16 --full
rsync -a $XIOS_DIR/bin $XIOS_DIR/inc $XIOS_DIR/lib $XIOS_INSTALL

cd $WORK

To compile FABM and ERSEM (NOTE: only one ERSEM module is used in this configuration) you need to run the make_fabm_cray.sh script.

This will first load an additional module:

module load cmake

make sure that the same modules you used to compile xios are still loaded (you can check with the command module list), if not load them:

module swap craype-network-ofi craype-network-ucx
module swap cray-mpich cray-mpich-ucx
module load cray-hdf5-parallel/1.12.0.7
module load cray-netcdf-hdf5parallel/4.7.4.7

export the directories for the installation (note that the ersem and fabm directories should already exist, so you are only exporting them, while the fabm-cray directory is created now).

ERSEM_DIR=$CODE_DIR/ersem
FABM_DIR=$CODE_DIR/fabm
FABM_INSTALL=$CODE_DIR/fabm-cray

mkdir $FABM_INSTALL

the move into the FABM_INSTALL directory for the installation, and compile:

cd $FABM_INSTALL
cmake $FABM_DIR/src -DFABM_HOST=nemo -DFABM_ERSEM_BASE=$ERSEM_DIR -DFABM_EMBED_VERSION=ON -DCMAKE_INSTALL_PREFIX=$FABM_INSTALL -DCMAKE_Fortran_COMPILER=ftn #-DCMAKE_BUILD_TYPE=debug
make
make install -j4

cd $WORK

IMPORTANT: NEMO must be compiled after XIOS and FABM

Finally, the last thing you have to compile is NEMO; to do this use the script make_nemo_cray.sh.

Make sure that the correct modules are loaded. If you have just compiled XIOS and FABM they should already be, if not load them:

module load cmake

module swap craype-network-ofi craype-network-ucx
module swap cray-mpich cray-mpich-ucx
module load cray-hdf5-parallel/1.12.0.7
module load cray-netcdf-hdf5parallel/4.7.4.7

Then export the directories you need and move into your $NEMO_DIR directory before installing:

export FABM_HOME=$CODE_DIR/fabm-cray
export XIOS_HOME=$CODE_DIR/xios-cray
NEMO_DIR=$CODE_DIR/nemo
cd $NEMO_DIR

Copy the Config.pm file in your Fcm directory:

cp $CODE_DIR/extra-files/nemo/Config_cray.pm $NEMO_DIR/ext/FCM/lib/Fcm/Config.pm

Before compiling, make sure that in the Config.pm you have the line FC_MODSEARCH => '-J',. It is possible that you have FC_MODSEARCH => '', instead; if this is the case, edit it to be FC_MODSEARCH => '-J',.

NOTE: in the example the $CODE_DIR/extra-files/*** directory was cloned from the AMM7_NEMO_FABM git extra-files. You will need to change the path depending on where you are copying the files from. In this wiki the Config.pm file is here.

Then, export the directories:

CFG=SANH_FABM
ARCH=X86_ARCHER2-Cray_FABM
REF=AMM7_FABM

and, making sure you are still in the $NEMO_DIR directory, compile NEMO and get back in your main work space:

printf 'y\nn\nn\ny\nn\nn\nn\nn\n' |./makenemo -n $CFG -r $REF -m $ARCH -j 0
./makenemo -n $CFG -r $REF -m $ARCH -j 4 clean
./makenemo -n $CFG -r $REF -m $ARCH -j 4 

cd $WORK