4. Make open boundary conditions for the physics and biogeochemistry - NOC-MSM/Regional-NEMO-Medusa GitHub Wiki
We made the open ocean boundary conditions using the PyNEMO tool. This includes time varying fields (SSH and barotropic currents) and tidal harmonics (which NEMO reconstructs on-the-fly). An overview of PyNEMO usage.
For tidiness, this section focus only on the generation of the physical and biological tracers 3D boundaries. To generate the tidal harmonics, see the previous section 3.
Even for a simulation with frozen boundaries, you need a coordinates.bdy.nc file to tell NEMO where the frozen boundaries are..
Workflow:
- Build and install PyNEMO
- 3D Lateral boundaries datasets : biological tracers
- 3D Lateral boundaries datasets : physical tracers
- Summary for all the data paths extracted from JASMIN
- Prepare PyNEMO files
- Run PyNEMO
- Make sure any output is where it is supposed to end up
For the generation of the biological tracers, James has developped a specific branch called valegu_tmp.
I installed PyNEMO valegu_tmp branch on a local machine (here livljobs7).
The file name used for that is Install_pynemo_valegu_tmp (located here: /projectsa/NEMO/valegu/BDY_VALEGU_TMP):
vi Install_pynemo_valegu_tmp
# Install Pynemo, testing the branch valegu_tmp
#cd /work/$USER
#--> Set up the directory for Pynemo
mkdir BDY_VALEGU_TMP
cd BDY_VALEGU_TMP
#--> Obtain the code from the git repository
git clone https://github.com/NOC-MSM/PyNEMO.git
#--> Load the correct branch
# cd /work/$USER/BDY_VALEGU_TMP/PyNEMO/
git checkout valegu_tmp
git status
module load anaconda/3-2018_12
conda env create -f pynemo_39.yml
source activate pynemo
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.x86_64/
python setup.py build
export PYTHONPATH=/work/valegu/valegu-conda/pynemo/lib/python3.9/site-packages:$PYTHONPATH
python setup.py install --prefix /work/valegu/valegu-conda/pynemo
For subsequent uses of PyNEMO, rebuilding is not necessary but path setting and environment activation are: E.g.
module load anaconda/3-2018_12
source activate pynemo
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.x86_64/
export PYTHONPATH=/work/valegu/valegu-conda/pynemo/lib/python3.9/site-packages:$PYTHONPATH
pynemo -h
Summary : It explains where the data for 3D boundaries come from and the issues I had for some parent variables in the year 2013.
The biological tracers come from the Global model ORCA0083-N06. I have extracted the files with the extension *d05P.nc from JASMIN. The path of the data for the year 2013 on JASMIN is :
/gws/nopw/j04/nemo_vol1/ORCA0083-N006/medusa/2013
It is worth noting that one of the d05P.nc file for 2013 has corrupted variables. This is the file ORCA0083-N06_20130902d05P.nc. The variables that are corrupted on this file are : CHN, CHD, PHN, PHD and ZMI. The rest of the variables are good.
To fix this, I replaced this corrupted file with the average of the previous and following file, i.e. ORCA0083-N06_20130828d05P.nc and ORCA0083-N06_20130907d05P.nc.
ncra ORCA0083-N06_20130828d05P.nc ORCA0083-N06_20130907d05P.nc AV_ORCA0083-N06_20130902d05P.nc
mv AV_ORCA0083-N06_20130902d05P.nc ORCA0083-N06_20130902d05P.nc
The surface values of the variables that were previously corrupted now look like this :
Concerning the physical tracers, they also come from the Global model ORCA0083-N06. I have extracted the files with the extension *d05T.nc, *d05U.nc and *d05V.nc from JASMIN. The path of the data for the year 2013 on JASMIN is :
/gws/nopw/j04/nemo_vol1/ORCA0083-N006/means/2013
There is also issues with variables U and V (but T is fine).
It is worth noting that 3 files of the d05U.nc and d05V.nc files for 2013 have corrupted variables. Those files are ORCA0083-N06_20130902d05U.nc, ORCA0083-N06_20130907d05U.nc, ORCA0083-N06_20130912d05U.nc, ORCA0083-N06_20130902d05V.nc, ORCA0083-N06_20130907d05V.nc and ORCA0083-N06_20130912d05U.nc.
Fixing the d05U.nc files:
ncks -x -v uo,uos,tauuo ORCA0083-N06_20130902d05U.nc out.nc
ncks -A -v uo,uos,tauuo ORCA0083-N06_20130828d05U.nc out.nc
ncks -A -v time_counter ORCA0083-N06_20130902d05U.nc out.nc
mv out.nc ORCA0083-N06_20130902d05U.nc
ncks -x -v uo,uos,tauuo ORCA0083-N06_20130912d05U.nc out.nc
ncks -A -v uo,uos,tauuo ORCA0083-N06_20130917d05U.nc out.nc
ncks -A -v time_counter ORCA0083-N06_20130912d05U.nc out.nc
mv out.nc ORCA0083-N06_20130912d05U.nc
ncra ORCA0083-N06_20130902d05U.nc ORCA0083-N06_20130912d05U.nc AV_ORCA0083-N06_20130907d05U.nc
mv AV_ORCA0083-N06_20130907d05U.nc ORCA0083-N06_20130907d05U.nc
Fixing the d05V.nc files:
ncks -x -v vo,vos,tauvo ORCA0083-N06_20130902d05V.nc out.nc
ncks -A -v vo,vos,tauvo ORCA0083-N06_20130828d05V.nc out.nc
ncks -A -v time_counter ORCA0083-N06_20130902d05V.nc out.nc
mv out.nc ORCA0083-N06_20130902d05V.nc
ncks -x -v vo,vos,tauvo ORCA0083-N06_20130912d05V.nc out.nc
ncks -A -v vo,vos,tauvo ORCA0083-N06_20130917d05V.nc out.nc
ncks -A -v time_counter ORCA0083-N06_20130912d05V.nc out.nc
mv out.nc ORCA0083-N06_20130912d05V.nc
ncra ORCA0083-N06_20130902d05V.nc ORCA0083-N06_20130912d05V.nc AV_ORCA0083-N06_20130907d05V.nc
mv AV_ORCA0083-N06_20130907d05V.nc ORCA0083-N06_20130907d05V.nc
The surface values of the variables that were previously corrupted now look like this. I only show surface data for uo and vo for the 3 dates previously corrupted:
Important note on the physical data for the year 2012:
On Jasmin : /gws/nopw/j04/nemo_vol1/ORCA0083-N006/means/2012
I noticed that the name of the physical variables (temperature, salinity and sea elevation) are named potemp, salin and ssh for the year 2012, whereas for the years after 2012, they are named votemper, vosaline and sossheig.
For Pynemo to work well, we need to provide the last files from the previous year (i.e. 2012 here). So I had to rename the variables potemp,salin and ssh into respectively votemper, vosaline and sossheig inside the file ORCA0083-N06_20121230d05T.nc.
ncrename -v potemp,votemper -v salin,vosaline -v ssh,sossheig ORCA0083-N06_20121230d05T.nc
The variables (uo and vo) for the current inside ORCA0083-N06_20121230d05U.nc and ORCA0083-N06_20121230d05V.nc have a consistent name for the years after 2012.
BIOGEOCHEMICAL TRACERS ORIGINS ON JASMIN
2012 (only ORCA0083-N06_20121230d05P.nc for Pynemo) : /gws/nopw/j04/nemo_vol1/ORCA0083-N006/medusa/2012/
2013 (all *d05P.nc) : /gws/nopw/j04/nemo_vol1/ORCA0083-N006/medusa/2013/
2014 (all *d05P.nc) : /gws/nopw/j04/nemo_vol1/ORCA0083-N006/medusa/2014/ (in progress)
2015 (all *d05P.nc) : /gws/nopw/j04/nemo_vol1/ORCA0083-N006/medusa/2015/ (in progress)
Note : No data available on Jasmin after 2015.
BIOGEOCHEMICAL TRACERS LOCATION INSIDE LIVLJOBS7
/projectsa/NEMO/valegu/gws/nopw/j04/nemo_vol1/ORCA0083-N006/medusa/2013/
/projectsa/NEMO/valegu/gws/nopw/j04/nemo_vol1/ORCA0083-N006/medusa/2014/ (in progress)
/projectsa/NEMO/valegu/gws/nopw/j04/nemo_vol1/ORCA0083-N006/medusa/2015/ (in progress)
The namelist_FES2014.bdy drives PyNEMO and has input files which point to the data sources sn_src_hgr, sn_src_zgr, sn_src_msk. You can reuse the same namelist as the one used to generated the tides.
Modify namelist_2013.bdy so that it points to the appropriate files.
vi namelist_2013.bdy
!!>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
!! NEMO/OPA : namelist for BDY generation tool
!!
!! User inputs for generating open boundary conditions
!! employed by the BDY module in NEMO. Boundary data
!! can be set up for v3.2 NEMO and above.
!!
!! More info here.....
!!
!!>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
!-----------------------------------------------------------------------
! vertical coordinate
!-----------------------------------------------------------------------
ln_zco = .false. ! z-coordinate - full steps (T/F)
ln_zps = .true. ! z-coordinate - partial steps (T/F)
ln_sco = .false. ! s- or hybrid z-s-coordinate (T/F)
rn_hmin = -10 ! min depth of the ocean (>0) or
! min number of ocean level (<0)
!-----------------------------------------------------------------------
! s-coordinate or hybrid z-s-coordinate
!-----------------------------------------------------------------------
rn_sbot_min = 0.52 ! minimum depth of s-bottom surface (>0) (m)
rn_sbot_max = 5902. ! maximum depth of s-bottom surface
! (= ocean depth) (>0) (m)
ln_s_sigma = .false. ! hybrid s-sigma coordinates
rn_hc = 39.0 ! critical depth with s-sigma
!-----------------------------------------------------------------------
! grid information
!-----------------------------------------------------------------------
sn_src_hgr = '/projectsa/NEMO/valegu/gws/nopw/j04/nemo_vol1/ORCA0083-N006/mesh_hgr.nc'
sn_src_zgr = '/projectsa/NEMO/valegu/BDY_VALEGU_TMP/mesh_zgr_renamed.nc'
sn_dst_hgr = '/projectsa/NEMO/valegu/BDY_VALEGU_TMP/domain_cfg.nc'
sn_dst_zgr = '/projectsa/NEMO/valegu/BDY_VALEGU_TMP/inputs_dst.ncml'
sn_src_msk = '/projectsa/NEMO/valegu/gws/nopw/j04/nemo_vol1/ORCA0083-N006/mask.nc'
sn_bathy = '/projectsa/NEMO/valegu/BDY_VALEGU_TMP/bathy_meter.nc'
!-----------------------------------------------------------------------
! I/O
!-----------------------------------------------------------------------
sn_src_dir = 'NCML/ORCA_2013.ncml' ! src_files/'
sn_dst_dir = 'OUTPUT_VALEGU_TMP_2013/'
sn_fn = 'EA_TMP' ! prefix for output files
nn_fv = -1e20 ! set fill value for output files
nn_src_time_adj = 0 ! src time adjustment
sn_dst_metainfo = 'metadata info: valegu'
!-----------------------------------------------------------------------
! unstructured open boundaries
!-----------------------------------------------------------------------
ln_coords_file = .true. ! =T : produce bdy coordinates files
cn_coords_file = 'coordinates.bdy.nc' ! name of bdy coordinates files (if ln_coords_file=.TRUE.)
ln_mask_file = .false. ! =T : read mask from file
cn_mask_file = '/projectsa/NEMO/valegu/Test_Pynemo/bdy_mask.nc' ! name of mask file (if ln_mask_file=.TRUE.)
ln_dyn2d = .true. ! boundary conditions for barotropic fields
ln_dyn3d = .true. ! boundary conditions for baroclinic velocities
ln_tra = .true. ! boundary conditions for T and S
ln_ice = .false. ! ice boundary condition
nn_rimwidth = 9 ! width of the relaxation zone
!-----------------------------------------------------------------------
! unstructured open boundaries tidal parameters
!-----------------------------------------------------------------------
ln_tide = .false. ! =T : produce bdy tidal conditions
clname(1) = 'M2'
clname(2) = 'S2'
clname(3) = 'N2'
clname(4) = 'K2'
clname(5) = 'K1'
clname(6) = 'O1'
clname(7) = 'P1'
clname(8) = 'Q1'
clname(9) = 'M4'
ln_trans = .false.
sn_tide_h = '/work/jelt/tpxo7.2/h_tpxo7.2.nc'
sn_tide_u = '/work/jelt/tpxo7.2/u_tpxo7.2.nc'
!-----------------------------------------------------------------------
! Time information
!-----------------------------------------------------------------------
nn_year_000 = 2013 ! year start ! val: updated
nn_year_end = 2013 ! year end ! val: updated
nn_month_000 = 1 ! month start (default = 1 is years>1)
nn_month_end = 1 ! month end (default = 12 is years>1)
sn_dst_calendar = 'gregorian' ! output calendar format
nn_base_year = 2013 !1900 ! base year for time counter in output data
sn_tide_grid = '/work/jelt/tpxo7.2/grid_tpxo7.2.nc'
nn_src_time_adj = 0 != -3168000 - 86400 ! N01: fix to align model time stamp
!-----------------------------------------------------------------------
! Additional parameters
!-----------------------------------------------------------------------
nn_wei = 1 ! smoothing filter weights
rn_r0 = 0.041666666 ! decorrelation distance use in gauss
! smoothing onto dst points. Need to
! make this a funct. of dlon
sn_history = 'bdy files produced by jelt from ORCA0083-N01'
! history for netcdf file
ln_nemo3p4 = .true. ! else presume v3.2 or v3.3
nn_alpha = 0 ! Euler rotation angle
nn_beta = 0 ! Euler rotation angle
nn_gamma = 0 ! Euler rotation angle
rn_mask_max_depth = 300.0 ! Maximum depth to be ignored for the mask
rn_mask_shelfbreak_dist = 60 ! Distance from the shelf break
Create directory that will contain the 3D outputs:
mkdir OUTPUT_VALEGU_TMP_2013
Run on livljobs7:
module load anaconda/3-2018_12
source activate pynemo
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.x86_64/
export PYTHONPATH=/work/valegu/valegu-conda/pynemo/lib/python3.9/site-packages:$PYTHONPATH
pynemo -s namelist_2013.bdy
This creates files in OUTPUT_VALEGU_TMP_2013:
I copied all the 3D bdy data into /work/valegu/TRANSITION/, then from there, I transferred them into ARCHER2 :
Go inside ARCHER2 on the directory you want your data, then type :
cd /work/n01/n01/valegu/EA_R12_BIO/nemo_vr4.06/cfgs/ORCA2_MEDUSA/EXP_FINAL/BDY_JAMES/
rsync -zav [email protected]:/work/valegu/TRANSITION/* ./