WIP: Messing about with PyNEMO - NOC-MSM/SEAsia GitHub Wiki

Notes setting up PyNEMO on ARCHER2

Install miniconda, as described in ...

Install PyNEMO. (Trying a version of master)

cd /work/n01/n01/$USER
git clone https://github.com/NOC-MSM/PyNEMO.git

cd PyNEMO
git checkout master-jp
conda env create -f pynemo_39.yml
conda activate pynemo3

# Build and install pynemo
#export JAVA_HOME=/usr/lib64/jvm/java  # Already set, but in the past has not being set has caused trouble.

python setup.py clean --all
python setup.py build
export PYTHONPATH=/work/n01/n01/$USER/miniconda3/envs/pynemo3/lib/python3.9/site-packages:$PYTHONPATH
python setup.py install

On mac:

conda env remove --name pynemo
conda env create -f pynemo_39.yml
conda activate pynemo

setenv JAVA_HOME /Library/Java/JavaVirtualMachines/jdk-17.0.1.jdk/Contents/Home

python setup.py clean --all
python setup.py build
setenv PYTHONPATH /Users/$USER/opt/anaconda3/envs/pynemo/lib/python3.9/site-packages
python setup.py install

Verify it works:

pynemo -h

(NB it takes a few minutes for the path to update and the command to be findable)

On ARCHER2:

cd /work/n01/n01/$USER
git clone https://github.com/NOC-MSM/PyNEMO.git

cd PyNEMO
git checkout master
#conda env remove --name pynemo
conda env create -f pynemo_39.yml
conda activate pynemo

#export JAVA_HOME=/usr/lib64/jvm/java  # Already set, but in the past has not being set has caused trouble.

python setup.py clean --all
python setup.py build
export PYTHONPATH=/work/n01/n01/$USER/miniconda3/envs/pynemo/lib/python3.9/site-packages:$PYTHONPATH
python setup.py install

Verify it works:

pynemo -h

Create link to FES2014 dir:

ln -s /work/n01/n01/shared/jelt/FES2014 inputs/.

Make new output directory:

mkdir outputs

Try it out:

pynemo -s inputs/namelist_local.bdy

On livljobs8:

module load anaconda/3-2018_12
conda env create -f pynemo_39.yml

permissions issue. Manaully fix chmod -R u+x /work/jelt/conda-env/pynemo/bin.

source activate pynemo

Then manually pip install the remaining packages: pip install pyjnius seawater thredds-crawler

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.x86_64/

python setup.py build
export PYTHONPATH=/work/jelt/conda-env/pynemo/lib/python3.9/site-packages:$PYTHONPATH
python setup.py install

Verify it works:

pynemo -h

Link in benchmarking resources, and make outputs dir

ln -s /work/jdha/PyNEMO/inputs/benchmark /login/jelt/work/GitHub/PyNEMO/.
mkdir /work/jdha/PyNEMO/outputs

NB links to benchmarking in src_data_remote.ncml need to be correct. Either absolute or relative.

Try it out:

pynemo -s inputs/namelist_local.bdy
Didn't find a proxy environment variable
OrderedDict([('zco', False), ('zps', True), ('sco', False), ('hmin', -10), ('sbot_min', 10.0), ('sbot_max', 7000.0), ('s_sigma', False), ('hc', 150.0), ('src_hgr', './benchmark/grid_low_res_C/mesh_hgr.nc'), ('src_zgr', './benchmark/grid_low_res_C/mesh_zgr.nc'), ('dst_hgr', './benchmark/grid_C/mesh_hgr_zps.nc'), ('dst_zgr', './benchmark/grid_C/mesh_zgr_zps.nc'), ('src_msk', './benchmark/grid_low_res_C/mask.nc'), ('bathy', './benchmark/grid_C/NNA_R12_bathy_meter_bench.nc'), ('src_dir', './inputs/src_data_local.ncml'), ('dst_dir', './outputs/'), ('fn', 'NNA_R12'), ('fv', -1e+20), ('src_time_adj', 0), ('dst_metainfo', 'Benchmarking Data'), ('coords_file', 'coordinates.bdy.nc'), ('mask_file', 'mask.nc'), ('dyn2d', False), ('dyn3d', False), ('tra', True), ('ice', False), ('rimwidth', 9), ('tide', False), ('tide_model', 'FES'), ('clname', {'1': "'M2'", '2': "'S2'", '3': "'K2'"}), ('trans', True), ('year_000', 1979), ('year_end', 1979), ('month_000', 11), ('month_end', 11), ('dst_calendar', 'gregorian'), ('base_year', 1960), ('tide_grid', './src_data/tide/grid_tpxo7.2.nc'), ('tide_h', './src_data/tide/h_tpxo7.2.nc'), ('tide_u', './src_data/tide/u_tpxo7.2.nc'), ('wei', 1), ('r0', 0.041666666), ('history', 'Benchmarking test case'), ('nemo3p4', True), ('alpha', 0), ('beta', 0), ('gamma', 0), ('mask_max_depth', 100.0), ('mask_shelfbreak_dist', 20000.0)])
./inputs/src_data_local.ncml
/vkamino/work/jelt/GitHub/PyNEMO/pynemo/nemo_bdy_extr_tm3.py:651: RuntimeWarning: invalid value encountered in true_divide
  dst_bdy = (np.nansum(sc_bdy[vn,:,:,:] * dist_wei, 2) /
/vkamino/work/jelt/GitHub/PyNEMO/pynemo/nemo_bdy_extr_tm3.py:676: RuntimeWarning: invalid value encountered in true_divide
  dst_bdy = (np.nansum(dst_bdy.flatten('F')[self.id_121] *
Execution Time: 20.159401178359985

---

Subsequent use of PyNEMO build might require explicitly updating the PYTHONPATH variable. E.g.

conda activate pynemo3
export PYTHONPATH=/work/n01/n01/$USER/miniconda3/envs/pynemo3/lib/python3.7/site-packages:$PYTHONPATH

---