Running at NCI - GeoscienceAustralia/tcrm GitHub Wiki
Here we describe how to run TCRM on one of the machines at the NCI. TCRM has been successfully run on raijin with up to 256 cores.
TCRM can be installed by cloning the GitHub repository:
git clone http://github.com/GeoscienceAustralia/tcrm
Compile the C extensions:
cd tcrm
python installer/setup.py build_ext -i
Load up the latest versions of Python, OpenMPI and netCDF:
~$ module load python/2.7.15
~$ module load openmpi/1.6.3
~$ module load netcdf/4.3.0
~$ module load hdf5/1.8.7
~$ module load pypar
Check that they are loaded:
~$ module list
Currently Loaded Modulefiles:
1) pbs 6) python/2.7.15
2) dot 7) netcdf/4.3.0
3) intel-fc/12.1.9.293 8) pypar/30May-2.7.6-1.8
4) intel-cc/12.1.9.293 9) szip/2.1
5) openmpi/1.6.3 10) hdf5/1.8.7
Build a local copy of the netcdf4-python library. Install netcdf4-python either using pip:: pip install netCDF4 --user
Or by building the latest copy of the netcdf4-python library:
git clone https://github.com/Unidata/netcdf4-python
cd netcdf4-python/
python setup.py install --user
Install the remaining required python modules locally:
pip -v install -r requirements.txt --user
Download basemap tar-ball and unpack in a tmp directory on NCI:
tar -xzvf basemap-1.0.7.tar.gz
cd basemap-1.0.7
export GEOS_DIR=$GEOS_ROOT
cc=gcc python setup.py build
python setup.py install --user
Create a job file called tcrm with the following content:
#!/bin/bash
#PBS -q normal
#PBS -l walltime=00:45:00,mem=3GB
#PBS -l ncpus=64
#PBS -j oe
module purge
module load openmpi/1.8.4
module load python/2.7.6
module load python/2.7.6-matplotlib
module load pypar/26Feb15-2.7.6-1.8.4
module load geos
module load gdal/1.11.1-python
NCPUS=${PBS_NCPUS-8}
cd ~/tcrm
mpirun -n $NCPUS python tcrm.py -c ./example/port_hedland.ini
Submit the job:
qsub tcrm