Running hyspdisp on the Harvard Odyssey cluster

Hysplit

The Hysplit exec folder is:

/n/regal/zigler_lab/software/hysplit/trunk/exec

Installing R packages on Odyssey

Create the folder apps/R in your home folder

mkdir apps
mkdir apps/R

Modify your .bashrc file as follows:

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

# User specific aliases and functions
source new-modules.sh

module load R

# replace username with your username.  You may have to change home03 too.
export R_LIBS_USER=/n/home03/username/apps/R:$R_LIBS_USER

SplitR

In an R session:

devtools::install_github("lhenneman/SplitR")

You can test with:

library(SplitR)

trajectory1 <- 
  hysplit_trajectory(
    lat = 42.83752,
    lon = -80.30364,
    height = 50,
    duration = 24,
    run_period = "2005-02-02",
    daily_hours = c(0, 12),
    direction = "forward",
    met_type = "reanalysis",
    exec_dir = "/n/regal/zigler_lab/software/hysplit/trunk/exec")

rgdal

The instructions are adapted from https://portal.rc.fas.harvard.edu/apps/modules/rgdal and should be run from a terminal.

To install rgdal

mkdir build_temp
cd build_temp
module load hdf5/1.10.1-fasrc03 gdal/2.3.0-fasrc01  proj/5.0.1-fasrc01
module load gcc/7.1.0-fasrc01 R/3.4.2-fasrc02
export R_LIBS_USER=$HOME/R:$R_LIBS_USER
wget https://cran.r-project.org/src/contrib/Archive/rgdal/rgdal_1.2-20.tar.gz
tar xf rgdal_1.2-20.tar.gz
R CMD INSTALL rgdal --configure-args="--with-proj-share=$PROJ_HOME/share/proj --with-proj-include=$PROJ_HOME/include --with-proj-lib=$PROJ_HOME/lib"

To use rgdal, from a terminal before starting R:

module load hdf5/1.10.1-fasrc03 gdal/2.3.0-fasrc01  proj/5.0.1-fasrc01
module load gcc/7.1.0-fasrc01 R/3.4.2-fasrc02
R

then from R

library(rgdal)

It is not tested yet, but a good option might be to specify the modules in .bashrc:

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

# User specific aliases and functions
source new-modules.sh

module load hdf5/1.10.1-fasrc03 gdal/2.3.0-fasrc01  proj/5.0.1-fasrc01
module load gcc/7.1.0-fasrc01 R/3.4.2-fasrc02

# replace username with your username.  You may have to change home03 too.
export R_LIBS_USER=/n/home03/username/apps/R:$R_LIBS_USER

SLURM jobs

Overview

Information on partitions cane be found at https://www.rc.fas.harvard.edu/resources/running-jobs/.

Interactive jobs

# Example: we request a total of 2GB on 2 nodes with 20 cores per node in the shared partition
srun -p shared  --mem 2g -t 0-06:00 -c 20 -N 2 --pty /bin/bash

# Example: we request a total of 300GB on 2 nodes with 32 cores per node in the bigmem partition
srun -p bigmem --pty --mem 300g -t 0-06:00  -c 32 -N 2 /bin/bash

Batch job

To create a batch job from file test_batch.sh:

#! /usr/bin/env Rscript
#SBATCH -n 1               # (Max) number of tasks per job, for R usually 1
#SBATCH -o out-%a.txt      # File for the standard output
#SBATCH -e err-%a.txt      # File for the standard error
#SBATCH -p serial_requeue  # Partition to use
#SBATCH --mem-per-cpu=1024 # Memory required per CPU, in MegaBytes
#SBATCH -a 10-20           # Array of 11 jobs, with ids 10, 11, ..., 20

if (Sys.getenv("SLURM_JOB_ID") != "") {
  
  meanDist <- function(n, nsamp = 500) {
    replicate(nsamp, mean(rnorm(n)))
  }
  
  my_id <- as.numeric(Sys.getenv("SLURM_ARRAY_TASK_ID"))
  
  print(meanDist(my_id))
}

You can estimate when the resources will become available with:

sbatch --test-only test_batch.sh 
# sbatch: Job 51033437 to start at 2018-08-17T15:43:16 using 1 processors on holy2a08205

and submit the job with:

sbatch test_batch.sh