Job Templates - QuantGen/HPCC GitHub Wiki

Shell Scripts

Regular Jobs

#!/usr/bin/env bash
#SBATCH --job-name=myJob
#SBATCH --time=4:00:00
#SBATCH --mem=20gb
#SBATCH --account=quantgen
#SBATCH --output=/mnt/research/quantgen/logs/%j

echo "Hello World!"

Array Jobs

#!/usr/bin/env bash
#SBATCH --job-name=myJob
#SBATCH --time=4:00:00
#SBATCH --mem=20gb
#SBATCH --account=quantgen
#SBATCH --output=/mnt/research/quantgen/logs/%A_%a
#SBATCH --array=1-5

echo "Hello World!"

R Scripts

These templates combine submission script and R code. You can still have a separate submission script that makes use R CMD BATCH or preferably Rscript.

Regular Jobs

#!/usr/bin/env Rscript
#SBATCH --job-name=myJob
#SBATCH --time=4:00:00
#SBATCH --mem=20gb
#SBATCH --account=quantgen
#SBATCH --output=/mnt/research/quantgen/logs/%j

message("Hello World!")

Array Jobs

#!/usr/bin/env Rscript
#SBATCH --job-name=myJob
#SBATCH --time=4:00:00
#SBATCH --mem=20gb
#SBATCH --account=quantgen
#SBATCH --output=/mnt/research/quantgen/logs/%A_%a
#SBATCH --array=1-5

message("Hello World!")

How to pass arguments

There are two ways you can pass arguments to R:

• Using arguments:
  • Set arguments: sbatch script.R args
  • Get arguments: args <- commandArgs(trailingOnly = TRUE) (consider optparse package or alternatives)
• Using environment variables (name-value pairs):
  • Set arguments:
    • NAME=VALUE sbatch script.R
    • export NAME=VALUE; sbatch script.R
    • sbatch --export=ALL,NAME=VALUE script.R (in this case you MUST specify ALL otherwise the job will execute in an incomplete environment that does not contain R)
  • Get arguments: Sys.getenv("NAME")

Job Defaults

For typical R jobs, there is no need to specify the following parameters:

• --nodes=1
• --ntasks=1
• --cpus-per-task=1

If you feel the need to throw more nodes, tasks, or threads at a problem, consider a split-compute-combine approach using array jobs first. For R jobs, this approach is generally faster and results in shorter queue times.