Working on the cluster

Here we provide basic commands for connecting and submitting jobs on the cluster. See https://docs.massive.org.au/ for more information on the cluster.

# Connect to cluster
ssh [username]@m3.massive.org.au # Mac users
ssh -l [username] m3.massive.org.au # Linux users

# Start a new smux session (n) for example with 20 cpus (--ntasks) for 1 day (--time)
smux n --ntasks=20 --time=1-00:00:00

# List ongoing smux sessions
smux l

# Attach to an ongoing smux session
smux a [job ID]

# List ongoing jobs
show_job

# Load modules
module load [module]

# Cancel a smux session
scancel [jobID]

# View specific information of a job
scontrol show job [jobID]

# Cancel all PENDING jobs for the user
scancel -t PENDING -u [username]

Preparing a bash script file for sbatch

You can create a script file for submission with the cluster sbatch command by including a few header lines as shown in an example below. See https://docs.massive.org.au/ for more information on the options.

#!/bin/bash
#SBATCH --job-name=anvio
#SBATCH --account=of33
#SBATCH --time=4:00:00
#SBATCH --mem-per-cpu=8G
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=12
#SBATCH --partition=genomics
#SBATCH --qos=genomics

IMPORTANT: if you have access to the genomics partition, then you can add the bottom two lines (otherwise omit them). Keep in mind that each node on the cluster has 48 CPUs and 320GB of RAM. You must attempt where possible to divide CPUs and RAM evenly due to a quirk of the scheduling system on this partition, i.e. 12 CPUs with 80GB RAM total, 24 CPUs with 160GB RAM total etc.

Downloading data from BaseSpace

See https://developer.basespace.illumina.com/docs/content/documentation/cli/cli-overview for downloading data from BaseSpace straight to the cluster.

# Download data from BaseSpace
./bs -c Australia download project -i [projectID] -o [directory]