Software - uwsph/hpcusers GitHub Wiki

The cluster has a selection of preinstalled software, ranging from data analysis tools to compilers. Software is installed into the base system, environment modules, and containers. Users and groups needing software that isn't already provided are encouraged to use containers to package their tools. Alternatively, you could install software directly into your home directory or project folder. If you'd like to see something added to the base system, or other managed software, please contact IT for assistance.

[!NOTE] Containers (Apptainer) are the recommended way for users to install software. It simplifies dependency management, and makes your software more portable.

Base

The base system includes commonly needed tools like text editors, file transfer utilities, compilers, and some data analysis tools. The base system is primarily aimed at helping users either create their own containers, environment modules, or run provided environment modules.

Modules

Modules or environment modules enable you to load or unload software packages into your shell environment. However, when possible, you should use containers, as they make it easier to maintain a toolchain, especially something that is complex or has many dependencies.

Command Definition
module avail Listing available Environment Modules (available software)
module list List currently loaded Environment Modules
module load R/R-3.5.3 Loading an environment (adding the application or library to your path)
module unload R/R-3.5.3 Unloading an environment (removing an application or library from your path)
module help R/R-3.5.3 Viewing an Environment Module's help information (not all environments contain help information)

Containers

Apptainer was previously known as Singularity. It's a container platform designed for high performance computing environments. With Apptainer, you can either build containers using its native format (SIF) or convert the more common Docker (OCI) containers. With a container, you can package an entire suite of tools with their dependencies. Then, take that container from the cluster to another computing environment, and run it, without modification. Through containers, you can preserve the environment used to run your work, making it easier to reproduce your work in the future. Or, to ensure everyone in your group is using the same tools and version.

Command Definition
apptainer pull docker://rocker/verse:4.4.1 Download the Rocker/Verse (version 4.4.1) container from Dockerhub
apptainer exec ./rocker_verse_4.4.1.sif /bin/bash Run Bash within the container
apptainer build rstudio_custom.sif rstudio_custom.def Build a container based on the rstudio_custom.def definition file

Example: Custom R Studio Container

  1. Login to the cluster, either SSH or Shell Access
  2. Request an interactive session on a compute node
salloc -A sph -p 12c128g -N 1 -c 1 --mem 4G -t 2:00:00
  1. Build a definition file (for example rstudio_custom.def) for the new container. We'll base it on an existing container.
Bootstrap: docker
From: rocker/verse:4.4.1

%post
        apt -y update
        apt -y install wget
  1. Use Apptainer to build it
apptainer build rstudio_custom.sif rstudio_custom.def