Tutorials ‐ QIIME2 - uwsph/hpcusers GitHub Wiki

This quick tutorial aims provide an overview of the complete process from connecting to the cluster, and converting a Docker container into an Apptainer image. For this tutorial, we'll be using QIIME 2.

Requirements

For this tutorial you'll need the following:

An account on the SPH HPC environment
Husky OnNet VPN (Big IP Edge Client)
An SSH client
Basic familiarity with a Linux command line

Connecting

If not already installed. You'll need the Husky OnNet VPN client. For SPH provided computers, this should already be installed. For non-SPH systems, you'll need to download the installer from UW-IT
TODO add VPN setup....

Once connected to the VPN, using your SSH client, connect to login.hpc.sph.washington.edu with your UW NetID and password.

Converting the Container

Request an interactive session on a compute node. Converting a container isn't typically resource intensive, so we'll request 1 node (-N), 1 CPU core (-c), 4GB of RAM (--mem) and 2 hours of runtime (-t). You'll want change the account (-A sph), and partition (-p 12c128g) to reflect those that you have access to.

salloc -A sph -p 12c128g -N 1 -c 1 --mem 4G -t 2:00:00

Use Apptainer to pull QIIME 2 Amplicon distribution, release 2024.5

apptainer pull docker://quay.io/qiime2/amplicon:2024.5

As the command runs, you should see a status output appear. It'll look something like:

INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
Copying blob 534fa1df7054 done   | 
Copying blob 7e8b39fc3597 done   | 
Copying blob a34e9b8a4f34 done   | 
Copying blob 64441255033a done   | 
Copying config c54069c148 done   | 
Writing manifest to image destination
2024/10/11 09:01:42  info unpack layer: sha256:7e8b39fc359757df5d27446bf6acae9b69fa5549e0cc293522a40c6aafad802f
2024/10/11 09:01:44  info unpack layer: sha256:534fa1df7054c6daf1ca1048e5192867c94f9d0752dae314fc2b70d17ea5aee2
2024/10/11 09:01:49  info unpack layer: sha256:64441255033a114d47e741ba12cdc8d0bd92d4b764f087256ff9328a47fd946b
2024/10/11 09:01:58  info unpack layer: sha256:a34e9b8a4f34bada507afaeda33966922f2f60367304effe161eca37d946bc55
INFO:    Creating SIF file...

When the process finishes, you should have an Apptainer image amplicon_2024.5.sif of the Docker container.

You can verify that the process worked, with:

apptainer run -B $(pwd):/data amplicon_2024.5.sif qiime

And, if the command works, you should get an output that looks like:

Usage: qiime [OPTIONS] COMMAND [ARGS]...

  QIIME 2 command-line interface (q2cli)
  --------------------------------------

  To get help with QIIME 2, visit https://qiime2.org.

  To enable tab completion in Bash, run the following command or add it to
  your .bashrc/.bash_profile:

      source tab-qiime

  To enable tab completion in ZSH, run the following commands or add them to
  your .zshrc:

      autoload -Uz compinit && compinit
      autoload bashcompinit && bashcompinit
      source tab-qiime

Options:
  --version   Show the version and exit.
  --help      Show this message and exit.

Commands:
  info                Display information about current deployment.
  tools               Tools for working with QIIME 2 files.
  dev                 Utilities for developers and advanced users.
  alignment           Plugin for generating and manipulating alignments.
  composition         Plugin for compositional data analysis.
  cutadapt            Plugin for removing adapter sequences, primers, and
                      other unwanted sequence from sequence data.
  dada2               Plugin for sequence quality control with DADA2.
  deblur              Plugin for sequence quality control with Deblur.
  demux               Plugin for demultiplexing & viewing sequence quality.
  diversity           Plugin for exploring community diversity.
  diversity-lib       Plugin for computing community diversity.
  emperor             Plugin for ordination plotting with Emperor.
  feature-classifier  Plugin for taxonomic classification.
  feature-table       Plugin for working with sample by feature tables.
  fragment-insertion  Plugin for extending phylogenies.
  longitudinal        Plugin for paired sample and time series analyses.
  metadata            Plugin for working with Metadata.
  phylogeny           Plugin for generating and manipulating phylogenies.
  quality-control     Plugin for quality control of feature and sequence data.
  quality-filter      Plugin for PHRED-based filtering and trimming.
  rescript            Pipeline for reference sequence annotation and curation.
  sample-classifier   Plugin for machine learning prediction of sample
                      metadata.
  taxa                Plugin for working with feature taxonomy annotations.
  vsearch             Plugin for clustering and dereplicating with vsearch.

Ending your session

Once your done, it's a good idea to properly exit your session. This will free up the computing resources immediately, allowing others to use them.

At the shell prompt, enter the exit command. After running, you should see an output similar to the following:

exit
salloc: Relinquishing job allocation 229
salloc: Job allocation 229 has been revoked.
jtyocum@sph-login0:~$