QuNex container development - ULJ-Yale/qunexsdk GitHub Wiki

The QuNex container framework supports both Apptainer (previously Singularity) and Docker deployments. Below is a brief overview of steps for how to build and tag a new container against a Docker registry. For any container development questions please email [email protected].

QuNex container versioning

Just like the QuNex suite, the QuNex container uses the semantic versioning: MAJOR.MINOR.PATCH (see (link) for details). The QuNex container version tag should match the version tag of the QuNex suite incorporated in the container. If QuNex container code changes while the QuNex suite code remains the same (e.g. the HCP pipelines or FSL included in the container is upgraded) then a letter is appended to the QuNex container tag, for example the tag 0.61.9 becomes 0.61.9a. With the release of the next QuNex suite version (e.g. 0.61.10) the appended letter is removed and the container is again tagged with the same tag as the suite.

Docker container build & tag

# -- Run docker build using the desired final layer docker file (here dockerfile_qunex_suite)
TAG=<QuNex version tag>
DOCKER_BUILDKIT=1 docker build --no-cache -f dockerfile_qunex_suite -t docker.io/qunex/qunex_suite:${TAG} .

# -- Make sure that you are logged into the relevant docker registry
docker login docker.io

# -- Push the version
docker push docker.io/qunex/qunex_suite:${TAG}

Singularity / Apptainer container build

General:

# -- Set tag
TAG=<QuNex version tag>

# -- Set Docker username
export APPTAINER_DOCKER_USERNAME=qunex
export APPTAINER_DOCKER_PASSWORD=<QuNex docker password>
apptainer build --docker-login qunex_suite-${TAG}.sif docker://qunex/qunex_suite:${TAG}

# -- Move to Apptainer (Singularity) folder

On Grace:

# -- Log in to Grace
ssh <netID>@grace.hpc.yale.edu

# -- Switch to a compute node
srun --pty -p day -C cascadelake -c40 --mem=128GB -t8:00:00 bash

# -- Set scratch dir to a location with enough space
SCRATCH_DIR=/home/`whoami`/palmer_scratch
cd ${SCRATCH_DIR}
export APPTAINER_TMPDIR=${SCRATCH_DIR}/apptainer
export APPTAINER_CACHEDIR=${SCRATCH_DIR}/apptainer

# -- Set tag
TAG=<QuNex version tag>

# -- Set Docker username
export APPTAINER_DOCKER_USERNAME=qunex
export APPTAINER_DOCKER_PASSWORD=<QuNex docker password>
apptainer build --docker-login qunex_suite-${TAG}.sif docker://qunex/qunex_suite:${TAG}

# -- Move to Apptainer (Singularity) folder on grace
mv qunex_suite-${TAG}.sif ${QX_CONTAINERS}/qunex_suite-${TAG}.sif

# -- At this point we usually AT the container, before publishing it.

Docker container build & tag a legacy version

First, you need to prepare dockerfile_qunex_update by specifying the starting container version, the QuNex branch to use and the library submodule branch to use. Next execute the code below.

# -- Log into dev server
ssh <netID>@10.5.38.115

# -- Run docker build using the desired final layer docker file (here dockerfile_qunex_suite)
TAG=<QuNex version tag>
DOCKER_BUILDKIT=1 docker build --no-cache -f dockerfile_qunex_update -t docker.io/qunex/qunex_suite:${TAG} .

# -- Make sure that you are logged into the relevant docker registry
docker login docker.io

# -- Push the version
docker push docker.io/qunex/qunex_suite:${TAG}
⚠️ **GitHub.com Fallback** ⚠️