QuNex container development - ULJ-Yale/qunexsdk GitHub Wiki

The QuNex container framework supports both Apptainer (previously Singularity) and Docker deployments. Below is a brief overview of steps for how to build and tag a new container against a Docker registry. For any container development questions please email [email protected].

QuNex container versioning

Just like the QuNex suite, the QuNex container uses the semantic versioning: MAJOR.MINOR.PATCH (see (link) for details). The QuNex container version tag should match the version tag of the QuNex suite incorporated in the container. If QuNex container code changes while the QuNex suite code remains the same (e.g. the HCP pipelines or FSL included in the container is upgraded) then a letter is appended to the QuNex container tag, for example the tag 0.61.9 becomes 0.61.9a. With the release of the next QuNex suite version (e.g. 0.61.10) the appended letter is removed and the container is again tagged with the same tag as the suite.

Docker container build & tag

# -- Run docker build using the desired final layer docker file (here dockerfile_qunex_suite)
TAG=<QuNex version tag>
DOCKER_BUILDKIT=1 docker build --no-cache -f dockerfile_qunex_suite -t docker.io/qunex/qunex_suite:${TAG} .

# -- Push the version
docker push docker.io/qunex/qunex_suite:${TAG}

# -- Tag as latest
docker tag docker.io/qunex/qunex_suite:${TAG} docker.io/qunex/qunex_suite:latest

Singularity / Apptainer container build

General:

# -- Set tag
TAG=<QuNex version tag>

# -- Build
apptainer build qunex_suite-${TAG}.sif docker://qunex/qunex_suite:${TAG}

# -- Move to Apptainer (Singularity) folder
mv qunex_suite-${TAG}.sif $QX_CONTAINERS/qunex_suite-${TAG}.sif

# -- Tag as latest
ln -sf $QX_CONTAINERS/qunex_suite-${TAG}.sif $QX_CONTAINERS/qunex_suite-latest.sif

Using local Docker registry:

# -- Set tag
TAG=<QuNex version tag>

# -- Save docker image to a local file
docker save qunex/qunex_suite:${TAG} -o qunex_suite-${TAG}.tar

# -- Build sif
apptainer build qunex_suite-${TAG}.sif docker-archive://qunex_suite-${TAG}.tar

# -- Move to Apptainer (Singularity) folder
mv qunex_suite-${TAG}.sif $QX_CONTAINERS/qunex_suite-${TAG}.sif

# -- Remove the local tar
rm qunex_suite-${TAG}.tar

# -- Tag as latest
ln -sf $QX_CONTAINERS/qunex_suite-${TAG}.sif $QX_CONTAINERS/qunex_suite-latest.sif

On Grace:

# -- Log in to Grace
ssh <netID>@grace.hpc.yale.edu

# -- Switch to a compute node
srun --pty -p day -C cascadelake -c40 --mem=128GB -t8:00:00 bash

# -- Set scratch dir to a location with enough space
SCRATCH_DIR=/home/`whoami`/palmer_scratch
cd ${SCRATCH_DIR}
export APPTAINER_TMPDIR=${SCRATCH_DIR}/apptainer
export APPTAINER_CACHEDIR=${SCRATCH_DIR}/apptainer

# -- Set tag
TAG=<QuNex version tag>

# -- Build
apptainer build qunex_suite-${TAG}.sif docker://qunex/qunex_suite:${TAG}

# -- Move to Apptainer (Singularity) folder on grace
mv qunex_suite-${TAG}.sif ${QX_CONTAINERS}/qunex_suite-${TAG}.sif

Docker container build & tag a legacy version

First, you need to prepare dockerfile_qunex_update by specifying the starting container version, the QuNex branch to use and the library submodule branch to use. Next execute the code below.

# -- Log into dev server
ssh <netID>@10.5.38.115

# -- Run docker build using the desired final layer docker file (here dockerfile_qunex_suite)
TAG=<QuNex version tag>
DOCKER_BUILDKIT=1 docker build --no-cache -f dockerfile_qunex_update -t docker.io/qunex/qunex_suite:${TAG} .

# -- Push the version
docker push docker.io/qunex/qunex_suite:${TAG}
⚠️ **GitHub.com Fallback** ⚠️