Spack Environments - EpiModel/EpiModeling GitHub Wiki

spack offers an environment feature. This is similar to renv for R. Where only a subset of the available modules are available to the environment and the configuration is shareable.

`loadR.sh` with `spack* environment

Considering a loadR.sh script like so:

#!/bin/bash
. /projects/epimodel/spack/share/spack/setup-env.sh
spack load [email protected]
spack load [email protected]

with an environment called epimodel, the script would be:

#!/bin/bash
. /projects/epimodel/spack/share/spack/setup-env.sh
spack env activate epimodel
spack load r
spack load git

The only difference is the activation of the environment spack env activate epimodel and the removal of the version for the modules loaded. As the environment contains only one version of each package.

Creating an environment

Like git branches or renv environments, spack environments are cheap. They share the same modules as the global spack installation.

To create an epimodel environment, we run:

spack env create epimodel

We activate and deactivate it with:

spack activate epimodel
spack deactivate

Within an environment, spack add [email protected] will add R version 4.3.x to the environment. spack install then resolve the dependencies and install the missing modules. Installed modules are also available outside of the environment but the version (@x.x.x) needs to be specified.

Example

Let's create an epimodel environment with gcc@13, [email protected], [email protected] and a few R packages.

spack env create epimodel
spacke env activate epimodel

# the compiler is added first, in case it needs to be compiled
spack add gcc@13
spack install
spack load gcc
spack compiler find

# then the rest of the packages are installed
spack add [email protected]%gcc@13 [email protected]%gcc%13 r-renv r-tidyverse r-ergm
spack install

From there on, the packages can be loaded without mentioning the version as there will be only one available.

In this example, we did not specify [email protected] or [email protected]. spack will use the latest version matching [email protected] and gcc@13. However, the spack.lock file will contain the exact version installed.

Multiple environments, for different versions of R can be maintained on a single HPC. The user would only choose the right environment to activate for the project and the rest of the commands would be unchanged. Also, the admin could install update to some software without breaking the users projects (going from [email protected] -> [email protected] for example).

Advantages

Environments allow the isolation of packages and simplifies the usage of spack for end users.

Similarly to how renv allows multiple project to use different versions of R libraries, spack environments extend this to non R software. This becomes important where multiples languages are used in a project or when some R packages require older versions of some low level libraries in order to be compiled.

For end users, the self contained nature of environments means that they can activate them and load the packages without the need to find the compilers or know the exact version of each software. This task is only left to the spack admin that will create the environments.