Maintaining Spack Buildcache - GEOS-ESM/MAPL GitHub Wiki

This page details the efforts to create a buildcache for use in Spack CI tests in MAPL.

Useful Links

• Spack Buildcache Documentation
• setup-spack GitHub Action
• Possible Example

Building Packages on AWS

Preliminaries

Padded Length

Testing showed we apparently need to increase the length of the padding in the install tree. This is what I ran:

spack config add config:install_tree:padded_length:256

packages.yaml

My current packages.yaml is set as:

packages:

  gcc:
    externals:
    - spec: [email protected] languages='c,c++,fortran'
      prefix: /usr
      extra_attributes:
        compilers:
          c: /usr/bin/gcc
          cxx: /usr/bin/g++
          fortran: /usr/bin/gfortran
        flags: {}
        environment: {}
        extra_rpaths: []
    - spec: [email protected] languages='c,c++,fortran'
      prefix: /usr
      extra_attributes:
        compilers:
          c: /usr/bin/gcc-14
          cxx: /usr/bin/g++-14
          fortran: /usr/bin/gfortran-14
        flags: {}
        environment: {}
        extra_rpaths: []
    buildable: false

  all:
    require: target=x86_64_v3
    providers:
      mpi: [openmpi, intel-oneapi-mpi]
      blas: [openblas, intel-oneapi-mkl]
      lapack: [openblas, intel-oneapi-mkl]

  gcc:
    buildable: false
  openmpi:
    require:
    - any_of: ['%gcc']
      message: Only use Open MPI with GCC

  intel-oneapi-mpi:
    require:
    - any_of: ['%oneapi']
      message: Only use Intel MPI with Intel

  hdf5:
    variants: +fortran +szip +hl +threadsafe +mpi
    # Note that cdo requires threadsafe, but hdf5 doesn't
    # seem to want that with parallel. Hmm.
  netcdf-c:
    variants: +dap
  esmf:
    variants: ~pnetcdf ~xerces
  cdo:
    variants: ~proj ~fftw3
    # cdo wanted a lot of extra stuff for proj and fftw3. Turn off for now
  pflogger:
    variants: +mpi
  pfunit:
    variants: +mpi +fhamcrest
  fms:
    require: '@2024.03 ~gfs_phys +pic constants=GEOS precision=32,64 +deprecated_io ~yaml target=x86_64_v3'
    # NOTE: I have to re-add the target here otherwise spack seemed to not build for a generic processor
  mapl:
    variants: +extdata2g +fargparse +pflogger +pfunit ~pnetcdf

repos.yaml

Spack is now split with packages in a separate repo. To be consistent, we manually clone and put it in a place easy to maintain. As such, our repos.yaml is:

repos:
  builtin:
    git: [email protected]:spack/spack-packages.git
    destination: /home/ubuntu/spack-packages
  geosesm: /home/ubuntu/geosesm-spack

Building Packages

We want all the dependencies of GEOS, so we need to do:

spack install --only dependencies mapl
spack install mepo openblas [email protected]

This (should) get us everything that MAPL and GEOS needs to build.

Building the Buildcache

Create Spack Environment

It turns out, you can't build or push a buildcache unless you are in a Spack environment. So we need to create one:

spack env create geos-buildcache
spack env activate -p geos-buildcache

Add Packages to Environment

spack install --add [email protected] gftl gftl-shared fargparse pflogger pfunit yafyaml ecbuild udunits mepo openblas [email protected]
spack concretize -f

Add mirror for buildcache

spack mirror add --oci-username mathomp4 --oci-password-variable GEOS_BUILDCACHE_TOKEN --unsigned geos-buildcache oci://ghcr.io/GEOS-ESM/geos-buildcache

Pushing the Buildcache

spack buildcache push --update-index oci://ghcr.io/GEOS-ESM/geos-buildcache

ESMF 9

There is now a need to have ESMF 9 in a buildcache. But, testing shows that you can't really have two versions of ESMF in the same Spack environment. You should, but the fact that ESMF v9.0.0b03 is not a "real" release but a git commit makes it tricky.

So, let's make a new Spack environment and buildcache just for ESMF 9.

Add ESMF 9 to Spack

First, I did spack edit esmf and added the 9.0.0b03 version to the package.py file (see below).

diff --git a/repos/spack_repo/builtin/packages/esmf/package.py b/repos/spack_repo/builtin/packages/esmf/package.py
index 0db3a6a22e..183b4953fb 100644
--- a/repos/spack_repo/builtin/packages/esmf/package.py
+++ b/repos/spack_repo/builtin/packages/esmf/package.py
@@ -31,6 +31,7 @@ class Esmf(MakefilePackage, PythonExtension):
     # Develop is a special name for spack and is always considered the newest version
     version("develop", branch="develop")
     # generate chksum with 'spack checksum [email protected]'
+    version("9.0.0b03", tag="v9.0.0b03")
     version("8.9.0", sha256="586e0101d76ff9842d9ad43567fae50317ee794d80293430d9f1847dec0eefa5")
     version("8.8.1", sha256="b0acb59d4f000bfbdfddc121a24819bd2a50997c7b257b0db2ceb96f3111b173")
     version("8.8.0", sha256="f89327428aeef6ad34660b5b78f30d1c55ec67efb8f7df1991fdaa6b1eb3a27c")

Create New Spack Environment

spack env create geos-buildcache-esmf9
spack env activate -p geos-buildcache-esmf9

Add ESMF 9 to Environment and make Buildcache

spack install --add [email protected] gftl gftl-shared fargparse pflogger pfunit yafyaml ecbuild udunits mepo openblas [email protected]
spack concretize -f

spack mirror add --oci-username mathomp4 --oci-password-variable GEOS_BUILDCACHE_TOKEN --unsigned geos-buildcache-esmf9 oci://ghcr.io/GEOS-ESM/geos-buildcache-esmf9
spack buildcache push --update-index oci://ghcr.io/GEOS-ESM/geos-buildcache-esmf9