Maintaining Spack Buildcache - GEOS-ESM/MAPL GitHub Wiki

This page details the efforts to create a buildcache for use in Spack CI tests in MAPL.

Useful Links

• Spack Buildcache Documentation
• setup-spack GitHub Action
• Possible Example

Building Packages on AWS

Preliminaries

Padded Length

Testing showed we apparently need to increase the length of the padding in the install tree. This is what I ran:

spack config add config:install_tree:padded_length:512

packages.yaml

My current packages.yaml is set as:

packages:
  all:
    require: target=x86_64_v3
    compiler: [gcc@=14.2.0]
    providers:
      mpi: [openmpi, intel-oneapi-mpi]
      blas: [openblas, intel-oneapi-mkl]
      lapack: [openblas, intel-oneapi-mkl]

  gcc:
    buildable: false
  openmpi:
    require:
    - any_of: ['%gcc']
      message: Only use Open MPI with GCC

  intel-oneapi-mpi:
    require:
    - any_of: ['%oneapi']
      message: Only use Intel MPI with Intel

  hdf5:
    variants: +fortran +szip +hl +threadsafe +mpi
    # Note that cdo requires threadsafe, but hdf5 doesn't
    # seem to want that with parallel. Hmm.
  netcdf-c:
    variants: +dap
    #variants: +hdf4 +dap
  esmf:
    #variants: ~pnetcdf ~xerces ~external-parallelio
    variants: ~pnetcdf ~xerces
  cdo:
    variants: ~proj ~fftw3
    # cdo wanted a lot of extra stuff for proj and fftw3. Turn off for now
  pflogger:
    variants: +mpi
  pfunit:
    variants: +mpi +fhamcrest
  fms:
    variants: ~gfs_phys +pic +yaml constants=GEOS precision=32,64 +deprecated_io
  mapl:
    variants: +extdata2g +fargparse +pflogger +pfunit ~pnetcdf

Building Packages

We want all the dependencies of MAPL, so we need to do:

spack install --only dependencies mapl
spack install mepo

This (should) get us everything that MAPL needs to build.

NOTE: Make sure you are using ESMF 8.8.0. This is needed because of the sprintf issue Atanas found. It's triggered in ubuntu 24

Building the Buildcache

Create Spack Environment

It turns out, you can't build or push a buildcache unless you are in a Spack environment. So we need to create one:

spack env create mapl-buildcache
spack env activate -p mapl-buildcache

Add Packages to Environment

spack install --add esmf gftl gftl-shared fargparse pflogger pfunit yafyaml ecbuild udunits mepo
spack concretize -f

Add mirror for buildcache

spack mirror add --oci-username mathomp4 --oci-password-variable MAPL_BUILDCACHE_TOKEN --unsigned mapl-buildcache oci://ghcr.io/mathomp4/mapl-buildcache

Pushing the Buildcache

spack buildcache push --update-index oci://ghcr.io/mathomp4/mapl-buildcache