Setting up a Development Environment in your $HOME and submitting PRs [WIP] - NERSC/spack GitHub Wiki
TL;DR
git clone https://github.com/NERSC/spack.git
export SPACK_ROOT=$PWD/spack
. spack/share/spack/setup-env.sh
cd $SPACK_ROOT/etc/spack/
if [ x"$method" == "copy" ](/NERSC/spack/wiki/-x"$method"-==-"copy"-); then
grep -v craype_link_type ~swowner/spack/etc/spack/packages.yaml.backup >packages.yaml
# craype_link_type was removed from the schema in commit e697b385
else
cat >packages.yaml <<EOF
(paste contents of packages.yaml here)
EOF
fi
git checkout -b user/[package-name]
spack install [package-name]
spack edit [package-name]
git add $SPACK_ROOT/var/spack/repos/builtin/packages/[package-name]
git commit -m "Update [package-name]"
git push origin user/[package-name]
Git Clone
This first thing you want to do for setting up an environment in your home:
git clone https://github.com/NERSC/spack.git
Once you git clone the NERSC repository, your main branch will be nersc-spack. nersc-spack is the main branch for our fork which is out of sync with upstream. This is okay since the only time an update should be made is when changes to the core code are merged. Most commits being merged upstream are for new packages. We will explain how to find new packages later in this guide.
Adding Spack to your $PATH
Once you have git cloned the repository you will want to add spack to your PATH.
Note: Hopefully you're a bash user
export SPACK_ROOT=/path/to/spack
source "$SPACK_ROOT/share/spack/setup-env.sh"
Setting up a configuration
Spack has a number of configuration files and it does seem a bit confusing. Here is the order of precedence that Spack has for it's config.
TL;DR: command line > user > site > system > defaults
-
defaults: Stored in $(prefix)/etc/spack/defaults/. These are the “factory” settings. Users should generally not modify the settings here, but should override them in other configuration scopes. The defaults here will change from version to version of Spack.
-
system: Stored in /etc/spack. These are settings for this machine, or for all machines on which this file system is mounted. The site scope can be used for settings idiosyncratic to a particular machine, such as the locations of compilers or external packages. These settings are presumably controlled by someone with root access on the machine.
-
site: Stored in $(prefix)/etc/spack/. Settings here affect only this instance of Spack, and they override defaults. The site scope can can be used for per-project settings (one spack instance per project) or for site-wide settings on a multi-user machine (e.g., for a common spack instance). user: Stored in the home directory: ~/.spack/. These settings affect all instances of Spack and take higher precedence than site or default scopes. command line: Optionally specified by the user on the command line. These settings take the highest precedence. If multiple scopes are listed on the command line, they are ordered from lowest to highest precedence.
-
user: Stored in the home directory: ~/.spack/. These settings affect all instances of Spack and take higher precedence than site or default scopes. command line: Optionally specified by the user on the command line. These settings take the highest precedence. If multiple scopes are listed on the command line, they are ordered from lowest to highest precedence.
-
command line: Optionally specified by the user on the command line. These settings take the highest precedence. If multiple scopes are listed on the command line, they are ordered from lowest to highest precedence.
Note that there are also platform specific configuration files. These are located in ~/.spack/cray/.
NERSC external defaults
As of Aug 28th, this is the current defaults set for packages.yaml
packages:
all:
compiler: [[email protected], [email protected], [email protected]]
variants: ~shared
# craype_link_type: "static"
providers:
mpi: [mpich, openmpi]
mkl: [intel-mkl]
blas: [intel-mkl, cray-libsci]
scalapack: [intel-mkl, cray-libsci]
pkgconfig: [pkg-config]
pkg-config:
buildable: false
paths:
[email protected]: /usr
cmake:
buildable: false
paths:
[email protected]: /usr
mpich:
buildable: false
modules:
[email protected]%gcc: cray-mpich
[email protected]%intel: cray-mpich
[email protected]%cce: cray-mpich
intel-mkl:
buildable: false
paths:
[email protected]%intel: /opt/intel
[email protected]%gcc: /opt/intel
[email protected]%cce: /opt/intel
fftw:
buildable: false
modules:
[email protected]%gcc+openmp: cray-fftw
[email protected]%intel+openmp: cray-fftw
[email protected]%cce+openmp: cray-fftw
hdf5:
buildable: false
modules:
hdf5%intel~mpi+hl: cray-hdf5
hdf5%gcc~mpi+hl: cray-hdf5
hdf5%cce~mpi+hl: cray-hdf5
hdf5%intel+mpi+hl: cray-hdf5-parallel
hdf5%gcc+mpi+hl: cray-hdf5-parallel
hdf5%cce+mpi+hl: cray-hdf5-parallel
petsc:
buildable: false
modules:
[email protected]%gcc~complex~int64 : cray-petsc
[email protected]%intel~complex~int64 : cray-petsc
[email protected]%cce~complex~int64: cray-petsc
[email protected]%gcc+complex~int64 : cray-petsc-complex
[email protected]%intel+complex~int64 : cray-petsc-complex
[email protected]%cce+complex~int64: cray-petsc-complex
[email protected]%gcc~complex+int64 : cray-petsc-64
[email protected]%intel~complex+int64: cray-petsc-64
[email protected]%cce~complex+int64: cray-petsc-64
[email protected]%gcc+complex+int64 : cray-petsc-complex-64
[email protected]%intel+complex+int64 : cray-petsc-complex-64
[email protected]%cce+complex+int64: cray-petsc-complex-64
cray-libsci:
buildable: false
modules:
[email protected]%gcc: cray-libsci
[email protected]%intel: cray-libsci
[email protected]%cce: cray-libsci
netcdf:
buildable: false
modules:
[email protected]%gcc+parallel-netcdf+mpi: cray-netcdf-hdf5parallel
[email protected]%intel+parallel-netcdf+mpi: cray-netcdf-hdf5parallel
[email protected]%cce+parallel-netcdf+mpi: cray-netcdf-hdf5parallel
[email protected]%gcc~parallel-netcdf~mpi: cray-netcdf
[email protected]%intel~parallel-netcdf~mpi: cray-netcdf
[email protected]%cce~parallel-netcdf~mpi: cray-netcdf
netcdf-fortran:
buildable: false
modules:
[email protected]%intel: cray-netcdf-hdf5parallel
[email protected]%cce: cray-netcdf-hdf5parallel
[email protected]%gcc: cray-netcdf-hdf5parallel
papi:
buildable: false
modules:
[email protected]%gcc: papi
[email protected]%intel: papi
[email protected]%cce: papi
trilinos:
buildable: false
modules:
[email protected]%gcc: cray-trilinos
[email protected]%intel: cray-trilinos
[email protected]%cce: cray-trilinos
perl:
compiler: [[email protected], gcc, cce]
python:
variants: +shared
This should live in your ${spack_prefix}/etc/spack/ directory. By having it here, we setup site-wide defaults.
Compiler configuration
When you first run a spack install command, spack will go and find compilers for you. The compiler meta-data is then placed in ~/.spack/cray/. These are platform specific configuration files. If you want to modify some compiler information, you can go to this location or use spack config edit compilers.
I had to remove 3 duplicates of the "[email protected]" compiler by deleting the entire section.
Finding your package
Now that you have your externals setup, you can look at whether your package is available. Run spack list [package-name] to see if a package recipe is already available. Most likely it will be (Spack has over 2k packages and growing). If it isn't then you will have to spack create [URL] and start editing. If it is, then go on to the next step.
Installing package
spack install [package_name] will go on and install the package for you. Default location is $spack_root/opt/, this can be changed in the config.yaml configuration file. This can be placed in ~/.spack/.
Commit and submit PR
First create a new branch with git checkout -b [branch-name]. By now your package should be installing. Go ahead and git add and git commit the package. Then when you are on github, submit the PR. Someone will review it (probably Mario Melara) and then merge the PR.