installed software build scripts - raeker/ARC-Wiki-Test GitHub Wiki
Scripts for building software in /sw/arcts/centos7 on the Great Lakes cluster are maintained in the git repository, arc_glb_build_scripts, on Bitbucket. See https://bitbucket.org/um_flux_software/arc_glb_build_scripts/src/master/. Installers in the UM_Flux_software group on Bitbucket should clone the git repository to /var/software/${USER} where ${USER} is the installers username for their *sw account:
git clone [email protected]:um_flux_software/arc_glb_build_scripts.git
The *sw accounts must be used in order to have write permission in the /sw/arcts/centos7 directory, where software managed by Lmod reside.
Build scripts on Great Lakes have a specific format that requires sourcing a common configure script, build.cfg, providing a number of functions that automatically determine installation paths, creation of log directories, etc.
The installer should set the variable SCRIPT_ROOT in their .bashrc file to point to the cloned build script repository as follows:
export SCRIPT_ROOT=/var/software/${USER}/arc_glb_build_scripts
With SCRIPT_ROOT set in a users .bashrc file, each build script begins by confirming that the variable is set. Then, the build.cfg file is sourced to make predefined build functions available to the package build script:
source $SCRIPT_ROOT/build.cfg
Within each package build script, modules are purged, correct ones loaded, and then loaded modules are listed. The STACKS variable must be set to "yes" if the module will be part of a module hierarchy (described below) or "no" if not. Depending on the given package, an extension for the source file (TAR_TYPE) may be set, as well as any special CONFIG_OPTIONS. A basic installation, without any unusual steps or naming conventions, would then call the build_setup function from build.cfg which will set up the build directory, set up the install directory, and unpack the tar ball before continuing with the remaining package specific build steps. Following the standard build script approach, settings are confirmed before proceeding with installation and log files are saved when installation completes.
We have chosen to organize our package installation tree based on
package dependencies such as compiler type and version and MPI type
where relevant, similar to the way our modulefiles hierarchy has been
implemented. All examples assume you start at the top of the stacks
directory. This is what the stacks directory tree looks like with
only a few applications shown for example.
/sw/arcts/centos7
stacks
├── gcc
│ ├─── 4.8.5
│ │ └── hdf5
│ │ └── openmpi
│ └─── 8.2.0
│ ├── hdf5
│ ├── openmpi
│ └── mpi-pkgs
│ ├── impi
│ │ └── 2018.4.274
│ │ └── hdf5
│ └── openmpi
│ └── 3.1.4
│ └── hdf5
├── intel
│ ├─── 14.0.2
│ │ ├── hdf5
│ │ ├── openmpi
│ │ └── mpi-pkgs
│ │ └── openmpi
│ │ └── 3.1.4
│ │ └── fftw
│ ├─── 18.0.5
│ │ ├── hdf5
│ │ ├── openmpi
│ │ └─── mpi-pkgs
│ │ ├── impi
│ │ │ └── 2018.4.274
│ │ │ └── hdf5
│ │ └── openmpi
│ │ └── 3.1.4
│ │ └── hdf5
│ └─── 19.1
│ └── hdf5
│ └── openmpi
└── pgi
└─── 19.1
└── openmpi