Conference call notes 20210106 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 164th EasyBuild conference call, Wednesday January 6th 2021 (16:00 UTC - 17:00 CET)

Attendees

Alphabetical list of attendees (11):

Sebastian Achilles (Jülich Supercomputing Centre, Germany)
Simon Branford (University of Birmingham, UK)
Alex Domingo (Vrije Universiteit Brussel, Belgium)
Fotis Georgatos (SDSC, Switzerland)
Kenneth Hoste (HPC-UGent, Belgium)
Adam Huffman (Big Data Institute, Oxford, UK)
Terje Kvernes (University of Oslo, Norway)
Kurt Lust (Univ. of Antwerpen - LUMI)
Robert Mijakovic (LuxProvide)
Bart Oldeman (ComputeCanada)
Jörg Saßmannshausen (NIHR Biomedical Research Centre, UK)

Agenda

update on recent developments
Q&A

Recent developments

next release (v4.3.3): before EUM'21?
recent changes
- framework
  - bug fixes
    - add --init and --recursive options to 'git submodule update' command that is used when creating source tarball for specific commit (PR #3537)
    - filter out duplicate paths in RPATH wrapper script (PR #3538)
  - enhancements
    - add support for --accept-eula configuration option + 'accept_eula' easyconfig parameter (PR #3535)
      - used for AOCC, will also be useful for Intel oneAPI (and maybe also NVHPC, see PR #11920)
- easyblocks
  - bug fixes
    - only configure PETSc with --with-numpy or --with-mpi4py if those Python packages are available, and only if Python is not a build-only dependency (PR #2299)
    - fix handling of build deps in PETSc/SLEPc/Trilinos easyblocks (PR #2299, PR #2300, PR #2301)
  - enhancements
    - set $TF_GPU_COUNT and $TF_TESTS_PER_GPU for TensorFlow tests (PR #2292)
    - update BerkeleyGW (PR #2297) and QuantumESPRESSO (PR #2298) easyblocks to fix compilation issues with GCC 10.x
  - new software
    - custom easyblock for AOCC (PR #)
- easyconfigs
  - bug fixes
    - fix source URL in expat easyconfigs (and consistently add custom sanity_check_paths) (PR #11960)
  - enhancements
  - new software
    - gobff toolchains (PR #11761)
    - AOCC (PR #11868)
    - AMD-LibM (PR #11929 + PR #11933)
  - software updates
    - CP2K 7.1 with foss/2020b (PR #11669) and intel/2020b (PR #11670)
    - NVHPC 20.9 (PR #11694)
    - update Java/1.8 wrapper to JDK 8u271 (PR #11928)
    - OpenMPI 4.1.0 (PR #11943)
    - QuantumESPRESSO 6.6 w/ foss/2020b (PR #11946)
    - PyTorch 1.6.0 w/ fosscuda/2019b + imkl (PR #11785)
      - imkl is required for FFT support in PyTorch
  - changes
    - (nothing special)
to merge/fix/tackle soon
- framework (v4.3.3 milestone)
  - support additional features in easystack files (see issues #3468, #3512, #3513, #3516)
- easyblocks (v4.3.3 milestone)
  - bug fixes
    - correctly determine path to active binutils in TensorFlow easyblock (PR #2218)
    - fix taking into account --sysroot when installing/using CMake
      - two options:
        
        use toolchain file in CMakeMake (PR #2247)
        
        patch CMake installation (PR #2248)
  - enhancements
    - enhance OpenBLAS easyblock to make it aware of optarch (PR #1946)
      - needs testing...
    - add support for statically linking Bazel (PR #2272)
    - set $PYTHONNOUSERSITE in PythonBundle.extensions_step (PR #2272)
    - improve Bazel easyblock: add support for running tests, enabl static linking, use build dir rather than tmpdir, verbose output (PR #2285)
    - add support for skipping steps in Extension PythonPackages (PR #2290)
  - changes
    - create less temporary directories for TensorFlow by (only) using --output_user_root (PR #2293)
  - new software
    - custom easyblock for ADIOS (PR #2070), JAX (PR #2262), RELION (PR #2274)
- easyconfigs (v4.3.3 milestone)
  - bug fixes
    - added missing space in configopts in ParaView 5.8.0 easyconfigs using 2020a toolchain (PR #10989)
      - (cfr. discussion in previous conf calls)
  - enhancements
    - (nothing special)
  - new software
    - Intel oneAPI
      - there will be no further updates to Intel Parallel Studio (apparently)
  - software updates
    - PyTorch 1.7.1 (PR #11636)
    - TensorFlow 2.4.0 (PR #11637)

Q&A

Sebastian: AMD toolchains
- what should be in it as compiler: AOCC or Clang (+ flang) + AMD LibM
  - performance differences between AOCC and standard Clang seem small
  - Clang-based toolchain is more portable (and open source)
  - AOCC may be used for vendor support reasons
- should we consult AMD or George @ LUMI?
  - cfr.
- Kurt: similar question w.r.t. MPI library
  - we're currently focused a lot on OpenMPI
  - but we should consider MPICH too, for performance and compability with vendor MPIs (Cray MPI, Intel MPI, etc.)
- Robert: ATOS seems to favor OpenMPI
- Bart: AMD-optimized HPL is built on top of OpenMPI
- do we need to reconsider the approach of common toolchains going forward?
  - increasing variety in CPUs (Intel, AMD, Arm, POWER, ...)
  - probably implies we need more diverse toolchains w.r.t. compiler & libraries
  - another Clang-based common toolchains?
    - since NVHPC, AOCC, Intel oneAPI, Cray are all based on Clang/LLVM...
    - MPI should be MPICH2 due to arguments above
      - what about MVAPICH2? (ping Xavier)