Conference call notes 20220831 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 205th EasyBuild conference call, Wednesday 31 August 2022 (15:00 UTC)

Attendees

List of attendees (15):

  • Simon Branford (Univ. of Birmingham, UK)
  • Em Dragowsky (Case Western Reserve University, Ohio, US)
  • Jasper Grimm (University of York, UK)
  • Thomas Hayward-Schneider (Max Planck Institute for Plasma Physics, Germany)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Sam Moors (Vrije Universiteit Brussel, Belgium
  • Abhishek Mukherjee (New Jersey Institute of Technology, US)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Bart Oldeman (Digital Research Alliance of Canada)
  • Jörg Saßmannshausen (Imperial College London, UK)
  • Åke Sandgren (Umeå University, Sweden)
  • Alexandre Strube (JSC, Germany)
  • Sebastien Moretti (SIB, Switzerland)
  • Davide Vanzo (Microsoft Azure)
  • Matthew West (University of Exeter, UK)

Agenda

  • overview of recent developments + outlook to next EasyBuild release
  • how to communicate that some software packages have been replaced going forward (pkg-config, GTK+, ...)
  • NVHPC toolchains
  • backwards-incompatible changes being considered for EasyBuild v5.0
  • Q&A

Recent developments

  • release timeline
    • latest release: EasyBuild v4.6.0 (8 July 2022)
    • ETA next release (EasyBuild v4.6.1): ASAP
  • progress on problems with older Intel MPI versions on RHEL 8.6 (see issue #15651)
    • kernel patch to fix the hangs is being worked into upcoming RHEL 8.6 kernel release
  • recent changes
    • framework
      • bug fixes
        • ...
      • enhancements
        • ...
      • changes
        • use SYSTEM constant for dependency that uses system toolchain in dumped easyconfig PR #4069
          • just so we can use SYTEM to indicate that a dependency should resolved with system toolchain without easyconfigs test suite complaining: 
            dependencies = [
    ('Java', 11, '', SYSTEM),
]
    • easyblocks
      • bug fixes
        • use lib* in post_install step of FFTW.MPI easyblock to fix paths not being found on Linux distros favouring lib64 (like Suse/SLES) (PR #2771)
        • tweak Amber(Tools) easyblock to run tests from top-level directory (PR #2781)
      • enhancements
        • make Amber(Tools) easyblock aware of FlexiBLAS (PR #2720)
        • make $LD_LIBRARY_PATH detection more robust for LAMMPS (PR #2765)
        • enhance NVHPC easyblock to avoid superfluous warning (PR #2767)
        • set $NVHPC_CUDA_HOME for NVHPC 22.7+ (PR #2776)
      • updates
        • ...
      • changes
        • update CI workflows to use Ubuntu 20.04 (since Ubuntu 18.04 is deprecated) (PR #2779)
      • new software
        • ...
    • easyconfigs

      • ~75 easyconfig PRs were merged since last conf call

      • bug fixes

        • use correct Matlab Runtime Compiler (v8.4) for FreeSurfer v7.1.1 (PR #13375)
        • add libffi and elfutils as dependency for Clang v11.0.1 + v12.0.1 (PR #15935)
          • should also be fixed for other Clang easyconfigs
        • add patch to fix broken test on POWER for numpy in SciPy-bundle 2022.05 (PR #15968)
        • refactor checksum test for extensions to use collect_exts_file_info (PR #15973)
        • exclude failing test in TensorFlow 2.4.1 (PR #16042)
        • skip NASA performance test in netCDF v4.9.0 (PR #16050)
        • add missing BCFtools dependency for recent medaka versions (PR #16107)
        • disable remote server tests for netCDF 4.9.0 (PR #16158)
        • add GTK2 as dependency for Ghostscript v9.56.1 (PR #16112)

      • enhancements

        • add pigz dependency for cutadapt v3.4 + v3.5 (PR #16056)

      • (noteworthy) new software

      • noteworthy software updates

        • LAMMPS v23Jun2022 with CUDA support (PR #15900)

      • changes

        • speed up OpenMPI 4.1.4 configure by not running "autogen.pl --force", but only running required Autotools commands (PR #15957)
        • replace sed commands by upstreamed patches for BLIS built with intel-compilers toolchain (PR #15958)
        • make check for toolchain value in dependency spec in easyconfigs test suite aware that dumped easyconfig uses SYSTEM constant (PR #16126)
        • update CI workflows to use Ubuntu 20.04 (since Ubuntu 18.04 is deprecated) (PR #16070)
    • framework
      • reported bugs / bug fixes
        • ensure we call EasyBlock.patch_step for postinstallpatches (PR #4063)
        • fix leaked stty handle (PR #4066)
        • fix type-checking of patches to allow dict values (PR #4070)
      • enhancements
        • add support for easystack file that contains easyconfig filenames + implement parsing of configuration options (PR #4021)
        • adding support for the use of --from-pr _and other options) in easystack files (WIP PR #4057)
      • changes
        • print potential errors after running a command only once (PR #4065)
        • fix run_cmd error handling (PR #4067)
        • run python in the same process as eb wrapper script by using exec (PR #4048
        • update CI workflows to use Ubuntu 20.04 (since Ubuntu 18.04 is deprecated) (PR #4064)
          • breaks container tests because yum can no longer be installed easily in Ubuntu 20.04 via apt...
    • easyblocks
      • bug reports/fixes
        • don't enable building of ld.gold when installing binutils on a RISC-V system + don't configure GCC to use gold as default linker on a RISC-V system (PR #2780)
      • enhancements
        • ...
        • use det_cmake_version function to determine CMake version in CMakeMake generic easyblock (PR 2772)
      • updates
        • ...
      • new software
        • ...
      • changes
        • ...
    • easyconfigs
      • close to 700 open easyconfig PRs...
      • bug fixes
        • fix OpenBLAS 0.3.15 patch to correctly set the CPU core type for Tiger Lake (PR #15845)
        • add and fix patches for PyTorch 1.9.0 on POWER (PR #15919)
        • use versioned syms in system level ncurses (PR #16064)
        • fix checksum for Stacks v2.62 (due to silent re-release without version bump) (PR #16134)
        • fix libsanitzer for glibc 2.36 to build GCCcore 11.3.0 (PR #16145)
        • fix top level Makefile for AmberTools 20/21 (PR #16150, PR #16151, PR #16152)
        • remove modextrapaths to add top-level install directory to $PATH for InterProScan v5.55-88.0 (now done by default by Binary easyblock) (PR #16167)
        • make sure that InterProScan installation directory is writable before creating symlink to data in InterProScan_data easyconfig (PR #16169)
      • new software
      • noteworthy software updates
      • changes
        • simplify AlphaFold foss/2021a easyconfigs by using a fleshed out patched OpenMM dependency (PR #15981)
        • use OpenJDK for Java 8 (PR #16062)
        • drop HDF5 from 1.13.1 to 1.12.2 (PR #16153)
        • inconsistent use of versionsuffix for Java (issue #16038)

How to communicate that some software packages have been replaced going forward (pkg-config, GTK+, ...)

  • make easyconfigs test suite complain in PRs
  • also add comment in easyconfigs?
  • README in easyconfigs directory for that software
  • also in documentation? (and refer to that from everywhere)
  • other cases: util-linux, texinfo (makeinfo), numpy/scipy/pandas now in SciPy-bundle, ...

NVHPC toolchains

  • cfr. https://github.com/easybuilders/easybuild-easyconfigs/issues/16066
  • Add CUDA as versionsuffix to NVHPC easyconfigs, to make it clear which CUDA is being used
  • drop nvompic in favor of nvompi + CUDA (similar to recent foss + CUDA)
  • should current NVHPC be renamed (to nv-compilers) so we can use nvhpc as toolchain name for a full toolchain?
  • or use nvoff as full toolchain name?
    • nv for NVHPC, o for OpenMPI, ff for FlexiBLAS + FFTW
  • should NVHPC that is used in a toolchain be named nv-compilers, while also keeping NVHPC modules stand-alone?

Backwards-incompatible changes being considered for EasyBuild v5.0

Q&A

  • ...