Conference call notes 20250521 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 270th EasyBuild conference call, Wednesday 21 May 2025 (08:00 UTC / 10:00 CEST)

Attendees

List of attendees (15):

Alex Domingo (Vrije Universiteit Brussel, Belgium)
Loris Ercole (CECA)
Davide Grassano (CECAM)
Jasper Grimm (University of York, UK)
Alexander Grund (ZIH, Dresden, Germany)
Leonardo Honfi Camilo (Wageningen University, The Netherlands)
Kenneth Hoste (HPC-UGent, Belgium)
Kurt Lust (UAntwerpen / LUMI)
Mikael Öhman (Chalmers University of Technology, Sweden)
Jure Pečar (EMBL, Germany)
Andrea Piserchia (E4)
Jan Reuter (JSC, Germany)
Jörg Saßmannshausen (Imperial College London, UK)
Alain van Hoof (TU Eindhoven, Netherlands)
Cintia Willemyns (Vrije Universiteit Brussel, Belgium)

Agenda

overview of recent developments
2025a update of common toolchains
Q&A

Recent developments

latest EasyBuild release: EasyBuild v5.0.0 (18 March 2025) :partying_face:
next (stable) EasyBuild release: EasyBuid v5.1.0 (hopefully finally this week...)
- milestones are still being updated ("next release (5.1.0)" vs "release after 5.1.0")
- "release after 5.0.0" milestones are being renamed to "5.0.1" + some issues/PRs are being moved to new "release after 5.0.1" milestone
- thinking about trying to shift to releases on fixed schedule
  - block enough time in specific week to wrap up on a release

News

easyconfigs merge sprint scheduled for Monday 16 June 2025
- main goal is to push down on number of open easyconfig PRs
- sync meeting to get organised at 10:00 CEST
- split in groups?
  - closing old/stale PRs (cfr. toolchain support policy)
  - helping out "whales" (people with lots of open PRs)
  - recent PRs
- focus effort on easyconfigs PRs (which don't need changes in framework/easyblocks)
- trim down # configurations in CI workflows for easyconfigs repo

Merged PRs

(changed made in PRs marked with * are included latest EasyBuild stable release)

docs (merged PRs)
- ...
framework (merged PRs)
- bug fixes
  - restore original value for non-list easyconfig parameter values that are considered for iterating over (PR #4848)
  - use new ModEnvVarType.STRICT_PATH_WITH_FILES with CMAKE_LIBRARY_PATH environment variable (PR #4858)
    - fixes issue #4844 Regression: CMAKE_LIBRARY_PATH=lib added to modulefiles
  - Fix is_patch_for for patch dicts (PR #4865)
  - Update fake module for each extension installed (PR #4868)
  - use develop branch for PRs that target removed 5.0.x branch in fetch_files_from_pr (PR #4879)
  - implement exponential backoff in download_file + allow max. 6 attempts to download PR diff in fetch_files_from_pr and fetch_files_from_commit (PRs #4870 + #4880)
    - workaround for issue #4869 (failing --from-pr and/or --include-easyblocks-from-pr)
  - count the data_sources (PR #4873) + include data_sources when checking checksums (PR #4874)
  - also ignore errors raised during test step when --ignore-test-failure is used (PR #4881)
  - fix download progress bar (PR #4885)
  - check whether exts_defaultclass is set after call to prepare_for_extensions (PR #4888)
    - fixes regression introduced in PR #4868
- enhancements
  - add a CUDA device code sanity check (PR #4692)
  - use full words in summary (PR #4875)
  - obtain PR/commit diff via GitHub API rather than downloading *.diff file via github.com (PR #4878)
    - proper fix for issue #4869 (failing --from-pr and/or --include-easyblocks-from-pr)
  - Support options for patch command (PR #4886)
- changes
  - ...
- code cleanup
  - ...
- CI
  - ...
easyblocks (merged PRs)
- bug fixes
  - Enhance LLVM EasyBlock to better handle offload builds (PR #3675)
  - various fixes for the LLVM easyblock (PR #3706)
    - still missing: making sure that we don't pick up on system CUDA/ROCm
- enhancements
  - ...
- updates
  - minor updates to ABAQUS easyblock to make hotfixes 2441 and 2424 working (PR #3512)
  - Update TensorFlow easyblock for v2.18.x (PR #3699)
- changes
  - deprecate the JAR easyblock (PR #3721)
- new easyblocks
  - add generic Dataset easyblock (PR #3246)
- code cleanup
  - Remove Python2 style super()-calls and fix ExtensionEasyBlock (PR #3681)
- CI
  - ...
easyconfigs (merged PRs)
- ~XXX easyconfig PRs were merged since last conf call
- bug fixes/reports
  - stick to QIIME2 2024.2.0 with foss/2023a, since recent versions (like 2024.10.0) require a more recent version of pandas (PR #22904)
- enhancements
  - add patch to add zen5 support to OpenBLAS 0.3.27 (PR #22850)
- (noteworthy) new software
  - ...
- noteworthy software updates
  - LLVM 20.1.5 (PR #22903)
- cleanup
  - ...
- changes
  - update easyconfigs to use Binary instead of JAR easyblock (PR #22900)
  - use Java 17 as default Java version in all 2024a generation easyconfigs (PR #22862)

Open (active) PRs

docs (open PRs + issues)
- there still are some gaps in documentation that covers changed in EasyBuild 5.0.0
  - for example: https://docs.easybuild.io/interactive-debugging-failing-shell-commands
framework (open PRs + issues)
- PRs marked with (***) still being considered to be included with EasyBuild v5.1.0 (others are not)
- bugs
  - ...
- enhancements
  - (***)? add support for environment variables in set_environment of module generators (PR #4855)
  - (***) Avoid leaking keys by mistake with --upload-test-report (PR #4877)
- code cleanup
  - (***) move EasyBlock.expand_module_search_path into ModuleEnvironmentVariable.expand_paths (PR #4859)
- changes
  - ...
easyblocks (open PRs + issues)
- bug fixes/reports
  - run pip check only once for PythonBundle (PR #3432)
  - Avoid failure in Python package installation and sanity check when $PIP_REQUIRE_VIRTUALENV is set (PR #3460)
  - Unset PMIX variables when configuring OpenMPI (PR #3511)
  - Fix access to unassigned variable when installing Python < 3.9.1 with using sysroot (PR #3686)
  - fix using LLVM easyblock for bundle component + don't build GO tests by default for LLVM versions < 16 (PR #3690)
  - Set Rust channel to stable to prevent usage of rust-lld as default linker (PR #3691)
    - impact on easyconfigs that use Rust is unclear, some creates may use that nightly channel is used? => to be tested...
  - Avoid leaking file handles in PythonPackage (PR #3704)
  - Rebuild C++ API for Gurobi (PR #3714)
  - fix dry run for openssl_wrapper (PR #3719)
  - allow downloaded Python packages if implicitely requested (PR #3722)
  - Improve reliability of PyTorch test reporting (PR #3723)
    - required for PyTorch 2.6
  - Fix failure in sanity-check-only/module-only rebuilds of OpenMPI (PR #3724)
- enhancements
  - LAMMPS: generate version from the source files when using software-commit (PR #3485)
  - Let CMakeMake set the Python paths for CMake (PR #3694)
  - Handle single-line members in Cargo workspaces (PR #3705)
  - Fix Python shebangs for Python and Conda-like modules (PR #3717)
- updates
  - Update Mathematica easyblock to support versions > 14.1 (PR #3707)
  - update Amber easyblock for version 24.x and newer (PR #3685)
- changes
  - ...
- code cleanup
  - ...
- new easyblocks
  - add custom easyblock for Term::ReadLine::Gnu Perl module (PR #3712)
easyconfigs (open PRs + issues)
- ~1k open easyconfig PRs
- bug fixes/reports
  - ...
- enhancements
  - ...
- (noteworthy) new software
  - ...
- software updates
  - PyTorch 2.6.0 w/ foss/2024a (PR #22824)
  - foss/2025a (PR #22879)
- changes
  - add easybuild package to EasyBuild/5.0.0 (PR #22774)
    - see also change to EasyBuildMeta easyblock in easyblocks PR #3700

`2025a` common toolchains

(2024b is skipped to catch up with original schedule for defining common toolchain versions)
- EasyBuild v5.0 is ideal excuse for that break in continuity...
GCC 14.2 as a base (see easyconfigs PR #21114)
we should define candidate toolchains using latest version of all components
foss/2025a:
- easyconfigs using GCCcore/14.2.0 toolchain available for Python 3.13.1, Perl + Perl-bundle-CPAN 5.40.0
- easyconfigs PR #22125 merged for FlexiBLAS, OpenBLAS, BLIS
- (easyconfigs PR #22349) merged for cryptography, required for Python-bundle-PyPI, after fixing issue with RPATH linking of Rust crates
- gfbf/2025.02 + FFTW merged in easyconfigs PR #22126
intel/2025a:
- updates for intel-compilers and impi merged in PR #22657
- TODO: imkl
- separate package for intel-shmem, doesn't really belong in intel toolchain
GCC 14.3 will be released (really) soon
for intel/2025a, we can pick up on 2025.1.1 releases

Q&A / others

(Jan) fighting with PySide6 (PR #22906)
(Alex) still looking into NVHPC-based toolchains in EasyBuild
- VASP fails to built with our current NVHPC-based toolchains
  - works fine when using libraries that come with NVHPC itself
  - (Jörg) VASP supports GPUs via OpenACC, so could work on AMD GPUs?
    - (Kurt) not really, partial support for OpenACC on AMD GPUs
      - Fortran compiler support also matters a lot
      - VASP really only supports building with NVHPC for (NVIDIA) GPUs
      - OpenACC is "an NVIDIA standard" by now
      - GCC also supports OpenACC in C++/Fortran, but support is less good than in NVHPC
- looking into making toolchain with nvidia-compilers + NVHPC on top of it
  - OK to remove non-compiler stuff from nvidia-compilers installation?
    - install dir is ~21GB, so worth considering
    - kicking out anything non-compiler from nvidia-compilers makes sense
  - Jan: sticking to a single CUDA variant of NVHPC would also help to reduce disk space
    - especially old CUDA versions like 11.8
    - or supply CUDA as proper EasyBuild dependency (and delete all CUDA versions bundled with NVHPC)
    - JSC has seen tickets when combining separate CUDA with NVHPC (instead of one of the bundled ones)
- Davide has seen similar issues with QuantumESPRESSO & MetalWalls
- Jörg: one specific CUDA compute capability can be used when installing NVHPC?
  - Jan: probably not actually be necessary, NVHPC easyblock may need to be relaxed
- having to support lots of old GPUs makes this more painful, requires older CUDA versions