Conference call notes 20250212 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 264th EasyBuild conference call, Wednesday 12 Feb 2025 (16:00 UTC / 17:00 CET)

Attendees

List of attendees (17):

  • Em Dragowsky (Case Western Reserve University, Ohio, US)
  • Davide Grassano (CECAM)
  • Jasper Grimm (University of York, UK)
  • Alexander Grund (ZIH, Dresden, Germany)
  • Leonardo Honfi Camilo (Wageningen University, The Netherlands)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Georgios Kafanas (University of Luxembourg)
  • Kurt Lust (UAntwerpen / LUMI)
  • Davide Obbi (E4)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Bart Oldeman (McGill University, Digital Research Alliance of Canada)
  • Andrea Piserchia (E4)
  • Jan André Reuter (JSC, Germany)
  • Jörg Saßmannshausen (Imperial College London, UK)
  • Roberto Scipioni (Red Oak Consulting, UK)
  • Cintia Willemyns (Vrije Universiteit Brussel, Belgium)

Agenda

  • overview of recent developments
  • outlook to EasyBuild 5.0 release
  • 2025a update of common toolchains
  • Q&A

Recent developments

  • latest EasyBuild release:
  • next (stable) EasyBuild release:
    • EasyBuild v5.0.0 :fire:
    • additional EasyBuild 4.9.x versions could still be done via 4.9.x branches, but none are planned currently
  • EasyBuild v5.0.0
    • project board: https://github.com/orgs/easybuilders/projects/18/views/2
    • detailed notes on latest developments here
    • to test development version of EasyBuild 5.0: 
      # set up Python virtual environment, and jump into it
python3 -m venv eb5
source eb5/bin/activate

# install EasyBuild 5.0 development version into it
pip install https://github.com/easybuilders/easybuild-framework/archive/5.0.x.tar.gz
pip install https://github.com/easybuilders/easybuild-easyblocks/archive/5.0.x.tar.gz
pip install https://github.com/easybuilders/easybuild-easyconfigs/archive/5.0.x.tar.gz

# go!
eb --version

Merged PRs

  • docs (merged PRs)

    • ...

  • framework (merged PRs)

    • bug fixes
      • [develop] Avoid making build directory read-only (PR #4736)
      • [develop] show readable error message when applying patch without (extracted) source (PR #4738)
      • [5.0.x] Adapt module show command run to cope with non-zero exit code for non-existing module (required for Environment Modules v5.5+ and Lmod 8.7.56+) (PR #4739)
      • [5.0.x] enable non-blocking reads for streaming outputs (framework PR #4755)
      • [5.0.x] add BOTH_TO_DIR state to LibSymlink to fix case where both lib and lib64 are symlinks to other paths(framework PR #4761)
    • enhancements
      • [5.0.x] add TODO and warning about non-path variables not being currently handled in module load environment (PR #4748)
      • [5.0.x] add module-search-path-headers configuration option to control how modules set search paths to header files (PR #4655)
      • [5.0.x] Small improvements to ModuleLoadEnvironment (PR #4754)
    • changes
      • ...
    • code cleanup
      • [5.0.x] Remove superflous is_string method (PR #4756)

  • easyblocks (merged PRs)

  • easyconfigs (merged PRs)

    • ~20 easyconfig PRs were merged since last conf call
    • bug fixes/reports
      • move download of data files to source step for Casacore v3.5.0 (PR #22201)
      • add patch for WIEN2k 24.1 to fix bug in symmetry determination (PR #22234)
      • add include + lib under grass84 subdirectory to $CPATH, $LD_LIBRARY_PATH, $LIBRARY_PATH for GRASS 8.4.0 (PR #22271)
    • enhancements
      • ...
    • (noteworthy) new software
      • ...
    • noteworthy software updates
      • ...
    • cleanup
      • ...
    • changes
      • ...

Open (active) PRs

  • docs (open PRs + issues)

    • ...

  • framework (open PRs + issues)

    • bug fixes
      • [5.0.x] Add context manager for allowing unresolved templates and make the state members private (PR #4735)
      • incompatibility with Lmod 8.7.56 due to module show producing non-zero exit code for non-existent module (issue #4759)
      • Ignore unmodified files when using --new-pr or --update_pr (PR #4753)
      • trouble with huge build directories (issue #22247)
        • can be disabled via --disable-keep-debug-symbols
          • using that for Clang results in 10x smaller build directory
    • enhancements
      • [5.0.x] Copy build log and artifacts to a permanent location after failures (PR #4601)
        • mostly new code, so worth considering to include in EasyBuild v5.0.0 (but not a priority)
      • Problem using $CPATH in modulefiles overwriting system paths (issue #3331)
      • [develop] initial work towards integrating easy_update functionality (PR #4714)
      • enhance apply_regex_substitutions (PR #4758)
    • code cleanup
      • ...
    • changes
      • [5.0.x] Deprecate use of parallel easyconfig parameter and fix updating the template value (PR #4580)
        • (Roberto) in some situations the auto-detection done by EasyBuild on how many cores are available seems incorrect, only one core detected
      • [5.0.x] Let jobs retweak easyconfigs themselves (PR #4669)
      • With new clang based intel compilers (ifx, icx, icpx) we should use -march=native (issue #4744)

  • easyblocks (open PRs + issues)

    • bug fixes/reports
      • [develop] fix enhance_sanity_check for PythonPackage (PR #3477)
      • Fix building PyTorch when using setup.py as the build command (PR #3574, fixes issue #3570)
      • changes to Cargo easyblock break installation of polars (issue #3567)
      • [5.0.x] Fix $PYTHONPATH for hermetic python in TensorFlow builds with EB 5.x (PR #3568)
      • [develop] Set Cargo variables also for extensions (PR #3576)
      • [develop] Restructure RPATH and elf interpreter setting for Java, and add the LIBRARY_PATH to the additional RPATH and separate java awt libs (PR #3583)
    • enhancements
      • [develop] Explicitely mention that the PyTorch easyblock needs updating when failing for this reason (PR #3255)
      • [develop -> 5.0.x] enhance LLVM easyblock for compilation of clang/flang + other llvm-projects (PR #3373)
      • [develop] LAMMPS: generate version from the source files when using software-commit (PR #3485)
      • [5.0.x] Add build_target parameter to PythonPackage (PR #3575)
        • to help deal with issue #3570
        • unclear whether build_target would cover use case where a build command needs to be run for PyTorch while it has use_pip enabled (so build step is skipped)
      • [develop] add support for neoverse_n1 instances of Azure (cortex-a72) to LAMMPs easyblock (PR #3580)
      • [5.0.x] enhance FlexiBLAS easyblock to add support for AOCL-BLAS backend (PR #3589)
    • updates
      • ...
    • changes
      • Use context managers for templating changes in Bundle easyblock (PR #3547)
      • updates to easyblock required because make_module_req_guess is being deprecated:
        • split SystemCompilerGCC out of SystemCompiler and adopt module_load_environment (PR #3559)
        • honor --module-search-path-headers in easyblocks that hardcode CPATH (PR #3584)
        • fix module load environment in conda and Anaconda easyblock (PR #3585)
        • adopt module_load_environment in Clang easyblock (PR #3586)
        • adopt module_load_environment in CUDA easyblock (PR #3587)
      • update easyblocks to use EasyConfig.parallel property (PR #3557)
    • code cleanup
      • ...
    • new
      • ...

  • easyconfigs (open PRs + issues)

2025a common toolchains

  • (2024b is skipped to catch up with original schedule for defining common toolchain versions)
    • EasyBuild v5.0 is ideal excuse for that break in continuity...
  • GCC 14.2 as a base (see easyconfigs PR #21114)
  • we should define candidate toolchains using latest version of all components
  • easyconfigs using GCCcore/14.2.0 toolchain available for Python 3.13.1, Perl + Perl-bundle-CPAN 5.40.0
  • effort sort of on hold until EasyBuild v5.0.0 is released...
  • easyconfigs PR #22125 for FlexiBLAS, OpenBLAS, BLIS
    • (Bart) no AMD Zen4 (Genoa) or Zen5 (Turin), which support AVX-512, support yet in current BLIS releases (and even the upcoming BLIS 2.0)

Q&A / others

  • 8th EasyBuild User Survey, please fill it out! (closes 28 Feb'25)
  • (Roberto) eb --check-contrib is useful to check for code style issues + missing checksums (includes eb --check-style)
  • confusion on LLVM/Clang in HOOMD (easyconfig PR #22013)
  • check for not allowing multiple variants of dependencies can be improved
    • see also easyconfigs #12687
    • can be done without a hardcoded list
    • extend the check to only dependencies, but sticking to a single software version per easyconfigs generations, yet allow to make exceptions
    • run this check via a dedicated workflow that doesn't need to be run multiple times for different Lmod versions
      • could allow pushing comments to PR from workflow