Conference call notes 20240313 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 242nd EasyBuild conference call, Wednesday 13 Mar 2024 (09:00 UTC)

Attendees

List of attendees (18):

Raj Ayyampalayam
Alex Domingo (Vrije Universiteit Brussel, Belgium)
Em Dragowsky (Case Western Reserve University, Ohio, US)
Davide Grassano (CECAM, Switzerland)
Jasper Grimm (University of York, UK)
Alexander Grund (TU Dresden, Germany)
Kenneth Hoste (HPC-UGent, Belgium)
Adam Huffman (Big Data Institute, Oxford, UK)
Ian Kirker (University College London, UK)
Sam Moors (Vrije Universiteit Brussel, Belgium)
Sébastien Moretti (SIB, Switzerland)
Mikael Öhman (Chalmers University of Technology, Sweden)
Bart Oldeman (Digital Research Alliance of Canada)
Jörg Saßmannshausen (Imperial College London, UK)
Åke Sandgren (Umeå University, Sweden)
Cintia Willemyns (Vrije Universiteit Brussel, Belgium)
Iker
Lars Viklund (Umeå University, Sweden)

Agenda

overview of recent developments
Q&A

Recent developments

latest EasyBuild release: 4.9.0 (30 Dec 2023)
- see release notes + highlights
next EasyBuild release: EasyBuild v4.9.1 (ASAP)

ETA for EasyBuild 5.0 release: in the coming weeks...

started doing short sprint meetings, each Monday at 15:00 CE(S)T to set next 5 goals to tackle that week
monthly sync meetings every 1st Wed of the month at 14:00 CE(S)T: next today Wed 7 Feb'24 at 14:00 CET
see also https://hackmd.io/tZtz6vMmRfGL6N9Rf__OEA

to test development version of EasyBuild 5.0:

# set up Python virtual environment, and jump into it
python3 -m venv eb5
source eb5/bin/activate

# install EasyBuild 5.0 development version into it
pip install https://github.com/easybuilders/easybuild-framework/archive/5.0.x.tar.gz
pip install https://github.com/easybuilders/easybuild-easyblocks/archive/5.0.x.tar.gz
pip install https://github.com/easybuilders/easybuild-easyconfigs/archive/5.0.x.tar.gz

# go!
eb --version

recent changes
- docs (merged PRs)
  - split out default config changes and remove proposed changes for EasyBuild v5.0 (PR #242)
- framework (merged PRs)
  - bug fixes
    - fix test_github_merge_pr by using more recent easyconfigs PR (PR #4470)
    - add workaround for 404 error when installing packages in CI workflow for testing Apptainer integration (PR #4472)
  - enhancements
    - Improve logging & handling of (empty) --optarch values (PR #4481)
  - changes
    - ...
  - code cleanup
    - ...
  - EasyBuild 5.0 (to 5.0.x branch)
    - bug fixes
      - update easyblocks for toy extensions to make sure that asynchronous installation command is run in correct working directory (PR #4471)
      - fix easyconfig parameter deprecation (PR #4479)
      - clean up parameter deprecation warnings, and update easybuild doc urls (PR #4480)
    - changes
      - rename unclear *run* methods to *install_extension* + rename install_extensions to install_all_extensions (PR #4400)
      - eliminate distutils.util.strtobool (PR #4477)
      - eliminate use of distutils.version.StrictVersion, use LooseVersion instead (PR #4476)
        
        see also: stop relying on distutils (will no longer be available in Python 3.12) (issue #3963)
    - enhancements
      - replace log.warn with log.warning (PR #4473)
- easyblocks (merged PRs)
  - bug fixes
    - only run torchvision w/cuda sanity check command test if GPU is present (PR #3236)
    - Remove wrong configure option for LibInt 2.6.x (PR #3249)
  - enhancements
    - use proxy in sanity check of EB_OpenSSL_wrapper (PR #2922)
  - updates
    - fix OCaml easyblock for recent versions (PR #3235)
  - changes
    - ...
  - new easyblocks
    - ...
  - EasyBuild 5.0 (to 5.0.x branch)
    - bug fixes:
      - update RPackage to make sure that asynchronous installation command is run in correct working directory (PR #3242)
    - changes:
      - rename run method to install_extension, and likewise for prerun to pre_install_extension, postrun to post_install_extension, and run_async to install_extension_async (PR #3064)
      - replace run_cmd with run_shell_cmd in various custom easyblock for LAMMPS (dozens of PRs)
      - remove unused custom easyblock for libsmm (PR #3216), NEMO (PR #3202)
    - enhancements:
      - replace log.warn with log.warning (PR #3245)
- easyconfigs (merged PRs)
  - ~75 easyconfig PRs were merged since last conf call
  - bug fixes/reports
    - add patch to fix failing tests with OpenSSL 3.x for Net-SSLeay v1.92 extension in Perl-bundle-CPAN v5.36.1 #20090
  - enhancements
    - ...
  - (noteworthy) new software
    - ...
  - noteworthy software updates
    - ...
  - changes
    - ...
  - EasyBuild 5.0 (to 5.0.x branch)
    - migrate archive of easyconfigs to easybuild-easyconfigs-archive (PR #20006)
      - see https://github.com/easybuilders/easybuild-easyconfigs-archive
- docs (open PRs + issues)
  - project board to perform yearly review cycle of all documentation pages: https://github.com/orgs/easybuilders/projects/17/views/1
  - document download_instructions enhancements in EasyBuild 5.0 (PR #243)
- framework (open PRs + issues)
  - reported bugs / bug fixes
    - ...
  - enhancements
    - add support for data installations (PR #4474)
    - add support for --from-commit and --include-easyblocks-from-commit (WIP PR #4468)
  - changes
    - ...
  - EasyBuild 5.0 (to 5.0.x branch)
    - active PRs
      - change tar command used in get_source_tarball_from_git to get reproducible tarballs (PR #4248)
        
        awaiting feedback from contributor on Alex' suggestions...
      - implement support for running interactive commands with run_shell_cmd (WIP PR #4453)
        
        TODO: support for qa_wait_patterns (WIP) + some more tests + testing with actual easyblock that uses interactive commands
      - rename easyconfig parameters (PR #4483)
        
        see also: refactor easyconfig parameters and template constants to use a consistent naming style (issue #4464)
      - add support for disabling strict RPATH sanity check (PR #4475)
        
        to deal with mixing of RPATH and non-RPATH installations
        
        needs more work (incorrect regex)
      - Remove distutils use from setup.py (PR #4478)
    - dormant issues/PRs (for now)
      - deprecate failure to resolve a template value (PR #3285)
      - drop load storm safe guard for Environment Modules v4.2.4+ (PR #4373)
        
        we should also enable depends_on by default when using Lmod as modules tool in EasyBuild 5.0
      - replace parallel easyconfig parameter by maxparallel (PR #4398)
      - bump minimum required Tmod (4.x) to 4.3.0 (PR #4425)
        
        requires that PR #4415 is merged first
      - deprecate run_cmd and run_cmd_qa & co, move them to easybuild._deprecated module (WIP) (PR #4433)
        
        requires that implementation of run_shell_cmd is complete, incl. support for asynchronously running commands and running interactive shell commands (WIP)
      - support uninstalling software + module (issue #590)
      - clean up SLURM/Batchsystem environment before doing builds (issue #4434)
        
        unset all $SLURM_* environment variables to avoid trouble with srun/MPI
        
        should be done opt-in (--clean-slurm-env configuration option)
- easyblocks (open PRs + issues)
  - bug reports/fixes
    - remove trailing slash in SLEPC_DIR (PR #3229)
    - fix OpenFOAM easyblock to correctly have is_dot_org as True when v >=11 and fix motorBike example for v >=11 (PR #3232)
    - ensure localrc is created in the correct subdir for NVHPC v22.9+ (PR #3240)
    - Feature improve QuantumESPRESSO easyblock (PR #3241)
      - see also issue #3234
    - Don't rename GCCs include-fixed directory by default and add script to regenerate it (PR #3254)
      - see also isuse #3253
    - RPATH sanity check fails for OCaml (issue #3247)
  - enhancements
    - Support GROMACS builds with SYCL (AdaptiveCpp implementation) (PR #3077)
    - Score-P easyblock: Select the oneAPI compiler suite when enabled in the Intel compiler toolchain (PR #3228)
    - enhance CMakeMake easyblock to check whether correct Python installation was picked up by cmake (PR #3233)
    - Make OpenBLAS easyblock aware to --optarch=GENERIC (issue #3230)
    - make a project environment and manage LOAD_PATH for JuliaPackage (PR #3239)
    - allow overriding testinstall in PyTorch easyconfigs + use --verbose option when using pip (PR #3250)
    - Explicitely mention that the PyTorch easyblock needs updating when failing for this reason (PR #3255)
  - updates
    - Update namd.py to run successfully NAMD-3.0b6-foss-2023a-mpi.eb (PR #3248)
  - new easyblocks
    - add generic Dataset easyblock (PR #3246)
  - changes
    - Always set $EBPYTHONPREFIXES instead of $PYTHONPATH (issue #2887)
  - EasyBuild 5.0 (to 5.0.x branch)
    - fix extension filter for Perl packages (PR #2699)
    - update ConfigureMake easyblock to error out on unknown configure args (PR #3025 + #3026)
    - set CMake installation LIBDIR to lib by default in CMakeMake easyblock (PR #3227)
- easyconfigs (open PRs + issues)
  - bug fixes/reports
    - RPATH sanity check fails for LAMMPS when (not all) dependencies are installed with RPATH (issue #20051)
    - Quantum espresso 7.2 gives segmentation faults if compiled with OpenMP (issue #20077)
    - add wrapper for Julia with linking safeguards and delegate environment setup to JuliaPackage (PR #20103)
    - strip Suprressor out of ResistanceGA (PR #20106)
    - add multiarch support for RISC-V to all GCCcore 12.x and 13.x easyconfigs (PR #20035)
  - enhancements
    - Correct configuration of QuantumESPRESSO for 6.7/7.2 (PR #20070)
  - new software
    - ...
  - noteworthy software updates
    - OpenSSL/3 (PR #19623)
    - Qt5 v5.15.11 w/ GCCcore/13.2.0 (for foss/2023b) (WIP PR #19320)
    - jax v0.4.24 w/ foss/2023a + CUDA 12.1.1 (PR #19841)
    - CUDA v12.4.0 + GCCcore/13.2.0 (PR #20107)
    - Added QuantumESPRESSO easyconfigs for 7.3 (both intel and foss) (PR #20105)
  - changes
    - unify Z3 easyconfigs into a single one with Python bindings (PR #20050)
  - EasyBuild 5.0 (to 5.0.x branch)
    - replace parallel by maxparallel (PR #19375)
    - detect easyconfigs where missing information about downloading sources (PR #19881)
  - closed PRs
    - ...

Q&A / others

EasyBuild User Meeting 2024 - agenda
- https://easybuild.io/eum
- suggestions for talk welcome!
- draft agenda is being puzzled together behind the scenes
- suggestions/ideas for talks still welcome!
Does the generoso test cluster have a GPU?
- No :)
Any reason PR for OpenMPI 5.0.2 isn't merged (PR #17561)
- no, can be merged if test bots are happy with it