Conference call notes 20230705 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 224th EasyBuild conference call, Wednesday 5 July 2023 (15:00 UTC)

Attendees

List of attendees (15):

• Sebastian Achilles (JSC, Germany)
• Simon Branford (Univ. of Birmingham, UK)
• John Dey (Fred Hutch, Seattle, US
• Alex Domingo (Vrije Universiteit Brussel, Belgium)
• Jasper Grimm (University of York, UK)
• Thomas Hayward-Schneider (Max Planck Institute for Plasma Physics (IPP))
• Adam Huffman (Big Data Institute, Oxford, UK)
• Kenneth Hoste (HPC-UGent, Belgium)
• Kurt Lust (UAntwerpen, Belgium + LUMI User Support Team)
• Sebastien Moretti (SIB, Switzerland)
• Mikael Öhman (Chalmers University of Technology, Sweden)
• Bart Oldeman (Digital Research Alliance of Canada)
• Jörg Saßmannshausen (Imperial College London, UK)
• Alexandre Strube (JSC, Germany
• Davide Vanzo (Microsoft)

Agenda

• overview of recent developments + outlook to next release
• progress on 2023a update of common toolchains
• Q&A

Recent developments

• easyconfigs merge sprint: 21 + 28 June 2023
  • was successful, managed to bring down number open easyconfig PRs from ~850 to ~700
    • ~90 PRs closed
    • ~180 PRs merged
    • => ~270 PRs processed
  • we should probably have an easyconfigs merge sprint on a regular basis (monthly? every other month?)
• request for feedback (in context of EasyBuild 5.0 effort)
  • minimum supported Lmod version (issue #871)
    • we will probably start requiring a recent Lmod 8.x version (8.3+) in EasyBuild 5.0 - more feedback still welcome!
    • Debian stable (12.x) has Lmod 8.x packaged
  • toolchain support policy (issue #872)
    • common ground found on this, being carved in stone in docs PR #200
      • 6 most recent (common) toolchain versions supported (easyconfigs actively maintained + accepting PRs)
      • next 2 toolchain versions deprecated (PRs are closed, but still included in EasyBuild releases)
      • easyconfigs using older toolchain versions are archived (=> still there for reference, not supported)
      • sliding window as new common toolchain versions are defined (twice a year)
• release timeline
  • latest release: EasyBuild v4.7.2 (28 May 2023)
  • ETA next release (EasyBuild v4.7.3?): end of June'23
• progress on EasyBuild 5.0
  • see updates in https://github.com/easybuilders/easybuild/wiki/EasyBuild-5.0-working-group
• recent changes
  • docs (merged PRs)
    • ...
  • framework (merged PRs)
    • bug fixes
      • fix regex for extracting glibc version from output of 'ldd --version' in Gentoo Linux (PR #4290)
      • allow versionsuffix to be set to None in det_full_ec_version (PR #4292, fixes issue #4281)
    • enhancements
      • use version suffixes in PR title instead of Python versions when using --new-pr (PR #4253 + PR #4286)
    • changes
      • ...
    • EasyBuild 5.0 (to 5.0.x branch)
      • enable --trace by default (PR #4250)
      • update check_python_version for removed Python 2 support (PR #4270)
  • easyblocks (merged PRs)
    • bug fixes
      • reset modules loaded by PythonPackage to let ExtensionEasyBlock handle multi_deps correctly (PR #2951)
    • enhancements
      • add support for install_cmds in Binary easyblock (PR #2953)
    • updates
      • ...
    • changes
      • ...
    • new easyblocks
      • add PerlBundle generic easyblock for installing a bundle of Perl modules (PR #2945)
      • add custom easyblock for Rust (PR #2952)
    • EasyBuild 5.0 (to 5.0.x branch)
      • ...
  • easyconfigs (merged PRs)
    • over 250 easyconfig PRs were merged since last conf call!
    • bug fixes
      • add patches to fix TensorFlow 2.7.1 on POWER (PR #16795)
      • add pkgconfig as build dependency for nodejs 18.12.1, required for finding ICU dependency (PR #18007)
      • add missing pkgconf build dependency for XML-LibXML v2.0208 (PR #18159)
      • update homepage for NWChem 7.x (PR #18184)
      • update homepage for SUNDIALS 5.x and 6.x (PR #18186)
      • add missing source_urls for SimpleITK 2.1.1.2 (PR #18212)
      • patch oxDNA/oxpy to use correct pip install prefix (PR #18214)
      • update source_urls for Velvet 1.2.10 (PR #18237)
      • use patch files for Qt5 v5.17.7 to fix compatiblity with glibc 2.34 (PR #18230)
      • fix incorrect version for GMAP-GSNAP 2021-12-17 (was 2021-21-17) (PR #18231)
      • fix CUDA build of recent TensorFlow easyconfigs when using compiler symlinks (PR #18235)
      • use upstream patch for symbol compatibility in XZ 5.2.5 (PR #18250)
    • enhancements
      • add CI check for -DCMAKE_BUILD_TYPE, should use build_type instead (PR #14008)
      • enhance sanity check for recent LibTIFF easyconfigs (PR #17975)
      • update JupyterLab v3.5.0 to set $JUPYTERLAB_SETTINGS_DIR and $JUPYTERLAB_WORKSPACES_DIR on loading of module (PR #17982)
      • update easyconfig for Nextflow 23.04.2 to use install_cmds (PR #18173)
    • (noteworthy) new software
      • see also https://github.com/easybuilders/easybuild-easyconfigs/pulls?q=is%3Apr+is%3Amerged+label%3Anew
      • Perl-bundle-CPAN v5.36.1 (PR #17970)
    • noteworthy software updates
      • see also https://github.com/easybuilders/easybuild-easyconfigs/pulls?q=is%3Apr+is%3Amerged+label%3Aupdate+
      • Rust 1.70.0 w/ GCCcore/12.3.0 (PR #18016 + PR #18167)
    • changes
      • allow multiply dependency variants of ncbi-vdb for SRA-Toolkit per easyconfigs generation, as long as versions match (PR #18098)
      • use custom easyblock for Rust 1.65.0 (PR #18174)
      • bump OpenSSL 1.1 wrapper fallback to 1.1.1u (PR #18188)
      • use patch to make ncbi-vdb 2.11.2 compatible with HDF5 1.12.2 (PR #18236)
• work in progress
  • docs (open PRs + issues)
    • project board to perform yearly review cycle of all documentation pages: https://github.com/orgs/easybuilders/projects/17/views/1
    • ...
  • framework (open PRs + issues)
    • reported bugs / bug fixes
      • re-initializing easyblocks modifies easyconfig input parameter (issue #4289)
      • nested temporary directories for easystack items (issue #4291)
    • enhancements
      • suggest default title in --review-pr (PR #4287)
    • changes
      • ...
    • EasyBuild 5.0 (to 5.0.x branch)
      • initial implementation of run function (WIP) (PR #4284, see also issue #4252)
      • small PRs to clean up deprecated stuff:
        • remove support for 32-bit targets (PR #4272)
        • remove deprecated desc option for simple_option (PR #4273)
        • remove deprecated Toolchain.add_dependencies (PR #4274)
        • remove deprecated functionality from filetools (PR #4275)
        • remove deprecated log_error option from which command (PR #4276)
        • remove deprecated skip_lower option from template_constant_dict (PR #4277)
        • removed deprecated mod_exists_regex_template to exist (PR #4279)
        • remove deprecated options from easyconfig.py (PR #4278)
  • easyblocks (open PRs + issues)
    • bug reports/fixes
      • fixes issue in sympy test step when tmpdir is a symlink (PR #2949)
        • awaiting test report to confirm the fix + accompanying PR for sympy easyconfigs to make them use the new custom easyblock
      • disable disk space check in STAR-CCM+ installer (PR #2956)
    • enhancements
      • improve cpu-arch optimization settings for OpenCV (PR #2954)
    • updates
      • update MXNet easyblock + don't try to install R extension by default (PR #2955)
    • new easyblocks
      • ...
    • changes
      • drop unnecessary CUDA stub libraries from $LIBRARY_PATH (PR #2793)
  • easyconfigs (open PRs + issues)
    • still over 800 open easyconfig PRs...
    • bug fixes/reports
      • ...
    • enhancements
      • improve check for multi-variant dependencies per generation of easyconfigs (PR #12687)
    • new software
      • see https://github.com/easybuilders/easybuild-easyconfigs/labels/new
    • noteworthy software updates
      • PyTorch 1.13.1 (PR #17155 + PR #17156)
        • Are these good to go?
          • Probably just missing some patches that were added recently in existing PyTorch easyconfigs
        • Is anyone working on PyTorch 2.x?
          • Simon has briefly looked into this
          • CPU version is pretty much a copy of PyTorch 1.13.1
          • GPU distributed tests were failing, related to NCCL
            • BEAR is currently falling back to binary wheel for PyTorch 2.0
    • changes
      • ...

2023a update of common toolchains

• foss/2023a and intel/2023a toochains should be included in EasyBuild v4.7.3 release
  • foss/2023.05 can be promoted in-place to foss/2023a (PR #18264)
    • GCC 12.3.0 (latest, except for GCC 13.1.0)
    • OpenMPI 4.1.5 (latest)
    • OpenBLAS 0.3.23 + FlexiBLAS 3.3.1 (latest)
    • FFTW 3.3.10 (latest)
    • ScaLAPACK 2.2.0 (latest)
  • intel-compilers 2023 update 1 is still altest release, so intel/2023.03 can be copied to intel/2023a only after GCCcore base compiler is changed to 12.3.0 first (PR #18263)
    • GCCcore/12.3.0 as base
    • Intel oneAPI compilers 2023.1.0 (latest)
    • Intel MPI 2021.9.0 (latest)
    • Intel MKL 2023.1.0 (latest)
• see PRs marked with 2023a label
• open PRs
  • bundles for Python extensions (PR #18208)
    • cryptography should be in a stand-alone easyconfig to have full control over installation using CargoPythonPackage?
    • will most likely not be included yet in EasyBuild v4.7.3 release
• CUDA version for 2023a?
  • CUDA 12.1.x should be compatible with GCC 12.x, so OK
  • support for sm35 (K40) + sm37 (K80) were dropped (since CUDA 11.7)
  • easyconfig for CUDA 12.1.1 merged (PR #18092)

Q&A

• ...