Conference call notes 20210707 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 176th EasyBuild conference call, Wednesday July 7th 2021 (08:00 UTC)

Attendees

Alphabetical list of attendees (9):

• Sebastian Achilles (Jülich Supercomputing Centre, Germany)
• Erica Bianco (HPCNow!)
• Simon Branford (Univ. of Birmingham, UK)
• Alex Domingo (Vrije Universiteit Brussel, Belgium)
• Jorge Guerra (Universidad Politécnica de Madrid, Spain)
• Alexander Grund (TU Dresden, Germany)
• Victor Holanda (CSCS, Switzerland)
• Robert Mijakovic (LuxProvide)
• Jörg Saßmannshausen (NIHR Biomedical Research Centre, UK)

Agenda

• EasyBuild 4.4.1
• overview of recent developments
• Q&A

Recent developments

• last release: EasyBuild v4.4.1 (July 6)
  • Release Notes: https://docs.easybuild.io/en/latest/Release_notes.html
• ETA next release: TBD
• recent changes
  • framework
    • bug fixes
      • avoid checking msg attribute of GitCommandError (PR #3756)
      • Consider sources provided as dict in EasyBlock.check_checksums_for (PR #3758)
      • minor fixes to output of test reports when using multiple PRs (PR #3741)
      • don't make changes to software installation directory when using --sanity-check-only (PR #3761)
      • honor specified easyblock for extensions (PR #3762)
      • Error out when passing a list to run_cmd* to avoid running wrong command unintended, unless shell=True is used (PR #3737)
      • minor fixes to output of test reports when using multiple PRs (PR #3741)
    • enhancements
      • add toolchain nvompic (PR #3735)
      • warn about generic milestone in --review-pr and --merge-pr (PR #3751)
      • Add option to ignore failing test step (PR #3732)
    • changes
      • (none)
  • easyblocks
    • bug fixes
      • always add distinct_host_configuration=false to build command for TensorFlow (PR #2459)
      • Don't overwrite all of exts_default_options in TensorFlow easyblock (PR #2494)
      • always strip output from "gcc -print-multiarch" in intel-compilers easyblock (PR #2489)
      • enhance GCC easyblock to make sure that system GCC provides LTO support, and disable LTO when building MPFR if not (PR #2498)
      • rework the dependency handling of OpenMPI to use explicit configure options to disable features if required dependency is not provided (PR #2500)
      • correctly check whether --with-ucx is used as OpenMPI configure option (taking into account --with-ucx=no) (PR #2501)
      • Fix CPU-only runtime for dpcpp-generated executables in custom easyblock for intel-compilers (PR #2457)
      • Enhance sitecustomize.py script installed by Python easyblock to fix support for virtualenv (PR #2483)
      • change Bundle easyblock to also collect altroot and altversion in the module step so they are set when running --module-only (PR #2485)
      • don't use list value for command to detect installed Python packages in TensorFlow easyblock (PR #2469)
    • enhancements
      • enhance FFTW easyblock to skip tests if --mpi-tests EasyBuild configuration option is disabled (PR #2490)
      • Boost:
        • update CMakeMake to handle old and new Boost/Boost.Python builds using custom easyblock for Boost (PR #2461)
        • update custom easyblock for Boost to always build single and multi threaded versions (PR #2456)
      • Use PYPI_SOURCE as the default for source_urls of extensions of Python easyconfigs (PR #2493)
      • enhance Hypre easyblock to enable CUDA support (PR #2482)
    • new easyblocks
      • add custom easyblock for building UCX plugins (PR #2491)
    • changes
      • (none)
  • easyconfigs
    • over 100 easyconfig PRs merged since last conf call!
    • bug fixes
      • add patch for GCCcore 10.3.0 to prevent issues when compiling with nvcc (PR #13310)
    • enhancements
      • UCX-CUDA (PR #13260)
    • new software
      • freud-analysis (PR #13354), Vampire (PR #13352), LADR (PR #13351), DosageConvertor (PR #13278), OpenMM-PLUMED (PR #13268), RagTag (PR #13247), phototonic (PR #13241), HAL (PR #13267)
    • noteworthy software updates
      • (nothing major)
    • noteworthy changes
      • (nothing major) pull/13188))
  • documentation
    • (none)
• to merge/fix/tackle soon
  • framework
    • reported bugs / bug fixes
      • (nothing major?)
    • enhancements
      • support additional features in easystack files
        • support for filtering via labels (PR #3620)
      • avoid using a priority in prepend_module_path (Lmod) to avoid costly module calls (PR #3636)
      • add support for installing extensions in parallel (WIP) (PR #3667)
      • finding modules with multiple modulepaths and HMNS (issue #3703)
      • add support for NVHPC-based toolchains (PR #3736)
      • add --review-pr-max and --review-pr-filter options to limit easyconfigs shown in multi-diff (WIP) (PR #3754)
    • changes
      • also enable static analysis for Python 2.7 (PR #3725)
        • catches accidentally overwriting local variables in list comprehensions (like --from-pr --include-easyblocks-from-pr bug)
      • add per-extension timing (PR #3734)
      • raise error for patches without .patch extension and no destination given (WIP) PR #3738)
  • easyblocks
    • bug fixes
      • treat files/directories of unpacked sources equally in PackedBinary (PR #2306)
      • --module-only doesn't always work as expected
        • we need a better way of catching this in tests
        • problem is that you typically need an actual installation to catch these problems, so can't be done in easyconfigs or easyblocks test suite run in CI
        • test installations done on generoso via boegelbot could be enhanced to catch problems with --module-only?
      • explicitly use only OpenBLAS for PyTorch if MKL is not used (PR #2448)
      • fixed sanity check for later versions of molpro (PR #2499)
      • Handle failure of running nvidia-smi in TensorFlow tests (PR #2506)
    • enhancements
      • enhance test and install step of CMakePythonPackage easyblock (PR #2318)
      • add support for installing R extensions in parallel (WIP) (PR #2408)
        • requires https://github.com/easybuilders/easybuild-framework/pull/3667
      • allow for Perl modules being part of other, already installed Perl modules (PR #2386)
      • including FlexiBLAS as the default BLAS in foss will require easyblock changes (issue #2421)
      • should set BLA_VENDOR in CMakeMake easyblock if BLAS is in the toolchain (PR #2420)
      • enable make check and sanity check exec for OpenMPI (PR #2444)
      • enhance Java easyblock to define %(jdkarch)s template (PR #2472)
    • changes
      • (nothing major)
    • new software
      • custom easyblocks for RepeatModeler (PR #2470), crispr-dav (PR #2487), ORCA (PR #2504)
  • easyconfigs
    • bug fixes
      • (nothing major)
    • enhancements
      • (nothing major)
    • new software
      • (nothing major)
    • software updates
      • SciPy-bundle with intel/2021a (PR #12964)
        • need to look into handful of failing tests...

2021a update of common toolchains

• TODO: fosscuda/2021a foss/2021a + CUDA
  • collapsing foss and fosscuda toolchains
  • see https://github.com/easybuilders/easybuild-easyconfigs/issues/12484
    • UCX Plugins PR #2491)
    • UCX-CUDA easyconfigs PR #13260)
    • gcc (10.3) / nvcc fix (easyconfigs PR #13310)
  • OSU-Micro-Benchmarks v5.7.1, NCCL v2.9.9, magma v2.6.0 (easyconfigs PR #13282)

Q&A

• Robert: PyQt5 sources not available
  • Alex: LTS - available; other versions - not available
    • This is the download page for supported versions of Qt5: https://download.qt.io/archive/qt/
    • And this is the one for the rest of versions: https://download.qt.io/new_archive/qt/
• Jörg: Older easyconfig PRs
  • Simon: trying to deal with these
  • Where there is a new version - replace old PR with new PR for the new version
• Alex: UCX-CUDA and GDRCopy
  • UCX-CUDA depends on GDRCopy