Conference call notes 20200527 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 148th EasyBuild conference call, Wednesday May 27th 2020 (15:00 UTC - 17:00 CEST)

Attendees

Alphabetical list of attendees (13):

• Simon Branford (University of Birmingham, UK)
• Fotis Georgatos (SDSC, Switzerland)
• Kenneth Hoste (HPC-UGent, Belgium)
• Adam Huffman (Big Data Institute, Oxford, UK)
• Terje Kvernes (University of Oslo, Norway)
• Alan O'Cais (JSC, Germany)
• Mikael Öhman (Chalmers University of Technology, Sweden)
• Bart Oldeman (ComputeCanada)
• Victor Holanda Rusu (CSCS, Switzerland)
• Åke Sandgren (Umeå University, Sweden)
• Jörg Saßmannshausen (NIHR Biomedical Research Centre, UK)
• Alexandre Strube (JSC, Germany)
• Lars Viklund (Umeå University, Sweden)

Agenda

• recent developments
• EasyBuild tutorial
• policy on Conda (package) contributions (Mikael) e.g. https://github.com/easybuilders/easybuild-easyconfigs/pull/9778
• OpenMPI/UCX/PMIx presentation on Wed June 24th (5pm CEST)
• Q&A

Recent developments

• EasyBuild v4.2.1 was released last week (May 27th)

  • merged shortly before release:
    • framework
      • fix for Java wrappers in HMNS (PR #3337)
      • style cleanup in easybuild.tools (PR #3282)
      • support for copying tweaked easyconfig with --copy-ec (PR #3332)
    • easyblocks
      • let CMakeMake remove 'easybuild_obj' build directory if it already exists (PR #2032)
      • rewrite GROMACS easyblock to build all four variations in the same build (PR #1991)
      • update config.guess for all R packages (PR #1949)
      • fix for --module-only for GCC (PR #2059)
    • easyconfigs
      • (nothing shocking)

• recent changes

  • framework
    • fix for broken Singularity container recipes (PR #3347)
    • develop was broken for a couple of hours due to changes in PR #3344, fixed in PR #3345
      • slipped through due to incomplete test coverage
  • easyblocks
    • new easyblock for LLVM (PR #2065)
  • easyconfigs
    • avoid MPI_Init on 'import h5py' (PR #10687)

EasyBuild tutorial

• date set: Tue June 23rd 2020 (afternoon)
• half-day tutorial (remote)
  • pretty much what was accepted for ISC'20 (postponed to ISC'21)
• tutorial is being prepared at https://github.com/boegel/easybuild-tutorial
  • preview at https://boegel.github.io/easybuild-tutorial
    • current state: only introduction part is really done
  • plan is to move this to easybuilders GitHub org later
  • also integrate it with EasyBuild documentation
• incl. hands-on exercises
  • installation & configuration of EasyBuild
  • basic usage of eb CLI
  • writing easyconfigs for prepared toy software packages:
    • library (ConfigureMake)
    • C++ app (CMakeMake), which depends on the library
    • Python package (stand-alone + as extension)
• controlled tutorial environment:
  • Docker (& Singularity?) container with CentOS 7 + foss/2020a + Python 3 software stack pre-installed
  • VMs on AWS for tutorial attendees (just to run Docker container)

Policy on Conda package contributions

• question by Mikael
• example PR: https://github.com/easybuilders/easybuild-easyconfigs/pull/9778
• several reasons to not merge PRs like these:
  • can't guarantee that these easyconfigs will keep working
  • generically optimized binaries :(
  • but how much effort is required to do this "properly"?
• Terje: why do people even want to use EasyBuild to install conda packages?

OpenMPI/UCX/PMIx presentation

• date set: Wed June 24th 2020 (5pm CEST)
  • replaces EasyBuild conf call
• remote presentation (via WebEx), recorded (will be uploaded to YouTube)
• presenters: Jeff Squyres (Cisco, Open MPI) & Ralph Castain (PMIx)
• topics: Open MPI (MCA, MTL, ...) & related libraries (UCX, PMI, PMIx, PSM, PSM2, verbs, libfabric/OFI), integration with Slurm, CUDA, etc.
  • suggestions for specific topics welcome via https://github.com/easybuilders/easybuild/issues/630

Q&A

• Alexandre: fosscuda/2020a?
  • waiting for new CUDA release that supports GCC 9.3
• Mikael: let bot submit output of eb --review-pr into PR itself?
  • amount of easyconfigs we compare with should be limited
  • also add labels automatically by bot for unlabeled PRs
• Mikael: Mesa is/was not compatible with GPU drivers, what's the current situation (does vglrun work)?
  • Åke: recent versions should be better w.r.t. vendor neutral dispatch (glvnd)?
  • Terje was GPU login nodes, but Mesa isn't used a lot there
    • already running CentOS 8/RHEL 8, which may help with this
  • Mikael: used to be true to filter out Mesa/libGLU to actually get things working (and build against system Nvidia libs instead)
    • affected ParaView/VTK for sure
  • Alan has asked JSC colleagues to open PR to get proper OpenGL support in EB
    • currently is a huge easyconfig, need some work to put into easyblock
    • see https://github.com/easybuilders/easybuild-easyconfigs/issues/10701
• Mikael: time to add more maintainers?
• Victor: Open MPI talk will be public?
• Jörg: more info on Singularity changes?
  • Kenneth: only easyconfigs for old Singularity versions were removed
  • support for building Ubuntu containers?
    • only yum-based for now, see https://github.com/easybuilders/easybuild-framework/issues/2923
• Mikael: time to get rid of Travis?
  • currently only for:
    • running tests with Python 2.6
    • comment made by @boegelbot for adding comment with info on failing tests
  • so, disable requirement for tests to pass in Travis CI, which is good enough short term