Conference call notes 20200108 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 139th EasyBuild conference call, Wednesday Jan 8th 2020 (17:00 - 18:00 CET)

Attendees

Alphabetical list of attendees (5):

• Victor Holanda Rusu (CSCS, Switzerland)
• Kenneth Hoste (HPC-UGent, Belgium)
• Alan O'Cais (JSC, Germany)
• Mikael Öhman (Chalmers University of Technology, Sweden)
• Bart Oldeman (ComputeCanada)

Agenda

• outlook to next release (EasyBuild v4.1.1)
• component updates for 2020a common toolchains
• Q&A

Outlook to next release

• EasyBuild v4.1.1, release (hopefully) next week
• framework (https://github.com/easybuilders/easybuild-framework/milestone/74)
  • recent (noteworthy) changes in develop:
    • ensure correct exec/search permissions on subdirectories
    • only useful entries in CPATH, PATH, etc. (+ fix)
    • support for eb --show-ec
    • support for eb --copy-ec
    • fix permission-related crashes when installing software with multiple different users (1, 2)
  • TODO:
    • no more "urgent"/ready-to-merge PRs?
• easyblocks (https://github.com/easybuilders/easybuild-easyblocks/milestone/63)
  • recent (noteworthy) changes in develop:
    • build recent Python versions with --enable-optimizations (>= 3.5.4) & --with-lto (>= 3.7), don't use non-existing configure options
    • use major/minor version of Python command being used if req_py_majver/req_py_minver are not specified (fixes installation of EasyBuild when running eb on top of python3)
    • derive easyblock for iccifort only from icc easyblock (not ifort)
    • don't set $CPATH and $LIBRARY_PATH anymore for GCC/GCCcore (not needed)
    • make sure $PYTHONNOUSERSITE it set when performing sanity check for (bundles of) Python package(s)
    • updated SAMtools easyblock for v1.10
    • change Bazel easyblock to prefer using Java dependency rather than included JDK
    • fixes for Bundle easyblock (1, 2)
    • updates/fixes for WRF/WPS easyblocks (1, 2, 3, 4)
    • updated FSL easyblock for versions => 6.0.2 + RPATH-related fixes for FSL
  • TODO:
    • issues/PRs tagged for v4.1.1 need to be reconsidered again (not all will be fixed in time)
• easyconfigs
  • recent (noteworthy) changes in develop:
    • Clang 9.0.1, FSL 6.0.3, Julia 1.3.1, OpenMM 7.4.1, OpenFOAM 7, OpenFOAM v1912, Salmon 1.0.0, WRF 4.1.3, WPS 4.1
    • add easyconfigs for intel/2020.00 toolchain
      • iccifort 2020.0.166 + impi 2019.6.166 + imkl
    • checksum fixes for kallisto, MASS extension in R 3.6.0 easyconfigs
    • updated Java 1.8 wrapper to Java/1.8.0_231
    • fixed source URLS for xorg-macros (1, 2)
    • require setting sanity_pip_check in easyconfigs for Python packages (only with use_pip = True)
    • add missing wcwidth extension to Python 2.7.15 + 2.7.16 easyconfigs & enable 'pip check' in sanity check
  • TODO:
    • recent TensorFlow easyconfigs do not specify CUDA compute capabilities anymore
    • iccifort update5 does not respect components to install (important for 2020a common toolchains?)

Component updates for 2020a common toolchains

	2019b	current latest version	2020a
GCC	8.3.0	9.2 (Aug'19)	9.2 (or 9.3?)
binutils	2.32	2.33.1 (Oc't19)	2.33.1 (or stick to 2.32?)
OpenMPI	3.1.4	4.0.2 (Oct'19)	4.0.2 (or 4.0.3?)
OpenBLAS	0.3.7	0.3.7 (Aug'19)	0.3.7 (or newer)
ScaLAPACK	2.0.2	2.1.0 (Nov'19)	2.1.0
FFTW	3.3.8	3.3.8 (May'18)	3.3.8
iccifort	2019 update 5	2020 initial (Dec'19)	2019 update 5 or 2020 initial?
impi	2018.5.288	2019 update 6 (Dec'19)	2019 update 6?
imkl	2019.5.281	2020 initial (Dec'19)	2019 update 5 or 2020 initial?

• https://easybuild.readthedocs.io/en/latest/Common-toolchains.html

• new GCC 9.x release soon (9.3)?

  • could be interesting w.r.t. OpenMP 5, OpenACC 2.5, C++17 stable support, OpenACC offload support (to NVIDIA GPUs), ...
  • Victor has a contact in CSCS who can figure out timeline for GCC 9.3 release

• OpenMPI 4.0.3 expected by end of Jan'20 (https://github.com/open-mpi/ompi/milestone/52)

• latest OpenBLAS is already quite old, ping for an update?

  • 0.3.8 milestone @ https://github.com/xianyi/OpenBLAS/milestone/27

• (still) stick to latest 2019 versions for intel/2020a?

  • Bart: iccifort/imkl 2020 initial release is not really a new major version, more like a minor update of 2019 update 5
  • problem with ignored components spec in iccifort 2019.5.281 doesn't occur anymore in 2020.0.166 version

• does latest Intel versions actually support GCC 9.x => reach out to Intel support?

• re-consider OpenMPI (MPICH2?) & OpenBLAS (BLIS?) for future common toolchains => to be discussed at EUM

Q&A

• Victor: organized way of testing performance for software with different toolchains?
  • ReFrame could be a driver for that
  • see https://github.com/eth-cscs/reframe/tree/master/cscs-checks/apps
    • see for example https://github.com/eth-cscs/reframe/blob/master/cscs-checks/apps/python/numpy_check.py.
  • interesting for:
    • including performance checks in EasyBuild regression tests
    • performance test reports for contributions (eb --from-pr --upload-perf-test-report)
    • evaluating new version of common toolchains
    • performance impact of swapping toolchain components (BLIS instead of OpenBLAS, for example)
    • post-maintenance testing of systems
  • open questions:
    • where to host repository with tests contributed by EasyBuild community?
      • in easybuild-checks subdirectory of ReFrame repository?
        • Kenneth: not sure if this is ideal w.r.t. keeping control & not relying on ReFrame developers too much (w.r.t. review/merging of PRs)
      • new repository under easybuilders organisation?