Conference call notes 20171206 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 91st EasyBuild conference call, Wednesday December 6th 2017 (5pm - 6pm CEST)

Attendees

Alphabetical list of attendees (8):

• Damian Alvarez (JSC, Germany) + guest Aurelio Rodriguez (CESGA)
• Maxime Boissonneault (McGill University, Canada)
• Fotis Georgatos (Illumina, UK)
• Kenneth Hoste (HPC-UGent, Belgium)
• Adam Huffman (Big Data Institute, University of Oxford)
• Teo Ioannis (CSCS, Switserland)
• Bart Oldeman McGill University, Canada
• Rafael Alejandro Sarmiento Perez (CSCS, Switserland)

Agenda

• outlook to EasyBuild v3.5.0
  • recent changes in develop + overview of remaining PRs planned for v3.5.0
• discussion on providing Python bindings for multiple Python versions in a single installation (Maxime, Compute Canada)
• early look at options for 2018a common toolchains
• Q&A

Notes

Outlook to EasyBuild v3.5.0

• ETA for v3.5.0:
  • everything merged by end of this week
  • regtesting over weekend
  • release (early) next week
• framework
  • https://github.com/easybuilders/easybuild-framework/milestone/50
  • support for hooks: blocker for v3.5.0
    • https://github.com/easybuilders/easybuild-framework/pull/2343
    • also needs docs update
• easyblocks
  • https://github.com/easybuilders/easybuild-easyblocks/milestone/42
  • TensorFlow easyblock
    • works for CPU-only build from source, not yet when CUDA is involved...
    • https://github.com/easybuilders/easybuild-easyblocks/pull/1287
  • bug fix for CP2K: pass down optimisation flags correctly
    • https://github.com/easybuilders/easybuild-easyblocks/pull/1293
  • fixes for VTune & co (Damian)
    • https://github.com/easybuilders/easybuild-easyblocks/pull/1229
  • building ScaLAPACK in parallel (Åke)
    • https://github.com/easybuilders/easybuild-easyblocks/pull/1288
  • already merged:
    • enhancements to R easyblock:
      • correct setting of BLAS/LAPACK configure options (+ checking)
        • https://github.com/easybuilders/easybuild-easyblocks/pull/1292 + https://github.com/easybuilders/easybuild-easyblocks/pull/1300
      • set configure options for included dependencies
        • https://github.com/easybuilders/easybuild-easyblocks/pull/1297 + https://github.com/easybuilders/easybuild-easyblocks/pull/1303
• easyconfigs
  • https://github.com/easybuilders/easybuild-easyconfigs/milestone/45
    • cleanup for R easyconfigs: https://github.com/easybuilders/easybuild-easyconfigs/pull/5478
    • PRs for version updates, minor bug fixes, new software that are ready to go
    • already merged:
      • R 3.4.3 easyconfig with intel/2017b: https://github.com/easybuilders/easybuild-easyconfigs/pull/5360
      • Caffe 1.0: https://github.com/easybuilders/easybuild-easyconfigs/pull/5363
      • easyconfigs for various updates & new software

Installing Python bindings for different Python versions

• $EBPYTHONPREFIXES instead of $PYTHONPATH
  • see recent post to EasyBuild mailing list on use of sitecustomize.py
    • https://lists.ugent.be/wws/arc/easybuild/2017-11/msg00044.html
  • $PYTHONPATH is set to pick up sitecustomize.py, requires for virtualenv
• example: https://github.com/ComputeCanada/easybuild-easyconfigs/blob/computecanada-master/easybuild/easyconfigs/o/OpenMM/OpenMM-7.1.1-iomkl-2016.4.11.eb
• better support for doing this cleanly?
  • custom PythonPackage easyblock to update $EBPYTHONPREFIXES rather than updating $PYTHONPATH
  • support to swap to different Pythons when building bindings
  • also inject a prereq on some Python version in the module
• consider extension approach, so additional bindings can be added later via eb --skip
• support something like bindings = [('Python', '2.7.14'), ('Python', '3.6.3'), ('Perl', '5.26.0')]?
• another example: https://github.com/ComputeCanada/easybuild-easyconfigs/blob/computecanada-master/easybuild/easyconfigs/s/Scipy-Stack/SciPy-Stack-2017b-dummy-dummy.eb
  • different use case: install multiple Py pkgs for different Python versions in one installation

2018a toolchains

• see post on mailing list: https://lists.ugent.be/wws/arc/easybuild/2017-12/msg00019.html
• probably sticking with GCC 6.4.0 (unless GCC 6.5.0 is released...)
  • GCC 7.x not supported yet in Intel 2018 update 1, nor in latest CUDA
• stick to binutils 2.28 so GCCcore/6.4.0 easyconfigs can be reused?
  • take another at release notes
  • no issues on Skylake with binutils 2.28 by JSC/ComputeCanada
• Open MPI: stick with stable 2.1.2
  • 3.0.0 is too fresh?
• OpenBLAS still at 0.2.20
  • should start considering alternatives...
    • ATLAS? BLIS?
  • Intel MKL could be considered since it's free now (but not open source though)?
  • very liberal license
  • some potential concerns on AMD systems (needs env vars)
  • would MKL + Open MPI work fine for ScaLAPACK?
    • BLACS in MKL is available for both impi and ompi
  • OpenBLAS 0.2.20 + patches doesn't build correctly on KNL
    • max. thread number (256) is exceeded for 68-core KNL (4-way mt)
• intel/2018a: 2018 update 1 versions of Intel compilers, MPI & MKL

Other topics

• issues with SurfSARA registration page for EasyBuild User Meeting in Amsterdam?
• support for filtering dependencies based on version to be followed up in the issue
  • see https://github.com/easybuilders/easybuild-framework/issues/2351