Conference call notes 20180523 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 102nd EasyBuild conference call, Wednesday May 23rd 2018 (5pm - 6pm CEST)

Attendees

Alphabetical list of attendees (6):

Damian Alvarez (JSC, Germany)
Fotis Georgatos (Illumina, UK)
Kenneth Hoste (HPC-UGent, Belgium)
Alan O'Cais (JSC, Germany)
Åke Sangren (Umeå University, Sweden)
Shahzeb Siddiqui (Pfizer)

Agenda

outlook to EasyBuild v3.6.1
progress towards making EasyBuild compatible with Python 3.x
Q&A

EasyBuild v3.6.1

ETA: this week (?)
merged PRs for EB v3.6.1
framework:
- all PRs targeted for EasyBuild v3.6.1 have been merged
- highlights:
  - fixed bug to ensure temporary module is picked up
    - see https://github.com/easybuilders/easybuild-framework/pull/2506
    - ml use with high priority + set default
    - same problem with user modules?
      - TODO: double check, update docs to mention priority
  - lib vs lib64 fallback in sanity check
    - see https://github.com/easybuilders/easybuild-framework/pull/2477
    - makes sanity check slightly more flexible
    - should help significantly on SuSE-based systems
  - no more blind overwrite of existing easyconfigs on --try-*
    - see https://github.com/easybuilders/easybuild-framework/pull/2504
easyblocks:
- all PRs targeted for EasyBuild v3.6.1 have been merged
- highlights:
  - fixed sanity check for imkl (but no "damage" done, sanity check still passes)
    - cfr. https://github.com/easybuilders/easybuild-easyblocks/pull/1392
  - provide control over adding of root install dir to $PATH in Binary generic easyblock
    - https://github.com/easybuilders/easybuild-easyblocks/pull/1426
  - support for auto-detecting downloading of dependency pkgs in PythonPackage easyblock
    - https://github.com/easybuilders/easybuild-easyblocks/pull/1377
    - we should enforce a policy to enable download_dep_fail in new easyconfigs that use PythonPackage
easyconfigs:
- mostly software updates, minor bug fixes and new software (incl. dotNET!)
- handful of PRs will get get merged (hopefully)
  - OpenBLAS 0.3.0 (just released): https://github.com/easybuilders/easybuild-easyconfigs/pull/6361
  - archive URLs for Bioconductor: https://github.com/easybuilders/easybuild-easyconfigs/pull/6311
  - GCC 8.1: https://github.com/easybuilders/easybuild-easyconfigs/pull/6296

Progress towards making EasyBuild compatible with Python 3.x

Vanessa Sochat started looking into making vsc-base compatible with Python 3.x
- https://github.com/hpcugent/vsc-base/pull/258
- after making vsc-install Py3 compatible
  - see https://github.com/hpcugent/vsc-install/pull/83
once vsc-base is compatible with Python 3, we can work towards making EasyBuild Python 3 compatible as well...
fueled by interest in:
- leveraging HPC Container Maker (https://github.com/NVIDIA/hpc-container-maker) in EasyBuild
  - was Python 3-only until very recently (cfr. https://github.com/NVIDIA/hpc-container-maker/pull/28)
- Python API for Singularity: https://singularityhub.github.io/singularity-cli/commands

Q&A

follow-up conf call w.r.t. container support tomorrow (Kenneth, Shahzeb, ...)
- https://github.com/easybuilders/easybuild-framework/pull/2479 should go in first?
Shahzeb:
- better way to handle Bioconductor
  - support installing biocViews?
    - https://bioconductor.org/packages/release/bioc/html/biocViews.html
  - no version control?
  - could be supported as alternate way of installing, but requires enhanced easyblock?
- can we support alternatives to CRAN?
  - miniCRAN:
  - MRAN: https://mran.microsoft.com/documents/rro/reproducibility (see checkpoint library)
    - https://mran.microsoft.com/documents/rro/reproducibility
  - pakrat:
  - could be useful for easy_update script (https://github.com/fizwit/easy_update)?
  - could existing R easyblock be enhanced to use local snapshot of source for packages?
- too much of a pain to deal with R packages right now...
- Fotis: could $HTTP_PROXY (or similar) be used to convince R to use sources from somewhere else