Conference call notes 20170816 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 83rd EasyBuild conference call, Wednesday August 16th 2017 (5pm - 6pm CEST)

Attendees

Alphabetical list of attendees (4):

• Damian Alvarez (JSC, Germany)
• Kenneth Hoste (HPC-UGent, Belgium)
• Davide Vanzo (Vanderbilt University)
• Fotis Georgatos (Illumina, UK)

Agenda

• update on upcoming EasyBuild 3.4.0
• update on intel/2017b + foss/2017b toolchains
• new features (WIP)
  • support for eb --trace
  • support for eb --inject-checksums
• make SHA256 checksums required in easyconfig pull requests?
• Q&A

Notes

EasyBuild v3.4.0

• bug fix for 'import check for extensions
  • see https://github.com/easybuilders/easybuild-framework/pull/2276
  • only relevant for R extensions (import check relies on stdin & is basically skipped), not for Python, Perl, etc.
• some bug fixes in easyblocks (cfr. PRs by Damian)
• Python in GCCcore by Jack + Python packages on top that have Python as build dep
  • see e.g. Cython easyconfig: https://github.com/easybuilders/easybuild-easyconfigs/pull/4966
  • also, hackish use of sed to get Python to build with RPATH linking
    • see https://github.com/easybuilders/easybuild-easyconfigs/pull/4975/files#r131548414
  • should get cleaned up for 3.4.0, see discussion in https://github.com/easybuilders/easybuild-easyconfigs/pull/4962
• milestones
  • framework: https://github.com/easybuilders/easybuild-framework/milestone/48?closed=1
  • easyblocks: https://github.com/easybuilders/easybuild-easyblocks/milestone/40?closed=1
  • easyconfigs: https://github.com/easybuilders/easybuild-easyconfigs/milestone/43?closed=1

2017b toolchains

• still waiting on (non-buggy) OpenBLAS release for foss/2017b...
  • 0.2.21 release with reverted cgroups support?
    • cfr. https://github.com/xianyi/OpenBLAS/issues/1258 & https://github.com/xianyi/OpenBLAS/pull/1247
  • also, we shouldn't replace LAPACK: https://github.com/xianyi/OpenBLAS/issues/1273
  • options:
    • stick with OpenBLAS 0.2.19 for now
      • not really a good option, too old (Sept'16), doesn't support e.g. Intel Skylake
    • go forward with available 0.2.20, including potentially unstable cgroups support
      • known problems on non-glibc systems: https://github.com/xianyi/OpenBLAS/issues/1252
    • go forward with 0.2.20 & apply reversion of cgroups support as a patch
    • wait for 0.2.21 (which may be more than simply 0.2.20 + reverted cgroups support)
  • this whole ordeal raises concerns w.r.t. stability & release process of OpenBLAS...
  • but there are no real alternatives (ATLAS is same-same release-wise, but a lot slower?)

New features (WIP)

• eb --trace
  • see https://github.com/easybuilders/easybuild-framework/pull/2285
  • positive feedback from Davide/Damian
  • some minor enhancements will be added on top of current state (+ unit test)
• eb --inject-checksums sha256
  • see https://github.com/easybuilders/easybuild-framework/pull/2286
  • Damian: back_up_file function from backup-modules PR can be used here
  • adds checksums for sources/patches, incl. for extensions
  • also fixes desired order for source_urls, sources, patches & checksums
  • any existing checksums are simply replaced (requires --force)
  • checksum type is a required argument, to avoid problems with 'store-or-none' type options (cfr. --robot) that eat arguments...
• PRs Damian:
  • support for userinGroup: https://github.com/easybuilders/easybuild-framework/pull/2274 needs (enhanced) test?
  • support for --backup-modules: https://github.com/easybuilders/easybuild-framework/pull/2134

Required SHA256 checksums in PRs?

• see https://github.com/easybuilders/easybuild-easyconfigs/pull/5005
• --inject-checksums is a blocker for this

Other

• Fotis: support for hooks? (pre/post steps)
  • https://github.com/easybuilders/easybuild-framework/issues/2287
  • would allow to verify checksums with external service, cfr. https://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API#ArtifactoryRESTAPI-FileInfo
  • Kenneth: this could actually be quite easy to support, since we have a central place where all steps are triggered...
• Damian: switching between EB versions can be annoying with having $EASYBUILD_* env vars set for new features
• Davide: how to retrigger Travis for PRs?
  • 'official' way is to close/open the PR...