Conference call notes 20160330 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 48th EasyBuild conference call, Wednesday March 30th 2016 (5pm - 5.30pm CEST)

Attendees

Alphabetical list of attendees (5):

• Damian Alvarez Mallon (JSC, Germany)
• Pablo Escobar (UniBas, Switzerland)
• Fotis Georgatos (Illumina, UK)
• Kenneth Hoste (HPC-UGent)
• Robert Schmidt (OHRI, Canada)

Agenda

• support for using PGI in compiler toolchains
  • https://github.com/hpcugent/easybuild-framework/pull/1342 (& co)
• automated style checking of (easyconfig) pull requests
  • https://github.com/hpcugent/easybuild-framework/pull/1618

Notes

• PGI compiler in toolchain

  • support for PGI compiler in framework has been WIP for a long time
    • https://github.com/hpcugent/easybuild-framework/pull/1342
  • other PRs already building on top of it, should go in ASAP
  • very close to being ready, needs some more attention to details + testing
  • PGI compiler already used at JSC in toolchain w/ MVAPICH2 (binary) and Intel MKL
    • issues with getting OpenBLAS(?) built
  • also used at OHRI: MVAPICH2 built with PGI (no BLAS/LAPACK yet)
  • goal would be to compose easyconfigs for PGI-based toolchain + HPL on top of it as a test case

• automated style checking: needs to be followed up with Ward

  • https://github.com/hpcugent/easybuild-framework/pull/1618
  • needs a bit more work to 'mature'
  • would be good to switch to this early in development cycle and enable style checks in test suite

• switch to Travis for tests, cfr. https://travis-ci.org/boegel/easybuild-framework

  • WIP by Kenneth
  • Jenkins testing setup has been a PITA
    • limited resources
      • VM with 4 cores
      • Java being a huge hog on CPU resources
    • test configurations are not tracked through version control
    • multiple copies of 'same' test configuration for develop/master/PRs
    • test environment grew historically, very hard to reproduce again...
    • only way to test contributions is to push to central test setup
  • advantages of using Travis:
    • (also) free for open source projects
    • cloud infrastructure, so way more resources

      • 3x speedup for testing framework branches

    • single test configuration that is under version control (.travis.yml in repository)
    • empowers users to test their work prior to issuing PRs
      • very little (trivial) setup requirements, from zero to testing in minutes through travis-ci.org
  • current status: existing test environment/configurations reproduced in .travis.yml
  • to be figured out:
    • reporting back to pull requests
    • triggering easyblocks/easyconfigs tests after changes to framework
  • switch to Travis in coming weeks?

• slower EB when Lmod is being used

  • due to overzealous module use commands being run over and over again?
  • check 'profiling' support for module commands again done in https://github.com/hpcugent/easybuild-framework/pull/946

• Damian: eb --try-recent-dep-versions

  • use most recent available version for deps (within same toolchain)
  • some dependencies may be installed as hidden modules
  • installed modules vs available easyconfigs vs updates upstream?
  • Python 2 vs Python 3 => stick to same major version?
    • enhance easyconfigs to include more info on versions (what to fix, what's flexible, ...)
  • default: stick to available modules, keep major version number fixed
  • use case: bump Python version
    • bump version of one specific dep?
  • --try-recent-dep-versions[=modules,major_ver@Python,Boost] or --try-recent-dep-versions[=cfg.ini] (preferred)
  • should be easier than manually adjusting easyconfig file...

• Fotis: some progress with EasyBuild/Lmod on Bright