Conference call notes 20150812 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the 34th EasyBuild conference call, Wednesday August 12th 2015 (5.00pm - 5.40pm CET)

Attendees

Alphabetical list of attendees (7):

  • Petar Forai (IMP/IMBA, Austria)
  • Fotis Georgatos (freelancer)
  • Ryan Goggin (IQC, University of Waterloo, Canada)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Riccardo Murri (University of Zurich, Switzerland)
  • Alan O'Cais (JSC, Germany)
  • Robert Schmidt (OHRI, Canada)

Agenda

  • outlook to EasyBuild v2.2.1 (Kenneth)
  • Q&A
outlook to EasyBuild v2.2.1
  • bug fix release
  • support for --package never worked because it tried to include the temporary 'fake' module file in the package, rather than the final one
  • the unit tests for --package missed this due to heavy mocking of fpm :(
  • fixed in
  • issues with --job
  • hidden dependencies end up not being installed as hidden, because --hidden doesn't get added to the eb cmdline of the job being submitted
  • --job and --try don't work well together
  • tweaked easyconfigs are stored in --tmpdir by default, and thus typically not available anymore in the submitted job
    • workaround is to use a shared directory as --tmpdir when using --job
    • actual solution would be to only apply --try after submitting the job
      • can this actually work (w.r.t. dep resolution, etc.)?
  • --try options are still being passed down, despite a tweaked easyconfig being used in the job (??)
  • also take a look at reported issue https://github.com/hpcugent/easybuild/issues/135 ?
  • enhancements to EasyConfig.dump: formatting according to 'code' style for easyconfigs + retaining comments
  • include easyconfigs for 2015b common toolchains + software built with either foss/2015b or intel/2015b
Other topics
  • more flexibility should be supported w.r.t. what goes into module files generates by EasyBuild, and how they are composed
    • required in order to get TACC to pick up EasyBuild
    • this could be tackled by introducing the notion of a module file 'template'
    • either an actual file that can be provided, or support for providing a 'module generator plugin' similar to the support for custom module naming schemes
    • implementing this should be fairly straightforward, it basically boils down to letting module_generator construct a dict with values that can be used to complete a supplied template
    • this would most likely be accompanied by additional easyconfig parameters (e.g., usage, helptxt, ...)
    • also missing: support for Lmod specific features like prereq (vs load), module families, module properties, etc.
  • EasyConfig.dump() doesn't include the list of hiddendependencies in dependencies, even though that's required => https://github.com/hpcugent/easybuild-framework/issues/1350
    • relates to support for making dependency resolution aware of subtoolchains (WIP by Alan, see
  • Fotis: ping on support for easily overriding easyconfig parameters
    • see https://github.com/hpcugent/easybuild-framework/pull/712
    • Kenneth: needs to be looked at again
    • some concerns w.r.t. combo with --try, reproducibility of builds (stored easyconfig files don't include modifications), ability to only override via environment variable(s) (no cmdline or cfgfile support), etc.