Conference call notes 20160111 - easybuilders/easybuild GitHub Wiki

(back to Conference calls)

Notes on the EasyBuild conference call dedicated to discussing implementation of RPATH support, Monday January 11th 2016 (5pm CET)

Attendees

Alphabetical list of attendees (10):

  • Xavier Besseron (Uni.lu, Luxembourg)
  • Timothy Brown (Uni. of Colorado @ Boulder, US)
  • Pablo Escobar (UniBas, Switzerland)
  • Elisabeth Fisher (NASA, US)
  • Todd Gamblin (LLNL, US)
  • Fotis Georgatos (freelancer)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Lumír Jasiok (IIT4Innovations, Czech Republic)
  • Alan O'Cais (JSC, Germany)
  • Ward Poelmans (HPC-UGent, Belgium)
  • Robert Schmidt (OHRI, Canada)

Agenda

  • implementation of support for RPATH linking in EasyBuild
    • different available options?
    • overall approach for implementing option of choice
    • who does what?

Notes

Options

  • compiler wrapper script (cfr. Spack)

  • ld wrapper script (cfr. Spack, script from Swedish colleagues)

    • should be sufficient (?)

  • patchelf

    • doesn't work on OS X
    • 'dummy' values need to be included that are long enough to hold actual path

  • $ORIGIN

    • can be used to support relative RPATH'ing
    • allows relocating installations later
    • advantage of Conda over Nix
    • not used in Spack today

  • sanity check?

  • RPATH support means sort of re-architecting EB design?

    • RPATH support is step towards dropping modules as part of build process?
    • modules are still useful for users

  • $LD_LIBRARY_PATH may still be needed

    • e.g. for compilers (depends on how they were built)
    • Intel compilers are patched at LLNL to use RPATH

  • use of RPATH will fix problems caused by loading modules

    • loading CMake (which loads ncurses) breaks htop on the system
    • also allows using tools built with different toolchains/versions
      • linking to two different versions of a library in the same binary may not work though...
    • comes close to 'toolchain-neutral' software (cfr. https://github.com/hpcugent/easybuild-framework/issues/570)
    • can also helps with keeping size of environment under controle
      • important for MPI, job launchers, performance (library lookup)

  • RPATH vs RUNPATH

    • if both are set, RUNPATH wins
      • if you do RUNPATH, ld will actually do both (by default)
    • $LD_LIBRARY_PATH wins over RUNPATH, so to much potential issues
    • supporting both should be fairly trivial?

  • default on OS X is to use full paths to libraries

  • RPATH & CMake (Todd)

    • cmake by default will re-write RPATHs on install.
    • so if you use an ld wrapper it will not cover CMake in the standard case
    • (ironic b/c cmake has really good rpath support, but it requires a special case if YOU want to implement good RPATH support)
    • so you have to set CMAKE_INSTALL_RPATH on your cmake builds to tell it to write the right thing:
    • https://github.com/LLNL/spack/blob/develop/lib/spack/spack/build_environment.py#L215
    • the reason it does that is b/c it sets the RPATH at build time to the build directory (nice for developers)
    • but it rewrites on install so that the install works
Implementation
  • specify in easyconfigs which libraries to RPATH against it?
  • --rpath-deps?
  • also support to specify "RPATH all the things"
  • concern of reproducibility of build where RPATH was used
  • which deps were RPATH'ed should be dumped to the easyconfig copied in the install dir/archive
Inspiration
Who?
  • Rob
  • Ward (help with wrapper script)
Follow-up