Sync meeting on EESSI test suite (2023 10 04) - EESSI/meetings GitHub Wiki

EESSI test suite sync meetings

Planning

  • every 2 weeks on Thursday at 14:00 CE(S)T
  • next meetings:
    • Thu 19 Oct 14:00 CEST: OK for all
    • Wed 8 Nov 14:00 CEST
    • to be discussed
      • Wed 22 Nov 14:00 CEST
      • Wed 6 Dec 14:00 CEST
      • Wed 20 Dec 14:00 CEST

Meeting (2023-10-04)

  • PR to properly cover test suite in docs is ready to merge (docs PR #110)
  • OSU test (PR #54)
    • covers both point-to-point and collectives (all-to-all + allreduce)
    • for pt2pt
      • 2 cores are required (one same node, or on 2 cores)
      • only valid scale tags: 2_cores, 2_nodes, 1_node (using only 2 cores)
      • 1_2_node is not necessarily valid (think laptop with 2 cores)
      • should we generate tests and skip if notes enough cores are available?
      • CI tag should be equivalent with 2_cores?
    • ready for review
    • ready to test on other systems
      • [Caspar] Vega (EuroHPC)
      • [Lara] Hortense (UGent)
      • [Sam] Hydra (VUB)
    • additional custom scale(s) for OSU (either general or test-specific)
      • 1 core per node, 2 nodes => 1_core_per_node_2_nodes
        • this is not possible in SURF, submission will be blocked
        • 1_core_per_node_4_nodes can also be useful for collectives
      • 1 GPU per node, 2 nodes => tag?
      • these could also be useful for other tests like GROMACS
      • also impacts logic needed in our hooks
      • Sam is up for looking into this
    • docstring can be clarified + also cover in "Available tests" section in docs
  • CI tag should imply a scale tag? (1_node?)
  • Caspar: separate tag for regression tests in EESSI?
    • currently using --tag CI --tag='1_node|2_nodes' on Vega => fine
    • separate regtest tag could still make sense, could define per test what regtest means
  • for which EESSI versions will we test?
    • latest + latest-1?
    • all available versions in CVMFS repo?
  • pip install fixed (PR #86)
  • v0.1.0 release
    • only missing tag + release + release notes
    • introduce develop branch next to main branch
    • publish to PyPI
      • python setup.py sdist
      • twine upload dist/eessi-test-suite-0.1.0.tar.gz
      • can we reserve the eessi-* namespace in PyPI?
  • eb --sanity-check-only test (Kenneth)
  • Espresso test
    • probably promised in the context of MultiXscale somewhere => not really (not before M18, so mid 2024)
    • Satish could look into this, based on meeting with Jean-Noël, see meeting notes
  • test step in build-and-deploy step
    • run test suite with --tag CI --tag 1_node (for all tests, all available modules)
    • install EESSI test suite as a module in EESSI
  • ReFrame v4.3.4 now also included hpctestlib in PyPI package
    • but EasyBuild still installs from GitHub sources using bootstrap.sh script to install required Python packages

Previous meetings