Sync meeting on EESSI test suite (2023 03 30) - EESSI/meetings GitHub Wiki

EESSI sync meeting on software testing (2023-03-30)

Attending

Caspar, Sam, Satish, Kenneth

Notes

PRs
- #22 just does some code cleanup
  - merged!
- #23 add support for custom executable opts
  - needs to be synced with main by Sam after merging of #22
  - enables control over additional GROMACS options
  - Satish hopes to find time to take a look
- #24 add hortense config file
  - Kenneth will test & merge
Is GROMACS test good enough as example for other tests?
- There's still room for improvement
  - cfr. https://github.com/EESSI/test-suite/issues/18
  - my_find_modules should be moved to utils
  - come up with multiple general scales
    - 1_core (single-core)
    - sanity_check (1 node)
    - 1_node (1 node, all available cores)
    - 2_nodes (2 nodes)
    - 4_nodes (4 nodes)
    - 8_nodes (8 nodes)
    - => should define a list of known scales somewhere centrally, which defines both tag + node count
    - make a utils function that tags the test with x_nodes tags (+ additional tags like 1_core, sanity_check, ...)
  - set_test_scale is a very general method
    - could be part of an EESSITestCommon base class (known_scales)? or a constant that it's imported and used in base class
    - using reframe.core.pipeline.RegressionMixin? see https://reframe-hpc.readthedocs.io/en/stable/regression_test_api.html#reframe.core.pipeline.RegressionMixin + https://reframe-hpc.readthedocs.io/en/stable/tutorial_advanced.html#grouping-parameter-packs
  - body of filter_tests could be hoisted into a general function that takes self.nb_impl as input
    - filter_tests_by_device_type(device_type, mod_name)
      - checks whether device_type == 'gpu'
      - checks whether mod_name has CUDA support
  - code refactoring options:
    - Class inheritance
    - Use fixtures
    - Move functionality to dedicated functions
feature request made to ReFrame
- cfr. https://github.com/EESSI/test-suite/issues/21 + https://github.com/reframe-hpc/reframe/issues/2820
- this is just "nice to have" to be able to filter more easily
adding two/three additional tests could help figure out which parts of a test are common, and can be put in a base class
- (Caspar) TensorFlow - quite similar to GROMACS on CPU/GPU aspect
- (Satish) OSU - quite different compared to GROMACS + TensorFlow
  - point-to-point
  - collectives: 1-to-many, many-to-1, all-to-all
  - small/large packet size
effort of trying to generalise (Sam) + add more tests can be done in parallel by different people (Caspar: TensorFlow, Satish: OSU)
(Kenneth) GitHub Actions workflow to test 'pip install' in old/new Ubuntu
next steps
- tests for software used in MultiXscale (ESPResSo, waLBerla, LAMMPS)
- integration with the bot (test step in between build/deploy)