Sync meeting on EESSI test suite (2025 10 08) - EESSI/meetings GitHub Wiki

Planning

  • every 2 weeks on Thursday at 14:00 CE(S)T
  • next meetings:
    • Th 23rd of October: Sam, Satish, Caspar (Lara is on holiday)
    • Wed 12th of November, @14:00: Satish, Lara, Caspar, Sam may be too busy

Meeting (2025-10-08)

Attending: Lara, Sam, Caspar

  • BLAS test can use a (re)-review, should be done

    • Remark from SAM: you may remember that I saw bad performance for BLIS before. That was because of a name conflict with a single-threaded library with the same name.

  • TensorFlow test is done. We should test carefully - either on an offline system, or check the logs carefully to verify that it doesn't actually (re)download the data if the environment variable is set.

    • At least when the data is not there, it should print an informative error. That can be checked easily. If we then put the data in place and the error is resolved again, that proves the functionality.

  • MareNostrum config can be merged - we don't have access, so we cant test, but there's no point in keeping it open until we do.

    • We can at least test similar config arguments for e.g. Snellius and see if it gives the expected behavior

  • 1M case for OpenFOAM final comments from Sam

    • Plan is to introduce the base class in a follow-up PR

  • 64M case for OpenFOAM

    • Ready for testing

  • Plan: it would be nice to have a standardized way to skip tests if a certain resource X (GPUs, CPUs, memory) doesn't meet a certain requirement. So move that from the test, to the mixin class. That way the error messages on the skip are standardized, instead of different for each test.

    • In the mixin class, we can define one function for which the arguments are the resource, count and limit, and have that print a standardized message

  • Discussed various changes to the dashbaords:

    • Symbol S now indicates skipped tests
    • Symbol A for aborted tests

  • Discussed QuantumESPRESSO test, but that was because of the missing hook https://www.eessi.io/docs/known_issues/eessi-2023.06/#failed-to-modify-ud-qp-to-init-on-mlx5_0-operation-not-permitted

  • Updated the ingestion script so that it can now also ingest data from ReFrame Data Version of the report file 4.1: https://github.com/EESSI/dashboard-ingestion/pull/7