How to set up a required matrix workflow on GitHub actions - magma/magma GitHub Wiki

Motivation

• You want to run a GitHub actions job as a matrix workflow, but the job should be required (via a branch protection rule).
• The current behavior is that the overall job does not return a status anymore, as there are now many shards that return an individual status, and this leads to the job staying with an "expected" status forever.
• Here we take the bazel workflow as an example (link to PR).
  • The bazel build and test job should be run as a matrix workflow with the configs native, asan and production.

    strategy:
      fail-fast: false
      matrix:
        include:
          - bazel-config: ""
            bazel-target: "//..."
          - bazel-config: "--config=asan"
            bazel-target: "`bazel query 'kind(cc.*, //...)'`"
          - bazel-config: "--config=production"
            bazel-target: "`bazel query 'kind(cc.*, //...)'`"

How to set up a required matrix workflow on GitHub actions

• :exclamation: For the required status of the matrix job to work additional jobs are needed.
  • See blog post https://brunoscheufler.com/blog/2022-04-09-the-required-github-status-check-that-wasnt for the general setup.
  • There is one job that only runs if all shards of the matrix are successful

 if_bazel_build_and_test_success:
    name: Run when bazel successful
    needs: bazel_build_and_test
    runs-on: ubuntu-latest
    if: success() # Only run after all matrix jobs have passed
    # Store success output flag for report_result_bazel_build_and_test job
    outputs:
      success: ${{ steps.setoutput.outputs.success }}
    steps:
      - id: setoutput
        run: echo "::set-output name=success::true"

• And there is one job that always runs and evaluates the results. This final job is the one that should be required.
  • The approach from the blog needs to be adapted if there is a path filter step, which may be skipped itself or lead to a skipped bazel job. This can be handled by modifying the if conditional in the last job.

  report_result_bazel_build_and_test:
    name: Bazel build and test status
    runs-on: ubuntu-latest
    if: always()
    # This job always needs to run. It will be green if the bazel_build_and_test
    # job was successful in all matrix jobs or if the job was skipped.
    needs: [path_filter, bazel_build_and_test, if_bazel_build_and_test_success]
    steps:
      - run: |
          bazel_build_and_test_success="${{ needs.if_bazel_build_and_test_success.outputs.success }}"
          files_changed="${{ needs.path_filter.outputs.files_changed }}"
          echo "bazel_build_and_test_success: $bazel_build_and_test_success"
          echo "files_changed: $files_changed"
          if [ $bazel_build_and_test_success == "true" ](/magma/magma/wiki/|-$files_changed-!=-'true'-);
          then
            echo "Bazel build and test job passed or was skipped"
            exit 0
          else
            echo "Bazel build and test job failed"
            exit 1
          fi

Expected behaviour

With these additional jobs in place we now find that the Bazel build and test status job always returns the correct result.

• If all shards are successful the status job will be green (and the required status will be green):
  • 
• If the build step is skipped it will still return green at the end:
  • 
• If one of the shards fails the job will fail (and the required status will be blocked):
  • 

Needed changes to the branch protection rule

• :warning: Before a PR that creates a required matrix workflow is merged the branch protection settings need to be changed to require the following jobs:
  • In this example the report_result_bazel_build_and_test / Bazel build and test status, which is the final job after the matrix job.
  • NOT the overall workflow or matrix job, e.g. in this example the Bazel Build & Test Job (this is blocking the PR and will lead to wrong requirements).