Magma CI - magma/magma GitHub Wiki

This page aims to document important CI workflows as of 01/2023. This information is prone to becoming outdated but should still cover important jobs at Magma and how they are organized. Nevertheless, if you see outdated parts, feel free to update them.

The CI is based on GitHub workflows. An overview of all runs can be found on https://github.com/magma/magma/actions. The checked-in definitions can be found in the magma repository in .github/workflows.

We distinguish between pre-merge and post-merge workflows.

Pre-merge workflows

Pre-merge workflows run on pull requests (based on forks). This implies:

  • These workflows should run sufficiently fast so that developers do not have to wait too long for feedback.
  • Some of these workflows are required to pass before a pull request is able to be merged.
  • The workflows usually do not need to publish artifacts to external locations (this should only happen based on reviewed and merged code).
  • The workflows relevant for this documentation do not have access to Magma GitHub secrets - this is for security reasons. This is not needed pre-merge, because there is no need to publish artifacts to external locations for not merged pull requests. Without access to secrets, notifications via Slack can also not be sent.

These workflows cover, among others, unit tests, linters, code coverage checkers, and the building of certain artifacts (without publishing them).

Post-merge workflows

Post-merge workflows also include almost all pre-merge workflows. In addition, multiple long running integration tests and workflows that build and publish artifacts are executed - let's call those "strict-post-merge" workflows. The statuses of strict-post-merge workflows are also displayed on https://magma-ci.web.app/ for each commit (merge) on master or on one of the release branches. The extra tooling to display the workflow results is used because GitHub currently does not offer such a nice overview as this and we also want to display results of CI runs that are not modeled as GitHub workflows (e.g., 5g tests). For more information on this, see its section of the Technical Infrastructure page.

In the following we describe the strict-post-merge workflows, based on their names on https://magma-ci.web.app/, in more detail.

Builds

These are the workflows that create and publish build artifacts. AGW, FEG, ORC8R, NMS and CWAG are listed separately, but these builds are all covered by one workflow called "Magma Build & Publish" that is modeled in .github/workflows/build_all.yml. The created artifacts are published to the magma artifactory. For FEG, ORC8R, NMS and CWAG these are docker images. For AGW debian artifacts created with make are published.

Workers

Almost all workflows here are executing tests vs artifacts that were created by the workflow described in "Builds".

SUDO PYTHON

These are tests with a character somewhere between unit tests and integration tests (pipelined tests that need ovs, mobilityd tests that require a certain network setup). The workflow is given by .github/workflows/sudo-python-tests.yml.

LTE INTEG CONTAINERIZED

Integration tests vs a containerized AGW. The workflow is modeled by .github/workflows/agw-build-publish-container.yml. This workflow also runs on PRs - but there the AGW containers are only built (and not published). If this workflow runs post-merge, the AGW containers are created and published. Subsequently, .github/workflows/lte-integ-test-containerized.yml is triggered for different sets of tests to run in parallel.

LTE INTEG DEBIAN

Integration tests vs an installed magma debian artifact built with Bazel. The workflow is found in .github/workflows/lte-integ-test-bazel-magma-deb.yml. This workflow is actually triggered from the workflow in .github/workflows/bazel.yml. It creates the debian artifact which is published to the artifactory. The integration test workflows downloads and installs the respective artifact.

Note that there is also a workflow that runs the integration tests vs an environment where a debian artifact created with make is installed. This workflow is modeled in .github/workflows/lte-integ-test-magma-deb.yml. Results of these workflow are not published to https://magma-ci.web.app/. This workflow is triggered by the build_all.yml workflow mentioned above, after the AGW debian artifact has been created and published.

FEG INTEG

Integration tests executed vs a federated gateway. The workflow is specified in .github/workflows/federated-integ-test.yml.

CWF INTEG

Integration tests executed vs a carrier Wi-FI gateway. The workflow is specified in .github/workflows/cwf-integ-test.yml.

External workflows

In the pre-merge workflows, some OAI specific tests are run on an OAI hosted Jenkins (https://jenkins-oai.eurecom.fr). Also code coverage is analyzed by codecov.io (https://app.codecov.io/gh/magma/magma).

For the strict-post-merge CI we currently have "WL 5G" (also displayed on https://magma-ci.web.app/) that displays results of 5G specific tests.

GitHub Action Cache

A 10GB cache is reserved to store data from all workflows (https://github.com/magma/magma/actions/caches). This is mostly used to cache the VM base images which are needed for these workflows to reduce build time and possible rate limits to the Vagrant Cloud and for bazel caching, also to reduce build time. Reading and writing the cache from the CI workflows is done via GitHub's action/cache (https://github.com/actions/cache).

Least recently used caches will be automatically removed to keep the cache storage below the data limit, but can be manually removed as well.