How to tune CI Docker image - Open-Quantum-Platform/openqp GitHub Wiki

The Continuous Integration (CI) pipeline for Open Quantum Platform (OQP) runs within Docker images that are stored in the project's GitLab registry. Occasionally, new software, such as BLAS/LAPACK, MPI, or compiler suites, needs to be pre-installed to build and run OQP. This requires modifying the Docker image stored in the registry.

To create or update a Docker image, you need to make or edit a Dockerfile. This file contains several Docker commands executed sequentially during the image preparation step. Below is an example of a Dockerfile used as of August 2024:

FROM ubuntu:20.04

ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=Etc/UTC
RUN apt update \
   && apt install -y build-essential gcc gfortran g++ \
                     python3-dev python3-pip \
                     libopenmpi-dev \
                     libopenblas-dev libopenblas64-dev \
   && apt clean

RUN pip3 --no-cache-dir install cmake ninja cffi black numpy scipy

CMD ["/bin/bash"]

• FROM: Specifies the base image, in this case, a public Ubuntu 20.04 image from Docker Hub. You can choose another version by checking available options here.

• ARG and ENV: Used to set environment variables during the build (ARG) and runtime (ENV) stages. In this example, DEBIAN_FRONTEND and TZ are set to ensure non-interactive installations and timezone settings.

• RUN: Executes shell commands inside the container during the image build. The commands in this example install essential build tools, compilers, MPI libraries, BLAS/LAPACK, and Python packages.

• CMD: Defines the default command to be executed when the container starts. Here, it launches a Bash shell.

Note: Each RUN command creates a new layer in the Docker image, so try to combine commands where possible to minimize the number of layers.

For more details and command syntax, refer to the Docker manual.

To create or update the Docker image used in CI, follow these steps:

1. Install Docker: Ensure Docker is installed on your computer. You can find installation instructions here.

2. Set Up Your Workspace:

   • Create a new empty directory and navigate to it in your terminal.
   • Inside this directory, create a file named Dockerfile and populate it with the relevant commands.

3. Log In to the Registry:

   • Run docker login qchemlab.knu.ac.kr:5050 to authenticate with the registry. Use your GitLab credentials for this step.

4. (Optional) Update the Ubuntu Image:

   • If necessary, update the base Ubuntu image on your computer using docker pull <image>, where <image> could be ubuntu:20.04.

5. Build the Docker Image:

   • Run the following command to build the Docker image:

     docker build -t qchemlab.knu.ac.kr:5050/open-quantum-package/modules/ci-ubuntu:<version> .

     Replace <version> with a descriptive tag, such as 2024.8. The dot at the end indicates the current directory (where the Dockerfile is located).

6. Tag the Image as latest:

   • If the build succeeds, tag the image as latest:

     docker tag qchemlab.knu.ac.kr:5050/open-quantum-package/modules/ci-ubuntu:<version> qchemlab.knu.ac.kr:5050/open-quantum-package/modules/ci-ubuntu:latest

   • Alternatively, you can use the image ID instead of old_tag, which can be found with docker image ls.

7. Push the Image to the Registry:

   • Push your versioned image to the registry:

     docker push qchemlab.knu.ac.kr:5050/open-quantum-package/modules/ci-ubuntu:<version>

   • To test the new image, you can update the first line of the .gitlab-ci.yml file in your branch with your specific version.

8. Finalize the Image:

   • If everything works as expected, push the latest tag to the registry:

     docker push qchemlab.knu.ac.kr:5050/open-quantum-package/modules/ci-ubuntu:latest

   • This will make the image the default for all branches during CI. Don’t forget to revert any changes you made to the .gitlab-ci.yml file in your branch.

By following these steps, you can effectively manage and update the Docker images used in the OQP CI pipeline, ensuring that all necessary software and tools are available for building and running the platform.

The Continuous Integration (CI) pipeline for the Open Quantum Platform (OQP) runs within Docker images that are stored in the project's GitHub Container Registry. Occasionally, new software, such as BLAS/LAPACK, MPI, or compiler suites, needs to be pre-installed to build and run OQP. This requires modifying the Docker image stored in the registry.

To create or update a Docker image, you need to create or edit a Dockerfile. This file contains several Docker commands executed sequentially during the image preparation step. Below is an example of a Dockerfile used as of August 2024:

FROM ubuntu:20.04

ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=Etc/UTC
RUN apt update \
   && apt install -y build-essential gcc gfortran g++ \
                     python3-dev python3-pip \
                     libopenmpi-dev \
                     libopenblas-dev libopenblas64-dev \
   && apt clean

RUN pip3 --no-cache-dir install cmake ninja cffi black numpy scipy

CMD ["/bin/bash"]

• FROM: Specifies the base image, in this case, a public Ubuntu 20.04 image from Docker Hub. You can choose another version by checking available options here.

• ARG and ENV: Used to set environment variables during the build (ARG) and runtime (ENV) stages. In this example, DEBIAN_FRONTEND and TZ are set to ensure non-interactive installations and timezone settings.

• RUN: Executes shell commands inside the container during the image build. The commands in this example install essential build tools, compilers, MPI libraries, BLAS/LAPACK, and Python packages.

• CMD: Defines the default command to be executed when the container starts. Here, it launches a Bash shell.

Note: Each RUN command creates a new layer in the Docker image, so try to combine commands where possible to minimize the number of layers.

For more details and command syntax, refer to the Docker manual.

Going back to Administration Documentation