Understanding the stuff. - omair18/numpy-docker GitHub Wiki

This page will be focusing on understanding that how build-kite works, what's inside the repo, how to create a docker, use docker-compose and run docker-compose with nvidia support so that it utilized the host GPU.

build-kite.

Buildkite uses two files, a) template.yml b) pipeline.yml located inside the hidden folder .builtkite of this repo.
template.yml is used for the creation of testing pipeline. As explained on page-2. Buildkite reads the command.
pipeline.yml contains the commands used for the testing. Once the pipeline is created and repos are cloned inside running agent, buildkite will run the command. Buildkite is using the docker plugin in this repo. So once the docker-image is built, container is running, it'll execute the command inside the docker. In this case, py.test will run all the python test scripts located inside the working directory.

docker

The Dockerfile used in this repo looks like this.
ARGS represent the variables.
FROM is used when we want a pre-built docker container to be used as base image. Docker will look for this image inside the local host, if not found, it will fetch it from server. It's more like an import command used in python.
ENV represent the environment variables to be used while running this docker image.
RUN are the sequence of commands used while building this docker image.
WORKDIR is like pwd. It will switch to this directory once docker container is up and running.
COPY is used to copy the contents from local directory to docker image.
CMD is the command run when docker is up.
To build the docker image. Run the following command inside the folder where Dockerfile is located.
docker build -t numpy-docker .
It would take some good amount of time to download and setup docker image.
Once build, run the docker image using following command.
docker run -it --rm numpy-docker
To run the docker with nvidia support (so that you're able to use host machine's GPU inside docker) run the following command.
docker run --runtime=nvidia -it --rm numpy-docker

docker-compose.

Build-kite uses docker-compose instead of docker.
docker-compose is a handy utility of managing and running multiple docker images/containers using only 1 command. docker-compose uses the file docker-compose.yml to read the configurations.
docker-compose file used in this repo looks like this.
docker-compose reads the Dockerfile to build a container.
services are the containers you want to run. You can add more if you need.
build is the same as running the docker build command.
runtime is the same as explained above.
command is the command run once the container is up.
To build a docker-image using docker-compose, run the following command.
docker-compose --build
To run the docker-image.
docker-compose up
To stop the container.
docker-compose down