16 ‐ Docker - CloudScope/DevOpsWithCloudScope GitHub Wiki

• What is Docker?
  • Docker is a platform that enables developers to build, package, and distribute applications in a lightweight, portable container.
  • It helps in standardizing environments, ensuring that applications run the same regardless of where they are deployed.
• Docker Components
  • Docker Engine: The runtime that runs containers.
  • Docker Image: A read-only template that contains the application and its dependencies.
  • Docker Container: A running instance of a Docker image.
  • Docker Registry: A repository (e.g., Docker Hub) to store and share images.

• Docker Installation
  • Install Docker on various operating systems (Linux, Windows, macOS).
  • Key commands for installation: apt-get install docker-ce (Ubuntu), brew install docker (macOS), or using Docker Desktop on Windows/Mac.
• Docker Daemon & CLI
  • docker is the client tool for interacting with Docker Daemon.
  • Docker Daemon (dockerd) is responsible for managing containers and images.

• Dockerfile Basics
  • A Dockerfile is a script that contains a set of instructions to build a Docker image.
  • Common instructions: FROM, RUN, COPY, WORKDIR, EXPOSE, CMD, ENTRYPOINT.
• Example Dockerfile

  FROM ubuntu:20.04
  RUN apt-get update && apt-get install -y python3
  COPY . /app
  WORKDIR /app
  CMD ["python3", "app.py"]

• Build an Image

  docker build -t myapp:latest .

  • -t: Tags the image with a name (myapp:latest).

• Creating & Running Containers
  • Start a container from an image:

    docker run -d --name mycontainer myapp:latest

  • -d: Runs the container in detached mode.
  • --name: Specifies the container name.
• Interactive Mode

  docker run -it ubuntu bash

  • -it: Runs the container in interactive mode with a terminal (bash shell).

• List Running Containers

  docker ps

• List All Containers (Including Stopped)

  docker ps -a

• Stopping & Removing Containers

  docker stop <container_id>
  docker rm <container_id>

• Removing Images

  docker rmi <image_id>

• Pruning Unused Images, Containers, Networks

  docker system prune

• Purpose of Volumes
  • Docker volumes are used to persist data generated by and used by Docker containers.
  • Volumes are stored outside of the container filesystem and survive container restarts.
• Creating and Using Volumes

  docker volume create myvolume
  docker run -v myvolume:/data myapp

• Listing Volumes

  docker volume ls

• Removing Volumes

  docker volume rm myvolume

• Docker Networks
  • Docker containers can communicate with each other via networks.
  • Docker supports multiple network drivers (bridge, host, none, overlay).
• Create Custom Network

  docker network create mynetwork

• Run Containers on a Specific Network

  docker run --network=mynetwork myapp

• Inspect Networks

  docker network inspect mynetwork

• What is Docker Compose?
  • Docker Compose allows you to define and manage multi-container Docker applications with a simple YAML configuration file.
• docker-compose.yml Example

  version: '3'
  services:
    web:
      image: nginx
      ports:
        - "8080:80"
    db:
      image: mysql
      environment:
        MYSQL_ROOT_PASSWORD: example

• Commands
  • Start services: docker-compose up
  • Start in detached mode: docker-compose up -d
  • Stop services: docker-compose down

• Image Security
  • Always use trusted base images (official images from Docker Hub).
  • Keep images updated to avoid vulnerabilities.
• Run Containers with Least Privilege
  • Avoid running containers as root unless absolutely necessary.
  • Use USER in Dockerfile to specify non-root user.
• Seccomp, AppArmor, and SELinux Profiles
  • Use security profiles to limit the system calls available to containers.
• Docker Content Trust (DCT)
  • Enable Docker Content Trust (DOCKER_CONTENT_TRUST=1) to ensure that only signed images are used.

• Docker in CI/CD Pipeline
  • Docker is a critical part of modern CI/CD workflows as it ensures that the code works in the same environment from development to production.
• Common CI/CD Tools
  • Jenkins: Integrate Docker within Jenkins pipelines to automate testing and deployment.
  • GitLab CI/CD: Define docker as a service in .gitlab-ci.yml file for running tests in isolated environments.
• Example Jenkins Pipeline

  pipeline {
      agent any
      stages {
          stage('Build') {
              steps {
                  script {
                      docker.build('myapp')
                  }
              }
          }
          stage('Deploy') {
              steps {
                  script {
                      docker.image('myapp').push()
                  }
              }
          }
      }
  }

• What is Docker Swarm?
  • Docker Swarm is Docker's native orchestration tool, which allows you to deploy and manage a cluster of Docker nodes (machines).
• Creating a Swarm Cluster

  docker swarm init
  docker swarm join --token <token> <manager-node-ip>:2377

• Deploying Services in Swarm

  docker service create --name webserver -p 8080:80 nginx

• Scaling Services in Swarm

  docker service scale webserver=5

• Docker Stats
  • View real-time resource usage: docker stats
• Docker Logs
  • View container logs: docker logs <container_id>
• Docker Events
  • Stream Docker events for troubleshooting: docker events

• AWS Elastic Container Service (ECS)
  • Manage Docker containers at scale with ECS.
• Google Kubernetes Engine (GKE)
  • Run Docker containers with Kubernetes on Google Cloud.
• Azure Container Instances (ACI)
  • Deploy and manage Docker containers without the need for orchestration.

• Immutable Infrastructure: Treat Docker images as immutable and ensure reproducible builds.
• Layer Caching: Use Docker layer caching to speed up builds. Order your Dockerfile commands efficiently.
• Version Control for Dockerfiles: Store Dockerfiles in version control systems (e.g., Git).
• CI/CD Integration: Automate Docker build and push in your CI/CD pipeline.
• Logging & Monitoring: Use centralized logging and monitoring systems (e.g., ELK stack, Prometheus, Grafana) for Docker containers.

Creating an optimized and maintainable Dockerfile involves following a set of best practices to ensure your Docker images are efficient, secure, and easy to manage. Here are some of the key best practices for writing a good Dockerfile:

• Start with a minimal base image that includes only what is necessary for your application. For example, use alpine or debian for smaller images.
• For example, use:

  FROM python:3.9-slim

  instead of:

  FROM python:3.9

• Docker caches each layer of your image. Place frequently changed instructions, like COPY or ADD, towards the end to maximize cache usage for earlier steps (such as installing dependencies).
• Group together instructions that are less likely to change to avoid unnecessary rebuilds.

Example:

# Install dependencies
COPY requirements.txt /app/
RUN pip install -r /app/requirements.txt

# Copy the rest of your application
COPY . /app/

• Each instruction in the Dockerfile creates a new layer. Try to minimize the number of layers by combining related commands into a single RUN statement when possible.

Example:

# Good
RUN apt-get update && apt-get install -y \
    package1 \
    package2 \
    package3

This reduces the number of layers by combining the commands into one, compared to:

RUN apt-get update
RUN apt-get install -y package1
RUN apt-get install -y package2
RUN apt-get install -y package3

• Use a .dockerignore file to exclude unnecessary files and directories (e.g., .git, node_modules, test directories) from being copied into the Docker image. This reduces build time and image size.

Example .dockerignore:

.git
node_modules
*.log

• Always use official or well-maintained images as base images.
• Regularly update the base image and installed packages to ensure you're not running outdated software with security vulnerabilities.
• Use specific versions of dependencies rather than latest to ensure consistency.

Example for using specific tags:

FROM node:16-alpine

• Remove unnecessary files or caches after installing packages (e.g., clean up apt cache or package manager temporary files) to reduce the image size.

Example:

RUN apt-get update && apt-get install -y package1 package2 \
    && rm -rf /var/lib/apt/lists/*

• For complex builds, especially those involving compilation, use multistage builds to separate the build environment from the production environment. This reduces the size of the final image by excluding unnecessary build dependencies.

Example:

# Stage 1: Build stage
FROM golang:1.16-alpine as builder
WORKDIR /src
COPY . .
RUN go build -o myapp

# Stage 2: Production stage
FROM alpine:3.13
WORKDIR /app
COPY --from=builder /src/myapp /app/
CMD ["/app/myapp"]

• By default, Docker containers run as the root user, which is a security risk. Always create and switch to a non-root user if your application does not need root privileges.

Example:

RUN useradd -m myuser
USER myuser

• Use ENTRYPOINT for commands that always need to run when the container starts, and CMD for default arguments. If both are defined, the CMD will be passed as arguments to the ENTRYPOINT.

Example:

ENTRYPOINT ["python", "app.py"]
CMD ["--help"]

• Use LABEL instructions to add metadata to your images. This can be helpful for organizing, tagging, and managing images.

Example:

LABEL maintainer="[email protected]"
LABEL version="1.0"
LABEL description="This is a Python app"

• Use COPY to copy files from your local context, and reserve ADD for scenarios where you need features like unpacking a tarball or accessing a URL.

# Good: Simple copy of files
COPY . /app/

# Bad: Use ADD only when necessary
ADD . /app/

• Always comment on important steps in your Dockerfile to make it easier for others (and yourself) to understand and maintain the file.

Example:

# Install Python dependencies
COPY requirements.txt /app/
RUN pip install -r /app/requirements.txt

• If possible, add a HEALTHCHECK to verify that your container is running properly.

Example:

HEALTHCHECK CMD curl --fail http://localhost:8000/ || exit 1

• Make sure your Dockerfile and the app behave consistently across different environments. This includes things like environment variables and configuration files.
• Set sensible defaults using ENV for environment variables when possible.

Example:

ENV NODE_ENV production
ENV APP_PORT 8080