Docker compose - GradedJestRisk/cicd-training GitHub Wiki

https://docs.docker.com/compose/install/

Overview:

• all services share same network (log is Creating network (..) with the default driver )
• network aliases are created based on service's names and can be used anywhere in application source code
• commands ** start containers: docker-compose up ** start containers: docker-compose up -d ** check docker-compose.ymlk is valid docker-compose config ** build image, start container: docker-compose up --build ** list running containers docker-compose ps ** execute a command docker-compose exec <SERVICE_NAME> ** stop containers: docker-compose down

docker-compose.yml

Lint

<SERVICE_NAME>
   container_name : <CONTAINER_NAME> (optionnal)
   hostname: <HOST_NAME> (optional)
   build:
      context:    <BUILD_CONTEXT>
      dockerfile: <DOCKER_FILE>
   ports:
      - "<LOCAL_PORT>:<CONTAINER_PORT>" 
   volumes:
      -                 <PRESERVE_CONTAINER_FOLDER>
      - "<LOCAL_FOLDER>:<CONTAINER_FOLDER>" 
   environment:
      -<ENVIRONMENT_VARIABLE_NAME>=<<ENVIRONMENT_VARIABLE_VALUE>
   command: ["<COMMAND", "<PARAMETER_1>", "<PARAMETER_2>"]  

See here to get working example.

Syntax :

• $VARIABLE
• ${VARIABLE}
• "$$VARIABLE" to prevent interpolation and get $VARIABLE so it will be interpolated somewhere else

Default :

• ${VARIABLE:-default} evaluates to default if VARIABLE is unset or empty
• ${VARIABLE-default} evaluates to default only if VARIABLE is unset
• ${VARIABLE:-${FOO}} evaluate to $FOO if $VARIABLE is unset Mandatory :
• ${VARIABLE:?message}: if VARIABLE is unset or empty, exits with message
• ${VARIABLE?err}: if VARIABLE is unset, exits with message

https://docs.docker.com/compose/compose-file/12-interpolation/

Mind the precedence ! Having any ARG or ENV setting in a Dockerfile evaluates only if there is no Docker Compose entry. https://docs.docker.com/compose/environment-variables/envvars-precedence/

The order of precedence (highest to lowest) is as follows:

• docker compose run --env FOO=bar
• Substituted from your shell

<SERVICE_NAME>
   environment:
   - FOO=${FOO}

• Set using just the environment attribute in the Compose file

<SERVICE_NAME>
  environment:
    - FOO=BAR

• Use of the --env-file argument in the CLI

`docker compose --env-file .env

.env

FOO=bar

• Use of the env_file attribute in the Compose file

<SERVICE_NAME>
  env_file:
    - local.env

local.env

FOO=bar

• Set using an .env file placed at base of your project directory .env

FOO=bar

• Set in a container image in the ENV directive

ENV FOO=BAR
FROM alpine

https://docs.docker.com/compose/environment-variables/set-environment-variables/

You can read them from .env file

POSTGRES_DB=somedb

Reference them with ${} as regular environment variable docker-compose

environment:
  POSTGRES_DB: "MY-${POSTGRES_DB}"

Check result

docker compose config

If you want to use

• more than one .env file
• name it differently or store it elsewhere Then you can supply their name and mandatory status in the service

https://docs.docker.com/compose/environment-variables/set-environment-variables/#additional-information

<SERVICE_NAME>
    env_file:
    - path: ./default.env
      required: true # default
    - path: ./override.env
      required: false

You can always override these settings using CLI --env-file

If no name is given, the name will be $PROJECT_$SERVICE_1.

The project name comes from:

• COMPOSE_NAME environment variable;
• --project-name <PROJECT_NAME> parameter;
• folder name.

You can override this name by supplying a container_name element, but beware it cannot be scaled.

You can express dependencies: container A needs container B to start successfully.

Short form : service-a will start as soon as service b is started

  service-a :
    depends_on:
      - service-b

Long form : service-a will start as soon as s

• service b has started and exist successfully
• service c has started and healthcheck has completed successfully

  service-a:
    depends_on:
      service-b:
        condition: service_completed_successfully
      service-c:
        condition: service_healthy

example

Reuse a service definition from another file

<SERVICE_NAME>
    extends:
      file: <COMPOSE_FILE_PATH>
      service: <SERVICE_NAME>

While you can override some attributes, if something is not allowed you'll get a nice Go compilation error DOC

docker compose pull

Doc

In docker-compose.yml

Policy:

• always
• missing
• never https://github.com/compose-spec/compose-spec/blob/master/spec.md#pull_policy

!! The spec exists but does not seem to be implemented https://stackoverflow.com/questions/37685581/how-to-get-docker-compose-to-use-the-latest-image-from-repository

docker compose up --detach <OPTIONAL_SERVICES_NAME_LIST>

Doc

Pull (if needed - and this probably not work), build, start

docker compose up --pull missing --build <SERVICE>

--wait option: wait for ALL services in file to be healthy before returning. You'll need an healthcheck entry for each service.

Stateless: if an anonymous container is used, you can drop it using --renew-anon-volumes

Anonymous volumes are :

• reused if the service is stopped using docker compose stop <SERVICE>;
• discarded if the service is stopped using docker compose down.

See https://github.com/docker/compose/issues/7444

Containers lifecycle is influence by compose file. If it has changed, new containers will be created unless you provide --no-recreate If it has not changed, but you want fresh containers, use --force-recreate

Define healthcheck command: it will be run in the container

• postgresql : PGPASSWORD=<PASSWORD> psql -U <USER_NAME> -d <DATABASE_NAME>
• http (if wget is available) : sh -c "wget -q -O- http://localhost:8080/health"
• keycloak : ``

Test it: it should return 0 when the service is healthy

docker exec --tty --interactive <CONTAINER_NAME> bash
<COMMAND>
echo $?

Check how much time it take usually and enter it as start_period

In docker-compose.yml

• add healthcheck property to the service you want to wait for

    healthcheck:
      test: "PGPASSWORD=permissions psql -U permissions -d permissions_db"
      start_period: 1min
      interval: 10s
      timeout: 10s
      retries: 5

• add depends_on property to the service that will wait

    depends_on:
      <SERVICE_NAME>:
        condition: service_healthy

If no service is to wait, you can use hello-world image.

services:
  wait:
    image: hello-world
    container_name: wait   
    depends_on:
      <SERVICE_NAME>:
        condition: service_healthy

Then test

docker compose up --detach

If your service does not respond, you'll get an timeout error

 ✘ Container <CONTAINER_NAME>        Error                         162.8s   
 ⠦ Container wait                    Recreated                     152.5s 
dependency failed to start: container <CONTAINER_NAME> is unhealthy

If the logs show that it has started successfully, you can debug healthcheck

docker inspect --format "{{json .State.Health }}" <CONTAINER_NAME> | jq

You'll get

> 

{
  "Status": "starting",
  "FailingStreak": 1,
  "Log": [
    {
      "Start": "2024-02-28T15:58:54.948143982+01:00",
      "End": "2024-02-28T15:58:55.420688109+01:00",
      "ExitCode": 1,
      "Output": ""
    }
  ]
}

    deploy:
      restart_policy:
        condition: on-failure
        delay: 30s
        max_attempts: 10
        window: 5s

condition:

• "no"
• on-failure
• unless-stopped
• always

Doc

docker compose down 

Use stop.

Then you can :

• start it again start;
• restart it restart;
• delete if rm.

https://docs.docker.com/compose/compose-file/deploy/#resources

You can restrict memory and RAM, but not I/O https://superuser.com/questions/1306172/limit-usage-of-disk-i-o-by-docker-container-using-compose

Use deploy/resources/limits property

     deploy:
        resources:
           limits:
              cpus: '1'
              memory: 128m

In brief:

• Compose sets up a single (docker) network, each container join it.
• each container is discoverable on this network, by the swarm's members, using the service's name.
• containers are NOT discoverable outside the swarm (by the host, running dockerd) - except if exposing ports using ports mapping.

So you shouldn't use container_name or hostname property to pin down host name.

Long version

By default Compose sets up a single network for your app. Each container for a service joins the default network and is both reachable by other containers on that network, and discoverable by the service's name.

Your app's network is given a name based on the "project name", which is based on the name of the directory it lives in. You can override the project name with either the --project-name flag or the COMPOSE_PROJECT_NAME environment variable.

https://docs.docker.com/compose/how-tos/networking/

You can also give a custom host name in hostname property, so you can avoid using localhost to refer to it.

This will enforce 12-factor app rules.

Make the tool gap small (in Dev/Prod parity)

This will NOT resolve name from your OS, only from containers. To do it from your OS, you can use /etc/hosts

127.0.01 $HOST_NAME

Name services using kebab-case: my-service.

Publish as few ports as possible in ports, prefer not exposing them on host unless mandatory, as this impose that all ports are unique. But always declare ports used by your application using expose.

Use .env file to store common values.

Do NOT use container_name or hostname property (display will be less readable in docker ps, but you will not be annoyed when using the same name between project, as docker will refuse to start a service with the same name, unless you remove the container first).

Add healthchecks and use --wait on up.

If you need to start an application not in swarm, use hello-world with dependencies.

services:

  dummy-service-to-get-all-working-properly:
    image: hello-world
    depends_on:
     service-with-healthcheck:
        condition: service_healthy
     service-without-healthcheck:
        condition: service_started

Then start.

docker compose up dummy-service-to-get-all-working-properly

You will soon need these options also.

--detach --remove-orphans --force-recreate