Overview

Architecture:

Docker Command Line Interface (CLI) / Client
Docker machine / server / daemonCloud.mediawiki
Docker registry (eg: DockerHub)
Docker image
Docker container

Install

Windows

Overview:

install Follow this
check docker run hello-world

WSL

https://dev.to/bowmanjd/install-docker-on-windows-wsl-without-docker-desktop-34m9

Linux

3 ways:

snap, but you won't get last version
apt package, but you won't get last version
docker repo < = preferred version

Overview:

create group and add current user to id (if not done, would prevent you to build with Maven/Gradle)
- sudo addgroup \--system docker
- sudo adduser \$USER docker
- newgrp docker
logOut/logIN
check groups $USER give you docker
install with snap sudo snap install docker\
check sudo docker run hello-world

Problem after reboot:

check docker status: journalctl -u snap.docker.dockerd.service -b
you may get Error starting daemon: pid file found, ensure docker is not running or delete /var/snap/docker/423/run/docker.pid
remove file docker.pid
restart : sudo service snap.docker.dockerd start

Using Docker apt repository

Uninstall other versions

Check the service is not running.

docker ps
systemctl status docker

Uninstall all distributions Docker versions.

Uninstall snap version.

sudo snap remove --purge docker

Add repository

Add docker aptitude repository

Add packages

"Docker Engine on Linux, also known as Docker CE (Community Edition)" comes in 5 packages :

docker-ce : engine, as a daemon (service)
docker-ce-cli : docker CLI
containerd.io
docker-buildx-plugin
docker-compose-plugin

Hello, world

Start a test container.

sudo docker run hello-world

If you get this message, everything woks, go to next section.

Hello from Docker!
This message shows that your installation appears to be working correctly.

If you get this message.

docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.

Check docker daemon's log.

systemctl status docker

If you get this message.

oct. 09 11:23:40 octo-topi-laptop systemd[1]: Failed to start Docker Application Container Engine.

Start daemon in debug mode.

sudo dockerd --debug

network conflicts

When fixed in debug mode, start as service.

sudo systemctl start docker
systemctl status docker

Launch daemon at startup

sudo systemctl enable docker.service
sudo systemctl enable containerd.service

Run as non-root

sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker
docker run hello-world

You should get

Hello from Docker!
This message shows that your installation appears to be working correctly.

Help

Overview:

cli: on parameter docker command --help
online: Official Docker doc

Image

Overview

Iterative process:

get from registry docker pull <IMAGE_NAME>
list all docker image list
build an image docker build <BUILD_CONTEXT>, give you a image id
build an tagged image docker build -t <IMAGE_TAG> <BUILD_CONTEXT>, <IMAGE_TAG> being <DOCKER_ID>/<IMAGE_NAME>:<VERSION_NUMBER>

Connect to shell in image (to inspect content)

docker run --tty --interactive $IMAGE /bin/bash  
docker run --tty --interactive --entrypoint=/bin/bash $IMAGE

Pull

Docker use digest to download image if content has changed, even if the tag is the same, eg latest

IMAGE=<NAME:TAG>; docker pull $IMAGE

Build

Clean way

create Dockerfile
then docker build --build-arg SOME_ENV_VAR --tag <NAME>:<TAG> .

Dirty way:

get the previous (first) image
create a container
execute some command (mind the sequence to reduce unnecessary uncached rebuild)
snapshot the container into an (intermediate) image

Another dirty way:

start a container, with shell attached
execute some command in shell
open another terminal (outside docker) and get the container ID
execute docker commit -c 'CMD ["<COMMAND>"]' <CONTAINER_ID>

Structure

Validate Dockerfile Dockerfile overview:

base (image) FROM <BASE_IMAGE>, eg FROM alpine
(make AWS EBS expose network port EXPORT <PORT>, eg EXPORT 80)
command RUN, eg. RUN apk add --update redis
startup command CMD["<COMMAND>"], eg CMD["redis-server"]

# XX phase
FROM <BASE_IMAGE> (AS <PHASE_LABEL_1>)
RUN <COMMAND>
CMD ["<COMMAND>"] 

# YY phase
(FROM <BASE_IMAGE> (AS <PHASE_LABEL_2>)
COPY --from=<PHASE_LABEL_1> <SOURCE_FOLDER> <TARGET_FOLDER>
(CMD optional if base image initial command fits our needs )

Multi-stage

https://medium.com/@tonistiigi/advanced-multi-stage-build-patterns-6f741b852fae

Pass variable

https://www.baeldung.com/ops/docker-copy-variables-stages

Build phase : ARG - build argument

https://docs.docker.com/build/guide/build-args/

Supplying value in argument

Define the name

ARG FOO
FROM debian:buster-slim

Pass the value

docker build --build-arg="FOO=bar"  --tag <NAME>:<TAG> .

Supplying a reference

Pass the name and value ?

FOO=bar; docker build --build-arg $FOO  --tag <NAME>:<TAG> .

In multi-stage

Create Dockerfile

FROM alpine AS first-stage
ARG FOO
RUN echo FOO value is $FOO

FROM alpine AS second-stage
ARG FOO
RUN echo FOO value is $FOO

Build

docker build --build-arg FOO=bar --tag multi-stage-arg:latest .

Build and run phase : ENV - Environment variable

They are NOT passed in parameter in build stage. They can be static.

Environment variable is available in all stages referring the one who defined it using ENV.

They follow the usual shell interpolation rules.

FROM alpine:latest AS first-stage
ENV FOO="bar"

FROM first-stage AS second-stage
RUN echo ${FOO}

Or they can be dynamic: you can set them when running container:

by CLI, using --env parameter
reading a file, using --env-file parameter

docker run --env FOO=bar <IMAGE>
docker run --env-file .env.local <IMAGE>

But mind that docker compose may override these.

Inspect

Get sha1 from git source

docker inspect $IMAGE

[
    {
    "Id": "sha1",
    "RepoTags": [ sha2 ],

Debug

Build, keeping intermediate images, with legacy builder https://github.com/docker/buildx/issues/1104

DOCKER_BUILDKIT=0 docker build --tag multi-stage-env:latest --rm=false .

You'll get intermediate image hashes

Sending build context to Docker daemon  2.048kB
Step 1/4 : FROM alpine:latest as base
 ---> 05455a08881e
Step 2/4 : ENV FOO="bar"
 ---> Running in 1c2373a1783e
 ---> 16afb7b2fd83
Step 3/4 : FROM base
 ---> 16afb7b2fd83
Step 4/4 : RUN echo ${FOO}
 ---> Running in 29af56a6189a
bar
 ---> b1cee72a8376
Successfully built b1cee72a8376
Successfully tagged multi-stage-env:latest

Then run a container at the step it failed

docker run --tty --interactive 16afb7b2fd83 /bin/sh
echo $FOO

Base image

Linux :

alpine
alpaquita
debian

Java

Liberica : https://github.com/bell-sw/Liberica/blob/master/docker/repos/liberica-openjdk-alpine-musl/README.md

Liberica use busybox : https://busybox.net/downloads/BusyBox.html

docker run --tty --interactive bellsoft/liberica-openjre-alpine:21 /bin/sh

Container

Overview

List

rename: docker rename $FROM $TO
get a shell start -a busybox sh
(create and) run a container create
start (an attached) container start -a
copy a file: <code> docker cp

List:

create a container docker create <IMAGE_NAME>
create an container from an image, and start the container docker run <IMAGE_NAME>
create an container from an image, start the container, run a command docker run <IMAGE_NAME> <COMMAND>
(re) start a container
- detached docker start <CONTAINER_ID>
- attached to current shell docker start -a <CONTAINER_ID>
- attached to current shell, executing a command docker start -a <CONTAINER_ID> <COMMAND>
send a command to a running container
- staying detached docker exec <CONTAINER_ID> <COMMAND>
- attach beforehand docker exec -it <CONTAINER_ID> <COMMAND>
get logs docker logs <CONTAINER_ID> <COMMAND>
attach container to current terminal docker attach <CONTAINER_ID> <COMMAND>
stop a container
- gracefully docker stop <CONTAINER_ID>
- forcefully docker kill <CONTAINER_ID>
list containers
- active docker ps
- all docker ps --all
- format output (here port only): docker ps --format '{{.Ports}}'

start

docker run $IMAGE_NAME --name $CONTAINER_NAME

Be aware of entry-point which should behave as init in Linux systems :

does it forward signals ?
does it terminate on SIGTERM (no, by default for entry-points) ?
does it reap zombie children (process whose parents have died) ?

https://engineeringblog.yelp.com/2016/01/dumb-init-an-init-for-docker.html

You can use :

dumb-init
tini using --init parameter

Attach volumes

docker run --volume $OS_PATH:$CONTAINER_PATH $IMAGE

Peek into it

Running container

https://www.digitalocean.com/community/tutorials/how-to-use-docker-exec-to-run-commands-in-a-docker-container

Send a command to running container

docker exec exec --workdir /tmp $CONTAINER_ID pwd

Attach to a running container and send a command

docker exec --interactive --tty $CONTAINER_ID

Stopped container

In case you apply some configuration to your image and it crashed, running the image will not give you the same context.

Create an image from stopped container and run it

docker commit $CONTAINER_NAME test
docker run --interactive --tty --entrypoint=/bin/sh test

https://scriptcrunch.com/run-commands-stopped-container/

Image

Connect in shell to a new container of some image

docker run --interactive --tty --entrypoint /bin/sh $IMAGE

...as root

--user 0

Run parameter

Overview:

map network to the host docker run --publish <LOCAL_MACHINE_PORT>:<CONTAINER_PORT>
map filesystem docker run --volume <LOCAL_FOLDER>:<CONTAINER_FOLDER>
do not map filesystem (exception) docker run --volume <PRESERVE_CONTAINER_FOLDER>
map filesystem (full) docker run --volume <LOCAL_FOLDER>:<CONTAINER_FOLDER> --volume <PRESERVE_CONTAINER_FOLDER>

Lost ?

Overview:

who am I ? whoami
where am I ? head /etc/hostname
am I in a container ? head /proc/1/cgroup docker means you're in a container, you'll get / otherwise
I want to go out ! Ctrl-D or exit

Cleanup

Remove:

stop all running containers docker stop $(docker ps -q)
remove inactive containers docker rm $(docker ps -aq)
all (but docker) docker system prune

CLI

To connect to a different host, alter DOCKER_HOST variable.

Sample when using TLS for kubernetes:

export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.39.233:2376"
export DOCKER_CERT_PATH="/home/gradedjestrisk/.minikube/certs"
export DOCKER_API_VERSION="1.35"

Volume

Overview

Short: a mount point to local FS, mapped to container FS Doc

Overview from DockerDocs:

Create

Create : docker volume create

Get local FS folder: docker volume inspect <VOLUME_NAME> : you cannot

List all volumes: docker volumes ls

Remove

One volume

Remove a volume

docker volume remove <VOLUME_NAME>

Dangling

A dangling volume is a volume which was used by a container, but this container does not exists any more. The container is not stopped: it has been removed. Docker does not remove the volume by default.

Remove all dangling volumes

docker volume prune --all

All

Remove all volumes (think twice before doing it)

docker volume remove $(docker volume ls --quiet)

If volumes are used by a container (which may be stopped - the volume is not dangling)

you'll get a warning volume is in use
the volume will not be removed.

If you nevertheless want to remove them, remove all containers first (think twice before doing it)

docker remove --force `docker ps --all --quiet`

Implicit volume creation

[https://codeblog.dotsandbrackets.com/persistent-data-docker-volumes](Details here)

Short

MySQL dockerfile contains VOLUME /var/lib/mysql

At first container startup, Docker

generate a unique volume name (64 bytes long)
create new mount directory (a volume) in host FS (usually: /var/lib/docker/volumes/<VOLUME_NAME>)
copy whatever container had in /var/lib/mysql to the volume

Afterwards

will use the content from the volume, not container's FS

Full

VOLUME /var/lib/mysql creates a new volume attached to /var/lib/mysql. It behaves slightly like a regular directory mount, but actually is not quite the same. Whenever Docker sees a volume declaration, it'll generate a unique 64 byte name for it, create new mount directory (a volume) in host FS /var/lib/docker/volumes/%name%, and when container starts the first time, unlike with regular host directory mounts, it'll copy whatever container had in /var/lib/mysql to the volume, and after that will use the content from the volume, not container's FS. That has an important implication: when I create a container from newer mysql image that also has newer content in /var/lib/mysql, if the volume already exists, that new content will be ignored.

Hello, docker

docker volume create test_volume
docker run -it -v test_volume:/data alpine /bin/sh
touch /data/README.md
docker volume inspect test_volume
sudo ls <MOUNTPOINT>

Running docker in container

Docker in docker (DinD)

https://shisho.dev/blog/posts/docker-in-docker/

Be aware of:

security concerns
resource overhead

Docker in docker (DinD)

docker run --privileged --name dind -d docker:dind
docker exec -it dind sh
docker info
docker run -it --rm alpine

Docker outside Docker (DooD)

docker run -ti --rm -v /var/run/docker.sock:/var/run/docker.sock docker /bin/sh

Limit resources

Get used resources

To get CPU usage, Memory usage, Cumulated I/O (not speed)

docker stats

You'll get

CONTAINER ID   NAME         CPU %     MEM USAGE / LIMIT   MEM %     NET I/O          BLOCK I/O        PIDS
4dca1dd58e59   postgresql   0.08%     169.7MiB / 512MiB   33.14%    9.13kB / 4.8kB   872MB / 5.62GB   6

You can get metrics from file using container id

cat /sys/fs/cgroup/system.slice/docker-$ID.scope/cpu.stat

https://docs.docker.com/engine/containers/runmetrics/#cpu-metrics-cpuacctstat

Glances displays:

IO speed: IOR/s IOW/s
network speed: Rx/s Tx/s

Memory usage and CPU

Use --memory and --cpu parameters

docker run --rm       \
  --memory 512m       \
  ubuntu              \
  /bin/bash -c "grep MemTotal /proc/meminfo"

docker run            \
  --rm                \ 
  --memory 1G         \ 
  --cpus 1            \ 
  node:12.14.1-alpine \
  node -e 'console.log(require("v8").getHeapStatistics().heap_size_limit/1e6);'

You can change the value on-the-fly

docker update --cpuset-cpus "7" postgresql

https://docs.docker.com/reference/cli/docker/container/update/

I/O

Get device

sudo lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL

Will give you

nvme0n1              476,9G                                     
├─nvme0n1p1 vfat       512M /boot/efi                           
└─nvme0n1p2 ext4     476,4G /

Locate the device and append /dev, eg. /dev/nvme0n1 to get $DEVICE.

Then add --device-write-bps $DEVICE:$LIMIT

Compare both versions writing 1Gb to disk, limit to 50Mb/s and no limit

docker run -it --rm --device-write-bps /dev/nvme0n1:50Mb ubuntu /bin/bash -c "time dd if=/dev/zero of=test.out bs=1M count=1024 oflag=direct"
docker run -it --rm ubuntu /bin/bash -c "time dd if=/dev/zero of=test.out bs=1M count=1024 oflag=direct"

Source

Hello, world (hands-on)

Stateless?

Steps

think about container instance
- start an attached container with a shell
- create a file touch test.txt
- exit
- start an attached container with a shell
- ls to check if file exists
think about shell
- create a (background detached) container with a shell run -td -e FOO=bar busybox sh
- in a terminal, get a shell to the running container
- check FOO is bar and change its value FOO=barbar
- in another terminal, get a shell to the running container
- check FOO </code>
think about fs
- gentle way
  - create a (background detached) container with a shell run -td busybox sh
  - get a shell to the running container
  - create a file touch test.txt
  - exit
  - get a shell to the running container
  - ls
- forceful way
  - create a (background detached) container with a shell run -td busybox sh
  - get a shell to the running container
  - remove everything you can rm -rf /
  - try to ls
  - exit
  - get a shell to the running container

Run interactive one-shot container

Steps:

mkdir docker-java
cd docker-java
vi Dockerfile

 FROM java
 CMD java -version

docker build -t docker-java .
docker image ls
docker run -it docker-java

Run background container

Run background container (detached mode), -d, mapping ports (-p) between container and host:

docker run -d -p 80:8080 -p 443:8443 jetty wil give you a container id
look for container using docker ps

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                         NAMES
60aad7967123        jetty               "/docker-entrypoint.…"   13 seconds ago      Up 10 seconds       0.0.0.0:80->8080/tcp, 0.0.0.0:443->8443/tcp   flamboyant_hopper

check jetty's feedback on localhost:80

Make it stateful: use volumes

Integrate Docker with Maven/Gradle

Instead of building a java package, include it in a dedicated java Docker image:

git clone https://github.com/arun-gupta/docker-java-sample.git
cd docker-java-sample
build image mvn package -Pdocker
(you'll see the Dockerfile generated in log Step 1/5 : FROM openjdk:latest)
run container mvn install -Pdocker and check you get

[INFO] DOCKER> [hellojava:latest]: Start container 6038891f7f62
603889> Hello World!

check image

$ docker image ls
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
hellojava           latest              1a1f4f90d06c        9 minutes ago       986MB

check for container

$ docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS               NAMES
6767abd0e99b        1a1f4f90d06c        "/docker-java-sample…"   35 seconds ago      Exited (0) 33 seconds ago                       xenodochial_northcutt
(..)

Docker - GradedJestRisk/cicd-training GitHub Wiki

Overview

Install

Windows

WSL

Linux

Using Docker apt repository

Uninstall other versions

Add repository

Add packages

Hello, world

Launch daemon at startup

Run as non-root

Help

Image

Overview

Connect to shell in image (to inspect content)

Pull

Build

Structure

Multi-stage

Pass variable

Build phase : ARG - build argument

Supplying value in argument

Supplying a reference

In multi-stage

Build and run phase : ENV - Environment variable

Inspect

Debug

Base image

Container

Overview

start

Attach volumes

Peek into it

Running container

Stopped container

Image

Run parameter

Lost ?

Cleanup

CLI

Volume

Overview

Create

Remove

One volume

Dangling

All

Implicit volume creation

Short

Full

Hello, docker

Running docker in container

Docker in docker (DinD)

Docker outside Docker (DooD)

Limit resources

Get used resources

Memory usage and CPU

I/O

Hello, world (hands-on)

Stateless?

Run interactive one-shot container

Run background container

Make it stateful: use volumes

Integrate Docker with Maven/Gradle

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️