Docker storage volume - ghdrako/doc_snipets GitHub Wiki

The only time for attaching storage to containers is during initial creation, using options in the docker run command. There’s no going back to edit or add it later

Docker offers various strategies to persist the data outside image.

• tmpfs mounts
• secrets (podman)
• Bind mounts
• Volumes

Docker supports two types of mounts that are specific to the operating system used by the host system:

• tmpfs (or “temporary filesystem”) - are available when running Docker on Linux
• Named pipes are available on Docker for Windows, but they are typically not used in K8s

tmpfs creates a mount in a tmpfs, which is a temporary file storage facility. The directories mounted in tmpfs appear as a mounted filesystem but are stored in memory, not to persistent storage such as a disk drive.

tmpfs mounts are limited to Docker containers on Linux. A tmpfs mount is temporary and the data is stored in Docker’s hosts memory. Once the container is stopped, the tmpfs mount is removed and the files written to the tmpfs mount are lost.

docker run -it --name docker-tmpfs-test --tmpfs /tmpfs-mount ubuntu bash
docker inspect docker-tmpfs-test | jq ".[0].HostConfig.Tmpfs"

docker run -it --rm -d --name web -tmpfs /var/log/nginx nginx

tmpfs mounts are best for containers that generate data that doesn’tneed to be persisted and doesn’t have to be written to the container’s writable layer.

Tmpfs mounts are useful for applications that are written to persist a relatively small amount of data, especially sensitive data that you don’t want written to the host filesystem. Because the data is stored in memory, there is a side benefit of faster access.

Bind mount pozwala na podłączenie katalogu lub pliku z systemu hosta do kontenera. Używa się go głównie wtedy, gdy kontener wymaga bezpośredniego dostępu do plików na hoście.

docker run -v /ścieżka/na/hoście:/ścieżka/w/kontenerze -d nazwa_obrazu

Bind mounts map files and directories in containers to files and directories on hosts.

In bind mounts, the file/directory on the host machine is mounted into the container. In contrast, when using a Docker volume, a new directory is created within Docker’s storage directory on the Docker host and the contents of the directory are managed by Docker. We specify a bind mount with the --volume or -v option and the local filesystem path and container path to use

docker run -it --name bind-mount-container -v $HOME:/host-home ubuntu bash
docker inspect bind-mount-container | jq ".[0].Mounts"

docker run -it --rm -d --name web -v ~/site-content:/usr/share/nginx/html nginx 

Bind mounts are of immense help and are most often used during the development phase of an application. By having bind mounts, you can prepare the application for production by using the same container as production while mounting the source directory as a bind mount. This allows developers to have rapid code-test-iterate cycles without requiring the need to rebuild the Docker image.

If the local path directory does not already exist, the Docker runtime will create it. Docker allows you to create bind mounts with read-only or read-write permissions. Because the volume is represented as a directory, the application running in the container can put anything that can be represented as a file into the volume - even a database.

docker run -it --name read-only-bind-mount -v $HOME:/host-home:ro ubuntu bash # create a bind mount with a read-only option so that the directory
is mounted read-only

Using bind mounts is not suitable for a production environment since this leads to a container being dependent on a file being present in a specific host. This might be fine for a single machine deployment, but production deployments tend to be spread across multiple hosts. Another concern is the potential security hole that is presented by opening up access from the container to the host filesystem.

Wolumeny Dockera są lepszym rozwiązaniem dla przechowywania danych, ponieważ są zarządzane przez Dockera i mogą być współdzielone między kontenerami.

Docker Volumes are similar to bind mounts in some regards: each persists data outside the container’s union filesystem. The differences lie in capabilities and management. While bind mounts are directories on the host filesystem, Docker Volumes are Docker objects and must be created ahead of time,5 adding a step to container creation. The default location for Docker Volumes is the host’s private /var/lib/docker area. Using the /var/lib/docker for storage was one of the indictments against union filesystems, though! If volumes use this location, we’re back in the same boat, putting data in a limited destination! But that’s merely the default location. We have the opportunity to create volumes on the local filesystem, just like bind mounts. Volumes can also reference storage beyond the reach of binds, including Object Storage buckets on cloud sources.

Docker Volumes are objects in the Docker environment, managed through the Docker CLI, primarily the docker volume command.

Volumes created “on the fly” when calling docker run are called anonymous volumes. They’re assigned uniquely generated names, making them more difficult to associate with specific containers. Recommended to use named volumes.

Docker returns an error if you attempt to remove a volume that’s used by any containers—whether or not they’re running. The same isn’t true of manually managed directories!

Recommended method of persisting data stored in containers.Docker volumes are created and completly managed by Docker under a specific directory on the host filesystem.

• Volumes are easier to back up or transfer than bind mounts.
• Volumes work on both Linux and Windows containers.
• Volumes can be shared among multiple containers without problems.

The advantage of using Docker volumes is that Docker manages the filesystem access for containers, which makes it much simpler to enforce capacity and security restrictions on containers.

docker volume create --opt type=none --opt o=bind --opt device=/oradata

This volume is visible through Docker’s CLI and uses a directory on the host I can navigate with regular Linux commands

Opcje:

• --opt type=none → Określa typ systemu plików jako "none", co oznacza, że Docker nie używa standardowego systemu plików dla wolumenów.
• --opt o=bind → Mówi Dockerowi, aby używał bind mount, zamiast domyślnego mechanizmu wolumenów.
• --opt device=/oradata/ → Wskazuje ścieżkę katalogu na hoście, który ma być powiązany jako wolumen Dockera.

Docker exposes the Volume API as a series of subcommands. Theccommands are as follows:

• docker volume create
• docker volume inspect
• docker volume ls
• docker volume prune
• docker volume rm

docker volume create --name=<name of the volume> --label=<any extra metadata>
docker volume create --name=nginx-volume
docker volume create site-content  # If no name is specified, Docker assigns a random name

After creation, the resulting volume is available to mount in a container using the form -v VOLUME-NAME:CONTAINER-PATH.

docker volume inspect <volume-name>
docker volume inspect nginx-volume

docker volume create example-volume
docker volume ls --filter="name=example-volume" --format='{{json .}}'
# mount volume
docker run -it --rm --name example-volume-container --mount source=example-volume,target=/storage bash
# mount volume to anoter container
docker run -it --rm --name second-volume-container --mount source=example-volume,target=/storage bash
# mount volume as read-only
docker run -it --rm --name read-only-volume-container --mount source=example-volume,target=/storage,readonly bash

This command is useful when you want to copy/move/take a backup of a volume. The mount path property lists the location on the Docker host, which is where the file containing the data of the volume is saved.

docker volume ls

Docker considers volumes that are not used by at least one container unused. Since unused volumes can end up using a considerable amount of disk space, it’s not a bad idea to run the prune command at regular intervals, especially on local development machines. You can append --force to the end of command, which will not ask for confirmation of deletion when the command is run

docker volume prune

Docker will not remove a volume that is in use and will return an error. Even if the container is stopped, docker will consider the volume to be in use.

docker volume rm <name>
docker volume rm nginx-volume

Docker volume can be mounted in more than one container at once. Docker also supports a --mount syntax which allows you to specify the source and target folders more explicitly.

docker run --name container-with-volume -v data:/data ubuntu
docker inspect container-with-volume | jq ".[0].Mounts"
docker volume create info
docker volume inspect info
docker run -it --name info-container -v info:/container-info ubuntu bash
docker run -it --rm -d --name web -v site-content:/usr/share/nginx/html:ro nginx   # ro - read-only access

VOLUME ["/data-volume"]

the VOLUME instruction in a dockerfile doesn’t support named volumes. as a result, when the container runs, the volume name will be an autogenerated name.

A Docker volume is the Docker host's directory mounted inside the container. It allows the container to write to the host's filesystem as if it were writing to its own.

docker volume ls # wyswietlenie wolumenow dostepnych
--volume <host directory>:<target directory>
For windows convert C: -> //c/
docker run -it -v //d/projects/docker/flask/app:/app python:alpine sh

Instead of specifying a volume with the -v flag, it's possible to specify it as an instruction in the Dockerfile, as in the following example:

VOLUME /host_directory

-- specify working direcory

docker run -it -v //d/projects/docker/flask/app:/app -w  /app python:alpine sh run.sh
docker run -it -v $(pwd):/app -w  /app python:alpine sh run.sh  # dziala z konsoli git-bash

docker run -it -v $(pwd):/app  --name flask_test -p 9090:8080 -w /app python:alpine sh run.sh  # dziala z konsoli git-bash
curl -v $(docker inspect $(docker ps -a -q -f name=flask_test) {{.NettworkSettings.IPAddress}}):8080
curl -v 127.0.0.1:9090

A very common approach to data management with Docker is to introduce an additional layer, in the form of data volume containers. A data volume container is a Docker container whose only purpose is to declare a volume. Then, other containers can use it (with the --volumes-from option) instead of declaring the volume directly. Read more at https://docs.docker.com/storage/volumes/

# as root
sudo su

# grab the size and path to the largest overlay dir
du /var/lib/docker/overlay2 -h | sort -h | tail -n 100 | grep -vE "overlay2$" > large-overlay.txt

# make sure json parser is installed 
apt-get install jq -y 

# construct mappings of name to hash
docker inspect $(docker ps -qa) | jq -r 'map([.Name, .GraphDriver.Data.MergedDir]) | .[] | "\(.[0])\t\(.[1])"' > docker-mappings.txt

# for each hashed path, find matching container name
If you navigate into the ‘/var/lib/docker/overlay2’ directory, you will  see cryptic hashed ids representing the containers layers instead of a human readable name.  This can be resolved using docker inspect and some scripting.
cat large-overlay.txt | xargs -l bash -c 'if grep $1 docker-mappings.txt; then echo -n "$0 "; fi'