The local Docker registry - gipert/ECM GitHub Wiki

HTCondor needs to download Docker images from a Docker registry, which is currently configured on the nfs+docker-server instance using a container, distributed by Docker, called registry. On nfs-docker-server it is currently running with the following command:

$ sudo docker run --restart=on-failure:10 \
  -d -p 5000:5000 \
  -e standalone=True \
  -e disable_token_auth=True \
  -v /dataNfs/.docker-local-registry:/var/lib/docker \
  -v REGISTRY_STORAGE_DELETE_ENABLED=true \
  --name gerda-docker-hub-open \
  registry

Let's provide a bit of explanation. The --restart=on-failure:10 flag will cause the Docker service to attempt to restart the registry up to 10 times should it exit for any reason. This way your container will automatically start up at boot. The -d flag runs the container registry in a detached mode, while -p 5000:5000 means "map port 5000 on the host to port 5000 in the registry container. We will use it later to access the registry. The -e standalone=True -e disable_token_auth=True flag runs registry in stand-alone mode, while the -v /dataNfs/docker_reg:/tmp/registry flag sets the storage path of the images in the external volume. A name for the container is also set with the --name flag.

Finally, you should also let the other instances in the cluster access the 5000 port by editing the security group rules of the nfs-server's security group. Open it e.g. for the "default" security group.

Deploying an image on the hub

The gerda-master instance can be used do build and push the images on the hub. The reference Dockerfiles for the images with the GERDA software are stored in this repository. You can build a image with the command:

$ sudo docker build . -t 10.64.28.50:5000/gerda-sw-all:<tag>

and then push it to the local hub with:

$ sudo docker push 10.64.28.50:5000/gerda-swall:<tag>

The images must be tagged with 10.64.28.50:5000/gerda-sw:<tag> at the beginning to let Docker know where to push them (in this specific case, the same machine through the port 5000). Don't worry if you forgot to tag your image with the docker build command, you can later tag it with the docker tag command.

Inspecting the Docker hub

Type

curl http://10.64.28.50:5000/v2/_catalog

in a cluster machine to see the available images, then curl http://10.64.28.50:5000/v2/<image_name>/tags/list to list the tags available for a specific image <image_name>.

Deleting images in the Docker hub

Setting the REGISTRY_STORAGE_DELETE_ENABLED=true variable is essential here.

Get the manifest for <name>/<tag>:

curl -v --silent -H "Accept: application/vnd.docker.distribution.manifest.v2+json" \
     -X GET http://localhost:5000/v2/<name>/manifests/<tag> 2>&1 \
     | grep Docker-Content-Digest | awk '{print ($3)}'

you will get a sha256:<ID> string

Mark it for deletion:

curl -v --silent -H "Accept: application/vnd.docker.distribution.manifest.v2+json" \
     -X DELETE http://127.0.0.1:5000/v2/<name>/manifests/sha256:<ID>

Finally delete it:

sudo docker exec -it gerda-docker-hub-open bin/registry garbage-collect /etc/docker/registry/config.yml

Restart the registry container:

sudo docker restart gerda-docker-hub-open

Here's a complete script:

#!/bin/bash
#
# USAGE: ./delete-image <name> <tag>
#
id=$(curl -v --silent -H "Accept: application/vnd.docker.distribution.manifest.v2+json" \
          -X GET http://localhost:5000/v2/$1/manifests/$2 2>&1 \
          | grep Docker-Content-Digest | awk '{print ($3)}')
id=${id%$'\r'}

curl -v --silent -H "Accept: application/vnd.docker.distribution.manifest.v2+json" \
     -X DELETE "http://127.0.0.1:5000/v2/$1/manifests/${id}"

sudo docker exec -it gerda-docker-hub-open bin/registry garbage-collect /etc/docker/registry/config.yml
sudo docker restart gerda-docker-hub-open

Running a container

Refer to the README.md of this repository (note: remember to use the <ip-addr>:<port>/gerda-sw:<tag> syntax).

To display graphics within a container running on a vm you can use the following script [ref], available on gerda-master as dockerX11 [IMAGE] [ARGS]:

#!/bin/bash
#
# bash script to enable x11 forwarding from inside gerda-sw docker containter
# please call it without sudo!
#
if (( $EUID == 0 ))
  then printf "Please don't run this script as root! You will be prompted for the pw later."
  exit
fi

if (( $# == 0 ))
  then echo "Usage: dockerX11 [IMAGE] [ARGS]"; exit
fi

# Prepare target env
CONTAINER_DISPLAY="0"
CONTAINER_HOSTNAME=$(echo $1 | cut -d/ -f2)

mkdir -p ${HOME}/.display/socket
touch ${HOME}/.display/Xauthority

# Get the DISPLAY slot
DISPLAY_NUMBER=$(echo ${DISPLAY} | cut -d. -f1 | cut -d: -f2)

# Extract current authentication cookie
AUTH_COOKIE=$(xauth list | grep "^$(hostname)/unix:${DISPLAY_NUMBER} " | awk '{print $3}')

# Create the new X Authority file
xauth -f ${HOME}/.display/Xauthority add ${CONTAINER_HOSTNAME}/unix:${CONTAINER_DISPLAY} MIT-MAGIC-COOKIE-1 ${AUTH_COOKIE}

# Proxy with the :0 DISPLAY
socat TCP4:localhost:60${DISPLAY_NUMBER} UNIX-LISTEN:${HOME}/.display/socket/X${CONTAINER_DISPLAY} &

# Launch the container
sudo docker run -it --rm \
  -e DISPLAY=:${CONTAINER_DISPLAY} \
  -v ${HOME}/.display/socket:/tmp/.X11-unix \
  -v ${HOME}/.display/Xauthority:/root/.Xauthority \
  --hostname ${CONTAINER_HOSTNAME} \
  $@

rm ${HOME}/.display/socket/X${CONTAINER_DISPLAY} 2> /dev/null