Container Feature Design - RENCI-NRIG/orca5 GitHub Wiki

Documenting / discussing design decisions for #115.

Container Images

Currently undecided.

  1. User must post image to central Docker Hub Registry?
  2. Images will be managed by/via ImageProxy?
  3. Allow user- or RENCI- controlled private Docker Registries?

Image Caching

It's unclear exactly how big most user container images will be, but I would expect them to be 1+GB (it takes work to make small images). This means we would probably benefit from having some sort of image caching, which could be ImageProxy (after modification) or Docker Registry cache. This will probably affect/be affected by the decision on where images can/will be pulled from.

User Access

The current plan is to restrict implementation so that deployment is for one user only per node/VM. That user will be able to run one-or-more containers on that VM, but no other user will be given access to another user's VM. This simplifies some things (SSH, port access) while still providing some security. The user will not be given root access to the VM, but they will be placed in the docker group to have full access to start and stop containers [impact on docker pull probably needs to be explored].

SSH Access

User access via SSH will be a common although maybe not universal requirement. It may not make sense to require the user to incorporate an SSH server into their docker image. It was proposed to provide SSH access directly to a container via the Host SSH server and docker exec. The most straightforward way to accomplish this might have been to modify the user's shell in /etc/passwd to reflect the desired docker exec statement.

The main problem with the /etc/passwd shell method is that a user would only be able to run one container per host.

Other Services

Any externally available services (e.g. httpd, HTCondor) running in a user container would need to have host port numbers assigned. The mapping of container port to host port would need to be managed. Orca generally tries not to manage user applications [could it be argued that Orca now is doing this when starting containers?]

It is believed that Docker port mappings are not visible from inside the container. Thus, in order for the user to manage/view the port mappings, the user will need to have direct Host access.

Cleanup

Cleanup may be slightly messy if the user has direct access to the Host, but they will not have root access so their ability to change the local file system will be limited. Do we want to consider locking down the host file system more than usual to prevent user modification?

We should be able to accomplish most cleanup by deleting the user's home directory, and cleaning up docker state -- either using docker commands to delete containers and images, or possibly just wiping /var/lib/docker. If the user is allowed write access to any other parts of the filesystem, additional cleanup may be required (/tmp, others?).

Host management

Handler stuff?

is it possible to leverage xCAT and xCAT machinery for issuing nodes and IP addresses?