Feature: Kubernetes: Registry Admin - cockpit-project/cockpit GitHub Wiki

Scope: Search/List/Delete Images in Cluster Registry

Notes

Openshift, Origin and Atomic ship with a local image registry deployed as a service in the Kubernetes cluster.

Stories

Phillip J. Fry leads IT at a moderatly sized firm. His company has a modest data center, and has deployed Atomic with Kubernetes as a way of scheduling containers in the data center.

Phillip gets a complaint that one of of the containerized applications is out of date and not running the latest code. He wants to check that the required latest image tags are present.

Amy Wong is an admin and helps with security of an Atomic cluster hardware.

An audit of the images in the cluster shows that two images have been committed with secrets embedded in their layers. Amy wants to look at these images, their metadata, and then remove them from the cluster image registry.

Scruffy is the cluster janitor. It sucks to be Scruffy, but since the automated tooling at Planet Express has not yet been implemented ... he gets to clean up ancient docker image builds that the system has left around taking up space.

Sarah is a part-time backend developer, part-time sysadmin at a mid-sized software developer consultancy. They have moved most of their development to a container-based model. A lot of the projects they are working on are not public, and she's valuing her customers privacy, so she needs a self-hosted solution. She wants full control over what individuals or what teams gets view and write access to what repositories and images. Everyone in the company is authenticated against the company's domain controller. Their development workflow go in 3 distinct phases: Development, Testing and Production. Images need to be able to move between those.

Josh and Eric are devops at Sarah's company. Jane is their team lead

Greg and Frank are on the company's QA-team. They look very similar, and people mix them up all the time.

Workflows

Phillip: Search by Name, View Repository, View Tag

Phillip clicks the sidebar "Images" button.
He sees a paginated list of the images in the registry.
There is a search box above the paginated list.
He types the name of the image of his application component, and clicks Search
Since there's only one result, and he typed the name exactly, he sees the repository details view.
The view contains:
- Tags, identified their name. For each one
  - Its digest and hash of top layer
  - ImageStream that the tagged image is part of
    - Any nested tags of that ImageStream
  - The layers that make up the tag
Phillip can identify which tags are present and what their hashes are.

Amy: Search by Digest, Delete

Phillip clicks the sidebar "Images" button.
Amy pastes a digest (or image id?) into the search box
- TODO: Work out how she indicates that this is a digest or layer id
Because she typed a digest she only sees tags
Amy examines the list, and determines that she should delete them all
She clicks into quick delete mode.
Presses the trash icon next to each of them.

Scruffy: Delete stuff

Scruffy can delete whole repositories
Scruffy can delete tags
Scruffy can delete layers

This is a brand new repository server. It's joined to the domain hackershop.lan. Sarah logs in and creates the users "Josh", "Eric" and "Jane". She creates a repository called "Skyfield", same as the projects codename. Sarah gives all 3 users read/write access to the repository Skyfield. Greg and Frank from the QA team needs access as well, in order to test the images. Sarah creates their users and adds them to the group "QA". She sets the ability for the group to have read only access to the repository.

Sarah then goes over to Jane's cubicle and let her know that everything is set up. Jane fires up a terminal and run "docker login" against the domain address images.hackershop.lan Jane logs in, tags an fedora base image (does she tag it Skylake?) and pushes it.

Jane then asks Josh and Eric the address to the server and asks them to pull the skylake image. Eric makes some rudimentary changes, tags the image and pushes it to the server.

Two weeks later Greg is test-running the images, he tweaks a config file in the image to see if it does any difference to a thing that he's failing to get running. Apparently it doesn't, so he wants to pull it again. He accidentally writes push instead of pull, but since he only has write access he's unable to push his changes to the central repository.

Wireframes

Wireframes do not yet contain deletion aspect.

Implementation Notes and Technical Limitations

The registry is not an inherent part of Kubernetes but runs as a service in pods. So we should react intelligently if it's not running.
- Perhaps offer to add it to the cluster if not present?
The Openshift Images and ImageStream resources are the main source of information about images.
The Docker v2 REST Registry API is the basis for openshift's image registry backend.
- https://docs.docker.com/registry/spec/api/
Three possible ways to connect to the image registry:
- Reconfigure docker (and restart it) to do so
- Lookup endpoint and access that address directly
- Via a REST proxy URL in the API Server https://localhost:8443/api/v1/proxy/namespaces/default/services/docker-registry:5000-tcp/v2/
Authenticating against the registry:
- Registry doesn't seem to be listening on SSL
- Without SSL, authenticating against registry requires Authentication: Basic B64(unused:token)
- Want to have a token for access. When logged in with a token, we can read from kube config.
  - However when authenticated as system:admin no token is available.
    - Ensure serivce account exists: openshift-infra/registry-admin with access system:image-pruner
    - Lookup token for this service account and use it
All images are represented as Image objects in the openshift API, along with docker information.
The digest is the key to map between openshift API and registry API.
Search is harder to implement than just filtering client side.
Suggestion: Implement a listing/filtering of Image resources
- Supplement this with info from the registry API as necessary
Can we search docker registry based on an image id? Can we find all images that have a given layer in their graph.
- Docker v2 api does not support searching, you can list repos and list tags for repo, you cannot search for repos or find all repos with layer x.
- While the docker api supports listing by image digests they aren't the same as the hashed images ids you see when doing docker images for example. The api only supports entire hashes not partial ones like docker cli commands.
- Used to implement searching for and deleting an image that had bad/private data in it.
Deletes are not yet well supported upstream. When implemented by the backend only manifests and layers can be deleted.
deleting a layer in use the complete images will not be resolvable.
Unused layers are not cleaned up automatically.
You cannot currently delete a repo.