Current procedure for fetching and importing Docker images

All the required microservices for the TSD docker VM are automatically built by the GitHub-DockerHub CI/CD framework and made available in the publicly available global Docker Hub (https://hub.docker.com). Since TSD does not allow direct internet access, and thus a docker pull command is not allowed, we need to import them manually into TSD following these steps:

1. Pull images to a system outside TSD.
2. On this local system, do a docker save of the fetched images into a set of tgz files.
3. Upload the tgz files using the regular TSD import mechanisms (data.tsd.usit.no or any of the API clients).
4. The tgz files are then available on the inside in the /tsd/pxx/data/durable/file-api folder.
5. From a command line in the Docker VM, run the necessary docker load command to load the images into the Docker installation.

The actual microservice Docker images to load

• Ingestion/Data-In pipeline microservices (current prod) - Python version
• Ingestion/Data-In pipeline microservices (future prod) - GO version
• Data Out API - Java service

Possible future procedure for directly importing docker images

In the future, we could aim to use UiO's locally hosted Docker Registry instead of the global one. This is planned to be trusted by TSD, so that we can then load the images directly from that Hub instead, and avoid the manual export/import/load cycle above.