Context

The container solution present Mion is an custom built, proof of concept stack inherited from a previous project which is totally unsuitable for use in the project for the following reasons:

• It was built and intended for use in highly constrained environments - different from the powerful hardware mion currently targets
• It deviates significantly from the container functionality and ecosystems that end users will expect (docker, k8s, etc.) and is unusable for anything but the most trivial container setup
• Is completely insecure - largely defeating the purpose of having containers in the first place
• It adds significant complexity to the system - far too much to justify, given the limited functionality
• It is currently unmaintained (no commits during the last 3 release cycles) and would be extremely difficult to maintain or extend, considering the way it was implemented and given the projects priorities and limited resources
• The custom built containers are a mess - containing a full root file systems (not lightweight containers) including GRUB and a kernel

Potential Technical Solutions

Yocto has a number of container solutions available through meta-virtualisation which are maintained upstream and might be used to replace the current solution. Each of these solutions has a different set of PROs and CONs, and might be used in different situations.

1. containerd

• core service and software system used by docker and others to provide container functionality in modern linux systems but
• not clear if it supports container networking out of the box (forgot to test this)

PROs

• Can be used on its own to install and maintain containers, either through the provided command line utility or programmatically through the provided API - Well tested and well understood
• Appears to compile and work out of the box

CONs

• More complex to use on its own (lacks the abstraction/functionality of something like docker)
• requires additional packages to be installed which increases build time and rootfs size (go gcc runc git etc.)

2. docker

The de-facto standard in modern container runtime - working (installs and runs) recipe in meta-virt

PROs

• ubiquitous, well understood, massive ecosystem - might be expected
• seems to work out of the box to install and run containers from a registry

CONs

• security? leverages lots of third party stuff that we have no control over
• possibly a number of bugs to fix - not sure how well it'll integrate with Mion

3. kubernetes (k8s)

Built on the same technologies bit (AFAIK) is intended for a different use case -

• appears to build and install out of the box from meta-virt but i didnt test that it actually works

TODO

• test that it actually works
• compare and contrast with docker (and other options)

4. podman

Similar technology to docker but a slightly different implementation

• appears to build and install from meta-virt but did not test to see if it works as expected

5. k3s - lightweight Kubernetes

• in meta-virt but not in dunfell branch - would require backporting and maintaining or wait till we have a new lts branch

Next Steps

It is clear that we need to move away from the current container implementation as it is unmaintainable and not fit for purpose.

Without clear direction from the community on which of the potential solutions might meet their needs it is difficult to make a call as to which solution we should build into mion. It is also possible that we need to build and maintain multiple solutions if this is the expectation.

1. solicit feedback from the community and partners - if they intend to use containers, which one(s) meet their needs and why?

2. based on the most popular solution(s) - we will need to start estimating the work involved in removing the current container runtime and replacing it with something else