Containers Overview - egnomerator/misc GitHub Wiki

Containers Overview

An overview of containers, container specifications, and container engines (with emphasis on Windows)

note: this technology is very new and constantly evolving; most references include the date of the article when relevant

What is container technology

History

Containers did not originate with Docker

Virtualization

Containers are a form of Virtualization: Operating-system-level virtualization

  • VMs virtualize hardware; containers virtualize the OS layer

Isolation

  • A container is an isolated runtime (or user space instance) for its defined process(es) to run within
    • this runtime includes all the necessary dependencies for the process(es) to run
    • this runtime prevents its running process(es) from any awareness of outside processes
  • This level of isolation is secure, but not as secure as the level of isolation provided by VMs
    • containers share the kernel of their container host
    • see Windows Hyper-V Containers for a "hybrid" version of containerization which provides VM isolation

Application Containers VS System Containers

The difference is explained nicely in this article - 10/17/2017

An application container expects a single application to execute within the container. This application may be one process or many processes, however, under this paradigm there is a single entry point process. Docker and rkt are examples of container runtimes that are designed for this approach.

A system container can support multiple applications being executed within the container. The entry point is an init system residing within the container that facilitates process management (such as SysV, Upstart, and Systemd). Systemd-nspawn and LXC/LXD are examples of container runtimes that are designed for this approach.

Container Host

This is the machine (either physical or virtual) on which the containers run.

  • the host OS must be compatible with the container OS/kernel (Linux with Linux, Windows with Windows)

Container Specifications

Below are specifications for the image and runtime for each of the more prominent container technologies

  • there is much more to these technologies than the image and runtime
  • even within the scope of containers alone, there are many components other than

Docker: Image Spec | Runtime Spec (runc)

  • Docker donated the image specs and the runc code to the OCI
  • the OCI specs are not equal to but are based off of these initially donated specs

note: don't confuse containerd with runc

  • containerd is

    a daemon for Linux and Windows, which can manage the complete container lifecycle of its host system

  • runc is used by containerd

    to run containers according to the OCI specification.

OCI: Image Spec | Runtime Spec

Open Container Initiative Specifications are 1.0 - 07/19/2017

Demystifying the Open Container Initiative (OCI) Specifications

CoreOS: Image Spec (ACI) | Runtime Spec (appc)

NOTE: reference

Note that as of late 2016, appc is no longer being actively developed, and future versions of rkt will instead natively use OCI formats.

  • this same reference also has a good overview of rkt's ACI spec
  • this "Containers Overview" wiki page discusses CoreOS rkt in terms of its pre-OCI specs (which are currently being used still at the time of this writing)

CoreOS's container specification

appc spec repository on github

  • The Spec (spec | spec files)
  • There is a disclaimer section in this repo's README file that says, as of 2016, appc is no longer being actively developed, and encourages joining the OCI community
  • here is an article where CoreOS is supporting the OCI

Containers

Container Images

Images (high level based on Docker and OCI specs)

  • What is a Container Image
    • Technical
      • Basically, a container image is a root filesystem and collection of sequential "layers" - filesystem change sets - along with runtime execution parameters
      • At a higher level, a container image can be described as a series of layers where the first layer is the base image (a.k.a. base OS image), and each layer above this base image is a set of incremental changes from the layer before it
    • Conceptual
      • A container image is a blueprint defining an execution environment
      • Similar to a class defining an object instance in OOP
  • Images and Layering (via copy-on-write strategy)
    • Start with base image - e.g. Windows Server Core/Nano, GNU/Linux distro, etc.
    • Add new layer on top of this base image and build a new image
    • Continue adding layers and building images
      • Image Build Process Example:
        • On Windows Server Core base OS image, enable IIS components, build new image
        • On IIS image, install ASP.NET MVC website, build new image
        • On ASP.NET MVC image provide execution parameters, build new image

Container Runtimes

Containers (high level based on Docker and OCI specs)

  • Final/another step in Image Build Process Example:
    • Run the image providing any necessary arguments, now there is an execution environment (a container) in which will run the process(es) specified in the container image
  • What is a Container
    • In the final step of the above image build process example, the resulting execution environment (described by the container image) is a container
    • Technical
      • Execution environment
        • The container is the actual execution environment in which will run the process(es) specified by the container image
      • Writable Top Layer on Image
        • When a container is created, a new writable layer is created on top of the underlying layers of the image
        • After the process in the container stops running, the container (the execution environment) is destroyed
        • This "container layer" still exists though, persisting state for the next time the container is started
        • This "container layer" is preserved until the delete command is invoked
      • Given an image, a container is a running instance of that image - an isolated runtime environment on a (physical or virtual) host
      • The process(es) running within this runtime environment have no awareness of any other resources on the host
    • Conceptual
      • In the final step of the above image build process example, the resulting execution environment (described by the container image) is a container
      • So, a container is an instance of the execution environment defined by a container image blueprint
      • Similar to an object instance of a class in OOP
  • Isolation
    • The host isolates a container via namespace isolation and resource governance techniques
    • Containers share the host OS's kernel and libraries - no need to boot an OS or load those libraries
    • Hyper-V Containers
      • Stripped down minimal VM hosting a single container
      • Provides additional layer of isolation - regular containers may not address security concerns adequately in some cases (e.g. hostile multitenancy)
  • Space Consumption
    • Given a single container image, many containers can run on the same host taking only the additional space needed to run the contained application - due to the layering of container images
      • I.e. the many running containers can share the single common image rather than needing a duplicate image for each container

Kinds of Container Images

WSC (Windows Server Containers - including Hyper-V containers)

WSC Requirements

Windows Image File Format (WIM)

About Windows Containers

Running a container on Windows with or without Hyper-V Isolation is a runtime decision. You may elect to create the container with Hyper-V isolation initially and later at runtime choose to run it instead as a Windows Server container.

WSC with Hyper-V Isolation

  • essentially, this is a stripped-down, highly optimized VM hosting a container
  • the level of isolation provided is greater than that of regular containers, because within a Hyper-V container host a container has its own virtualized kernel
  • this isolation is designed to address the security concerns related to running containers in a hostile multitenant environment by providing the same level of isolation as VMs

WSC Version Compatibility

reference - 01/13/2018

Bottom line: the targeted OS version of a container image must be compatible with the container host's OS version Key takeaways:

  • Carefully review the version compatibility table and related explanations in the referenced article
  • Effort on container maintenance could potentially be reduced by utilizing Hyper-V isolation

note on base OS:

  • Windows containers require a base OS
  • Linux containers do not (although they normally do include a base OS)
    • see "from scratch" below

LXC (Linux Containers)

Various GNU/Linux distros

"from scratch"

Container Images Built with Container Tooling

Docker Images

These are images created via Docker tools resulting in a container image according to the Docker image spec

  • these can be built from WSC or LXC or even from "scratch"
  • Docker runs these images with their implementation of the runc runtime spec

CoreOS rkt Images

These are images created/converted via rkt tools resulting in a container image according to the ACI spec

  • rkt runs these ACI images with their implementation of the appc runtime spec

OCI

Just a note here on OCI: this is just a spec, not an actual type of image

Container Engines

note: using the term "container engine(s)" and "engine(s)" loosely

Container options

Engines with container formats they can run (not exhaustive)

engine runs LXC runs WSC runs Docker
CoreOS rkt yes Dockerized yes
Docker Dockerized Dockerized yes
LXC/LXD yes

* Dockerized = container image created as a Docker image via the Docker engine

note: CoreOS converts any non-ACI images into ACI images so it can run them (see this and this)

Engines with Host OSs they can be hosted on

engine runs on Linux runs on Windows runs on MacOS
CoreOS rkt yes Vagrant (or regular VM) Vagrant (or regular VM)
Docker yes yes yes
LXC/LXD yes VM VM

* bold yes = CentOS, Debian, Fedora, Ubuntu, more

Docker

Docker is a container platform (which includes the Docker engine) that supports hosting containers on just about any host OS.

Docker on Windows

A little history of progress:

Docker Engine on Windows

Docker is required in order to work with Windows Containers.

Currently Recommended Installation

  • Build and run your first Docker Windows Server container - 12/25/2017
    • Windows Server 2016 Installation
      • this shows a direct installation (via PowerShell) of the Docker Engine as a Windows Service on Windows Server 2016
        • clarification: this method of installation is done without Docker for Windows
    • Windows 10 Installation
      • if using above installation method for a Windows Server 2016 VM, this shows how to setup communication between Windows 10 host OS and the Docker Engine service on the guest OS
      • this also shows the installation method via Docker for Windows

CoreOS rkt

LXC/LXD

LXC Landing Page

LXD Landing Page

Others

list of OS-level virtualization implementations (some historical)

Cross Platform Containers

A container host cannot host a container of a different OS

  • Linux distros can host other Linux distros, but Windows cannot host Linux and Linux cannot host Windows

A VM has to be used to facilitate cross-platform container hosting.

Windows Containers on Linux

Haven't seen any tooling designed specifically for the purpose of facilitating the ability to run Windows containers on Linux

  • There are various generic virtualization options that can be used to facilitate this
  • Just have to install a Windows VM (via one of these generic virtualization options) on the Linux machine and run the Windows container on that VM

Linux Containers on Windows (LCOW)

Linux Containers on Windows - 11/22/2017

Hanselman running LCOW - 11/20/2017

  • a confusing quote from his article:

    no Virtual Machine or Hyper-V involved (unless you want), so Linux Containers run on Windows itself using Windows 10's built in container support.

  • actually it DOES involve Hyper-V - 11/22/2017
    • pre-LCOW architecture:

      a Hyper-V Linux VM runs a Linux Docker daemon, along with all your containers

    • LCOW architecture:

      the Docker daemon runs as a Windows process (same as when running Docker Windows containers), and every time you start a Linux container Docker launches a minimal Hyper-V hypervisor running a VM with a Linux kernel, runc and the container processes running on top.

Heterogenous deployments

Current

Combinations of various MSA technologies are required for building and maintaining deployments with both Linux and Windows infrastructure

  • Technologies such as
    • Docker Swarm, Docker Datacenter
    • Google Container Engine
    • Kubernetes
    • MS Service Fabric
    • Marathon (Apache Mesos)
    • Nomad, Otto, Vagrant (HashiCorp)

Future

Support for running Windows containers and Linux containers side-by-side on Windows is on the Docker road map

  • While Docker does not provide a way to run Windows containers and Linux containers side-by-side, it does provide a common set of tools and management interface for these different containers
  • Docker article: brief mention of future with Windows and Linux containers running in Parallel on Windows - 11/22/2017

    Because there’s only one Docker daemon, and because that daemon now runs on Windows, it will soon be possible to run Windows and Linux Docker containers side-by-side, in the same networking namespace. This will unlock a lot of exciting development and production scenarios for Docker users on Windows.

    • (this would still involve thin VMs for LXC to run on)

Resources

(some or all of these may be referenced throughout this article)

Terms

⚠️ **GitHub.com Fallback** ⚠️