What makes a well packaged Kubernetes app? - nomisbeme/acmecharts GitHub Wiki

Introduction

The Docker packaging guidelines describe how to apply 12-factor principles to generic Docker images suitable for use with any orchestrator, or with no orchestrator at all.

What's the equivalent set of best practices if I'm targeting a Kubernetes platform specifically? What is the pod/container level version of this?

Specific Questions

Health checking

Defining Liveness and readiness probes is a best practice and improves reliability of the application as a whole by enabling the scheduler to make better decisions. Kubernetes supports many types of health checks.

What constitutes a great health check?

Ref: https://www.ianlewis.org/en/using-kubernetes-health-checks

Performance monitoring

A performance monitoring solution (e.g. prometheus, dyantrace, datadog) is a requirement for a business-important application running on top of Kubernetes. In addition to monitoring the Kubernetes infrastructure the monitoring tool should be aware of application-level health and statistics too. How should this happen?

What statistics are generally useful for:

  1. uSvcs that contain business logic (stateless)
  2. uSvcs that contain business logic (stateful - StatefulSets or use PVs)
  3. uSvcs that don't contain app specific logic (e.g. message queues, memcache, etc)
  4. Services that the app depends on that are not running inside Kubernetes (e.g. the mainframe)

Ref: https://github.com/prometheus/prometheus/tree/master/documentation/examples/kubernetes-rabbitmq

Logging behavior

Logging to stdout is a best practice, as is using something like fluentd. But is there a "right" level of logging, is there a standard format for logs that makes ops easier?

Ref: https://lukemarsden.github.io/docs/getting-started-guides/logging/

Use of labels

What best practices around the use of labels e.g. http://label-schema.org/rc1/ exist to make life easier for ops?

Vic's talk on making good charts

  • Populate the description of why the app is useful. Keyword searchable so include them.
  • Include an icon

Referencing images

  • Point to the source for each of your container images (chart.yml)
  • Make the images configurable - make sure you can configure the repo, image and the version to use

Chart usability

  • Values.yml API for your app. Give users the knobs you need. Combine values+requirements for optional dependencies
  • Provide a notes.txt - usability key since it can sub values
  • Provide a simple "helm test" to smoke test the deployment
  • Use the optional feature idiom to create configurable solutions e.g optional prometheus monitoring sidecar support

Updates

  • Annotate your deployment with the sha256sum of the configuration so config changes updates the pods
  • Use maxUnavailable:0 to make rolling updates a bit safer

Lifecycle management

  • Use post-install step for setup e.g. create bucket
  • Check for existence of existing PVC, create if doesn't exist.
  • Allow people to configure annotations for ingress

Aim is to support two versions - default should work on 1.6 and 1.5 right now.

More details: https://github.com/kubernetes/helm/blob/master/docs/charts_tips_and_tricks.md

Ref: https://www.youtube.com/watch?v=16FU6U8eOdk&feature=youtu.be&t=12m34s

What else?