What makes a well packaged Kubernetes app? - nomisbeme/acmecharts GitHub Wiki
Introduction
The Docker packaging guidelines describe how to apply 12-factor principles to generic Docker images suitable for use with any orchestrator, or with no orchestrator at all.
What's the equivalent set of best practices if I'm targeting a Kubernetes platform specifically? What is the pod/container level version of this?
Specific Questions
Health checking
Defining Liveness and readiness probes is a best practice and improves reliability of the application as a whole by enabling the scheduler to make better decisions. Kubernetes supports many types of health checks.
What constitutes a great health check?
Ref: https://www.ianlewis.org/en/using-kubernetes-health-checks
Performance monitoring
A performance monitoring solution (e.g. prometheus, dyantrace, datadog) is a requirement for a business-important application running on top of Kubernetes. In addition to monitoring the Kubernetes infrastructure the monitoring tool should be aware of application-level health and statistics too. How should this happen?
What statistics are generally useful for:
- uSvcs that contain business logic (stateless)
- uSvcs that contain business logic (stateful - StatefulSets or use PVs)
- uSvcs that don't contain app specific logic (e.g. message queues, memcache, etc)
- Services that the app depends on that are not running inside Kubernetes (e.g. the mainframe)
Ref: https://github.com/prometheus/prometheus/tree/master/documentation/examples/kubernetes-rabbitmq
Logging behavior
Logging to stdout is a best practice, as is using something like fluentd. But is there a "right" level of logging, is there a standard format for logs that makes ops easier?
Ref: https://lukemarsden.github.io/docs/getting-started-guides/logging/
Use of labels
What best practices around the use of labels e.g. http://label-schema.org/rc1/ exist to make life easier for ops?
Vic's talk on making good charts
- Populate the description of why the app is useful. Keyword searchable so include them.
- Include an icon
Referencing images
- Point to the source for each of your container images (chart.yml)
- Make the images configurable - make sure you can configure the repo, image and the version to use
Chart usability
- Values.yml API for your app. Give users the knobs you need. Combine values+requirements for optional dependencies
- Provide a notes.txt - usability key since it can sub values
- Provide a simple "helm test" to smoke test the deployment
- Use the optional feature idiom to create configurable solutions e.g optional prometheus monitoring sidecar support
Updates
- Annotate your deployment with the sha256sum of the configuration so config changes updates the pods
- Use maxUnavailable:0 to make rolling updates a bit safer
Lifecycle management
- Use post-install step for setup e.g. create bucket
- Check for existence of existing PVC, create if doesn't exist.
- Allow people to configure annotations for ingress
Aim is to support two versions - default should work on 1.6 and 1.5 right now.
More details: https://github.com/kubernetes/helm/blob/master/docs/charts_tips_and_tricks.md
Ref: https://www.youtube.com/watch?v=16FU6U8eOdk&feature=youtu.be&t=12m34s
What else?
-
Prometheus best practices (https://prometheus.io/docs/practices/instrumentation/, https://prometheus.io/docs/practices/alerting/)