k8s_security - henk52/knowledgesharing GitHub Wiki

Kubernetes security

Introduction

References

Vocabulary

  • CIS - Center for Internet Security
    • CIS also offers dashboards to view the ongoing state of compliance and security considerations. Regular viewing of this information would be part of ongoing API server security[lfs260, ch5].
  • KMS - Key Management System
  • PSP - DEPRECATED; Pod security policies. See Pod Security Adminssion

securing the kube api server

There are three phases every API call goes through as part of the kube-apiserver[lfs260, ch5]:

  • Authentication

  • Authorization

  • Admission Control

  • unwanted traffic should be stopped at the edge of the organization via some combination of edge and interior firewalls[lfs260, ch5].

  • Every API server instance should always have its own firewall as well[lfs260, ch5].

  • Do not trust that someone else understands and is properly filtering your network traffic[lfs260, ch5].

Kubernetes exposes one or two ports

  • 6443 - TLS port
  • 8080 - bypasses authentication and authorazation(dont use?)

kube-apiserver.yaml

  • /etc/kubernetes/manifests/kube-apiserver.yaml

  • --advertise-address

    • This address is the one used by the entire cluster. If this variable is not set, it will use the bind-address variable.
      • If the bind-address is not set, it will use the default interface of the host.

  • --allow-privileged

    • Privileged containers are not isolated; they share a namespace with the host, and have near full root privilege on the host.
    • Seems to be sugested to be set to false.

  • --authorization-mode

  • --client-ca-file

    • Location of the file to be used for the Certificate Authority.
    • /etc/kubernetes/pki/ca.crt

  • --enable-admission-plugins

  • --enable-bootstrap-token-auth

  • --kubelet-preferred-address-types

  • Add audit logging

  - --audit-policy-file=/etc/kubernetes/simple-policy.yaml
  - --audit-log-path=/var/log/audit.log
  - --audit-log-maxbackup=2
  - --audit-log-maxage=7
  - --audit-log-maxsize=200
  • --audit-policy-filejulietlandau_De5BsXIUYAEY8pL.jpegside the container[lfs260, ch5].
    • This will also need a volume and a mount point configured[lfs260, ch5].
  • --audit-log-maxbackup
    • the maximum number of log files to retain[lfs260, ch5].
  • --audit-log-maxage
    • How many days the previous logs should be kept[lfs260, ch5].
  • --audit-log-maxsize
    • The size in megabytes at which the log file will be rotated[lfs260, ch5].

API auditing policy

  • When enabled, every API call goes through the audit process, even if eventually excluded by a filter[lfs260, ch5].
    • the kube-apiserver will consume more memory.

Every API call has three phases:

  • RequestReceived
  • ResponseStarted
  • ResponseComplete
  • and it might also have: Panic

The first of one or more rules which matches will set the audit level for the event:

  • None - Nothing sent to backend[lfs260, ch5].
  • Metadata - Metadata, but not request or response body, sent to backend[lfs260, ch5].
  • Request - Metadata and request, but not response body sent[lfs260, ch5].
  • RequestResponse - Metadata, request, and response bodies sent to backend[lfs260, ch5].

RBAC - Role Based Access Control

  • One or more rules can be bound to subjects which are users, service accounts, and groups[lfs260, ch5]-
  • A Role is a collection of rules which can be assigned within a single namespace[lfs260, ch5].
  • A ClusterRole makes the rules effective in all namespaces[lfs260, ch5].
  • RoleBinding is used to assign the Role, a subject(e.g. named user) [lfs260, ch5].
  • Service accounts are intended for intra-cluster processes running in pods. These are also namespaces accounts[lfs260, ch5].
  • Groups are used for assigning rules to groups of users[lfs260, ch5].

Pod security policies(PSP)

TODO figure out order

etcd

  • Special care must be taken to ensure the integrity and safety of etcd[lfs260, ch5].
  • The entire persistent state of the cluster is kept in etcd[lfs260, ch5].
    • In addition, secrets (which, by nature, should be protected) and configMaps (which are used to configure much of the cluster) must not be compromised[lfs260, ch5].

Storage encryption

  • When this is configured typically, the raw encryption key is kept as part of the EncryptionConfig. Use of a separate key management system (KMS) removes the local keys and instead keeps them in a distinct and, hopefully more secure, location. Several cloud providers and a couple of CNCF projects can provide KMS functionality.[lfs260, ch5].

  • Encryption providers

    • identity
    • aescbc
    • secretbox
    • aesgcm
    • kms

Encrypting secrets

Securing the network

[lfs260, ch6]

  • Ingress filtering - It is important to filter all ingress data not specifically required for services provided by the protected network to function (e.g., a web hosting provider, HTTP port 80)[lfs260, ch6].
    • Deciding which traffic to filter is done by creating selection criteria and then comparing the traffic attributes to that criteria.
  • Egress filtering - The filtering of packets originating inside the firewall. This is an important part of enforcing acceptable use policies, and reducing the risk that any malware or infected hosts can attack hosts outside the firewall[lfs260, ch6].
  • Brdging - aggregating multiple networks to appear as a single network, appearing as a direct connection of one network to the other.[lfs260, ch6].
  • Forwarding - the relaying of a packet from one node on a network to another[lfs260, ch6].
  • NAT - the process of modifying the IP address information in the TCP headers to hide a host or network in a private network from the public network outside the firewall or router[lfs260, ch6].

container security

host security

  • SELinux
  • AppArmor
  • seccomp
  • SELinux
    • semanage