Kubernetes Pod to Kube API Server Communication - CloudCommandos/JohnChan GitHub Wiki

For some use cases we would want to get our K8s cluster information or make changes to it from within a pod. This article will show you how to do that!

All pods are run with service accounts. As of now (06-08-2019) each pod can only run with one service account. If a service account is not specified for a pod, the default service account of the pod's namespace will be used. All namespaces have default service accounts that are created along with the namespaces.

Creating a Service Account

Creating a service account is easy. However, most likely you'll need to give specific permissions to the new service account for it to serve its purpose. Create Role, ClusterRole, RoleBinding, and/or ClusterRoleBinding based on your needs. The declaration below shows a ClusterRole that includes some of the permissions already granted by the Role just as an example.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: your-service-account-name
  namespace: your-namespace
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: your-namespace
  name: pod-access
rules:
- apiGroups: ["extensions"] # "" indicates the core API group
  resources: ["deployments", "deployments/scale", "replicasets"]
  verbs: ["*"]
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods", "pods/log", "pods/exec", "services", "secrets"]
  verbs: ["*"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: access-pods
  namespace: your-namespace
subjects:
- kind: ServiceAccount
  name: your-service-account-name
roleRef:
  kind: Role #this must be Role or ClusterRole
  name: pod-access # this must match the name of the Role or ClusterRole you wish to bind to
  apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: pod-access
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["nodes/proxy"]
  verbs: ["get", "watch", "list"]
- apiGroups: ["extensions"] # "" indicates the core API group
  resources: ["deployments", "deployments/scale", "replicasets"]
  verbs: ["*"]
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods", "pods/log", "pods/exec", "services"]
  verbs: ["*"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: access-pods
subjects:
- kind: ServiceAccount
  name: your-service-account-name
  namespace: your-namespace
roleRef:
  kind: ClusterRole #this must be Role or ClusterRole
  name: pod-access # this must match the name of the Role or ClusterRole you wish to bind to
  apiGroup: rbac.authorization.k8s.io

Specifying Service Account for a Pod

Use the serviceAccountName property to specify which service account to use. Kubernetes will use the service account matching the specified name under the pod's namespace.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: your-namespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.7.9
          imagePullPolicy: IfNotPresent
      serviceAccountName: your-service-account-name

How is a Service Account Related to a Pod?

When a service account is created, a corresponding secret will also be created. This secret contains the CA certificate of the cluster, the namespace of the service account, and a token. When a pod is created, the corresponding service account's secret contents will be copied into the pod at /var/run/secrets/kubernetes.io/serviceaccount/.

How to Query the Kube API Server From Within a Pod?

Using REST API

The K8s internal DNS record for the Kube API Server is kubernetes.default.svc.cluster.local. All pods can use their service account token to query the Kube API Server's REST API. Below is an example https request made from within a pod to get the cluster's API information.

curl -ik --header "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" https://kubernetes.default.svc.cluster.local/api

Refer to Kubernetes API Reference Docs for details on the available APIs.

Using Kubectl

There are container images in Docker Hub with pre-installed kubectl that you can use. Just run the pod using a service account that has the appropriate permissions. If you want to build your own container image with pre-installed kubectl you can use this Dockerfile:

FROM bash:4
RUN apk --no-cache add gettext ca-certificates openssl \
    && wget https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64 -O /usr/local/bin/dumb-init \
    && wget https://storage.googleapis.com/kubernetes-release/release/v1.14.1/bin/linux/amd64/kubectl -O /usr/local/bin/kubectl \
    && chmod a+x /usr/local/bin/kubectl /usr/local/bin/dumb-init \
    && apk --no-cache del ca-certificates openssl
ENTRYPOINT ["/usr/local/bin/dumb-init","--","/usr/local/bin/docker-entrypoint.sh"]
CMD ["/bin/bash","-c","while true; do sleep 30; echo sleeping; done;"]