aws_eks - henk52/knowledgesharing GitHub Wiki

AWS Elastic Kubernetes Service

Introduction

Purpose

Describe how to administrate Elastic Kubernetes Service(EKS) in AWS.

Vocabulary

EKS - Elastic Kubernetes Service.

Upgrade EKS

AWS Load Balancer controller

Allow to set up network path into the EKS using the AWS network services.

AWS Load Balancer controller - references

AWS Load Balancer controller - vocabulary

  • LBC - AWS Load Balancer Controller.

AWS Load Balancer controller - Overview

  • How it works

  • Kubernetes API Concepts

  • Kubernetes API

  • How is something allowed to register to watch

  • How is something allowed to make changes to the API flow?

  • the aws-load-balancer-controller

    • runs in a pod
    • in namespace kube-system
    • deployed via replica set
    • Service account: aws-load-balancer-controller
      • Cluster role.
      • Also ties into AWS via the metadata anotation: eks.amazonaws.com/role-arn: arn:aws:iam::REDACTED:role/aws-load-balancer-controller
    • Secrets: aws-load-balancer-tls TODO what is this used for?
    • ports
      • wekhook-server: 9443 - TODO is this used for receiving the changes from the kubernetes API server?
      • metric-server: 8080 - TODO how to scrap this? is there a grafana dashboard for it?

LBC Deployment

flowchart LR
    subgraph AWS["AWS Account"]
        subgraph EKS["EKS"]
            kubernetes_api[Kubernetes API]
            lbc[AWS Load Balancer Controller]
            cluster_role["Cluster role"]
        end

        sts[Security Token Service]
        aws_entities
    end

    kubernetes_api-->|events|lbc
    lbc---|patch|kubernetes_api
    lbc---aws_entities
    lbc---|get token|sts
    lbc---|authenticate with cluster role|kubernetes_api

LBC Installation

  • the work: helm chart
flowchart LR


  subgraph terraform
    helm_chart
  end

  subgraph ECR["public.ecr.aws"]
    lbc_image["/eks/aws-load-balancer-controller"]
  end

  subgraph AWS["AWS Account"]
      subgraph EKS["EKS"]
          lbc[AWS Load Balancer Controller]
          cluster_role["Cluster role"]
      end

      sts[Security Token Service]
  end

  terraform---|irsa|sts
  helm_chart---|deploy|lbc
  helm_chart---|create|cluster_role
  lbc---|pull image|lbc_image

Starting the LBC

sequenceDiagram
    participant LBC
    participant api as kubernetes api

    LBC->>api: watch ingress
    LBC->>api: watch service
    LBC->>api: watch gateway api

    LBC->>sts: get temporary token

Upgrade

cluster autoscaler

  • TODO how does it detect pod are not scheduled
  • TODO how does execut the scaling operations?
  • TODO how to get telemetry from this?
  • TODO where is the helm chart for this?
  • TODO where to find the image tag for container image?

cluster autoscaler references

Overview of cluster autoscaler

It watches for pods that fail to schedule and for nodes that are underutilized. It then simulates the addition or removal of nodes before applying the change to your cluster. The AWS Cloud Provider implementation within Cluster Autoscaler controls the .DesiredReplicas field of your EC2 Auto Scaling Groups.

  • The Autoscaler deployes a container
    • The container regularly monito the k8s API server and if 'too many' pods are not schedules then deploy more nodes.
  • There is a dependency on labels having to be available on nodes?

Upgrading the cluster autoscaler