Kubernetes VM Base Image Setup - doublebubblemethod/devsecops-cluster-infrastructure GitHub Wiki

VirtualBox Guest Utilities, basic tools

sudo apt-get install virtualbox-guest-utils

Check local time synchronization status
sudo systemctl status systemd-timesyncd
timedatectl

If there's a timezone offset, set it (example: Europe/Warsaw)
sudo timedatectl set-timezone Europe/Warsaw
sudo reboot

sudo apt install vim

Grant passwordless sudo for user "base" (edit sudoers)
sudo vim /etc/sudoers
Add this line:
base ALL=(ALL) NOPASSWD: ALL

For future network configuration:
sudo apt-get install iptables-persistent

Upgrade all packages (remove old)
sudo apt-get dist-upgrade -y

Install essential utilities
sudo apt-get install -y gnupg ca-certificates curl software-properties-common inetutils-traceroute net-tools nfs-common

Helm
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 chmod 700 get_helm.sh ./get_helm.sh

for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove -y $pkg done
Disable swap immediately and permanently
sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

Load required kernel modules
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sudo sysctl --system

Verify loaded modules
lsmod | grep br_netfilter
lsmod | grep overlay

Verify sysctl params
sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward

Prepare for k8s

Install Containerd Runtime
curl -LO https://github.com/containerd/containerd/releases/download/v1.7.24/containerd-1.7.24-linux-amd64.tar.gz
sudo tar Cxzvf /usr/local containerd-1.7.24-linux-amd64.tar.gz

curl -LO https://raw.githubusercontent.com/containerd/containerd/main/containerd.service
sudo mkdir -p /usr/local/lib/systemd/system/
sudo mv containerd.service /usr/local/lib/systemd/system/

sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml

sudo vim /etc/containerd/config.toml

sudo systemctl daemon-reload
sudo systemctl enable --now containerd
containerd --version

curl -LO https://github.com/opencontainers/runc/releases/download/v1.3.0/runc.amd64 sudo install -m 755 runc.amd64 /usr/local/sbin/runc runc -v

Install CNI Plugins

curl -LO https://github.com/containernetworking/plugins/releases/download/v1.7.1/cni-plugins-linux-amd64-v1.7.1.tgz
sudo mkdir -p /opt/cni/bin
sudo tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.7.1.tgz

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl --allow-downgrades --allow-change-held-packages
sudo apt-mark hold kubelet kubeadm kubectl

Test the results and check the version consistency
kubeadm version
kubelet --version
kubectl version --client

sudo chmod -R 775 /var/run/containerd/
sudo crictl config runtime-endpoint unix:///var/run/containerd/containerd.sock
sudo shutdown -h now

sudo systemctl restart containerd
sudo systemctl restart kubelet
sudo systemctl status containerd
sudo systemctl status kubelet

Setup Shared Folder (VirtualBox) for easy share of manifests

mkdir Kubernetes
sudo mount -t vboxsf -o uid=$USER,gid=base Kubernetes /home/base/Kubernetes
source ~/.profile

Troubleshooting

Kubelet status shows sth like: unset variable $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=255)

Then try these steps:
sudo rm -rf /etc/containerd/config.toml sudo systemctl restart containerd sudo systemctl daemon-reload sudo systemctl restart kubelet journalctl -u kubelet -xe

then do again: kubeadm init