Kubernetes - ILLYAKO/mywiki GitHub Wiki
Plan:
- Terminology and key features of the Kubernetes
- Create Kubernetes cluster locally
- Create and scale deployments
- Build custom Docler image and create deployment using published image
- Create services and deployments using YAML files
- Connect different deployments together
- Change container runtime from Docker to CRI-O
Kubernetes is a container orchestration system
- Automatic deployment of the containerized application across different servers
- Distribution of the load across multiple servers
- Auro_scaling of the deployed applications
- Monitoring and health checks of the containers
- Replacement of the failed containers
Container runtimes:
- Docker
- CRI-O
- containerd
Pod is the smallest unit in the Kubernetes world
Pod has:
-Containers (one or more)(One container per pod is the most common use case)
-Shared Volumes
-Shared IP Address
The containers in the same pod share Volumes and IP address
Kubernetes Cluster -> Node -> Pod -> Container
Node is a server (physical or cloud) that could be located in different places
Node inside Kubernetes Cluster
Pod inside Node
Container inside Pod
The Master Node has system pods that manage the worker nodes.
The worker Node has client containers.
Each Node (Master and Worker) has Kubernetes Services:
-kubelet
-kube-proxy responses for net communication inside the node and between nodes
-Container Runtime that is running the container inside each node
Master Node has:
-API Server that communicates with kubelet of each Node and manages Cluster by kubectl
-Scheduler that is responsible for planning and distribution of the load between different nodes in the cluster
-Kube Controller Manager that is responsible for Cluster what happened in the Cluster
-Cloud Controller Manager that is responsible for interaction with the cloud service provider where the nodes are stored
-etcd store all logs of Kubernetes Cluster
-DNS service for name resolution in Kubernetes Cluster
kubectl (kube control) is a Command Line Tool. it allows you to connect to a cluster and manage it remotely by HTTPS
Required software:
- minikube creates a Kubernetes cluster with a single Node(Master and Worker)
- Virtual machine or Container Manager(VirtualBox*, VMware*, Docker, HYper-V*, Parallels) *-recommended
- kubectl remote control
- Visual Studio Code + Kubernetes extension
2.1. Installing kubectl with Chocolatey package manager https://kubernetes.io
https://kubernetes.io/docs/tasks/tools/ https://kubernetes.io/docs/tasks/tools/install-kubectl-windows/#install-nonstandard-package-tools
> choco install kubernetes-cli
> kubectl version --client
output
Client Version: version.Info{Major:"1", Minor:"26",....
2.2. Install Hyper-V
https://www.youtube.com/watch?v=FaytvySV04s
winver
systeminfo
Hyper-V Requirements
pushd "%~dp0"
dir /b %SystemRoot%\servicing\Packages\*Hyper-V*.mum >hyper-v.txt
for /f %%i in ('findstr /i . hyper-v.txt 2^>nul') do dism /online /norestart /add-package:"%SystemRoot%\servicing\Packages\%%i"
del hyper-v.txt
Dism /online /enable-feature /featurename:Microsoft-Hyper-V -All /LimitAccess /ALL
pause
https://minikube.sigs.k8s.io/docs/start/
A Kubernetes cluster can be deployed on either physical or virtual machines. To get started with Kubernetes development, you can use Minikube. Minikube is a lightweight Kubernetes implementation that creates a VM on your local machine and deploys a simple cluster containing only one node. Minikube is available for Linux, macOS, and Windows systems. The Minikube CLI provides basic bootstrapping operations for working with your cluster, including start, stop, status, and delete.
> choco install minikube
> minikube version
output:
minikube version: v1.28.0
> minikube help
minikube status
3.1.2. Create Kubernetes Cluster with driver Hyper-V (create docker container inside Cluster, create cluster inside a virtual machine. Creating a virtual machine(cluster) 'minikube' with help of Hyper-V.
)
minikube start --driver=hyperv
minikube start --vm-driver hyperv -v 7 --alsologtostderr
minikube status
Normal Status:
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
3.1.4.1. Check Cluster IP to see which IP address was assigned to a virtual machine that is running our Kubernetes node
minikube ip
172.25.93.255
But Minikube also provides command to SSH into the local minikube node minikube ssh
If you set --driver=docker you should use minikube ssh because ssh docker@minikube IP will not work.
ssh [email protected]
default username: docker
default password: tcuser
3.1.4.4. Check the running docker container in the SSH tunnel to cluster (Docker is default container runner)
$ docker ps
Optional Note:
Minikube also provides kubectl tool and you could run commands for example like this
minikube kubectl --cluster -info
exit
kubectl cluster-info
kubectl get nodes
kubectl top nodes
kubectl get pods
kubectl get pods -owide
kubectl top pods
kubectl -n kube-system get pods
kubectl get namespaces
kubectl get pods --namespace=kube-system
The docker image will be pulled and the docker container will be created inside the pod.
The name of the Pod doesn't have to match with image name
kubectl run <pod_name> --image=<docker_image>
kubectl run nginx --image=nginx
output: pod/nginx created
kubectl get pods
kubectl describe pod <pod_name>
kubectl describe pod nginx
docker ps | grep nginx
There are two containers one is for nginx and the second is for post-container. the post-container is required to keep the name of the pod.
docker exec -it <container_id> sh
$ docker exec -it 0d1d403ea8af sh
# hostname output: nginx
# hostname -i
# curl 172.17.0.3 output HTML <title>Welcome to nginx!</title>
# exit
$ exit
kubectl get pods -o wide
If there are several containers in the Pod -they share the same IP address
kubectl delete pod <pod_name>
kubectl delete pod nginxoutput: pod 'nginx' deletedkubectl get podsoutput: No resources found in default namespace.
Linux
alias k="kubectl"
Powershell
Set-Alias -Name k -Value kubectl
A Kubernetes Deployment tells Kubernetes how to create or modify instances of the pods that hold a containerized application. Deployments can help to efficiently scale the number of replica pods, enable the rollout of updated code in a controlled manner, or roll back to an earlier deployment version if necessary.
kubectl create deployment nginx-deployment --image=nginx output:deployment.apps/nginx-deployment created
kubectl get deployments output: nginx-deployment
kubectl get pods output: nginx-deployment-5fbdf85c67-vsjvz
this pod is managed by deployments
kubectl describe deployment nginx-deployment
The connection between deployment and pods through the deployment-selector to pod-label
The Replicaset manages all pods related to the deployment
The Events id "ScalingReplicaSet"
NewReplicaSet: nginx-deployment-5fbdf85c67 (1/1 replicas created)
kubectl get pods output: nginx-deployment-5fbdf85c67-vsjvz
the pod prefix is equal to ReplicaSet
kubectl describe pod <pod_name>
kubectl describe pod nginx-deployment-5fbdf85c67-vsjvz
kubectl scale deployment <deployment_name> --replicas=<requered_number>
kubectl scale deployment nginx-deployment --replicas=5 output:deployment.apps/nginx-deployment scaled
kubectl get pods
Replica name: nginx-deployment-5fbdf85c67
Pod name : nginx-deployment-5fbdf85c67-9hv28
kubectl get pods -o wide
minikube ip
ssh [email protected]
Connect to pod from the node
curl 172.17.0.5
exit
the pod IP is dynamic and changed each time when pod is created.
Kubernetes services connect a set of pods to an abstracted service name and IP address. Services provide discovery and routing between pods.
Cluster IP is connected to a specific deployment.
The external IP address of the node
Load Balancer IP - from a cloud provider
kubectl get deploy = kubectl get deployment
kubectl expose deployment <deployment_name> --port=<deployment_external_port> --target-port=<container_port>
kubectl expose deployment nginx-deployment --port=8080 --target-port=80 output:service/nginx-deployment exposed
the default port of the nginx server is 80
kubectl get services or kubectl get svc
"kubernetes service" is defaulted system service
"nginx service" is a new service and has a virtual IP address created by Kubernetes which use to connect to any of the pods. Cluster IP allows connecting to any pod from inside the cluster only.
ssh [email protected]
curl 10.97.156.151:8080 output: HTML <h1>Welcome to nginx!</h1>
exit
curl <cluster_ip>:<explosed_port>
kubectl get svc
kubectl describe service nginx-deployment
Name: nginx-deployment
Namespace: default
Labels: app=nginx-deployment
Annotations: <none>
Selector: app=nginx-deployment
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.97.156.151
IPs: 10.97.156.151
Port: <unset> 8080/TCP
TargetPort: 80/TCP
Endpoints: 172.17.0.3:80,172.17.0.5:80,172.17.0.7:80
kubectl delete deployment nginx-deployment
kubectl delete service nginx-deployment
kubectl get deploy
kubectl get services
4.0.3. Create DockerHub account https://hub.docker.com/Create DockerHub account https://hub.docker.com/
4.0.3.1. Remote DocherHub repository https://hub.docker.com/repositories/illyako
mkdir kubernets-nodejs/k8s-web-hello
cd kubernets-nodejs/k8s-web-hello
npm init -y
npm install express
delete directory node-modules
delete directory kubernetes-notejs/k8s-web-hello
...
"scripts": {
"start": "node index.mjs"
},
...import express from 'express'
import os from 'os'
const app = express()
const PORT = 3000
app.get("/", (req, res) => {
const helloMessage = `VERSION 2: Hello from the ${os.hostname()}`
console.log(helloMessage)
res.send(helloMessage)
})
app.listen(PORT, () => {
console.log(`Web server is listening at port ${PORT}`)
})
FROM node:alpine
WORKDIR /app
EXPOSE 3000
COPY package.json package-lock.json ./
RUN npm install
COPY . ./
CMD ["npm", "start"]
cd .\k8s-web-hello\
docker build . -t illyako/k8s-web-hello
docker build . -t <dockerhube_account_name>/<repository_name>
docker images | grep k8s-web
docker login
docker push illyako/k8s-web-hello
docker push <dockerhube_account_name>/<repository_name>
https://hub.docker.com/repositories/illyako
kubectl get deploy
kubectl get services
kubectl create deployment k8s-web-hello --image=illyako/k8s-web-hello output: deployment.apps/k8s-web-hello created
kubectl create deployment <deploymetn_name> --image=<dockerhube_account_name>/<repository_name>
kubectl get pods
kubectl expose deployment k8s-web-hello --port=3000 output: "service/k8s-web-hello exposed"
Port 3000 is a port of the NodeJs server
kubectl get svc
minikube ip
ssh [email protected]
curl 10.107.250.45:3000
curl <cluster_ip>
kubectl get deploy
kubectl scale deployment k8s-web-hello --replicas=4
kubectl get pods -o wide
kubectl get svc
curl 10.107.250.45:3000; echo
output:
Hello from the k8s-web-hello-dcc56d869-vtjjf
Hello from the k8s-web-hello-dcc56d869-n9hvt
Hello from the k8s-web-hello-dcc56d869-z2zw9
kubectl delete svc k8s-web-hello
kubectl get svc
kubectl expose deployment k8s-web-hello --type=NodePort --port=3000
kubectl get svc
output:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
k8s-web-hello NodePort 10.96.44.253 <none> 3000:32137/TCP 2m52s
minikube ip output:172.25.93.255
4.11.3. Go to browser address 172.25.93.255:32137 output: "Hello from the k8s-web-hello-dcc56d869-z2zw9"
<node_ip>:<pandom_generated_port_in_service>
minikube service k8s-web-hello
the new tab in the browser is opened and there is a node server respond
minikube service k8s-web-hello --url
kubectl get svc
kubectl expose deployment k8s-web-hello --type=LoadBalancer --port=3000
kubectl get svc
minikube service k8s-web-hello
the new tab in the browser is opened and there is a node server respond
kubectl get svc
kubectl get deploy
kubectl describe deploy k8s-web-hello
StrategyType: RollingUpdate means that new pods will be created when old pods are running
docker build . -t illyako/k8s-web-hello:2.0.0
docker push illyako/k8s-web-hello:2.0.0
kubectl set image deployment k8s-web-hello k8s-web-hello=illyako/k8s-web-hello:2.0.0
kubectl rollout status deploy k8s-web-hello
kubectl get pods
kubectl get svc
minikube service k8s-web-hello
kubectl set image deployment k8s-web-hello k8s-web-hello=illyako/k8s-web-hello
kubectl rollout status deploy k8s-web-hello
kubectl get pods
kubectl delete pod <pod_name>
minikube dashboard
5.0. Delete all in the namespace (deployments, services, pods)(Kubernetes Service was deleted but after that, it was recreated)
kubectl delete all --all
5.1. Create deployment.yaml (https://kubernetes.io/docs/reference/kubernetes-api/)
apiVersion: apps/v1
kind: Deployment
metadata:
name: k8s-web-hello
spec:
replicas: 5
selector:
matchLabels:
app: k8s-web-hello
template:
metadata:
labels:
app: k8s-web-hello
spec:
containers:
- name: k8s-web-hello
image: illyako/k8s-web-hello
resources:
limits:
memory: "128Mi"
cpu: "250m"
ports:
- containerPort: 3000
cd kubernetes-nodejs
kubectl apply -f deployment.yaml
kubectl get deploy
apiVersion: v1
kind: Service
metadata:
name: k8s-web-hello
spec:
type: LoadBalancer
selector:
app: k8s-web-hello
ports:
- port: 3030
targetPort: 3000cd kubernetes-nodejs
kubectl apply -f service.yaml
kubectl get svc
minikube service k8s-web-hello
kubectl delete -f deployment.yaml -f service.yaml
kubectl get deployment
kubectl get service
...
app.get("/nginx", async (req, res) => {
const url = 'http://nginx' // connection to nginx service
const response = await fetch(url);
const body = await response.text();
res.send(body)
})
...cd k8s-web-to-nginx
npm install node-fetch // install nodejs package
delete directory node-modules
cd k8s-web-to-nginx
docker build . -t illyako/k8s-web-to-nginx
docker push illyako/k8s-web-to-nginx
apiVersion: v1
kind: Service
metadata:
name: k8s-web-to-nginx
spec:
type: LoadBalancer
selector:
app: k8s-web-to-nginx
ports:
- port: 3333
targetPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: k8s-web-to-nginx
spec:
replicas: 3
selector:
matchLabels:
app: k8s-web-to-nginx
template:
metadata:
labels:
app: k8s-web-to-nginx
spec:
containers:
- name: k8s-web-to-nginx
image: bstashchuk/k8s-web-to-nginx
resources:
limits:
memory: "128Mi"
cpu: "250m"
ports:
- containerPort: 3000apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
selector:
app: nginx
ports:
- port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 5
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
resources:
limits:
memory: "128Mi"
cpu: "125m"
ports:
- containerPort: 80cd kubernetes-nodejs
kubectl apply -f k8s-web-to-nginx.yaml -f nginx.yaml
kubectl get deploy
kubectl get svc
kubectl edit deploy <deployment_name>
get URL
minikube service k8s-web-to-nginx
connection from one service to other in index.mjs BY NAME OF the SERVICE
...
app.get("/nginx", async (req, res) => {
const url = 'http://nginx' // connection to nginx service
const response = await fetch(url);
const body = await response.text();
res.send(body)
})
...kubectl get pods
kubectl exec k8s-web-to-nginx-6ffcdb76-g6vr7 -- nslookup nginx
kubectl exec <pod_name> -- nslookup <server_name>
We try to resolve the nginx name from inside the container. The DNS server gives the IP address of nginx
kubectl get svc
there is a server IP which is like from the exec command
kubectl exec k8s-web-to-nginx-6ffcdb76-g6vr7 -- wget -qO- http://nginx
output: Html
kubectl delete -f nginx.yaml -f k8s-web-to-nginx.yaml
output:
service "nginx" deleted
deployment.apps "nginx" deleted
service "k8s-web-to-nginx" deleted
deployment.apps "k8s-web-to-nginx" deleted
minikube status
minikube stop
minikube delete
minikube start --driver=hyperv --container-runtime=cri-o
minikube status
minikube ip output: 172.25.84.144
ssh [email protected]
ssh docker@<cluster_ip>
docker ps output: "Cannot connect to the Docker daemon"
sudo crictl ps
In VS-Code terminal
kubectl apply -f k8s-web-to-nginx.yaml -f nginx.yaml
kubectl get deploy
kubectl get svc
kubectl get pods
minikube service <service_name>
minikube service k8s-web-to-nginx
The virtual machine on AWS-ES2
KUBEADM
EC2->Instanses->Lanche Instanses->Choose an Amazon Machine Image(AMI)->Ubuntu Server->Choose Instace Type->t3.micro->Number of Instances 3 ->Default setting-> Configure Security Group->Create a new key pair->Launch Instances
Security Group->Edit inbound rules->add SSH
Instances->rename instances(Master, Worker-1, Worker-2)
Steps and Commands:
(This assumes you have 3 servers up and running)
-
On each server, install Docker (Installation guide: https://docs.docker.com/engine/instal...) curl -fsSL https://download.docker.com/linux/ubu... | sudo apt-key add - sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" sudo apt-get update sudo apt-get install -y docker-ce
-
On each server, install kubernetes (Installation guide: https://kubernetes.io/docs/setup/prod...) curl -s https://packages.cloud.google.com/apt... | sudo apt-key add - cat << EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list deb https://apt.kubernetes.io/ kubernetes-xenial main EOF sudo apt-get update sudo apt-get install -y kubelet kubeadm kubectl
-
On each server, enable the use of iptables echo "net.bridge.bridge-nf-call-iptables=1" | sudo tee -a /etc/sysctl.conf sudo sysctl -p
-
On the Master server only, initialize the cluster sudo kubeadm init --pod-network-cidr=10.244.0.0/16
(After this command finishes, copy kubeadm join provided)
-
On the Master server only, set up the kubernetes configuration file for general usage mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf
$HOME/.kube/config sudo chown $ (id -u):$(id -g) $HOME/.kube/config -
On the Master server only, apply a common networking plugin. In this case, Flannel kubectl apply -f https://raw.githubusercontent.com/cor...
-
On the Worker servers only, join them to the cluster using the command you copied earlier. kubeadm join 172.31.37.80:6443 --token ... --discovery-token-ca-cert-hash ...
And that's it! You should now have a working kubernetes cluster!
https://docs.docker.com/desktop/kubernetes/
Docker Desktop->Setting->Kubernetes->check box Enable Kubernetes->Apply&restart
Context: docker-desktop
kubectl get all
output
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 19m
apiVersion: apps/v1
kind: Deployment
metadata:
name: order-deployment
labels:
app: orderlabel
spec:
replicas: 3
selector:
matchLabels:
app: orderlabel
template:
metadata:
labels:
app: orderlabel
spec:
containers:
- name: orderapi
image: nginx
resources:
limits:
memory: "128Mi"
cpu: "500m"
ports:
- containerPort: 80
kubectl apply -f deployment.yaml
kubectl get pods
apiVersion: v1
kind: Service
metadata:
name: order-service
spec:
selector:
app: orderlabel
type: LoadBalancer
ports:
- protocol: TCP
port: 8083
targetPort: 80
kubectl apply -f service.yaml
kubectl get services
output
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 41m
order-service LoadBalancer 10.103.199.240 localhost 8083:31371/TCP 82s
External link localhost:8083
kubectl get all