Skip to content

Azure Container Orchestratoin #2

manchiro edited this page Jul 15, 2020 · 15 revisions

day1 day2 day3

Azure ์‹ค์Šต

  • Azure๋Š” ๊ตฌ๋… ๊ธฐ๋ฐ˜์— ๊ณผ๊ธˆ ์ฐจ์ง•์„ ํ•จ.
  • AZure ํฌํƒˆ์—์„œ ๋ฆฌ์†Œ์Šค ๊ทธ๋ฃน > ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ์„œ๋น„์Šค ์ƒ์„ฑ > ์ปจํ…Œ์ด๋„ˆ ๋ ˆ์ง€์ŠคํŠธ๋ฆฌ ์ƒ์„ฑ
  • Azure (http://portal.azure.com) : admin5@gkn2019hotmail.onmicrosoft.com / skcc*
  • ๋ฆฌ์†Œ์Šค ๊ทธ๋ฃน๋ช… : admin5-Resource_Group
  • Kubernetes ์„œ๋น„์Šค : admin5-k8s
  • Azure ์ปจํ…Œ์ด๋„ˆ ๋ ˆ์ง€์ŠคํŠธ๋ฆฌ : admin5acr, admin5acr.azurecr.io

๋ณต์Šต

# az account set --subscription "์ข…๋Ÿ‰์ œ1"
# az aks get-credentials --resource-group admin5-Resource_Group --name admin5-k8s
# az acr login --name admin5acr
# az aks update -n admin5-k8s -g admin5-Resource_Group --attach-acr admin5acr

# nano Dockerfile - Dockerfile ์ƒ์„ฑ
# docker build -t admin5acr.azurecr.io/my-nginx:v2   - docker build
# docker images    : docker images ํ™•์ธ
# docker push admin5acr.azurecr.io/my-nginx:v2   : docker๋ฅผ ๋ฆฌํŒŒ์ง€ํ† ๋ฆฌ์— push
# kubectl create deploy my-nginx --image=admin5acr.azurecr.io/my-nginx:v2 -  AZure portal ์—์„œ ํ™•์ธ
# kubectl create -f nginx.yaml
# kubectl apply -f nginx.yaml
# kubectl ๋™์‚ฌ ๋ชฉ์ ์–ด
# kubectl get nodes   - ํด๋Ÿฌ์Šคํ„ฐ์— ๋งŒ๋“ค์–ด ์ง„ ๋…ธ๋“œ์˜ ์ˆ˜ (worker nodes. VM. ์„œ๋ฒ„)
# kubectl get pod
# kubectl get pod -o wide  - Output์„ ์ข€๋” ๋งŽ์ด ์ถœ๋ ฅ. node ์ •๋ณด๋„ ๊ฐ™์ด ๋ณด์—ฌ์ง.

1. Kubernetes Basic Objects

1.1 Master๊ฐ€ ํ•˜๋Š”์ผ

  • ์ปจํ…Œ์ด๋„ˆ์˜ ์ž๋™ ๋ฐฐํฌ
  • Self Healing - Nonn Terminating

1.2 ์ด๋ฏธ์ง€๋ฅผ ํ†ตํ•œ ์–ดํ”Œ๋ฆฌ์ผ€์ด์…˜ ๋ฐฐํฌ

# kubectl create deploy my-nginx --image=admin5acr.azurecr.io/my-nginx:v2
# kubectl expose deploy my-nginx --type=LoadBalancer --port=80  : ์™ธ๋ถ€์— ์˜คํ”ˆ. type ์„ LoadBalancer๋กœ ์ฃผ์–ด์•ผ External IP๋กœ ๋˜์–ด์•ผ ํ•จ.
skccadmin@SKCC18D00132:~/container-orchestration/yaml$ kubectl get all
NAME                                    READY   STATUS    RESTARTS   AGE
pod/my-nginx-74dcffd6ff-xcp4h           1/1     Running   0          3m5s
pod/nginx-deployment-54f57cf6bf-78pp9   1/1     Running   0          27m
pod/nginx-deployment-54f57cf6bf-svn8x   1/1     Running   0          18m

NAME                 TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)        AGE
service/kubernetes   ClusterIP      10.0.0.1      <none>         443/TCP        38m
service/my-nginx     LoadBalancer   10.0.159.90   40.82.129.79   80:30236/TCP   13s

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/my-nginx           1/1     1            1           3m5s
deployment.apps/nginx-deployment   2/2     2            2           27m

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/my-nginx-74dcffd6ff           1         1         1       3m5s
replicaset.apps/nginx-deployment-54f57cf6bf   2         2         2       27m
  • External IP ๋กœ ๋ธŒ๋ผ์šฐ์ €์—์„œ ์กฐํšŒํ•˜๋ฉด index.html ๋‚ด์šฉ์ด ์กฐํšŒ๋จ. http://40.82.129.79

1.3 Cloud Trouble Shooting

  1. kubectl describe pod pod๋ช… : ์–ด๋–ป๊ฒŒ ์ƒ์„ฑ๋˜์—ˆ๋Š”์ง€ ์ด๋ ฅ
  2. kubectl logs pod๋ช… -f : kube๋กœ๊ทธ๋กœ ํ™•์ธ
  3. kubectl exec -it pod๋ช… -- /bin/bash : POD๋กœ๊ทธ ํ™•์ธ

1.4 ํด๋ผ์šฐ๋“œ ์„œ๋น„์Šค ์„ฑ๋Šฅ ํ™•์ธ

  • Azure Portal ์—์„œ > ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ์„œ๋น„์Šค > ๋‚˜์˜ ํด๋Ÿฌ์Šคํ„ฐ์„œ๋น„์Šค ์„ ํƒ > ๋…ธ๋“œํ’€ > ๋‚ด๊ฐ€ ์„ ํƒํ•œ "admin5-k8s | ๋…ธ๋“œ ํ’€" ์„ ํƒํ•˜๋ฉด ์ƒ์„ธ ๋‚ด์šฉ ์กฐํšŒ ๊ฐ€๋Šฅ.
  • RAM(GiB) : 1์ดˆ/1byte๊ฐ€ ๋ˆ์ด๊ธฐ ๋•Œ๋ฌธ์— ์ •ํ™•ํ•œ ์‚ฌ์ด์ฆˆ๋ฅผ ํ‘œํ˜„ํ•˜๊ธฐ ์œ„ํ•ด์„œ ์‚ฌ์šฉํ•˜๋Š” ๋‹จ์œ„์ž„. Gibibyte.

1.5 ReplicaSets

  • kubectl scale type/type๋ช… --replicas=๊ฐœ์ˆ˜
    • kubectl scale replicaset/frontend --replicas=5 : replica ๊ฐœ์ˆ˜ ์กฐ์ •
    • kubectl scale deployment/my-nginx --replicas=3

1.6 Rolling update

  • ๋…ธ๋“œ์˜ ์ˆ˜๋ฅผ ๋Š˜๋ฆฌ๊ณ  ์ค„์ด๋Š” ๊ฒƒ์€ Scale out/in ์ด๊ณ ,
  • ๋…ธ๋“œ์˜ ์„ฑ๋Šฅ์„ ๋Š˜๋ฆฌ๊ณ  ์ค„์ด๋Š” ๊ฒƒ์€ Scale Up/Down ์ด๋‹ค
  • CF) deployment์˜ ๋ฒ„์ ผ์„ ๋ณ€๊ฒฝํ•˜๋Š” ๋ฐฉ๋ฒ•
    1. kubectl delete deployment ... ์‚ญ์ œ ํ•˜๊ณ  ๋‹ค์‹œ ์ƒ์„ฑ
    2. kubectl apply -f yaml ๋ช… --- ๋‹ค์‹œ ๋ฐ˜์˜
    3. kubectl set image .. ๋กœ ๋ณ€๊ฒฝ

    kubectl set image deployment/nginx deployment nginx=nginx:1.9.1

  • ๋ณ€๊ฒฝํ•˜๊ธฐ

    CF) kubectl edit deployment nginx-deployment

skccadmin@SKCC18D00132:~/container-orchestration/yaml/replicaset$ kubectl set image deploy nginx-deployment nginx=nginx:1.9.1
skccadmin@SKCC18D00132:~/container-orchestration/yaml/replicaset$ kubectl get deployment -o wide
NAME               READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                             SELECTOR
my-nginx           2/2     2            2           3h17m   my-nginx     admin5acr.azurecr.io/my-nginx:v2   app=my-nginx
my-nginx3          0/1     1            0           169m    nginx        manchiro/nginx                     app=my-nginx3
nginx-deployment   2/2     2            2           3h41m   nginx        nginx:1.9.1                        app=nginx
# kubectl rollout history deploy nginx-deployment
deployment.apps/nginx-deployment
REVISION  CHANGE-CAUSE
1         <none>
2         v2 is nginx:1.9.1
# kubectl rollout undo deploy nginx-deployment  - ์ด์ „๋ฒ„์ „์œผ๋กœ ๋‹ค์‹œ ์˜ฌ๋ฆผ
deployment.apps/nginx-deployment
REVISION  CHANGE-CAUSE
2         v2 is nginx:1.9.1
3         <none>
  • (์˜คํ†  ์Šค์ผ€์ผ๋ง ์„ค์ •, hpa: HorizontalPodAutoscaler )
  > kubectl top pods
  > kubectl top nodes
  > kubectl apply -f https://k8s.io/examples/application/php-apache.yaml  - apache ์ƒ์„ฑ : 
  > kubectl autoscale deployment php-apache --cpu-percent=20 --min=1 --max=10
  > kubectl get hpa php-apache -o yaml
spec:
  maxReplicas: 10
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: php-apache
  targetCPUUtilizationPercentage: 20
status:
  currentCPUUtilizationPercentage: 0
  currentReplicas: 1
  desiredReplicas: 1
  • ๋ถ€ํ•˜๋ฅผ ๋ฐœ์ƒํ•˜๋Š” ๋ฐฉ๋ฒ• : ์ค‘ ์กฐ์‹ฌํ•ด์•ผ ํ• ๊ฒƒ . External IP ๋กœ ๋งŒ๋“  ํ›„ ํ˜ธ์ถœํ•˜์ง€ ๋ง์•„๋ผ.
    • ํด๋ผ์šฐ๋“œ ๋‚ด์—์„œ ํ…Œ์ŠคํŠธ๋ฅผ ํ•ด๋ผ.
  > kubectl create -f siege.yaml
  > kubectl exec -it siege -- /bin/bash
  > siege -c30 -t30S -v http://php-apache  : -C 30๋ช…์˜ ์‚ฌ์šฉ์ž๊ฐ€ -t 30์ดˆ ๋™์•ˆ http ์„œ๋น„์Šค๋ฅผ ํ˜ธ์ถœํ•ด๋ผ.

1.7 ๋ฐฐํฌ์ „๋žต

  • Recreate : ๋‹ค์šดํ•˜๊ณ  ๋ฐฐํฌํ•˜๊ณ  ์Šคํƒ€ํŠธ.
  • Ramped (=Rolling Update) : ๊นœ๋ฐ•์ด๋ฉด์„œ, ํ•˜๋‚˜์”ฉ ์ˆœ์ฐจ์ ์œผ๋กœ ๋ฐฐํฌํ•จ (k8s์˜ ๊ธฐ๋ณธ ๋ฐฐํฌ ์ „๋žต).
  • Blue/Green : ํ˜„์žฌ ์„œ๋น„์Šค๋Š” Blue. ์ƒˆ๋กœ์šด ์„œ๋น„์Šค๋Š” Green. Zero Downtime์ด ๋ฐœ์ƒํ•˜์ง€๋งŒ ๋น„์šฉ์ด ๋ฐœ์ƒํ•จ. (*์˜ˆ์ƒ๋ฌธ์ œ)
  • Canary : ์ƒˆ๋กœ์šด ๋ฒ„์ ผ์„ ๋ฐฐํฌ ํ›„, ํŠน์ • Request๋งŒ ์‚ฌ์šฉํ•˜๋„๋ก ์„ค์ •.
  • A/B Testing
  • Shadow

1.8 Namespaces

# kubectl create ns admin5
# kubectl create deploy my-nginx4 --image=nginx -n admin5
# kubectl expose deploy my-nginx3 --type=LoadBalancer --port=80 -n=admin5
#  kubectl get all -n=admin5
skccadmin@SKCC18D00132:~/container-orchestration/yaml/replicaset$ kubectl get all -n=admin5
NAME                             READY   STATUS    RESTARTS   AGE
pod/my-nginx3-9d6bb446b-7qsdw    1/1     Running   0          2m38s
pod/my-nginx3-9d6bb446b-l8vtj    1/1     Running   0          2m38s
pod/my-nginx3-9d6bb446b-m7f6c    1/1     Running   0          6m56s
pod/my-nginx3-9d6bb446b-str45    1/1     Running   0          2m38s
pod/my-nginx3-9d6bb446b-whdfg    1/1     Running   0          2m38s
pod/my-nginx4-6fc7f8b4f8-9wz9v   1/1     Running   0          5m18s

NAME                TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/my-nginx3   LoadBalancer   10.0.242.183   <pending>     80:31012/TCP   3s

NAME                        READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/my-nginx3   5/5     5            5           6m56s
deployment.apps/my-nginx4   1/1     1            1           5m18s

NAME                                   DESIRED   CURRENT   READY   AGE
replicaset.apps/my-nginx3-9d6bb446b    5         5         5       6m56s
replicaset.apps/my-nginx4-6fc7f8b4f8   1         1         1       5m18s
# kubectl delete deploy,service,pod --all

1.9 Services

1.9.1 Service Discovery

  • Service ๋Š” ํด๋ผ์ด์–ธํŠธ๊ฐ€ ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์— ์ ‘๊ทผํ•˜๋Š” Kubernetes ์˜ ์ฑ„๋„๋กœ ๋Ÿฐํƒ€์ž„์‹œ , ์ด๋ฅผ ๊ฒ€์ƒ‰ํ•  ์ˆ˜ ์žˆ๋Š” ๋ฐฉ๋ฒ•์ด ํ•„์š”

1.9.2 ServiceType

  • Service ์ƒ์„ฑ ์‹œ IP ์ฃผ์†Œ ํ• ๋‹น ๋ฐฉ์‹๊ณผ ์„œ๋น„์Šค ์—ฐ๊ณ„ ๋“ฑ์— ๋”ฐ๋ผ 4 ๊ฐ€์ง€๋กœ ๊ตฌ๋ถ„
    • ClusterIP
    • NodePort
    • LoadBalancer
    • ExternalName
  • G/W ์—ญํ• ์„ ํ•˜๋Š” ํ•œ๊ฐœ๋งŒ ๋งŒ๋“ค๊ฒŒ ๋จ. ๋น„์šฉ๊ณผ ๊ด€๋ จ์ด ์žˆ์Œ. ๋ณด์•ˆ. ๋‚˜๋จธ์ง€ ํŒŒํŠธ๋Š” ํ† ํฐ๋งŒ ์ฒดํฌํ•˜๋„๋ก ํ•˜๋ฉด ๋จ.
  • IngressControlor๋Š” : NodePort Type์œผ๋กœ ์ƒ์„ฑํ•จ.
  • ํด๋ผ์šฐ๋“œ๋‚ด ์„œ๋น„์Šค URL
    • --- : ๊ฐ์ฒด์˜ ๊ตฌ๋ถ„์ž
    • -- : array

    curl [์„œ๋น„์Šค๋ช…].[namespace].[์„œํ•์Šค]

    • ์˜ˆ

    curl http://php-apache.default.svc.cluster.local

1.10 Volumes

  • ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค๋Š” ์—ฌ๋Ÿฌ ํ˜ธ์ŠคํŠธ์— ๊ฑธ์ฒ˜ Stateless ํ•œ ์ปจํ…Œ์ด๋„ˆ๋ฅผ ๋งˆ์ดํฌ๋กœ์„œ๋น„์Šค๋กœ ๋ฐฐํฌํ•˜๋Š” ๊ฒƒ์ด ๋ชฉํ‘œ์ด๊ธฐ์— ์˜์†์„ฑ ์žˆ๋Š” ์ €์žฅ์žฅ์น˜ (Persistent Value)๋ฅผ ๊ณ ๋ คํ•ด์•ผ ํ•จ.

1.10.1 emptyDir (P.126)

1.10.2 PersistentVolume & PersistentVolumeClaim

  • PV vs PVC
    • PV : ์ •์˜๋œ ์šฉ๋Ÿ‰๋งŒํผ pod๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Œ.
  • provisioning : POD๊ฐ€ ์›ํ•˜๋Š” ๋งŒํผ์˜ ์Šคํ† ๋ฆฌ์ง€๋ฅผ ์ธํ”„๋ผ ์ ์œผ๋กœ ์„ค์ •ํ•˜๋Š” ๊ฒƒ.
skccadmin@SKCC18D00132:~/container-orchestration/yaml$ kubectl get storageclass  (=sc)
NAME                PROVISIONER                AGE
azurefile           kubernetes.io/azure-file   23h
azurefile-premium   kubernetes.io/azure-file   23h
default (default)   kubernetes.io/azure-disk   5h55m
managed-premium     kubernetes.io/azure-disk   5h55m  (SSD  ๊ธฐ๋ฐ˜)

1.11 ConfigMap

1.11.1

  • ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€๋กœ๋ถ€ํ„ฐ ์„ค์ • ์ •๋ณด๋ฅผ ๋ถ„๋ฆฌํ•  ์ˆ˜ ์žˆ๊ฒŒ ํ•ด์คŒ.
  • ๋™์ผํ•œ ์ด๋ฏธ์ง€๋กœ ๊ฐœ๋ฐœ/์šด์˜์— ๋ถˆ๋ณ€ํ•˜๊ฒŒ ์„ค์ •ํ•˜๊ธฐ ์œ„ํ•จ.
  • https://workflowy.com/s/msa/27a0ioMCzlpV04Ib#/6ce6be6e8804
$ docker build -t admin5acr.azurerc.io/cm-sandbox:v1 .
$ docker push admin5acr.azurerc.io/cm-sandbox:v1
# ์œ„ 2 ๋ช…๋ นํ–‰์„ ์•„๋ž˜ ํ•˜๋‚˜์˜ ๋ช…๋ น์œผ๋กœ ์ˆ˜ํ–‰ ๊ฐ€๋Šฅ
$ az acr build --registry admin5acr --image admin5acr.azurecr.io/cm-sandbox:v1 .
$ 

1.12 Secrets

  • ConfigMap ์ด ์ผ๋ฐ˜์ ์ธ ํ™˜๊ฒฝ ์„ค์ • ์ •๋ณด๋‚˜ Config ์ •๋ณด๋ฅผ ์ €์žฅํ•˜๋„๋ก ๋””์ž์ธ ๋˜์—ˆ๋‹ค๋ฉด , ๋ณด์•ˆ์ด ์ค‘์š”ํ•œ ํŒจ์Šค์›Œ๋“œ๋‚˜ API ํ‚ค , ์ธ์ฆ์„œ ํŒŒ์ผ๋“ค์€ Secret ์— ์ €์žฅ
skccadmin@SKCC18D00132:~/container-orchestration/yaml/configmap$ echo test | base64
dGVzdAo=
skccadmin@SKCC18D00132:~/container-orchestration/yaml/configmap$ echo dGVzdAo= | base64 --decode
test
$ az acr build --registry admin5acr --image cm-sandbox:v2 .
  • Master์— ์ €์žฅ๋˜๊ณ  2M์˜ ์ œ์•ฝ์ด ์žˆ์Œ.

1.13 Liveness Probes & Readiness Probes

  • pod๊ฐ€ platform์—๊ฒŒ ์ƒํƒœ๋ฅผ ์ฃผ๊ธฐ์ ์œผ๋กœ ์•Œ๋ ค์ฃผ๋Š” ์„œ๋น„์Šค์ž„.
  • Probe Types : ์ฒดํฌํ•˜๋Š” ๋ฐฉ์‹ - Command probe, HTTP probe, TCP probe
  • Liveness Probes
  • Readiness Probes : ์ค€๋น„๊ฐ€ ๋˜์—ˆ๋Š”์ง€ ํ™•์ธํ•ด์„œ ์ค€๋น„๊ฐ€ ๋˜๋ฉด Traffice์„ ์ „๋‹ฌํ•จ.
    • autoscale out ๋˜๋Š” ์ค‘ ์„œ๋น„์Šค๋ฅผ ๋ณด๋‚ด์ง€ ์•Š์Œ.
  • Difference between Liveness and Readiness
    • Liveness probe ๋Š” ์ปจํ…Œ์ด๋„ˆ์˜ ์ƒํƒœ๊ฐ€ ๋น„์ •์ƒ์ด๋ผ๊ณ  ํŒ๋‹จํ•˜๋ฉด , ํ•ด๋‹น Pod ๋ฅผ ์žฌ์‹œ์ž‘ํ•˜๋Š”๋ฐ ๋ฐ˜ํ•ด
    • Readiness probe ๋Š” ์ปจํ…Œ์ด๋„ˆ๊ฐ€ ๋น„์ •์ƒ์ผ ๊ฒฝ์šฐ์—๋Š”, ํ•ด๋‹น Pod ๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์—†์Œ์œผ๋กœ ํ‘œ์‹œํ•˜๊ณ  , ์„œ๋น„์Šค๋“ฑ์—์„œ ์ œ์™ธํ•œ๋‹ค.
Clone this wiki locally