KR_K8s_Troubleshooting - somaz94/DevOps-Engineer GitHub Wiki

์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ํŠธ๋Ÿฌ๋ธ”์ŠˆํŒ… ์‹ค์ „ (Q31-Q35)

ํŠธ๋Ÿฌ๋ธ”์ŠˆํŒ… ์‹ค์ „ (31~35๋ฒˆ)


Q31. CPU Throttling ๋ฌธ์ œ์™€ CFS Quota ์ตœ์ ํ™”๋Š”?

  • CPU Limits ์„ค์ • ์‹œ CFS(Completely Fair Scheduler) Quota๋กœ ์ œํ•œ๋˜๋ฉฐ, ์‹ค์ œ ์‚ฌ์šฉ๋ฅ ์ด ๋‚ฎ์•„๋„ throttling์ด ๋ฐœ์ƒํ•  ์ˆ˜ ์žˆ๋‹ค.
  • container_cpu_cfs_throttled_seconds_total ๋ฉ”ํŠธ๋ฆญ์œผ๋กœ ๊ฐ์ง€ํ•œ๋‹ค.

ํ•ด๊ฒฐ:

  • โ‘  Limits๋ฅผ ์ œ๊ฑฐํ•˜๊ฑฐ๋‚˜ ์ถฉ๋ถ„ํžˆ ๋†’๊ฒŒ ์„ค์ •(Guaranteed QoS ํ•„์š” ์—†์œผ๋ฉด) โ†’
  • โ‘ก CPU Requests๋งŒ ์„ค์ •ํ•˜์—ฌ ์Šค์ผ€์ค„๋ง ๋ณด์žฅ โ†’
  • โ‘ข --cpu-cfs-quota=false ๋กœ Quota ๋น„ํ™œ์„ฑํ™”(๋น„๊ถŒ์žฅ).
  • Java ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์€ -XX:ActiveProcessorCount ๋กœ CPU ๊ฐœ์ˆ˜ ์ธ์‹ ์กฐ์ •ํ•œ๋‹ค. Requests๋Š” ์‹ค์ œ ํ‰๊ท  ์‚ฌ์šฉ๋Ÿ‰, Limits๋Š” spike ํ—ˆ์šฉ ๋ฒ”์œ„๋กœ ์„ค์ •ํ•œ๋‹ค.

Q32. DNS ํ•ด์„ ์ง€์—ฐ๊ณผ ndots=5 ๋ฌธ์ œ ํ•ด๊ฒฐ์€?

  • Pod์˜ /etc/resolv.conf์— ndots:5 ์„ค์ •์œผ๋กœ ์ธํ•ด ์ตœ๋Œ€ 6๋ฒˆ์˜ DNS ์ฟผ๋ฆฌ๊ฐ€ ๋ฐœ์ƒํ•œ๋‹ค.
  • example.com ์กฐํšŒ ์‹œ example.com.ns.svc.cluster.local โ†’ example.com.svc.cluster.local โ†’ example.com.cluster.local โ†’ example.com ์ˆœ์„œ๋กœ ์กฐํšŒํ•œ๋‹ค.

ํ•ด๊ฒฐ:

  • โ‘  FQDN ์‚ฌ์šฉ(์˜ˆ: api.example.com.) โ†’
  • โ‘ก dnsConfig๋กœ ndots:2 ์„ค์ • โ†’
  • โ‘ข CoreDNS autopath plugin ํ™œ์„ฑํ™” โ†’
  • โ‘ฃ NodeLocal DNSCache ์‚ฌ์šฉ(๋…ธ๋“œ๋ณ„ DNS ์บ์‹œ, UDP โ†’ eBPF).
  • coredns_dns_request_duration_seconds ๋ฉ”ํŠธ๋ฆญ์œผ๋กœ ์‘๋‹ต ์‹œ๊ฐ„ ๋ชจ๋‹ˆํ„ฐ๋งํ•œ๋‹ค.

Q33. ImagePullBackOff vs ErrImagePull ์ฐจ์ด์™€ Private Registry ์ธ์ฆ ๋ฌธ์ œ๋Š”?

ErrImagePull์€ ์ฒซ ๋ฒˆ์งธ ์ด๋ฏธ์ง€ pull ์‹คํŒจ, ImagePullBackOff๋Š” ์žฌ์‹œ๋„ ์ค‘ ๋ฐฑ์˜คํ”„ ์ƒํƒœ๋‹ค.

์›์ธ:

  • โ‘  ์ด๋ฏธ์ง€ ์กด์žฌ ์•ˆ ํ•จ โ†’
  • โ‘ก Private Registry ์ธ์ฆ ์‹คํŒจ โ†’
  • โ‘ข Rate Limit(Docker Hub) โ†’
  • โ‘ฃ ๋„คํŠธ์›Œํฌ ๋ฌธ์ œ.

ํ•ด๊ฒฐ:

  • โ‘  imagePullSecrets๋กœ Registry ์ธ์ฆ(docker-registry Secret ์ƒ์„ฑ) โ†’
  • โ‘ก ServiceAccount์— imagePullSecrets ์ถ”๊ฐ€ํ•˜์—ฌ ์ž๋™ ์ ์šฉ โ†’
  • โ‘ข ImagePullPolicy๋ฅผ IfNotPresent๋กœ ๋ณ€๊ฒฝํ•˜์—ฌ ๋กœ์ปฌ ์ด๋ฏธ์ง€ ์šฐ์„  ์‚ฌ์šฉ โ†’
  • โ‘ฃ Registry Mirror ๋˜๋Š” Harbor ๊ตฌ์ถ•.
  • kubectl describe pod๋กœ ์ •ํ™•ํ•œ ์—๋Ÿฌ ํ™•์ธํ•œ๋‹ค.

Q34. Node NotReady ์ƒํƒœ์˜ ์›์ธ ๋ถ„์„๊ณผ ๋ณต๊ตฌ ์ ˆ์ฐจ๋Š”?

Node NotReady ์›์ธ:

  • โ‘  kubelet ์ค‘์ง€/crash โ†’
  • โ‘ก ๋„คํŠธ์›Œํฌ ํŒŒํ‹ฐ์…˜ โ†’
  • โ‘ข ๋””์Šคํฌ/๋ฉ”๋ชจ๋ฆฌ ๋ถ€์กฑ โ†’
  • โ‘ฃ CNI ๋ฌธ์ œ.

์ง„๋‹จ:

  • โ‘  kubectl describe node๋กœ Condition ํ™•์ธ(DiskPressure, MemoryPressure, PIDPressure) โ†’
  • โ‘ก ๋…ธ๋“œ SSH ์ ‘์† ํ›„ systemctl status kubelet โ†’
  • โ‘ข journalctl -u kubelet๋กœ ๋กœ๊ทธ ํ™•์ธ โ†’
  • โ‘ฃ df -h, free -m๋กœ ๋ฆฌ์†Œ์Šค ํ™•์ธ.

๋ณต๊ตฌ:

  • โ‘  systemctl restart kubelet โ†’
  • โ‘ก ๋””์Šคํฌ ์ •๋ฆฌ(docker system prune, /var/log/* ์‚ญ์ œ) โ†’
  • โ‘ข CNI Pod ์žฌ์‹œ์ž‘ โ†’
  • โ‘ฃ ๋ณต๊ตฌ ์•ˆ ๋˜๋ฉด ๋…ธ๋“œ ๊ต์ฒด.
  • Pod๋Š” --pod-eviction-timeout(๊ธฐ๋ณธ 5๋ถ„) ํ›„ ๋‹ค๋ฅธ ๋…ธ๋“œ๋กœ ์žฌ์Šค์ผ€์ค„๋œ๋‹ค.

Q35. Persistent Volume์ด Terminating ์ƒํƒœ์—์„œ ๋ฉˆ์ถ˜ ๊ฒฝ์šฐ ํ•ด๊ฒฐ ๋ฐฉ๋ฒ•์€?

PV/PVC๊ฐ€ Terminating ์ƒํƒœ๋กœ ๋ฉˆ์ถ”๋Š” ์›์ธ์€ Finalizer ๋•Œ๋ฌธ์ด๋‹ค. Finalizer๋Š” ๋ฆฌ์†Œ์Šค ์‚ญ์ œ ์ „ ์ •๋ฆฌ ์ž‘์—…์„ ๋ณด์žฅํ•œ๋‹ค.

ํ•ด๊ฒฐ:

  • โ‘  kubectl patch pv <pv-name> -p '{"metadata":{"finalizers":null}}' โ†’
  • โ‘ก kubectl patch pvc <pvc-name> -p '{"metadata":{"finalizers":null}}'

์‚ฌ์šฉ ์ค‘์ธ Pod๊ฐ€ ์žˆ์œผ๋ฉด ๋จผ์ € ์‚ญ์ œํ•œ๋‹ค.

AWS EBS์˜ ๊ฒฝ์šฐ Volume์ด ์‹ค์ œ๋กœ detach๋˜์ง€ ์•Š์•„ ๋ฉˆ์ถœ ์ˆ˜ ์žˆ์œผ๋ฏ€๋กœ, AWS Console์—์„œ ์ˆ˜๋™ detachํ•œ๋‹ค.

Reclaim Policy๊ฐ€ Retain์ด๋ฉด PVC ์‚ญ์ œ ํ›„ PV๊ฐ€ Released ์ƒํƒœ๋กœ ๋‚จ์œผ๋ฏ€๋กœ ์ˆ˜๋™ ์ •๋ฆฌ๊ฐ€ ํ•„์š”ํ•˜๋‹ค.

volumeBindingMode: WaitForFirstConsumer ๋Š” Pod ์—†์ด PVC๋งŒ ์ƒ์„ฑ ์‹œ Pending ์ƒํƒœ๋กœ ์œ ์ง€๋œ๋‹ค.

๐Ÿ’ก ์šฉ์–ด ์„ค๋ช…:


โš ๏ธ **GitHub.com Fallback** โš ๏ธ