⚡ ~/naveed K8s Hub
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
Phase 2 — Cluster Administration Module 10 of 24 Free & Open Access

Kubernetes Control Plane Deep Dive

The Kubernetes control plane keeps the cluster at its desired state. The API server validates objects and stores them in etcd, the scheduler assigns Pods to nodes, and controllers reconcile reality toward the spec. This module goes inside each component, including admission and static Pods.

Last updated: · By Naveed Ahmed, Certified Kubernetes Administrator

10 - Control Plane Deep Dive

Why this matters

Topic 01 gave you the map. Now you actually open each component up. This is where "I use Kubernetes" turns into "I understand Kubernetes."

Read this first — Definitions & Explanations

etcd deep dive

Uses Raft consensus. Production etcd clusters usually have 3 or 5 members for quorum. Losing quorum means the control plane can’t reliably write state.

API server internals (mental model)

Request path roughly: authn → authz → mutation admission → object validation → etcd → validation admission / webhooks → response. Watches let clients stream changes efficiently.

Admission controllers

Plugins/webhooks that can mutate or validate API requests before persistence (enforce labels, block privileged Pods, inject sidecars).

Scheduler phases

  1. Filtering: eliminate impossible nodes
  2. Scoring: rank remaining nodes
    Then bind Pod to the winner.

Static Pods

Pod manifests read directly by kubelet from a local folder (often /etc/kubernetes/manifests). Control-plane components on kubeadm clusters commonly run this way. They appear in the API but aren’t managed by Deployments.

kube-proxy modes

Official docs (read for detail)

Key Concepts

YouTube search terms

Hands-on lab (on prod-sim)

# etcd: talk to it directly
minikube ssh -p prod-sim
  sudo ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
    --cacert=/var/lib/minikube/certs/etcd/ca.crt \
    --cert=/var/lib/minikube/certs/etcd/server.crt \
    --key=/var/lib/minikube/certs/etcd/server.key \
    member list -w table
  sudo ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
    --cacert=/var/lib/minikube/certs/etcd/ca.crt \
    --cert=/var/lib/minikube/certs/etcd/server.crt \
    --key=/var/lib/minikube/certs/etcd/server.key \
    get /registry/pods/default --prefix --keys-only
  exit

# Static pods: how the control plane bootstraps itself
minikube ssh -p prod-sim
  ls /etc/kubernetes/manifests/     # kubelet watches this dir directly, no API server needed
  sudo cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep -A2 enable-admission-plugins
  exit

# Admission controllers in action: try to create a pod violating a policy
kubectl run priv --image=nginx --overrides='{"spec":{"containers":[{"name":"priv","image":"nginx","securityContext":{"privileged":true}}]}}'
kubectl get pod priv   # likely allowed here (no PSA restricted policy yet) — revisit in topic 17

# Scheduler: force it to explain its decision
kubectl get events --field-selector reason=Scheduled
kubectl get events --field-selector reason=FailedScheduling

Common mistakes and how to avoid them

  • Ignoring etcd disk latency. Slow disks cause slow API responses and unstable leader elections. Run etcd on fast SSD storage.
  • Running an even number of etcd members. Quorum needs a majority, so four members tolerate no more failures than three. Use three or five.
  • Editing static Pod manifests carelessly. The kubelet restarts static Pods from /etc/kubernetes/manifests when the file changes, so a typo can take the API server down. Back up the file before editing.
  • Assuming admission controllers only validate. Mutating webhooks can change objects, and a failing webhook can block creates. Set failurePolicy and exclusions deliberately.
  • Forgetting certificate expiry. kubeadm certificates expire after one year by default. Renew with kubeadm certs renew and track the dates.

Quick answers

What is etcd and why does it matter?

etcd is a consistent key-value store that holds all cluster state, including every object definition. The API server is its only client. If etcd is lost or corrupt, the cluster loses its state, which is why regular snapshots and an odd number of members matter.

What are admission controllers?

They intercept requests to the API server after authentication and authorization but before the object is stored. Mutating controllers can change the object, such as injecting a sidecar, and validating controllers can reject it, such as for a policy violation.

What are static Pods?

Static Pods are managed directly by the kubelet from manifest files on a node, usually in /etc/kubernetes/manifests, not through the API server. kubeadm runs the API server, scheduler, controller manager and etcd this way, and the kubelet restarts them if they fail.

Notes

(fill in your own words after watching + labbing)

📋 Self-Assessment Mastery Checklist (4 Competencies)
🧠 Practice Exam Questions (Module 10 MCQs)
⚡ Take Quiz & Save Progress in Tracker

Review these sample exam questions out loud, test your retrieval, and then unlock official scoring in the interactive tracker.

Question 1: Static Pod manifests for control plane (kubeadm) usually live under:
  • A. /etc/kubernetes/manifests
  • B. /var/log/pods only
  • C. /home/ubuntu
  • D. /opt/cni/bin
✓ Correct Answer: A (/etc/kubernetes/manifests)
Option A ('/etc/kubernetes/manifests') is the standard production architectural best practice.
Question 2: If the API server is down:
  • A. kubectl mostly cannot manage the cluster
  • B. Pods still get new schedules normally
  • C. etcd becomes optional
  • D. Ingress still updates routes via API
✓ Correct Answer: A (kubectl mostly cannot manage the cluster)
Option A ('kubectl mostly cannot manage the cluster') is the standard production architectural best practice.
Question 3: controller-manager runs:
  • A. Controllers that reconcile desired state
  • B. Only kube-proxy rules
  • C. Only DNS
  • D. Only CSI drivers
✓ Correct Answer: A (Controllers that reconcile desired state)
Option A ('Controllers that reconcile desired state') is the standard production architectural best practice.
Question 4: Leader election is important for:
  • A. HA control-plane controllers
  • B. PVC binding speed
  • C. Image layers
  • D. NodePort allocation only
✓ Correct Answer: A (HA control-plane controllers)
Option A ('HA control-plane controllers') is the standard production architectural best practice.
Question 5: Audit logs help you:
  • A. See who did what via the API
  • B. Compress etcd
  • C. Replace CNI
  • D. Generate TLS automatically
✓ Correct Answer: A (See who did what via the API)
Option A ('See who did what via the API') is the standard production architectural best practice.
← Previous Module (09) Helm & Package Management Next Module (11) → Installing & Upgrading Clusters (kubeadm)