10 - Control Plane Deep Dive

Why this matters

Topic 01 gave you the map. Now you actually open each component up. This is where "I use Kubernetes" turns into "I understand Kubernetes."

Read this first — Definitions & Explanations

etcd deep dive

Uses Raft consensus. Production etcd clusters usually have 3 or 5 members for quorum. Losing quorum means the control plane can’t reliably write state.

API server internals (mental model)

Request path roughly: authn → authz → mutation admission → object validation → etcd → validation admission / webhooks → response. Watches let clients stream changes efficiently.

Admission controllers

Plugins/webhooks that can mutate or validate API requests before persistence (enforce labels, block privileged Pods, inject sidecars).

Scheduler phases

1. Filtering: eliminate impossible nodes
2. Scoring: rank remaining nodes
   Then bind Pod to the winner.

Static Pods

Pod manifests read directly by kubelet from a local folder (often /etc/kubernetes/manifests). Control-plane components on kubeadm clusters commonly run this way. They appear in the API but aren’t managed by Deployments.

kube-proxy modes

• iptables: simple, widely used
• IPVS: better performance/scale for huge numbers of Services

Official docs (read for detail)

• Control Plane Components
• Operating etcd clusters for Kubernetes
• Admission Controllers
• kube-scheduler
• Static Pods

Key Concepts

• etcd: Raft consensus, quorum, why you need an odd number of members (3/5), watch mechanism
• kube-apiserver: admission controllers (mutating/validating), API aggregation layer, watch/list mechanics, resource versioning
• kube-controller-manager: built-in controllers (node, replication, endpoint, service account, etc.) — one binary, many loops
• kube-scheduler: filtering (predicates) then scoring (priorities) phases, custom schedulers, scheduler extenders
• kubelet: PLEG (Pod Lifecycle Event Generator), cgroup driver (systemd vs cgroupfs — must match containerd config), static pods
• kube-proxy: iptables vs IPVS mode tradeoffs

YouTube search terms

• "etcd Raft consensus explained"
• "Kubernetes admission controllers explained mutating validating"
• "Kubernetes scheduler filtering scoring explained"
• "kubelet static pods explained"
• "kube-proxy iptables vs IPVS"

Hands-on lab (on prod-sim)

# etcd: talk to it directly
minikube ssh -p prod-sim
  sudo ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
    --cacert=/var/lib/minikube/certs/etcd/ca.crt \
    --cert=/var/lib/minikube/certs/etcd/server.crt \
    --key=/var/lib/minikube/certs/etcd/server.key \
    member list -w table
  sudo ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
    --cacert=/var/lib/minikube/certs/etcd/ca.crt \
    --cert=/var/lib/minikube/certs/etcd/server.crt \
    --key=/var/lib/minikube/certs/etcd/server.key \
    get /registry/pods/default --prefix --keys-only
  exit

# Static pods: how the control plane bootstraps itself
minikube ssh -p prod-sim
  ls /etc/kubernetes/manifests/     # kubelet watches this dir directly, no API server needed
  sudo cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep -A2 enable-admission-plugins
  exit

# Admission controllers in action: try to create a pod violating a policy
kubectl run priv --image=nginx --overrides='{"spec":{"containers":[{"name":"priv","image":"nginx","securityContext":{"privileged":true}}]}}'
kubectl get pod priv   # likely allowed here (no PSA restricted policy yet) — revisit in topic 17

# Scheduler: force it to explain its decision
kubectl get events --field-selector reason=Scheduled
kubectl get events --field-selector reason=FailedScheduling

Notes

(fill in your own words after watching + labbing)