12 - Networking Deep Dive (CNI, NetworkPolicy)
Why this matters
Networking is the #1 area where "it works" turns into a multi-hour debugging session. This topic is what actually separates people who can operate a cluster from people who can only deploy YAML into one someone else built.
Read this first — Definitions & Explanations
CNI (Container Network Interface)
Plugin interface that sets up Pod networking: interfaces, IPs, routes. Examples: Calico, Cilium, Flannel.
Pod network
Every Pod gets an IP. Nodes must route Pod ↔ Pod traffic according to the CNI design (overlay or routing-based).
NetworkPolicy
API for allowing/denying traffic to/from Pods. Only enforced if your CNI supports NetworkPolicy. Without support, policies may be stored but not applied.
Default-open reality
Many clusters allow all Pod-to-Pod traffic until you write restrictive NetworkPolicies. Don’t assume isolation exists by default.
CoreDNS
Cluster DNS. Services get DNS names; Pods resolve them via CoreDNS. DNS breakage looks like “random app failures.”
Overlay vs underlay
- Overlay: encapsulate Pod packets (e.g. VXLAN)
- Underlay/routing: route Pod IPs natively on the network fabric
Official docs (read for detail)
Key Concepts
- CNI plugin responsibilities: pod IP allocation, routing, sometimes NetworkPolicy enforcement
- Major CNIs: Calico, Cilium (eBPF-based), Flannel — know at least what each is known for
- NetworkPolicy: default-deny, ingress/egress rules, namespaceSelector vs podSelector
- Note: NetworkPolicy needs a CNI that supports it — Flannel alone does NOT enforce it
- Service mesh preview (deep dive in topic 23): mTLS, sidecar proxies
- DNS troubleshooting flow
YouTube search terms
- "Kubernetes CNI explained Calico Cilium Flannel"
- "Kubernetes NetworkPolicy tutorial default deny"
- "Cilium eBPF Kubernetes networking explained"
Hands-on lab (on prod-sim)
# Check current CNI
kubectl get pods -n kube-system | grep -i -E 'calico|cilium|flannel|kindnet'
# minikube's default is usually a simple bridge CNI - check what NetworkPolicy support it has
# Default-deny all ingress in a namespace, then allow one specific path
kubectl create namespace netpol-test
kubectl -n netpol-test create deployment web --image=nginx
kubectl -n netpol-test expose deployment web --port=80
kubectl -n netpol-test run client --image=busybox --restart=Never -- sleep 3600
# baseline: client can reach web
kubectl -n netpol-test exec client -- wget -qO- --timeout=2 web
cat <<EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: netpol-test
spec:
podSelector: {}
policyTypes: ["Ingress"]
EOF
kubectl -n netpol-test exec client -- wget -qO- --timeout=2 web # should now hang/fail
cat <<EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-client
namespace: netpol-test
spec:
podSelector:
matchLabels:
app: web
policyTypes: ["Ingress"]
ingress:
- from:
- podSelector:
matchLabels:
run: client
EOF
kubectl -n netpol-test exec client -- wget -qO- --timeout=2 web # works again
# DNS troubleshooting drill
kubectl -n netpol-test exec client -- nslookup web.netpol-test.svc.cluster.local
kubectl -n kube-system logs -l k8s-app=kube-dns --tail=50
Note: if your CNI doesn't enforce NetworkPolicy, install Calico for this lab:
minikube start -p prod-sim --cni=calico (recreate cluster with this flag).
Common mistakes and how to avoid them
- Assuming NetworkPolicy works on any cluster. It only takes effect if the CNI enforces it, as Calico and Cilium do. On a CNI without support the objects are accepted and silently ignored.
- Writing allow policies without a default deny. Pods that no policy selects stay fully open. Start with a default-deny policy, then add specific allows.
- Forgetting DNS in egress policies. Blocking egress also blocks name resolution. Allow traffic to the cluster DNS Service on port 53.
- Mixing up AND and OR in selectors. A
podSelectorandnamespaceSelectorin the same list item mean AND, while separate list items mean OR. This is a classic exam trap. - Overlapping Pod and Service CIDRs with the VPC or each other. Overlaps cause hard-to-trace routing failures. Plan the ranges before building the cluster, because they are difficult to change later.
Quick answers
What does a CNI plugin do?
A Container Network Interface plugin gives each Pod an IP address and connects it to the cluster network so Pods can reach each other across nodes. Calico, Cilium and Flannel are common examples. The container runtime calls the plugin when a Pod sandbox is created.
How do you create a default-deny NetworkPolicy?
Create a policy with an empty podSelector, which matches every Pod in the namespace, and list the policy types to block, such as Ingress and Egress, with no allow rules. Then add specific policies that open only the traffic you need.
Does NetworkPolicy work on every cluster?
No. The API accepts the object everywhere, but only a CNI that implements it, such as Calico or Cilium, enforces it. On a CNI without support, policies are silently ignored, so test with a connection that should be blocked instead of trusting the YAML.
Notes
(fill in your own words after watching + labbing)