19 - Runtime & Network Security
Why this matters
Prevention (topics 16-18) will eventually fail once — you need detection: something watching what's actually happening at runtime and alerting/blocking on suspicious behavior.
Read this first — Definitions & Explanations
Runtime security
Detect suspicious behavior while workloads run (unexpected process, shell in container, crypto miner patterns). Tools in the Falco family are examples of this class.
Egress control
NetworkPolicies can limit outbound connections so a compromised Pod can’t freely exfiltrate data.
mTLS between services
mutual TLS so services authenticate each other and encrypt traffic — often provided by a service mesh.
Segmentation
Separate sensitive workloads by namespace + network policy + strict RBAC to reduce blast radius.
Privileged containers
privileged: true / hostNetwork / hostPID are powerful and dangerous. Use only with strong justification.
Official docs (read for detail)
- Network Policies
- Security Checklist
- Falco documentation
- Kubernetes Security Concepts & Architecture Overview
Key Concepts
- Falco: runtime security — detects anomalous syscalls/behavior (shell spawned in a container, unexpected outbound connection, file write to sensitive path)
- Falco rules — how detections are written and tuned
- Network security beyond basic NetworkPolicy: egress restriction to prevent data exfiltration, DNS-based policy (Cilium), mTLS between services (ties into service mesh, topic 23)
- Incident response basics for a compromised pod: isolate (NetworkPolicy deny-all),
capture (logs,
kubectl cpforensics), don't just delete-and-forget
YouTube search terms
- "Falco Kubernetes runtime security tutorial"
- "Kubernetes egress NetworkPolicy data exfiltration prevention"
- "Kubernetes incident response compromised pod"
Hands-on lab (on prod-sim)
# Install Falco
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm install falco falcosecurity/falco -n falco --create-namespace \
--set tty=true
# Watch Falco's live output
kubectl -n falco logs -l app.kubernetes.io/name=falco -f &
# Trigger a real detection: spawn a shell inside a running container (classic red flag)
kubectl run victim --image=nginx
kubectl exec -it victim -- /bin/bash
whoami; exit
# check the Falco log tab — you should see a "Terminal shell in container" alert
# Trigger another: write to a sensitive path
kubectl exec victim -- sh -c "touch /etc/newfile"
# Falco should flag "Write below etc"
kill %1 # stop tailing falco logs
# Egress lockdown drill: deny-all egress except DNS + one destination
kubectl create namespace locked-egress
kubectl -n locked-egress run app --image=nginx
cat <<EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-egress
namespace: locked-egress
spec:
podSelector: {}
policyTypes: ["Egress"]
egress:
- to:
- namespaceSelector: {}
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
EOF
kubectl -n locked-egress exec app -- curl -m2 -sS http://example.com # should fail/timeout
kubectl -n locked-egress exec app -- nslookup example.com # DNS still works
Common mistakes and how to avoid them
- Running privileged containers. They have almost full access to the host. Grant the specific capability you need instead of
privileged: true. - Mounting the container runtime socket. Access to the Docker or containerd socket gives control of every container on the node. Avoid it, including for CI or monitoring helpers.
- Leaving egress open. A compromised Pod can then reach anything. Use a default-deny egress policy and allow only the destinations and DNS the app needs.
- Assuming a service mesh secures everything. mTLS only covers meshed workloads, and a permissive mode still accepts plain traffic. Check that policy is strict where it matters.
- Skipping a seccomp profile. Without one, containers can use the full set of system calls. Set
seccompProfile: RuntimeDefaultas a baseline.
Quick answers
What is runtime security in Kubernetes?
It covers controls that limit and detect what a container does while running. Seccomp and AppArmor profiles restrict system calls, read-only filesystems and dropped capabilities reduce what a process can change, and tools such as Falco alert on suspicious behaviour like a shell starting inside a container.
What is egress control and why use it?
Egress control limits which destinations Pods may connect to, usually with NetworkPolicy. It stops a compromised Pod from calling out to the internet or scanning other services, and it blocks data exfiltration. Allow only the needed destinations, plus DNS, and deny the rest.
What does mTLS between services give you?
Mutual TLS makes both sides of a connection present certificates, so traffic is encrypted and each service proves its identity. A service mesh such as Istio or Linkerd can add it automatically through sidecars, without changing application code.
Notes
(fill in your own words after watching + labbing)