19 - Runtime & Network Security

Why this matters

Prevention (topics 16-18) will eventually fail once — you need detection: something watching what's actually happening at runtime and alerting/blocking on suspicious behavior.

Read this first — Definitions & Explanations

Runtime security

Detect suspicious behavior while workloads run (unexpected process, shell in container, crypto miner patterns). Tools in the Falco family are examples of this class.

Egress control

NetworkPolicies can limit outbound connections so a compromised Pod can’t freely exfiltrate data.

mTLS between services

mutual TLS so services authenticate each other and encrypt traffic — often provided by a service mesh.

Segmentation

Separate sensitive workloads by namespace + network policy + strict RBAC to reduce blast radius.

Privileged containers

privileged: true / hostNetwork / hostPID are powerful and dangerous. Use only with strong justification.

Official docs (read for detail)

• Network Policies
• Security Checklist
• Falco documentation
• Linux kernel security features used by Kubernetes

Key Concepts

• Falco: runtime security — detects anomalous syscalls/behavior (shell spawned in a container, unexpected outbound connection, file write to sensitive path)
• Falco rules — how detections are written and tuned
• Network security beyond basic NetworkPolicy: egress restriction to prevent data exfiltration, DNS-based policy (Cilium), mTLS between services (ties into service mesh, topic 23)
• Incident response basics for a compromised pod: isolate (NetworkPolicy deny-all), capture (logs, kubectl cp forensics), don't just delete-and-forget

YouTube search terms

• "Falco Kubernetes runtime security tutorial"
• "Kubernetes egress NetworkPolicy data exfiltration prevention"
• "Kubernetes incident response compromised pod"

Hands-on lab (on prod-sim)

# Install Falco
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm install falco falcosecurity/falco -n falco --create-namespace \
  --set tty=true

# Watch Falco's live output
kubectl -n falco logs -l app.kubernetes.io/name=falco -f &

# Trigger a real detection: spawn a shell inside a running container (classic red flag)
kubectl run victim --image=nginx
kubectl exec -it victim -- /bin/bash
  whoami; exit
# check the Falco log tab — you should see a "Terminal shell in container" alert

# Trigger another: write to a sensitive path
kubectl exec victim -- sh -c "touch /etc/newfile"
# Falco should flag "Write below etc"

kill %1   # stop tailing falco logs

# Egress lockdown drill: deny-all egress except DNS + one destination
kubectl create namespace locked-egress
kubectl -n locked-egress run app --image=nginx
cat <<EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-egress
  namespace: locked-egress
spec:
  podSelector: {}
  policyTypes: ["Egress"]
  egress:
  - to:
    - namespaceSelector: {}
    ports:
    - protocol: UDP
      port: 53
    - protocol: TCP
      port: 53
EOF
kubectl -n locked-egress exec app -- curl -m2 -sS http://example.com   # should fail/timeout
kubectl -n locked-egress exec app -- nslookup example.com              # DNS still works

Notes

(fill in your own words after watching + labbing)