Kyverno: Zero to Hero

Policy-as-code for Kubernetes — written in YAML, not a new language. You'll install Kyverno, write policies that validate, mutate, and generate resources, verify image signatures, run policies in CI, and roll them out safely from audit to enforce. Hands-on throughout.

The journey:

1. Why a policy engine
2. Install Kyverno
3. Your first validate policy
4. Audit vs Enforce — rolling out safely
5. Mutate — fix resources automatically
6. Generate — create resources from policy
7. Verify images (supply chain)
8. Testing policies in CI
9. Production practices

---

Prerequisites

Kyverno needs no new language — if you can read Kubernetes YAML, you can write policies. Confirm your cluster: kubectl get nodes.

---

Part 1 — Why a policy engine

Admission control is the checkpoint between kubectl apply and the resource being stored. A policy engine hooks in there to reject, modify, or react to resources against your rules — enforcing standards no reviewer can consistently catch: "no :latest images," "every pod has resource limits," "only our registries," "required owner labels."

Kyverno's differentiator vs OPA/Gatekeeper: policies are Kubernetes resources in YAML with familiar match/pattern syntax, so there's no Rego to learn. It does four things — validate, mutate, generate, and verifyImages — covering most platform-guardrail needs.

---

Part 2 — Install Kyverno

helm repo add kyverno https://kyverno.github.io/kyverno
helm repo update
helm install kyverno kyverno/kyverno --namespace kyverno --create-namespace

kubectl get pods -n kyverno       # admission/background/cleanup controllers

Kyverno registers as a ValidatingWebhookConfiguration/MutatingWebhookConfiguration, so it sees resources as they're created or updated. Policies come as ClusterPolicy (cluster-wide) or Policy (namespaced) CRDs.

---

Part 3 — Your first validate policy

A validate rule checks a resource against a pattern and blocks (or audits) violations. Require every pod to set resource limits:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-limits
spec:
  validationFailureAction: Audit      # start in Audit (see Part 4)
  rules:
    - name: require-container-limits
      match:
        any:
          - resources: { kinds: [Pod] }
      validate:
        message: "Every container must set resources.limits."
        pattern:
          spec:
            containers:
              - resources:
                  limits:
                    memory: "?*"       # ?* = any non-empty value
                    cpu: "?*"

kubectl apply -f require-limits.yaml
kubectl run nginx --image=nginx        # a pod with no limits → policy report flags it
kubectl get policyreport -A            # violations show up here

The pattern is an "overlay" the resource must match; ?* means "must be present and non-empty." Kyverno reports violations to PolicyReports you can query and dashboard.

---

Part 4 — Audit vs Enforce — rolling out safely

validationFailureAction controls what a violation does:

• Audit — allow the resource, record a violation in PolicyReports. Use this first, everywhere.
• Enforce — reject the resource at admission.

The safe rollout: ship as Audit, watch PolicyReports for what would break, fix the offending workloads, then flip to Enforce. Flipping to Enforce blindly will block deploys and page you.

kubectl get clusterpolicyreport -o wide     # cluster-wide violation counts

You can also scope policies with match/exclude (skip kube-system, target only certain namespaces) so guardrails don't break platform components.

---

Part 5 — Mutate — fix resources automatically

A mutate rule modifies resources on admission — apply sane defaults instead of rejecting. Add a default securityContext so workloads are hardened by default:

spec:
  rules:
    - name: default-securitycontext
      match:
        any:
          - resources: { kinds: [Pod] }
      mutate:
        patchStrategicMerge:
          spec:
            securityContext:
              runAsNonRoot: true
              seccompProfile:
                type: RuntimeDefault

Mutation is powerful for platform ergonomics: inject labels, sidecars, imagePullSecrets, or default resources so teams get compliant defaults without editing every manifest. Prefer mutate (fix it) over validate (reject it) where a safe default exists.

---

Part 6 — Generate — create resources from policy

A generate rule creates new resources when a trigger appears — e.g. give every new namespace a default-deny NetworkPolicy and a resource quota automatically:

spec:
  rules:
    - name: default-netpol
      match:
        any:
          - resources: { kinds: [Namespace] }
      generate:
        apiVersion: networking.k8s.io/v1
        kind: NetworkPolicy
        name: default-deny
        namespace: "{{request.object.metadata.name}}"
        synchronize: true            # keep it in sync; recreate if deleted
        data:
          spec:
            podSelector: {}
            policyTypes: [Ingress, Egress]

generate turns "every namespace should have X" into an enforced guarantee, with synchronize: true re-creating the resource if someone deletes it. This is how platform teams bootstrap namespaces consistently.

---

Part 7 — Verify images (supply chain)

verifyImages blocks unsigned or untrusted images at admission — supply-chain security enforced in the cluster:

spec:
  rules:
    - name: verify-signature
      match:
        any:
          - resources: { kinds: [Pod] }
      verifyImages:
        - imageReferences: ["ghcr.io/acme/*"]
          attestors:
            - entries:
                - keyless:
                    subject: "https://github.com/acme/*"
                    issuer: "https://token.actions.githubusercontent.com"

Combined with signing in CI (cosign, keyless via OIDC), this guarantees only images your pipeline built and signed can run — closing the "someone deployed a random image" gap. Pair it with the least-privilege and provenance ideas from the GitHub Actions tutorial.

---

Part 8 — Testing policies in CI

Catch policy violations at PR time, not at admission, with the Kyverno CLI:

# Test that a policy gives the expected result on sample resources
kyverno apply require-limits.yaml --resource bad-pod.yaml
kyverno test .        # runs kyverno-test.yaml assertions in the repo

Wire kyverno apply/kyverno test into CI so a manifest that would be rejected fails the pull request. This shifts policy left — developers see the violation in their PR with the exact message, long before the cluster rejects it.

---

Part 9 — Production practices

• Start every policy in Audit, read PolicyReports, then flip to Enforce.
• Exclude system namespaces (kube-system, kyverno) so guardrails don't break the platform.
• Prefer mutate over validate where a safe default exists — fix, don't reject.
• Version policies in Git, review them, and test with the CLI in CI.
• Use generate + synchronize to guarantee per-namespace defaults (NetworkPolicy, quotas).
• Enforce image provenance with verifyImages + signing.
• Watch webhook latency/availability — a failing admission webhook can block all deploys (set failurePolicy deliberately).
• Layer with Pod Security Admission — PSA for the coarse baseline, Kyverno for everything PSA can't express.

---

Common mistakes to avoid

• Shipping straight to Enforce. You'll block deploys cluster-wide. Audit first, always.
• Not excluding system namespaces. Policies can break kube-system or Kyverno itself.
• failurePolicy: Fail without HA. If the webhook is down, every admission fails — cluster-wide outage.
• Rejecting when you could mutate. Adding a default label/securityContext is friendlier than a hard reject.
• Policies only in the cluster, not in CI. Developers hit failures late; test with kyverno test.
• Forgetting synchronize on generate. A deleted generated resource won't come back without it.
• Overlapping mutate rules. Order and idempotency matter; test the combined effect.

---

Practice: try it yourself

1. Install + validate. Helm-install Kyverno, apply a require-limits policy in Audit, create a bad pod, and find it in kubectl get policyreport.
2. Safe rollout. Fix the workload, flip the policy to Enforce, and confirm a non-compliant pod is now rejected with your message.
3. Mutate defaults. Write a mutate rule that injects runAsNonRoot: true and verify a plain pod gets it.
4. Generate. Add a generate rule that creates a default-deny NetworkPolicy in every new namespace; create a namespace and confirm.
5. Test in CI. Use kyverno apply against a good and a bad manifest and see the pass/fail.

Self-check:

• What are Kyverno's four rule types, and when do you reach for each?
• Why start in Audit before Enforce?
• What does generate + synchronize guarantee?
• How does Kyverno differ from OPA/Gatekeeper?

---

Where to go next

• Compare policy engines — the Open Policy Agent tutorial (Rego, more general).
• Generate common policies with the Kyverno Policy Generator, then validate manifests with the K8s YAML Validator.
• Practice with Kubernetes security interview questions in the question bank.

You now have the loop: install → validate → audit-then-enforce → mutate → generate → verify images → test in CI. That's Kubernetes policy-as-code in production.