KEDA: Zero to Hero

Kubernetes Event-Driven Autoscaling — scale workloads on what actually drives load (queue depth, Kafka lag, HTTP rate, a cron window), not just CPU, and all the way down to zero. You'll install KEDA, scale a Deployment on a Prometheus metric and a queue, scale jobs, wire up authentication, and adopt production patterns. Hands-on throughout.

The journey:

1. Why event-driven autoscaling
2. Install KEDA
3. Your first ScaledObject
4. Scalers — the trigger catalog
5. Scale to zero and back
6. Authentication for triggers
7. ScaledJobs for queue workers
8. How KEDA relates to the HPA
9. Production practices

---

Prerequisites

New to Kubernetes autoscaling? Skim the HPA section of the Kubernetes tutorial first. Confirm your cluster: kubectl get nodes.

---

Part 1 — Why event-driven autoscaling

The built-in HorizontalPodAutoscaler scales on CPU/memory — a poor proxy for many workloads. A queue consumer with 10,000 backlogged messages may be at 20% CPU; you want to scale on the backlog, not the CPU. KEDA lets you scale on the signal that actually represents demand:

• Kafka consumer lag, RabbitMQ/SQS queue length
• A Prometheus query (requests/sec, p99 latency)
• A cron schedule (pre-scale before a daily spike)
• 60+ built-in scalers for clouds, databases, and messaging

And critically, KEDA can scale to zero — run no pods when there's no work, then spin up on the first event. That's serverless-style economics on plain Kubernetes.

---

Part 2 — Install KEDA

helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm install keda kedacore/keda --namespace keda --create-namespace

kubectl get pods -n keda      # keda-operator + metrics-apiserver Running

KEDA installs two components: the operator (watches ScaledObjects and manages scaling) and a metrics adapter (feeds external metrics to Kubernetes' HPA machinery). It adds CRDs: ScaledObject, ScaledJob, and TriggerAuthentication.

---

Part 3 — Your first ScaledObject

A ScaledObject points at a Deployment and defines the triggers that scale it. Here, scale on a Prometheus query:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: api-scaler
spec:
  scaleTargetRef:
    name: api                 # the Deployment to scale
  minReplicaCount: 1
  maxReplicaCount: 20
  cooldownPeriod: 300         # wait before scaling back down
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus.monitoring:9090
        query: sum(rate(http_requests_total{service="api"}[1m]))
        threshold: "100"      # target ~100 req/s per replica

kubectl apply -f scaledobject.yaml
kubectl get scaledobject       # READY/ACTIVE columns
kubectl get hpa                # KEDA created an HPA under the hood

KEDA divides the metric by the threshold to compute desired replicas — 500 req/s ÷ 100 = 5 replicas. minReplicaCount: 1 keeps one pod warm; set it to 0 for scale-to-zero (Part 5).

---

Part 4 — Scalers: the trigger catalog

A scaler knows how to read one source. Common ones:

You can list multiple triggers on one ScaledObject — KEDA scales to satisfy the most demanding one. A cron trigger is great for known daily peaks:

triggers:
  - type: cron
    metadata:
      timezone: UTC
      start: "0 8 * * *"      # scale up at 08:00
      end: "0 20 * * *"       # back down at 20:00
      desiredReplicas: "10"

---

Part 5 — Scale to zero and back

Set minReplicaCount: 0 and KEDA removes all pods when the trigger reports no work — then activates the workload on the first event:

spec:
  minReplicaCount: 0
  maxReplicaCount: 30
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: kafka:9092
        consumerGroup: orders
        topic: orders
        lagThreshold: "50"

With zero replicas, KEDA (not the HPA) watches the source directly; when lag appears it scales 0→1, then the HPA takes over for 1→N. This is ideal for bursty consumers and batch work — you pay for pods only when there's a backlog. Note: scale-to-zero suits asynchronous workloads; a synchronous HTTP service scaled to zero adds cold-start latency (consider the HTTP add-on for that).

---

Part 6 — Authentication for triggers

Most real sources need credentials. Keep them out of the ScaledObject with a TriggerAuthentication that references a Secret:

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: kafka-auth
spec:
  secretTargetRef:
    - parameter: sasl
      name: kafka-secret
      key: sasl
    - parameter: password
      name: kafka-secret
      key: password
---
# reference it from the trigger
triggers:
  - type: kafka
    authenticationRef:
      name: kafka-auth
    metadata: { bootstrapServers: kafka:9092, topic: orders, consumerGroup: orders, lagThreshold: "50" }

TriggerAuthentication also supports pod identity (IRSA, workload identity) so cloud scalers assume a role instead of holding static keys — the least-privilege pattern from the Vault and AWS tutorials.

---

Part 7 — ScaledJobs for queue workers

For work that runs to completion (batch, video encode, one message = one job), use a ScaledJob — KEDA creates Kubernetes Jobs per unit of work instead of scaling a long-running Deployment:

apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: encoder
spec:
  jobTargetRef:
    template:
      spec:
        containers:
          - name: worker
            image: encoder:1.2
        restartPolicy: Never
  maxReplicaCount: 50
  triggers:
    - type: aws-sqs-queue
      metadata:
        queueURL: https://sqs.../jobs
        queueLength: "5"      # one job per ~5 messages

ScaledObject = scale a service; ScaledJob = spawn jobs for discrete work. Pick ScaledJob when each item should be processed by a fresh, isolated pod that exits when done.

---

Part 8 — How KEDA relates to the HPA

KEDA doesn't replace the HPA — it drives it. For a ScaledObject with minReplicaCount ≥ 1, KEDA creates and manages an HPA that scales on external metrics KEDA feeds via its metrics adapter. From 0→1 (activation), KEDA acts directly; from 1→N, the HPA does the math.

Practical consequences: don't create your own HPA for the same Deployment (they'll fight — the same lesson as HPA-vs-VPA); the ScaledObject owns the HPA. And CPU/memory triggers are just KEDA configuring the standard HPA resource metrics.

---

Part 9 — Production practices

• Scale on the real demand signal (lag, queue length, RPS), not CPU, for event-driven workloads.
• Set maxReplicaCount sensibly so a metric spike (or a broken source) can't scale you into a bill.
• Tune cooldownPeriod and pollingInterval to avoid flapping; add HPA behavior stabilization if needed.
• Never hand-create an HPA for a KEDA-managed target — the ScaledObject owns it.
• Use TriggerAuthentication + pod identity, never inline credentials.
• Scale-to-zero for async only; mind cold starts for synchronous services.
• Ensure the metric source is reliable — if Prometheus/Kafka is unreachable, scaling stalls; alert on it.
• Right-size pod requests so the HPA math and the scheduler behave.

---

Common mistakes to avoid

• Scaling on CPU when the real signal is a backlog. The whole point of KEDA is the demand-representative metric.
• Creating a separate HPA for the target. It conflicts with the one KEDA manages.
• No maxReplicaCount cap. A runaway metric scales you into an outage or a huge bill.
• Scale-to-zero on a synchronous HTTP service. Cold starts hurt user latency; use the HTTP add-on or keep min ≥ 1.
• Inlining secrets in the ScaledObject. Use TriggerAuthentication + a Secret or pod identity.
• Ignoring source outages. If the trigger source is down, scaling silently freezes — monitor it.
• Too-aggressive polling/cooldown. Causes replica flapping; tune the intervals.

---

Practice: try it yourself

1. Install + inspect. Helm-install KEDA and confirm the operator and metrics adapter are Running, and the CRDs exist.
2. Prometheus scaler. Point a ScaledObject at a Deployment with a prometheus trigger; generate load and watch replicas track the query.
3. Cron pre-scale. Add a cron trigger that scales to 10 during a window and confirm the schedule takes effect.
4. Scale to zero. Set minReplicaCount: 0 with a queue trigger; drain the queue, watch pods go to 0, then enqueue and watch 0→1.
5. ScaledJob. Convert a batch worker to a ScaledJob keyed on queue length and watch Jobs spawn per backlog.

Self-check:

• Why scale on queue lag instead of CPU for a consumer?
• What two components does KEDA install, and what does each do?
• How does KEDA relate to the HPA — who owns it?
• When do you use a ScaledJob instead of a ScaledObject?

---

Where to go next

• Feed it metrics — the Prometheus tutorial; most KEDA triggers read PromQL.
• Generate the YAML with the KEDA ScaledObject Generator.
• Practice with Kubernetes/SRE interview questions in the question bank.

You now have the loop: install → ScaledObject → pick a scaler → scale to zero → authenticate → ScaledJobs → coexist with the HPA. That's event-driven autoscaling in production.