How does the Horizontal Pod Autoscaler (HPA) scale workloads, and what metrics beyond CPU can drive scaling decisions?

The Horizontal Pod Autoscaler watches a scalable resource—typically a Deployment—and periodically evaluates metrics to decide whether to increase or decrease replica count. Every 15 seconds (by default), the HPA controller queries the metrics API. For resource metrics like CPU, it compares average utilization across Pods against a target percentage of each Pod's configured request. If payments-api Pods average 85% CPU against a 70% target with 3 replicas, HPA calculates desired replicas: ceil(3 × 85 / 70) = 4 and patches the Deployment.

CPU-based HPA requires resource requests to be set on containers; without requests, utilization percentages are meaningless. Memory-based HPA is supported but less common because memory is not compressible—a sudden spike can OOMKill before scale-out completes. Production teams often prefer custom metrics tied to business signals.

Custom metrics expose application-specific values through the custom.metrics.k8s.io API, usually via prometheus-adapter. For checkout-worker processing orders from a queue, scaling on rabbitmq_queue_messages_ready provides faster reaction than CPU lag. External metrics (external.metrics.k8s.io) reference metrics without associating them to Kubernetes objects—cloud provider queue length or Pub/Sub backlog.

HPA respects scale-down stabilization windows (default 300 seconds) to prevent flapping—rapid scale-up followed by immediate scale-down when a traffic spike subsides. Scale-up can be aggressive by default. Behavior policies let architects tune percentages and periods per direction.

Analogy: HPA is a thermostat for your fleet. CPU target is the temperature setpoint. When the room heats up (traffic increases), the AC adds capacity (replicas). Custom metrics are like humidity sensors—sometimes humidity, not temperature, is the better signal for comfort.

HPA does not create nodes; Cluster Autoscaler (or Karpenter) must provision capacity if pending Pods cannot schedule. Always verify the full loop: metric → HPA → Deployment → scheduler → nodes.

Multiple metrics in one HPA use OR logic—the highest calculated replica count wins. Object metrics (type: Object) scale on a single metric for an entire Service or Ingress rather than per-Pod averages. KEDA extends this model with event-driven scalers for Kafka, Redis, and cron schedules. When tuning payments-api from 3 to 12 replicas, ensure PDB minAvailable and node pool headroom can absorb the surge without scheduling delays.