Your HPA (Horizontal Pod Autoscaler) is active, but even when application traffic spikes and CPU usage hits 98%, no new pods are scaling up. Why might this happen?
Quick Answer
The most common cause is that the pod's containers have no CPU requests defined, and HPA calculates utilization as a percentage of the request, not the limit or the node's total capacity — with no request set, the percentage is undefined or treated as zero, so HPA has nothing to scale against even though the node's actual CPU usage is genuinely maxed out. Other frequent causes are the metrics-server not running or not reporting fresh metrics, the HPA already being at maxReplicas, or a scaling policy's stabilization window and behavior limits silently suppressing the scale-up you expect to see immediately.
Detailed Answer
Think of HPA like a thermostat that's supposed to turn on more air conditioners as a room gets hotter, but the thermostat isn't reading room temperature directly — it's reading the percentage difference from a baseline temperature you were supposed to write on a sticky note. If you never wrote a baseline down, the thermostat has no percentage to calculate, so it does nothing, no matter how hot the room actually gets.
Kubernetes designed HPA around relative resource requests rather than absolute usage, specifically so autoscaling behavior stays consistent regardless of node size or instance type — a pod requesting 250m CPU should scale the same way whether it lands on a small or huge node. But this design decision means HPA's core scaling signal, currentCPUUtilizationPercentage, is mathematically defined as (actual usage / requested CPU) × 100. If requests.cpu is unset on the container spec, that denominator is effectively undefined, and the Kubernetes HPA controller either reports the metric as unavailable or, in older versions, silently treats missing requests as a scaling blocker rather than erroring loudly.
Internally, the HPA controller polls the metrics API (backed by metrics-server, which itself scrapes kubelet's cAdvisor-derived stats) on a fixed interval, computes the ratio for each pod, averages across replicas, and compares that average to the target utilization threshold in the HPA spec, only issuing a scale-up when it's confidently above target for a sustained period. If metrics-server is crash-looping, unreachable due to a webhook or network policy misconfiguration, or simply hasn't scraped fresh data within its resync window, the HPA controller has stale or missing data and conservatively does nothing rather than guessing — from the outside this looks identical to 'HPA is broken,' when it's actually behaving exactly as designed given the input it has.
At production scale, teams also get bitten by the HPA behavior policies introduced in newer Kubernetes versions — scaleUp.stabilizationWindowSeconds and scaleUp.policies rate-limiting how many pods can be added per interval, meant to prevent scaling storms from a single traffic blip, but easy to configure so conservatively (say, only 1 pod per 60 seconds with a 5-minute stabilization window) that a genuine, sustained traffic spike scales up far slower than the incident actually requires, feeling like 'no scaling happened' even though it technically did, eventually.
The non-obvious gotcha: HPA respects maxReplicas as a hard ceiling with zero exceptions and zero alerting by default — a team hits their traffic spike, HPA scales correctly and quickly, but silently caps out at the configured maxReplicas of, say, 10, while traffic keeps climbing, and because there's no default alert for 'HPA is at max and still under target load,' the team can spend an hour debugging why 'scaling stopped working' when in reality it worked perfectly and simply ran out of configured headroom.