How do HPA and VPA work together for autoscaling in production?

Imagine a restaurant kitchen during peak hours. Horizontal scaling is hiring more cooks to handle more orders in parallel -- each cook handles a portion of the workload. Vertical scaling is upgrading your existing cooks to faster, more skilled chefs who can each handle more complex dishes. In practice, you need both strategies: more cooks for sheer volume, and better-equipped cooks so each one operates efficiently. That is the relationship between HPA and VPA in Kubernetes.

HPA watches specified metrics (CPU use, memory, or custom/external metrics) and adjusts the replica count of a Deployment, ReplicaSet, or StatefulSet. It runs a control loop every 15 seconds (configurable via --horizontal-pod-autoscaler-sync-period) that calculates the desired replicas using the formula: desiredReplicas = ceil(currentReplicas * (currentMetricValue / desiredMetricValue)). VPA, on the other hand, monitors actual resource consumption of Pods over time and recommends or automatically updates the resource requests and limits in the Pod spec. VPA has three modes: Off (recommendations only), Initial (sets requests at Pod creation), and Auto (evicts and recreates Pods with updated requests).

Internally, HPA queries the metrics API (metrics.k8s.io for resource metrics, custom.metrics.k8s.io for custom metrics, or external.metrics.k8s.io for external sources like Prometheus). The metrics-server or a Prometheus adapter populates these APIs. HPA calculates the ratio of current to desired metric values across all Pods, applies a tolerance (default 10%) to prevent flapping, and issues a scale request to the API server. VPA consists of three components: the Recommender (analyzes historical usage and computes recommendations), the Updater (evicts Pods whose requests deviate significantly from recommendations), and the Admission Controller (mutates Pod specs at creation time to inject recommended requests). The Recommender uses a decaying histogram of resource usage to generate its recommendations.

In production, running HPA and VPA together on the same metric (like CPU) creates a conflict. HPA sees high CPU and adds replicas; VPA sees high CPU and increases requests per Pod. Both react to the same signal, leading to over-provisioning or oscillation. The recommended pattern is to use HPA for CPU-based scaling and VPA in recommendation-only mode (mode: Off) so operators can manually adjust requests based on VPA suggestions. Alternatively, use HPA with custom metrics (like requests-per-second from Prometheus) and let VPA manage CPU and memory requests in Auto mode, since they are responding to different signals. Multidimensional Pod Autoscaler (MPA), available in some managed Kubernetes distributions, attempts to coordinate both axes natively.

A non-obvious gotcha is that VPA in Auto mode evicts Pods to apply new resource requests, which means it causes rolling restarts that can impact availability if your PodDisruptionBudget is not configured correctly. Another trap: HPA uses resource requests as the baseline for percentage calculations (e.g., 80% CPU target means 80% of the CPU request), so if VPA increases the request, the same absolute CPU usage now represents a lower percentage, potentially causing HPA to scale in and reduce replicas. This feedback loop can destabilize your scaling behavior. Always set VPA minAllowed and maxAllowed bounds to prevent runaway resource allocation, and use HPA stabilization windows (behavior.scaleDown.stabilizationWindowSeconds) to dampen rapid fluctuations.