Kubernetes Autoscaling: HPA, VPA, and Cluster Autoscaler

Autoscaling is where the declarative model earns its money: instead of you watching a dashboard and typing kubectl scale, controllers watch load and adjust the desired state for you. But "autoscaling" is not one thing - it is three separate controllers operating on three different axes, and mixing them up is where most teams get hurt. This guide separates the three, shows how each one actually decides, and walks through the manifests and the tuning knobs you will reach for. It leans on the resource-requests idea from the fundamentals guide, so if requests and limits are fuzzy, start there.

The three axes of scaling

There are three independent ways to give a workload more capacity, and each has its own controller:

• Scale out (more pods) - run more copies of the same pod. This is the Horizontal Pod Autoscaler (HPA). It changes a Deployment's replicas based on load. This is the one you use most.
• Scale up (bigger pods) - give each pod more CPU and memory by raising its requests and limits. This is the Vertical Pod Autoscaler (VPA). It is really about right-sizing, not bursting.
• Scale the cluster (more nodes) - add or remove worker machines so there is somewhere to put the pods. This is the Cluster Autoscaler (or its faster successor, Karpenter).

These stack. A typical setup runs HPA to add pods under load, and Cluster Autoscaler to add nodes when those new pods have nowhere to fit. VPA usually runs on the side for right-sizing, and - as you will see - it does not cooperate with HPA on the same metric. Keep the three axes distinct in your head and most autoscaling confusion evaporates.

Horizontal Pod Autoscaler: scaling pods out

The HPA is a controller that watches a metric across a Deployment's pods and adjusts the replica count to keep that metric near a target. The classic case is CPU: "keep average CPU utilization at 50 percent; if it climbs, add pods; if it drops, remove them." The core formula is simple:

desiredReplicas = ceil(currentReplicas * (currentMetric / targetMetric))

If you have 4 pods averaging 80 percent CPU and your target is 50 percent, the HPA wants ceil(4 * 80/50) = ceil(6.4) = 7 replicas. It scales up, the load spreads across more pods, average utilization falls back toward the target, and the loop settles.

Why it depends on metrics-server and correct requests

The HPA cannot see anything on its own. For CPU and memory it reads from metrics-server, a lightweight cluster add-on that scrapes the kubelet on each node and exposes pod resource usage through the metrics API. No metrics-server, no CPU-based HPA - the HPA just reports unknown for the metric and refuses to scale. On most managed clusters (EKS, GKE, AKS) you install or enable it explicitly; it is not there by default on vanilla Kubernetes.

The subtler dependency is resource requests. HPA's CPU target is a percentage of the pod's CPU request, not of a physical core. If a pod requests 200m and uses 100m, that is 50 percent utilization to the HPA - regardless of how big the node is. This has two consequences:

• If a container has no CPU request set, CPU-based HPA has no denominator and cannot compute utilization at all. It will not scale.
• If the request is wildly wrong, the percentages are meaningless. A too-low request makes every pod look pegged at 300 percent; a too-high request makes a busy pod look idle.

So correct requests are a prerequisite for HPA, not a nice-to-have. This is exactly why people are tempted to run VPA and HPA together, and exactly why that is a trap (more below).

A real HPA manifest

The modern API is autoscaling/v2, which supports multiple metrics and richer behavior. Here is a CPU-driven HPA for the web Deployment:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60

The Deployment those pods belong to must declare a CPU request, or averageUtilization has nothing to divide by:

resources:
  requests: { cpu: "250m", memory: "256Mi" }
  limits:   { cpu: "1",    memory: "512Mi" }

Check what the HPA sees with:

kubectl get hpa web
kubectl describe hpa web    # shows current vs target metric and recent scaling events

If the metric column reads <unknown>/60%, the problem is almost always one of two things: metrics-server is not installed/healthy, or the pods have no CPU request.

HPA on custom and external metrics

CPU is a poor proxy for load in plenty of workloads - a queue consumer or a latency-sensitive API might sit at 20 percent CPU while badly overloaded. autoscaling/v2 lets you scale on:

• Custom metrics - per-pod or per-object metrics from inside the cluster, like HTTP requests per second, exposed through a custom metrics adapter (commonly the Prometheus Adapter).
• External metrics - metrics from outside the cluster, like the depth of an SQS queue or a cloud pub/sub backlog.

  metrics:
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"       # aim for 100 rps per pod
    - type: External
      external:
        metric:
          name: sqs_queue_depth
          selector:
            matchLabels: { queue: orders }
        target:
          type: AverageValue
          averageValue: "30"        # ~30 messages per pod

When you list several metrics, the HPA computes a desired replica count for each and takes the largest - it scales to satisfy the most demanding signal. Custom and external metrics need an adapter serving the custom.metrics.k8s.io or external.metrics.k8s.io API; metrics-server alone only covers CPU and memory.

Tuning: stabilization and thresholds

Naive HPA thrashes: load spikes, it adds 10 pods, load drops, it removes them, load spikes again. Two levers control this.

Stabilization windows make scaling decisions look at a window of recent recommendations instead of the instantaneous value. Scale-down defaults to a 300-second window (it takes the highest recommendation over the last 5 minutes, so it is slow and cautious to remove pods); scale-up has no stabilization by default (it reacts fast). You can tune both, plus policies that cap how fast it moves:

  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100          # at most double the pods
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Pods
          value: 2            # remove at most 2 pods
          periodSeconds: 60

Threshold choice matters as much as stabilization. A target of 80 percent CPU leaves little headroom - by the time the HPA reacts and new pods become ready, you may already be dropping requests. A target of 50 to 60 percent gives room for the scale-up plus pod startup time to catch up. Remember that new pods are not instant: image pull, container start, and the readiness probe all have to pass before a new replica takes traffic, so size your threshold to cover that lag.

Vertical Pod Autoscaler: right-sizing requests

The VPA solves a different problem: not "how many pods" but "how big should each pod be." It watches actual CPU and memory usage over time and recommends (or automatically sets) the requests and limits that fit. It is the antidote to the two failure modes of hand-set requests - reserving far more than you use (wasted, expensive capacity) or reserving too little (pods packed too tight, OOMKills, CPU throttling).

VPA runs in one of a few modes, set by updatePolicy.updateMode:

• Off - compute and expose recommendations only, change nothing. The safest way to start: read the numbers, apply them yourself.
• Initial - set requests only when a pod is first created, never after.
• Auto / Recreate - actively update running pods by evicting and recreating them with the new requests. Powerful, but it means VPA will restart your pods to resize them.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  updatePolicy:
    updateMode: "Off"       # recommend only; you apply the numbers
  resourcePolicy:
    containerPolicies:
      - containerName: web
        minAllowed: { cpu: "100m", memory: "128Mi" }
        maxAllowed: { cpu: "2",    memory: "2Gi" }

Read its recommendation with:

kubectl describe vpa web    # shows Target / Lower Bound / Upper Bound per container

VPA is not built into core Kubernetes the way HPA is - you install it as an add-on (and GKE offers it as a managed feature). Start in Off mode on a real workload, watch the recommendations for a few days, and you will usually find your hand-set requests were off by 2-5x in one direction or the other.

Why HPA and VPA conflict on the same metric

Here is the rule that trips people up: do not run HPA and VPA on the same resource metric for the same workload. They fight.

HPA scales the pod count to hold CPU utilization near a target - it wants pods to sit at, say, 60 percent of their request. VPA in Auto mode watches CPU usage and adjusts the request to match observed usage - it wants the request to track actual consumption. Put them together on CPU and you get a feedback loop: VPA raises the request to match usage, which drops the utilization percentage the HPA sees, which makes the HPA remove pods, which raises per-pod load, which makes VPA raise the request again. The two controllers chase each other and neither settles.

The safe combinations:

• HPA on CPU or memory, VPA off (or VPA on the other dimensions only). VPA can right-size memory while HPA scales on CPU, for example, but only carefully.
• HPA on a custom metric (like rps or queue depth), VPA on CPU/memory. Because they now operate on different signals, they do not fight - VPA sizes each pod correctly while HPA decides how many you need. This is the combination that actually works well together.

When in doubt: HPA for how many, VPA for how big, and never let them arbitrate the same number.

Cluster Autoscaler: adding and removing nodes

HPA and VPA both assume there is somewhere to put the pods. When you scale out and the new pods have no node with enough unreserved capacity to satisfy their requests, they land in Pending - scheduled by no one, waiting. That is precisely the signal the Cluster Autoscaler watches for.

The mechanism is elegant and worth stating plainly: the Cluster Autoscaler does not watch CPU graphs. It watches for pods stuck in Pending because of insufficient resources, and when it sees them it adds a node (by growing a node group / autoscaling group in the cloud) so the scheduler has somewhere to place them. This is why correct requests matter yet again - a Pending pod only triggers a scale-up if its requests genuinely do not fit; a pod with no requests can be scheduled anywhere and will never trigger node scaling, no matter how much it actually uses.

Scale-down is the mirror image: the Cluster Autoscaler looks for underutilized nodes whose pods could all be rescheduled elsewhere, and after a cooldown it drains and removes them. Pods it cannot safely relocate (bare pods with no controller, pods using local storage, pods blocked by a restrictive PodDisruptionBudget) will pin a node and keep it alive - a common reason a cluster refuses to shrink.

The autoscaler itself is configured through your cloud's node groups plus flags on the controller, not a Kubernetes manifest you write per app. The knobs you touch most:

• min and max size per node group - the floor and ceiling on nodes.
• scale-down-utilization-threshold - how empty a node must be before it is a candidate for removal (default around 50 percent).
• scale-down-unneeded-time - how long a node must look unneeded before it is drained (default ~10 minutes), which prevents flapping.

You can protect critical workloads from disruptive scale-down with a PodDisruptionBudget, which caps how many pods of a set can be down at once:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web
spec:
  minAvailable: 2
  selector:
    matchLabels: { app: web }

Debugging a cluster that will not scale up is the same workflow as any Pending pod (see the fundamentals guide):

kubectl get pods --field-selector status.phase=Pending
kubectl describe pod <pending-pod>    # events explain why: insufficient cpu/memory, taints, etc.

If describe says "0/5 nodes are available: insufficient memory" and the autoscaler is not adding nodes, check that the node group has not hit its max size, and that nothing about the pod (a node selector, a taint toleration, an affinity rule) makes it un-schedulable on any node the autoscaler could add. Karpenter is the newer alternative on AWS that provisions right-sized nodes directly from Pending pods rather than scaling fixed node groups, and it is generally faster and packs better.

Event-driven scaling and scale-to-zero

Plain HPA has a hard floor: minReplicas cannot be less than 1, so you always keep at least one pod running and paying. For bursty or idle-most-of-the-time workloads (a nightly batch consumer, a rarely-used internal tool), that idle pod is pure waste.

KEDA (Kubernetes Event-Driven Autoscaling) fills this gap. It is an add-on that acts as a metrics source and a manager on top of HPA, with a long list of scalers for event sources - Kafka lag, RabbitMQ queue length, Redis list depth, cloud queues, cron schedules, Prometheus queries, and dozens more. Its headline feature is scale-to-zero: when there are no events (an empty queue), KEDA scales the Deployment down to 0 replicas, and spins the first pod back up the moment an event arrives.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: consumer
spec:
  scaleTargetRef:
    name: consumer
  minReplicaCount: 0        # scale all the way to zero when idle
  maxReplicaCount: 30
  triggers:
    - type: kafka
      metadata:
        topic: orders
        bootstrapServers: kafka:9092
        consumerGroup: orders-consumer
        lagThreshold: "50"     # ~50 messages of lag per pod

Under the hood KEDA generates and manages an HPA for you when replicas are 1 or more, and handles the 0-to-1 and 1-to-0 transitions itself (which vanilla HPA cannot do). For anything driven by a queue or an external event rather than CPU, KEDA is usually the right tool.

Common pitfalls

Most autoscaling incidents come from the same short list:

• No resource requests set. This breaks all three axes at once: CPU-based HPA has no denominator and does nothing, VPA has no baseline, and Cluster Autoscaler never sees a "does not fit" signal because a request-less pod fits anywhere. Set sane requests before you set up any autoscaler. It is the single most common cause of "my HPA does nothing."

• No metrics-server (or an unhealthy one). The HPA shows <unknown> for the metric and silently refuses to scale. Confirm with kubectl top pods - if that errors, metrics-server is your problem, not the HPA.

• Slow or stale metrics. metrics-server polls on an interval (~15s) and the HPA reconciles every ~15s, so there is inherent lag between a load spike and a scale-up, on top of pod startup time. If your traffic spikes faster than pods can come up, no HPA setting saves you - you need headroom in the threshold, faster-starting pods, or pre-warmed capacity.

• Thrashing. Aggressive thresholds with no stabilization make the replica count oscillate, which churns pods and destabilizes the service. Use the scale-down stabilization window (the 300s default is there for a reason) and rate-limit policies in behavior.

• HPA and VPA fighting on the same metric. Covered above - the feedback loop where neither settles. Split them across different signals or turn one off.

• A cluster that will not scale down. Almost always a pod that cannot be evicted - a bare pod, one using node-local storage, or one guarded by a PodDisruptionBudget that leaves no room to drain. Check which pods are pinning the underutilized node.

• Thresholds tuned with no headroom for startup. An HPA targeting 85 percent CPU has already run out of runway by the time new pods are Ready. Leave room for the reaction plus the pod's own boot time.

The shape of it

Autoscaling is three controllers on three axes, and keeping them straight is most of the battle. HPA changes how many pods you run, driven by a metric it reads through metrics-server (CPU/memory) or an adapter (custom/external), and everything hinges on correct resource requests because its targets are percentages of those requests. VPA changes how big each pod is by right-sizing requests, and it must not arbitrate the same metric as an HPA or the two loop forever - so pair them only across different signals. Cluster Autoscaler changes how many nodes exist, and it reacts not to CPU but to Pending pods that do not fit, which is one more reason requests have to be right. KEDA extends the picture to event sources and the scale-to-zero that plain HPA cannot do. Get requests right first, install metrics-server, then layer the autoscalers on - in that order autoscaling stops being magic and becomes just the control loop doing your capacity planning for you.