How Autoscaling Actually Decides to Scale

"Autoscaling" gets used as a single word for three different things, and conflating them is where most confusion starts.

• Horizontal scaling adds or removes running instances (Pods) to match demand.
• Vertical scaling changes the resources — CPU, memory — allocated to instances that are already running.
• Cluster-level scaling adds or removes the underlying nodes a cluster runs on, usually triggered when Pods can't be scheduled due to insufficient node capacity.

This piece covers the first one — Kubernetes' Horizontal Pod Autoscaler (HPA) — since it's the mechanism most people mean when they say "autoscaling," and its logic is publicly documented in enough detail to describe precisely rather than approximately.

The control loop, mechanically

The HPA is not a continuous, always-watching process. It's a control loop that runs intermittently, and the interval between runs is a configurable value on the cluster's controller manager, defaulting to 15 seconds per Kubernetes' current documentation.

On each cycle, the controller does four things in sequence:

1. Finds the target workload (a Deployment or StatefulSet) via the HPA's scaleTargetRef.
2. Selects the Pods belonging to that workload.
3. Pulls the relevant metric for each Pod — CPU and memory come from the built-in metrics API; anything else comes from a custom or external metrics API.
4. Computes a ratio between the current metric value and the target value, and uses that ratio to decide the new replica count.

Nothing scales between cycles. If a traffic spike hits at second 1 and resolves by second 14, the HPA may never even register it happened.

The actual formula

This is the part most explainers skip or approximate. Kubernetes' own documentation states the calculation directly:

desiredReplicas = ceil(currentReplicas × (currentMetricValue / desiredMetricValue))

If current CPU usage is 200m against a target of 100m, replicas double, since 200 ÷ 100 = 2.0. If current usage drops to 50m against the same target, replicas halve, since 50 ÷ 100 = 0.5.

Two details in the official docs matter more than the formula itself:

• A tolerance band prevents constant micro-adjustments. The control plane skips scaling entirely if the ratio is close enough to 1.0 — within a configurable tolerance that defaults to 0.1. Without this, a workload sitting almost exactly at target would flap up and down every cycle over noise.
• Not-ready and missing-metric Pods get handled conservatively, not ignored. If a Pod's metrics are missing, the controller sets it aside and recomputes more cautiously — assuming it's using 100% of target during a scale-down decision, and 0% during a scale-up decision — specifically to avoid overreacting to incomplete data.

Why scaling down is deliberately slower than scaling up

Kubernetes doesn't act on a single scaling recommendation the moment it's computed. Right before the HPA actually scales the target, the controller records the recommendation, then chooses the highest recommendation from within a configurable window — defaulting to 5 minutes — before acting on a scale-down. That's the downscale stabilization window, and its entire purpose is to prevent thrashing: without it, a brief dip in traffic would trigger an immediate scale-down, only for the next cycle's traffic spike to trigger an immediate scale-up again.

There's no equivalent forced delay on scale-ups by default — the asymmetry is intentional. Under-provisioning during real demand is a worse failure mode than briefly over-provisioning during a lull, so Kubernetes' defaults are biased toward reacting fast when load increases and reacting cautiously when load decreases.

Where this breaks down in practice

The mechanics above are precise, but the defaults built around them have real limits worth naming plainly rather than glossing over:

• The reaction is never instant. Between the 15-second poll interval, the time it takes new Pods to actually start serving traffic, and the tolerance band absorbing small fluctuations, there's an unavoidable lag between "load increased" and "capacity caught up." For workloads with CPU-based scaling specifically, a newly-started Pod also gets a grace period — a configurable initial readiness delay, defaulting to 30 seconds, and a longer CPU-initialization period, defaulting to 5 minutes — before its metrics are trusted at all.
• Averages hide spikes. The default calculation uses the mean metric value across all targeted Pods. A workload with wildly uneven request costs — some cheap, some expensive — can have a healthy average while a subset of Pods are actually overloaded. The HPA has no visibility into that distribution by default.
• CPU and memory aren't always the metric that matters. For request-latency-sensitive services, scaling on CPU alone can miss the actual bottleneck entirely; Kubernetes does support scaling on custom and external metrics for exactly this reason, but it has to be configured deliberately — it isn't the default.

I'd treat any claim that autoscaling "reacts instantly" or "just works" out of the box with suspicion. The tuning knobs exist precisely because the defaults are a reasonable general-purpose starting point, not a correct answer for every workload shape.

Quick reference: key defaults

---

A note on version history: older Kubernetes documentation and several still-circulating blog posts state the default sync period as 30 seconds. Kubernetes' current documentation states 15 seconds. If you're working against a specific cluster, check that cluster's actual kube-controller-manager flags rather than assuming either figure — defaults have changed across Kubernetes versions, and what's written here reflects the current upstream docs, not necessarily what an older managed cluster is running.

References

• Kubernetes, "Horizontal Pod Autoscaling" — official documentation, including the algorithm details section
• Google Cloud, "Horizontal Pod autoscaling | GKE"