Kubernetes Capacity Planning for High-Traffic Apps

Most teams do not discover their capacity plan is wrong during planning. They discover it during a sale, a product launch, or the four minutes after a marketing email goes out — when the dashboard shows CPU at 40% and the pods are still being OOM-killed.

Kubernetes capacity planning is the work of converting a traffic forecast into a defensible number of pods, nodes and spare capacity. It is not the same as autoscaling. Autoscaling reacts; capacity planning decides what the autoscaler is allowed to react to, how fast it can get there, and what happens when it can't.

This guide covers the arithmetic, the multipliers that plans usually omit, and the failure modes specific to high-traffic workloads on Kubernetes.

Why Capacity Planning Is Different on Kubernetes

On a VM fleet, capacity is one number: how many instances. Kubernetes splits that into two independent layers that fail in different ways.

The scheduler spends requests, not usage

The scheduler never looks at how much CPU your pod is actually burning. It looks at resources.requests and nothing else. A pod requesting 2 vCPU that idles at 0.05 still consumes 2 vCPU of schedulable space.

This produces the single most common symptom in over-provisioned clusters: nodes that are 90% "full" and 15% busy. Requests were copied from a template, nobody revisited them, and the cluster is now sized for a fiction.

The corollary matters just as much. If requests are set far below real usage, the scheduler will happily pack pods onto a node that then runs out of memory. CPU over-request wastes money; memory under-request causes outages.

Allocatable is always less than the instance you paid for

An 8 vCPU / 32 GiB node does not give you 8 vCPU and 32 GiB. The kubelet reserves capacity for the system and for eviction thresholds, and your DaemonSets take a slice before any application pod lands.

On a typical EKS m6i.2xlarge you can expect roughly 7.9 vCPU and ~29 GiB allocatable, dropping to around 7.4 vCPU and ~27 GiB once a log shipper, a CNI agent, a node exporter and a service mesh sidecar-injector have taken their share. Plan against allocatable, or your node count will be short by 10–15% before you start.

There is also a hard pod ceiling. Kubernetes defaults to 110 pods per node, and on EKS with the VPC CNI the real limit is often lower and governed by ENI and IP allocation on the instance type. For workloads with many small pods, that ceiling — not CPU or memory — becomes the binding constraint.

Three autoscalers, three different clocks

High-traffic planning fails most often on timing, not on totals.

If your traffic ramps faster than the slowest layer in the path, extra capacity has to already exist. This is the entire justification for headroom, and it is why "we have autoscaling" is not a capacity plan.

One useful change here: in-place pod vertical scaling went GA in Kubernetes v1.35, and v1.36 extended it to pod-level resource budgets. Adjusting a running pod's CPU and memory no longer requires a restart in most cases, which makes vertical rightsizing viable for stateful services that previously couldn't tolerate the churn. See the Kubernetes in-place resize documentation for the current behaviour and limits.

Translating Traffic Forecasts Into Pod and Node Requirements

The method below is four steps. The numbers are illustrative — substitute your own load-test results.

Step 1 — Find the peak, not the average

Use the busiest five-minute window, not the daily mean, and not the hourly mean. Hourly averaging routinely hides a 3× intra-hour spike.

Pull this from your existing metrics rather than from a spreadsheet. A max_over_time query against your request-rate metric across the last 90 days, bucketed at one minute, gives you both the peak and — more usefully — the shape of the ramp.

Two things to record alongside the number:

• Ramp rate. Going from 10k to 48k RPS over 40 minutes is a different problem from doing it in 90 seconds.
• Concurrency, if your workload is latency-bound. For long-lived connections, websockets, or streaming, requests per second is the wrong unit. Use concurrent connections or in-flight requests.

For the worked example: 48,000 RPS at peak, ramping over roughly six minutes.

Step 2 — Measure what one pod can actually do

This is the step people skip, and skipping it makes everything downstream fiction.

Run a single pod with the resource requests you intend to ship, behind a load generator, and increase load until P99 latency starts to bend upward — not until the pod falls over. The knee of that curve is your safe per-pod throughput. Record:

• Safe RPS at target latency
• CPU and memory consumed at that point
• Time from container start to first healthy request

Set requests near observed usage at the knee, with limits above it for CPU-bursty work. For the example: 220 RPS per pod at 0.5 vCPU and 1.2 GiB, with a 45-second warm-up before the JIT and connection pool settle.

A common trap: testing one pod in isolation ignores shared bottlenecks. If 200 pods will contend for the same database connection pool, per-pod throughput at scale will be lower than your single-pod test suggests. Validate at a realistic replica count before committing.

Step 3 — Replicas at peak

48,000 RPS ÷ 220 RPS per pod = 219 pods
219 pods × 0.5 vCPU  = 110 vCPU of requests
219 pods × 1.2 GiB   = 263 GiB of requests

This becomes your HPA maxReplicas — a derived number, not a round one. If maxReplicas is 200 because someone liked the look of it, you have capped yourself below peak by 9%.

Step 4 — Pods to nodes

Pods per node is the minimum of three limits: CPU, memory, and the pod-count ceiling.

Now the full chain, including the multipliers from the diagram:

Two distinct numbers come out of this, and conflating them is a classic error:

• 24 nodes is what you keep running at peak.
• 36 nodes is what the node pool must be able to reach. It is a limit, not a bill — you only pay for it during an actual failover.

A node pool capped at 24 will refuse to scale during a zone failure, and that refusal will look like an application bug for the first ten minutes of the incident.

Headroom, Overcommit and Bin-Packing Strategy

How much headroom, and why

Headroom is not a safety blanket. It is a purchased substitute for the time your autoscaler needs.

Work out how long it takes to add real capacity, then buy enough spare to cover demand growth during that window.

Realistic end-to-end: 1–8 minutes. If your traffic can double inside that window, headroom must cover the doubling.

Practical defaults, assuming a modern setup:

• Steady-state services: 15–20% headroom
• Spiky consumer traffic: 25–35%
• Known events — sales, launches, match days: pre-scale to forecast peak and disable scale-down for the window

Headroom that only exists on paper is worthless. The reliable mechanism is balloon pods: low-priority placeholder pods running pause, sized to your typical workload pod, that real workloads preempt instantly. The node is already warm, the image is already pulled, and you have converted a four-minute node provisioning delay into a two-second eviction.

Overcommit: yes for CPU, no for memory

CPU is compressible. A pod exceeding its CPU request gets throttled, which is bad for latency but not fatal. Memory is not compressible — exceeding available memory gets the pod OOM-killed, and if it happens at the node level it takes neighbours with it.

The rule that holds up in production:

• CPU: set requests at P50–P75 of observed usage, limits at P99 or omitted entirely for latency-critical services. Overcommitting CPU 1.5–2× across a node is normal and safe.
• Memory: set requests at P99 of observed usage, limits equal to requests. Do not overcommit memory. Ever.

Guaranteed QoS — requests equal to limits on both dimensions — is worth it for your handful of genuinely critical services. It costs more and it is the only way to be confident about eviction ordering.

Bin-packing: node size is a real decision

The same total capacity behaves very differently depending on how it's divided.

A workable heuristic: no single pod should request more than about 25% of a node's allocatable capacity. Above that, the scheduler starts leaving unusable gaps, and your effective packing efficiency collapses.

The biggest source of packing waste is heterogeneous pod sizes on homogeneous nodes. A node pool of 8 vCPU machines serving pods that request 0.5, 3 and 6 vCPU will strand capacity constantly. Either standardise pod sizes into two or three tiers, or use a provisioner that picks instance types per workload shape — which is exactly what Karpenter does, and where most of its cost advantage comes from.

Planning for Burst Traffic and Multi-AZ Failover

The burst is a latency problem

Everything above sizes for a peak you can see coming. Burst traffic is the peak you can't.

Three mechanisms, in order of how much they help:

1. Balloon pods — as above, the single highest-leverage change for burst response.
2. Aggressive scale-up, conservative scale-down. The HPA's default scale-up policy allows doubling or +4 pods per 15 seconds with no stabilisation window; scale-down has a 300-second stabilisation window by default. Keep scale-up fast, and lengthen scale-down rather than shortening it. Flapping during a burst is worse than holding capacity for an extra five minutes. The HPA documentation covers the full behaviour block.
3. Scale on the leading indicator. CPU is a lagging signal — by the time it rises, latency has already degraded. Scale on queue depth, in-flight requests, or an upstream metric. For queue-driven work, KEDA scaling on broker lag will beat CPU-based HPA every time.

Multi-AZ failover: the N+1 arithmetic

Spreading across three availability zones does not by itself survive losing one. It survives it only if the remaining two zones have the capacity to absorb the third's load.

For a three-AZ deployment, losing one zone removes 33% of capacity and redirects that traffic to the survivors. To stay within your latency target, the survivors need to reach roughly 150% of their steady-state size — hence the 36-node pool maximum in the earlier table.

Three things have to be true, and usually only the first one is:

• topologySpreadConstraints are set to DoNotSchedule, not ScheduleAnyway. The soft version is a hint, and under scheduling pressure it will be ignored — leaving you with 60% of your replicas in one zone and a much worse outage than you planned for.
• Node pool maximums allow the surge. Per-AZ node group maximums sized exactly to steady state will block the failover.
• PodDisruptionBudgets are set for the degraded state, not the healthy one. A PDB of minAvailable: 90% will block the drains and consolidation you need during recovery.

Test this. A zone failure that has never been rehearsed is a plan, not a capability — and rehearsing it is a core part of Kubernetes disaster recovery design.

Stateful services are the real constraint

Stateless pods scale in seconds. The database behind them does not. In almost every high-traffic incident we have reviewed, the application layer scaled correctly and the failure was connection pool exhaustion, replica lag, or a cache stampede against a cold tier.

Include in the plan: connection pool sizing against your maximum replica count, read replica capacity at peak, cache warm-up time after a cold start, and cross-AZ data transfer cost at 150% of normal traffic.

A Capacity Review Cadence That Actually Holds

Capacity plans decay. Traffic grows, code changes throughput per pod, and a new sidecar quietly eats 200 MiB of every node.

What works in practice:

• Monthly: compare requests against actual P99 usage per workload. Anything requesting more than 3× what it uses gets rightsized.
• Quarterly: re-run the per-pod throughput test. Application changes move that number more than people expect.
• Before every known event: pre-scale, raise node pool maximums, and disable scale-down for the window.
• Continuously: alert on cluster allocatable versus requested, not just node CPU. Getting to 85% requested is the signal to add nodes, and it fires long before any usage-based alert.

The metrics that make this possible have to already exist. If you cannot query requested-versus-allocatable per node pool, the first task is monitoring and observability, not capacity planning. A standard Prometheus and Grafana setup with kube-state-metrics gives you every input this guide uses.

Related reading: our Kubernetes production readiness checklist covers the controls that sit around this, and EKS vs GKE vs AKS compares how the managed platforms differ on autoscaling and node management.

The failures are consistent. Requests copied from a template. maxReplicas Set to a round number. Node pool maximums sized for a healthy day. Soft topology constraints that quietly collapse under pressure. None of them are visible until the traffic arrives.

If you are sizing a cluster for a launch, a sale, or sustained growth — or you suspect you are paying for nodes that are full of requests and empty of work — our managed Kubernetes services and 24×7 SRE team do this work continuously.

Talk to our team → We will start with your actual peak-hour metrics, not a template.