Right-Sizing Kubernetes Workloads: Data-Driven Approach

Introduction: The Right-Sizing Challenge

Right-sizing Kubernetes workloads—configuring CPU and memory resource requests and limits that align with actual application consumption—represents the fastest path to structural cost savings in container environments. Yet most organizations operate with massive over-provisioning: average request-to-usage ratios of 3-5x for CPU and 2-4x for memory. This means for every CPU core or gigabyte of memory actually consumed by applications, teams have requested and pay for three to five times that amount, resulting in billions of wasted CPU milliseconds and terabytes of unused memory capacity.

The over-provisioning epidemic stems from multiple causes: developers request resources conservatively "to be safe" without measuring actual needs, copy-paste inheritance from example manifests designed for different workload profiles, lack of production telemetry during initial deployment forcing engineers to guess, organizational risk aversion preferring waste over potential performance issues, and absence of feedback loops showing teams the financial impact of their resource requests. When compounded across hundreds of microservices and thousands of pods, cumulative waste reaches staggering proportions.

However, aggressive under-sizing poses equally serious risks: insufficient CPU causes throttling degrading latency and throughput, inadequate memory triggers OOMKilled terminations interrupting service, under-provisioned requests lead to poor scheduler decisions placing pods on nodes lacking actual capacity, and overly tight limits prevent handling legitimate traffic spikes. The key challenge is finding balance: resource configurations providing necessary safety buffers while eliminating structural waste.

This comprehensive guide provides a systematic, data-driven methodology for safely rightsizing Kubernetes workloads at scale using Prometheus for telemetry collection and tools like SpendZero that automate the analysis. You'll learn how to collect 30 days of accurate utilization data, analyze usage at appropriate percentiles (P95/P99) rather than misleading averages, calculate recommended requests and limits with workload-specific safety buffers, validate changes through testing before production rollout, implement gradually while monitoring impact, and establish continuous re-analysis as applications evolve.

Understanding Kubernetes Resource Management

Requests vs Limits: The Two-Tier Model

Kubernetes uses a dual-tier resource management system that many engineers misunderstand:

Resource Requests (Scheduling Guarantee):

• Define the minimum resources that Kubernetes guarantees the container
• Scheduler uses requests for pod placement decisions—only schedules pods on nodes with sufficient unreserved capacity
• Requests do NOT limit consumption—containers can use more if node capacity available
• Determine pod Quality of Service (QoS) class affecting eviction priority during resource pressure
• Cloud billing based on node capacity, but over-requesting drives unnecessary node scale-out increasing infrastructure spend

Resource Limits (Hard Ceiling):

• Define the maximum resources a container may consume
• CPU limits enforced through throttling—kernel pauses execution if container exceeds limit
• Memory limits enforced through OOMKill—kubelet terminates container immediately if it exceeds memory limit
• Limits do NOT affect scheduling—scheduler ignores limits when placing pods
• Setting limits too low causes artificial performance degradation even when node has spare capacity

Example Configuration:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-api
spec:
  replicas: 10
  template:
    spec:
      containers:
      - name: api
        image: payment-api:v2.4
        resources:
          requests:
            cpu: 500m        # Scheduler guarantees 0.5 CPU cores
            memory: 1Gi      # Scheduler guarantees 1 GB RAM
          limits:
            cpu: 1000m       # Maximum 1 CPU core (throttled if exceeded)
            memory: 2Gi      # Maximum 2 GB RAM (OOMKilled if exceeded)

Quality of Service (QoS) Classes

Resource configuration determines pod QoS class affecting eviction priority during node pressure:

Common Misconfiguration Patterns

Anti-Pattern 1: Massive Over-Requesting

# BAD: Developer "playing it safe" without data
resources:
  requests:
    cpu: 4000m      # 4 full CPU cores requested
    memory: 8Gi     # 8 GB RAM requested
  limits:
    cpu: 8000m
    memory: 16Gi

# Reality: Actual P99 usage is 450m CPU, 950Mi memory
# Result: 8.9x CPU over-provisioned, 8.6x memory over-provisioned
# Cost: $890/month per pod vs $98/month if properly sized
# Waste: $792/month per pod × 10 replicas = $7,920/month wasted

Anti-Pattern 2: No Resource Requests (BestEffort)

# BAD: No resource management
resources: {}  # No requests or limits

# Impact: Pod gets BestEffort QoS (evicted first during pressure)
# Risk: Unpredictable performance, scheduler places on any node regardless of capacity
# Problem: No cost attribution (can't track which team consumed resources)

Anti-Pattern 3: Limits Without Requests

# SUBOPTIMAL: Only limits defined
resources:
  limits:
    cpu: 2000m
    memory: 4Gi
  # No requests specified

# Behavior: Kubernetes automatically sets requests = limits
# Result: Forces Guaranteed QoS when Burstable would be more appropriate
# Impact: Wastes resources that could be shared across workloads

Phase 1: Collect 30 Days of Prometheus Telemetry

Prerequisites: Prometheus and Metrics Exporters

Right-sizing requires historical utilization data from Prometheus with these components:

• kube-state-metrics: Exposes resource requests, limits, and pod metadata
• node-exporter: Provides node-level CPU, memory, disk metrics
• cAdvisor: Embedded in kubelet, exposes container-level usage metrics

If Prometheus not deployed, install with 30-day retention minimum:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install prometheus prometheus-community/prometheus \
  --namespace monitoring \
  --create-namespace \
  --set server.retention=30d \
  --set server.persistentVolume.size=100Gi

Essential Metrics for Rightsizing

CPU Metrics (per container):

# Actual CPU usage (cores per second)
container_cpu_usage_seconds_total

# CPU requests (what was requested)
kube_pod_container_resource_requests{resource="cpu"}

# CPU limits
kube_pod_container_resource_limits{resource="cpu"}

# CPU throttling (indicates limits too low)
container_cpu_cfs_throttled_seconds_total
container_cpu_cfs_periods_total

Memory Metrics (per container):

# Actual memory usage (working set, excludes cache)
container_memory_working_set_bytes

# Memory requests
kube_pod_container_resource_requests{resource="memory"}

# Memory limits
kube_pod_container_resource_limits{resource="memory"}

# OOMKill events (indicates limits too low)
kube_pod_container_status_terminated_reason{reason="OOMKilled"}

Data Collection Best Practices

1. Minimum 14-Day Lookback (30 Days Recommended)

Collect at least two full business weeks to capture:

• Weekday vs weekend traffic patterns
• Peak load periods (morning login rush, nightly batch processing)
• Weekly cycles (Monday traffic spikes, Friday drop-offs)
• Monthly patterns (end-of-month reporting, payroll processing)

2. Scrape Interval: 15-60 Seconds

# Prometheus scrape configuration
scrape_configs:
  - job_name: 'kubernetes-pods'
    scrape_interval: 30s      # Balance granularity vs storage
    scrape_timeout: 10s
    kubernetes_sd_configs:
      - role: pod

3. Segment by Workload Criticality

# Label pods by criticality tier for differentiated safety buffers
kubectl label deployment payment-api tier=tier1-critical
kubectl label deployment analytics-batch tier=tier3-batch
kubectl label deployment internal-dashboard tier=tier2-standard

Phase 2: Analyze at Appropriate Percentiles

Why Percentiles Matter More Than Averages

Using average utilization for rightsizing is a critical mistake causing under-provisioning and performance degradation:

If you set CPU requests to 320m (average), application will be under-provisioned during 50% of operating time causing throttling and latency degradation. If you set to 2400m (max), you're over-provisioning for potentially anomalous spikes.

Recommended Percentile Targets by Workload Type

Prometheus Queries for Percentile Analysis

# Calculate P95 CPU usage over 30 days for payment-api
quantile_over_time(0.95,
  rate(container_cpu_usage_seconds_total{
    pod=~"payment-api-.*",
    container="api"
  }[5m])[30d:5m]
)

# Calculate P99 memory usage over 30 days
quantile_over_time(0.99,
  container_memory_working_set_bytes{
    pod=~"payment-api-.*",
    container="api"
  }[30d:1m]
)

# Identify top 20 over-provisioned workloads (request/usage ratio)
topk(20,
  kube_pod_container_resource_requests{resource="cpu"} /
  on(pod,container) rate(container_cpu_usage_seconds_total[7d])
)

Phase 3: Automated Analysis

Manual Analysis Pain Points

Rightsizing 100+ workloads manually requires:

• Writing and validating PromQL queries for each workload
• Exporting data to spreadsheets for percentile calculations
• Manually determining appropriate safety buffers per workload type
• Calculating cost impact using AWS/GCP pricing data
• Generating Kubernetes YAML patches for each service
• Total effort: 40-60 hours for 100 workloads (monthly re-analysis needed)

Automating the Analysis

There are two ways to take the manual work out of this phase. You can script it yourself — PromQL recording rules for the percentiles in Phase 2, a job that applies the buffers in Phase 4, and a report that prices each change — or you can use a tool that does the same calculation for you. Either way, the output should be the same: a ranked list of workloads with current and recommended requests and a dollar figure on each.

Example: SpendZero. SpendZero, the cost optimization tool SquareOps builds, automates this for EKS and GKE clusters in connected AWS and GCP accounts:

1. Connect (about 2 minutes): a read-only IAM role or GCP service account, created from a provided snippet. No agents and no sidecars in the cluster. Clusters in connected accounts are discovered automatically.
2. Inventory: workloads and nodes in each connected cluster, alongside 25+ cloud services in the account.
3. Detect: CPU and memory requests compared with 14-day p95 usage per container. Recommendations keep 30% headroom above p95 and never go below the workload's liveness-probe requirements. Idle namespaces and underused node groups are flagged in the same pass, each with an estimated monthly saving. The first report arrives in about five minutes.
4. Act: apply a request change as a one-click action — every action requires confirmation and is written to an audit log — or export the recommendations to the owning team.

Note the window: SpendZero sizes against 14 days of p95 usage. For workloads with monthly cycles — billing runs, month-end reporting — cross-check its recommendation against the 30-day analysis from Phase 2 before applying it, and keep the workload-specific buffers in Phase 4 for Tier-1 services.

What the output should look like

Whichever route you take, treat this output as a starting point for Phase 4 and Phase 5, not as a change to push straight to production.

Phase 4: Calculate Recommendations with Safety Buffers

Recommendation Formula

For CPU Requests:

Recommended_CPU_Request = P95_CPU_Usage × (1 + Safety_Buffer)

Example (standard tier-2 service):
  P95 CPU usage over 30 days: 680m
  Safety buffer: 20%
  Recommended CPU request: 680m × 1.20 = 816m (round to 850m)

For Memory Requests:

Recommended_Memory_Request = P95_Memory_Usage × (1 + Safety_Buffer)

Example:
  P95 Memory usage: 950Mi
  Safety buffer: 20%
  Recommended memory request: 950Mi × 1.20 = 1140Mi (round to 1200Mi)

For Limits (Burst Capacity):

Recommended_CPU_Limit = Recommended_CPU_Request × Burst_Multiplier

Typical burst multipliers:
  - Stateless APIs: 2.0x (allow 2x burst)
  - Databases: 1.5x (less burstable, consistent usage)
  - Batch jobs: 3.0x (highly variable, benefit from burst)

Example:
  Recommended request: 850m
  Burst multiplier: 2.0x
  Recommended limit: 1700m

Guardrails Preventing Under-Provisioning

# Minimum thresholds (prevent aggressive under-sizing)
MINIMUM_CPU_REQUEST = 100m      # Prevent excessive throttling
MINIMUM_MEMORY_REQUEST = 128Mi  # Prevent OOMKills

# Maximum single reduction (staged rollout)
MAX_REDUCTION_PERCENTAGE = 50%  # Don't cut more than 50% at once

# Exclusion criteria
EXCLUDE_IF:
  - Label: cost-optimization=disabled
  - Label: rightsizing=manual
  - Insufficient data: < 7 days metrics
  - High variance: stddev > 2x mean

Real Example: payment-api Container

# BEFORE (Over-Provisioned)
resources:
  requests:
    cpu: 2000m        # 2 full CPU cores
    memory: 4Gi       # 4 GB RAM
  limits:
    cpu: 4000m
    memory: 8Gi

# 30-DAY PROMETHEUS ANALYSIS
P50 CPU: 320m | P95 CPU: 680m | P99 CPU: 1150m | Max: 1680m
P50 Mem: 720Mi | P95 Mem: 950Mi | P99 Mem: 1280Mi | Max: 1520Mi

Workload tier: tier-2 (standard)
Target percentile: P95
Safety buffer: 20%

# RECOMMENDED (Data-Driven)
resources:
  requests:
    cpu: 850m         # 680m P95 + 25% = 850m (58% reduction)
    memory: 1200Mi    # 950Mi P95 + 26% = 1200Mi (71% reduction)
  limits:
    cpu: 1700m        # 2x requests for burst
    memory: 2400Mi    # 2x requests for burst

# COST IMPACT
Current cost: $24.50/month per pod
Recommended cost: $6.85/month per pod
Monthly savings: $17.65 per pod
Cluster savings (10 replicas): $176.50/month
Annual savings: $2,118/year for this one service

Phase 5: Validate Before Production

Staging Environment Testing

1. Apply recommendations to staging first
2. Run load tests mimicking production traffic
3. Monitor SLIs closely: P95/P99 latency, error rate, CPU throttling, OOMKills
4. Validate for 48-72 hours before production

# Load test with k6
k6 run --vus 500 --duration 30m load-test.js

# Monitor during load test
kubectl top pods -n staging | grep payment-api

# Check for throttling (should be < 5%)
kubectl exec -n staging prometheus-0 -- promtool query instant \
  'sum(rate(container_cpu_cfs_throttled_seconds_total{pod=~"payment-api-.*"}[5m])) /
   sum(rate(container_cpu_cfs_periods_total{pod=~"payment-api-.*"}[5m]))'

Progressive Production Rollout

Wave 1: 10% of Replicas (Week 1)

# Update 1 of 10 pods
kubectl patch deployment payment-api --type=json -p='[
  {"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value": "850m"},
  {"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/memory", "value": "1200Mi"}
]'

# Monitor for 48 hours
# Compare metrics between old (2000m/4Gi) and new (850m/1200Mi) pods

Wave 2: 50% of Replicas (Week 2, if Wave 1 successful)

Wave 3: 100% Rollout (Week 3, if Wave 2 successful)

Phase 6: Monitor and Iterate

Post-Deployment Monitoring

Critical SLIs to Track:

Continuous Re-Analysis Cadence

• Weekly: Review top 20 workloads for new optimization opportunities
• Monthly: Full cluster re-analysis with updated 30-day utilization (scheduled scans in a tool such as SpendZero keep the recommendations current between full reviews)
• After Major Releases: Re-baseline if application behavior changes significantly
• Seasonal Adjustments: Update safety buffers before known traffic spikes (Black Friday, tax season)

Expected Results and ROI

Typical Savings by Resource Type

Time Investment vs Savings

Manual Approach (Open Source Tools):

• Initial setup: 40-60 hours (Prometheus, VPA, custom dashboards)
• Monthly analysis: 20-30 hours (PromQL queries, spreadsheet analysis, YAML generation)
• Timeline to savings: 6-8 weeks
• Total: 100+ hours for first month

Automated Approach:

• Initial setup: minutes rather than days — with SpendZero, read-only credentials and no agents, and a first report in about 5 minutes
• Monthly analysis: reviewing and confirming ranked recommendations instead of writing PromQL and spreadsheets
• Timeline to savings: set by your validation and staged rollout (Phases 5 and 6), not by the analysis
• Engineering time goes into review and rollout, not data collection

Common Pitfalls and Solutions

Pitfall 1: Using Average Instead of Percentiles

Problem: Setting requests based on average under-provisions during 50% of operating time.

Solution: Always use P95 (standard services) or P99 (critical services) plus 20-30% safety buffer.

Pitfall 2: Ignoring Seasonal Patterns

Problem: Rightsizing based on current 30 days fails when Black Friday or tax season hits.

Solution: Include peak periods in analysis window, or add growth buffers for rapidly scaling services, re-baseline after major launches.

Pitfall 3: Optimizing Everything Simultaneously

Problem: Applying changes to 100+ services at once makes root cause impossible if issues occur.

Solution: Phased rollout—staging first (week 1), non-critical prod (week 2-3), critical services conservatively (week 4-5).

Pitfall 4: Setting CPU Limits Too Low

Problem: Aggressive CPU limits cause throttling even when node has spare capacity.

Solution: Set CPU limits 2-3x requests allowing burst. Consider removing CPU limits entirely for non-critical workloads (limits don't affect scheduling).

Conclusion: Rightsizing as Continuous Discipline

Right-sizing Kubernetes workloads delivers 35-55% cost reduction when done systematically with data-driven analysis rather than guesswork. The key success factors: collecting 30+ days of Prometheus metrics capturing weekly and monthly patterns, analyzing at appropriate percentiles (P95/P99) with workload-specific safety buffers, validating thoroughly in staging before production, rolling out progressively monitoring SLIs closely, and re-analyzing continuously as applications evolve.

Automating the analysis does not change the method — it removes the hours of query writing so that engineering time goes into validation and rollout. SpendZero, for example, connects with read-only credentials and no agents, compares requests with p95 usage for every container, and produces a ranked rightsizing report in about five minutes, with each change applied only after you confirm it.

Related Guides

• Kubernetes FinOps: Complete Guide for Engineering Teams — the FinOps framework rightsizing fits into.
• How to Fix Kubernetes OOMKilled Errors — avoid the failure mode of cutting memory too aggressively.
• SpendZero Kubernetes cost optimization — request rightsizing from p95 usage with safe headroom, namespace cost and idle-workload detection.

Ready to rightsize your Kubernetes workloads? Start a free SpendZero scan and get rightsizing recommendations from your clusters' real usage, with a first report in about 5 minutes. Read-only, no agents, no credit card.

Questions about rightsizing methodology? Schedule a consultation with our solutions team to review your specific workload patterns and optimization potential.