Kubernetes HPA in Production — Automatic Scaling That Cuts Costs • Meteora Web Agency

Does your Kubernetes cluster crash when Black Friday traffic hits? Or do you keep 20 pods always on when only 3 are needed? Both scenarios cost money: the first loses customers and revenue, the second burns cloud or on‑prem resources. At Meteora Web, we've seen SMEs spend 40% more than necessary on AWS because the Horizontal Pod Autoscaler was disabled or misconfigured. This guide shows you how to use HPA to automatically scale pods based on real load, without waste or surprises.

What Is the Kubernetes Horizontal Pod Autoscaler and Why Do You Need It in Production?

The Horizontal Pod Autoscaler (HPA) is a native Kubernetes controller that increases or decreases the number of pod replicas based on metrics like CPU, memory, or HTTP requests per second. It's not an external tool — it's part of the autoscaling/v2 API and runs inside your cluster. The real question: why should you care now? Because production load is never linear. An e‑commerce site might have 10 visitors in the morning and 10,000 at 6 PM. A booking backend explodes during events. Without automatic scaling, you have two choices: overprovision (and pay) or underprovision (and crash). We took over a cluster with 5 fixed pods for a CRM: 3 were enough at idle, but they crashed during morning logins. With HPA we fixed it in one day, saving 30% on compute costs.

HPA is not magic: it relies on metrics collected by the Metrics Server (mandatory) and modifies the replicas field of a Deployment or StatefulSet. You can target percentages, absolute values, or custom metrics via Prometheus adapter. But a poorly tuned HPA does more harm than no autoscaler. Aggressive scale‑up causes cost bursts and instability; slow scale‑down keeps idle resources running. The key is balancing reactivity with stability.

How to Configure the Kubernetes Horizontal Pod Autoscaler with CPU and Memory?

Start with the most common case: CPU utilisation as a metric. Assume a Deployment named my-api with resources.requests.cpu: 250m and limits: 500m. Create an HPA manifest file:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-api
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

Apply with kubectl apply -f hpa.yaml. The HPA calculates the number of replicas needed to keep average CPU at 70% and memory at 80%. When load increases, it scales up to 10 pods; when it drops, it scales back to 2. Important: without resources.requests in your containers, the HPA cannot calculate percentages. This is one of the most common mistakes we see in audits.

Check status: kubectl get hpa my-api-hpa -w shows target, current, and replicas in real time. If you see <unknown> in targets, Metrics Server is likely missing. Install it with kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml (on cloud it's often already enabled).

Which Advanced Metrics Should You Use for the Horizontal Pod Autoscaler?

CPU and memory work for general‑purpose workloads, but for web apps or APIs the best metric is requests per second (RPS). This is where custom metrics via Prometheus Adapter or KEDA (backed by Microsoft) come in. We prefer KEDA because it also handles event queues (Kafka, RabbitMQ) and more sophisticated schedulers.

Example HPA with a Prometheus metric using KEDA's ScaledObject:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: api-scaler
spec:
  scaleTargetRef:
    name: my-api
  minReplicaCount: 2
  maxReplicaCount: 20
  triggers:
  - type: prometheus
    metadata:
      serverAddress: http://prometheus.monitoring.svc:9090
      metricName: http_requests_per_second
      query: |
        sum(rate(http_requests_total{job="my-api"}[1m]))
      threshold: '100'
      activationThreshold: '10'

With threshold: 100, KEDA scales when the rate exceeds 100 req/s per pod. activationThreshold prevents scale‑up on noise. We used this for a logistics client: order notifications come in bursts, but with KEDA we scale in 30 seconds and then drop quickly. Result: zero timeouts and a 45% reduction in AWS compute costs compared to fixed replicas.

How to Test the Horizontal Pod Autoscaler Before Going to Production?

Don't trust theory. Stress‑test your cluster and observe. We use kubectl run -i --tty load-generator --image=busybox /bin/sh then run while true; do wget -q -O- http://my-api-svc; done. Alternatively, use hey (https://github.com/rakyll/hey) from an external node: hey -n 10000 -c 50 http://my-api-svc.cluster.local.

Monitor with kubectl top pods and kubectl describe hpa my-api-hpa. Check HPA controller logs on on‑prem clusters: kubectl logs -n kube-system deployment/kube-controller-manager | grep -i hpa. If you see too rapid scaling, add behavior to the HPA spec to stabilise:

behavior:
  scaleDown:
    stabilizationWindowSeconds: 300
    policies:
    - type: Pods
      value: 1
      periodSeconds: 60
  scaleUp:
    stabilizationWindowSeconds: 0
    policies:
    - type: Percent
      value: 100
      periodSeconds: 15

This blocks scale‑down for 5 minutes (prevents flapping) and allows aggressive scale‑up. We applied this to an e‑commerce client: during the Christmas peak the HPA went from 3 to 30 pods in 45 seconds without instability.

How to Optimize the Horizontal Pod Autoscaler to Reduce Costs?

An HPA that scales too often or too high costs money. Three practical tips:

Set realistic minReplicas: never leave it at 1 if you need redundancy. We always set at least 2 for critical workloads.
Pair with the Vertical Pod Autoscaler (VPA) to adjust requests/limits based on actual usage, so HPA works on more accurate metrics. Caution: VPA and HPA conflict on the same deployment if not managed correctly. Prefer VPA in Off mode that only suggests.
Set higher CPU targets (e.g., 80% instead of 50%) if your application is stateless and tolerates short bursts. Fewer idle pods, more savings – but watch for timeouts. We have a SaaS client that runs at 90% CPU target with HPA and saves 25% on cloud. It works because their code is optimised.

Remember: HPA is not just about saving money – it's about not losing revenue. As we tell our clients: a site that goes down during an ad campaign costs much more than a few extra pods.

Common Horizontal Pod Autoscaler Kubernetes Mistakes (and How to Avoid Them)

Missing Metrics Server: not installed or unreachable. Check with kubectl top nodes.
No resource requests: HPA ignores pods without requests. Every container MUST have resources.requests.
Stabilization window too low: causes flapping (scale up and down continuously). Set at least 2–3 minutes for scaleDown.
Ignoring namespace ResourceQuotas: HPA cannot exceed a quota limit. Check with kubectl describe quota.
Not testing in staging: we've seen clients apply HPA directly in production with wrong targets and crash the system. Always simulate with artificial load.

Summary — What to Do Next

The Kubernetes Horizontal Pod Autoscaler is a powerful tool, but it needs attention to detail. Here's what you can do right now:

Enable Metrics Server on your cluster if missing.
Set resource requests on all containers in your target deployment.
Create a first HPA with CPU at 70%, minReplicas 2 and maxReplicas 10.
Test with artificial load and adjust behaviour parameters.
Monitor costs: compare pod counts before and after over a 7‑day period.

If you manage a Kubernetes cluster, HPA is not optional – it's the difference between a system that holds up and one that collapses. At Meteora Web, we use it daily for our clients. For a deeper look at the entire Kubernetes ecosystem, read our pillar guide on Kubernetes and Container Orchestration (link placeholder – replace with actual pillar URL). For specific questions, reach out – we'll solve it together.