Under Load, Healthy Pods Keep Restarting. Add Replicas?
During traffic peaks, Kubernetes keeps killing pods that are not actually broken. A teammate wants to scale up replicas. Defend the real fix.
Your decision
During peak traffic your pods enter a restart loop and latency gets worse, even though the app is not crashing. kubectl shows liveness probe failures and restarts. A teammate says just increase the replica count to spread the load. Do you agree, and what do you do instead?
The situation
Your service runs on Kubernetes behind a Service and an ingress. During daily traffic peaks, latency climbs and then pods start cycling. kubectl get pods shows RESTARTS incrementing and kubectl describe pod reports Liveness probe failed with a timeout. The application logs show no panics or fatal errors around the restarts.
Context
The Deployment defines a single httpGet liveness probe with timeoutSeconds set to 1 and failureThreshold set to 1, checking a handler that also does application work. There is no readiness probe and no startupProbe. CPU sits near its limit at peak. A teammate is convinced the pods are simply overloaded and wants to raise the replica count to spread traffic.