Under Load, Healthy Pods Keep Restarting. Add Replicas?
During traffic peaks, Kubernetes keeps killing pods that are not actually broken. A teammate wants to scale up replicas. Defend the real fix.
Kubernetes
Observabilitythe decision you defend
During peak traffic your pods enter a restart loop and latency gets worse, even though the app is not crashing. kubectl shows liveness probe failures and restarts. A teammate says just increase the replica count to spread the load. Do you agree, and what do you do instead?
the situation
Your service runs on Kubernetes behind a Service and an ingress. During daily traffic peaks, latency climbs and then pods start cycling. kubectl get pods shows RESTARTS incrementing and kubectl describe pod reports Liveness probe failed with a timeout. The application logs show no panics or fatal errors around the restarts.
context
The Deployment defines a single httpGet liveness probe with timeoutSeconds set to 1 and failureThreshold set to 1, checking a handler that also does application work. There is no readiness probe and no startupProbe. CPU sits near its limit at peak. A teammate is convinced the pods are simply overloaded and wants to raise the replica count to spread traffic.
How this challenge works
Take a position on the decision above and defend it. A senior-engineer AI will push back over up to 5 rounds. When you are done, you are scored against a verified rubric so you can see exactly what a complete answer covers - these are learning prompts, not gotchas.