Guides/KubernetesKubernetes/Kubernetes Deployments and Rollout Strategies

Kubernetes Deployments and Rollout Strategies

Past replicas=3: how RollingUpdate works, maxSurge vs maxUnavailable, kubectl rollout status/undo/pause, probes gating a safe deploy, plus blue-green and canary.


A Deployment is the object you reach for on almost every stateless workload, and the fundamentals guide already covers what it is: desired state for "keep N identical pods running from this image, and manage changes to them safely." This guide is about that last clause. Changing an image tag and running kubectl apply is easy; understanding exactly what happens between "old version serving traffic" and "new version serving traffic" is where deploys go from a leap of faith to something you can reason about and control. We will pull apart the RollingUpdate strategy knob by knob, cover the kubectl rollout commands you use during a real deploy, see how readiness probes turn a rollout from optimistic to safe, and then step up to blue-green and canary and where vanilla Kubernetes stops and a dedicated tool takes over.

Deployment, ReplicaSet, Pod: the layer that makes rollouts possible

A quick recap, because the rollout mechanics live entirely in this hierarchy. A Deployment does not manage pods directly. It manages ReplicaSets, and each ReplicaSet manages pods. The ReplicaSet's only job is "keep exactly N pods matching this template alive." The Deployment's job is to manage the ReplicaSets over time.

The key insight: a rollout is the Deployment orchestrating two ReplicaSets at once. When you change the pod template (a new image, new env var, new resource limit), the Deployment creates a new ReplicaSet for the new template and gradually scales it up while scaling the old ReplicaSet down. At any moment mid-rollout you have two ReplicaSets, both partially populated, and the sum of their pods is roughly your desired replica count. When the new ReplicaSet is fully up and the old one is at zero, the rollout is done. The old ReplicaSet is not deleted - it is kept, scaled to zero, which is precisely what makes an instant rollback possible.

kubectl get replicasets -l app=web
# NAME             DESIRED   CURRENT   READY   AGE
# web-6d4b8f9c7c   3         3         3       2d      <- old, will scale to 0
# web-7f8c5d6b9d   1         1         1       15s     <- new, scaling up

Only a change to spec.template triggers a new ReplicaSet and a rollout. Changing spec.replicas does not - that is just scaling, handled by the current ReplicaSet with no new pods of a new version. This distinction matters: kubectl scale never rolls anything out, while kubectl set image always does.

RollingUpdate in depth

RollingUpdate is the default strategy, and for stateless services it is almost always the right one. The idea is to never take the whole service down: replace pods incrementally so there is always a healthy set serving traffic. The behavior is governed by two fields under strategy.rollingUpdate, and understanding what they trade off is the difference between a smooth deploy and a surprise mini-outage.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 10
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%          # how many EXTRA pods above replicas during the rollout
      maxUnavailable: 25%    # how many pods below replicas may be unavailable
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: web
          image: myapp:1.5.0
          ports:
            - containerPort: 8080

maxSurge is how many pods the Deployment may run above the desired replicas count during the rollout. With replicas: 10 and maxSurge: 25%, Kubernetes may run up to 13 pods at once (10 rounded up by 2.5 -> 3 extra). Surge is what lets the rollout bring new pods up before taking old ones down, so capacity never dips. Higher surge means faster rollouts and no capacity loss, at the cost of temporarily needing more cluster resources (and more room for the scheduler to place those extra pods).

maxUnavailable is how many pods, out of replicas, may be unavailable at any point during the rollout. With replicas: 10 and maxUnavailable: 25%, at least 8 pods must always be Ready (2 may be down mid-swap). Lower unavailable means you keep more capacity serving traffic throughout, but the rollout proceeds in smaller steps and takes longer.

The two work as a pair, and the trade-off is capacity versus speed versus headroom:

  • maxSurge: 25%, maxUnavailable: 0% (a common safe default) - never drop below full capacity. Kubernetes surges new pods up first, waits for them to be Ready, then removes old ones. Zero risk of reduced capacity, but you must have room in the cluster for the extra pods, and the rollout is a touch slower because it always waits for new pods before retiring old.
  • maxSurge: 0%, maxUnavailable: 25% - never exceed the replica count (useful when a resource quota or a fixed node pool means you literally cannot run extra pods). The trade-off is you do run below full capacity during the rollout, since old pods are removed to make room before new ones come up.
  • Both non-zero (the default 25%/25%) - fastest churn, but you can briefly be both over the replica count and below full readiness. Fine for most services, risky if you are tight on either capacity or cluster room.

One rule they must obey together: maxSurge and maxUnavailable cannot both be zero. That would mean "add no pods and remove no pods," which cannot make progress. Kubernetes rejects it.

A subtle gotcha with percentages and small replica counts: percentages round. maxUnavailable rounds down and maxSurge rounds up. With replicas: 2 and maxUnavailable: 50%, that is exactly 1. But with replicas: 3 and maxUnavailable: 25%, that rounds down to 0, meaning maxSurge must carry the whole rollout. For small deployments, set these as absolute integers (maxUnavailable: 1) rather than percentages so the behavior is obvious.

Recreate: when you want the opposite of rolling

The other built-in strategy is Recreate, and it is the blunt instrument: kill all the old pods, then start all the new ones. There is a gap in the middle where zero pods run and the service is down.

spec:
  strategy:
    type: Recreate

You use Recreate deliberately, when a rolling update is actually unsafe:

  • Old and new versions cannot run at the same time. The classic case is a database schema migration where v1.5 expects a new column that v1.4 would choke on, or vice versa. During a RollingUpdate both versions serve traffic simultaneously; if they are mutually incompatible, that window corrupts data or throws errors. Recreate guarantees only one version is ever live.
  • A singleton that cannot double-run. Something holding an exclusive lock, a single-writer process, or a workload backed by a ReadWriteOnce volume that only one pod can mount. Surging a second pod would fail or deadlock.

The cost is honest downtime for the duration of the swap, so Recreate is for internal tools, batch processors, and workloads where a brief outage is acceptable and correctness during the transition is not negotiable. For anything user-facing where you cannot accept downtime and cannot run two versions at once, that is your signal to reach for blue-green or a migration strategy that decouples the schema change from the code deploy (expand-and-contract), covered below.

Driving a rollout: the kubectl rollout commands

kubectl rollout is the family of commands for watching and controlling a deploy in flight. These are what you actually type during a release.

# Trigger a rollout (any of these changes the pod template)
kubectl set image deploy/web web=myapp:1.5.0
kubectl apply -f web.yaml
kubectl edit deploy/web

# Watch it happen - blocks until done or failed, exits non-zero on failure
kubectl rollout status deploy/web
# Waiting for deployment "web" rollout to finish: 6 of 10 updated replicas are available...
# deployment "web" successfully rolled out

kubectl rollout status is the one that makes a deploy scriptable. It blocks until the rollout completes and returns exit code 0, or gives up at the progress deadline and returns non-zero. In CI/CD you run kubectl apply then kubectl rollout status, and a non-zero exit fails the pipeline instead of silently reporting a green deploy on top of a stuck rollout.

History and rollback are where the retained old ReplicaSets pay off:

kubectl rollout history deploy/web
# REVISION  CHANGE-CAUSE
# 1         <none>
# 2         kubectl set image deploy/web web=myapp:1.5.0

kubectl rollout history deploy/web --revision=1   # inspect a specific revision's template

kubectl rollout undo deploy/web                   # roll back to the immediately previous revision
kubectl rollout undo deploy/web --to-revision=1   # roll back to a specific revision

rollout undo is fast because it is not rebuilding anything - it just scales the previous ReplicaSet back up and the current one down, another RollingUpdate in reverse. The CHANGE-CAUSE column is blank unless you record it; add kubectl annotate deploy/web kubernetes.io/change-cause="deploy 1.5.0" (or --record on older clusters) so history is readable months later.

How many revisions you can roll back to is capped by spec.revisionHistoryLimit (default 10). Beyond that limit the oldest zero-scaled ReplicaSets are garbage-collected. Set it to 0 and you keep no history and lose the ability to undo - do not do that.

Pause and resume let you stage changes or run a manual canary:

kubectl rollout pause deploy/web     # freeze the Deployment - changes accumulate but do not roll out
kubectl set image deploy/web web=myapp:1.6.0
kubectl set resources deploy/web -c web --limits=memory=512Mi
kubectl rollout resume deploy/web    # now apply everything at once, as ONE rollout

Pausing has two real uses. First, batching: if you need to change the image and the resources and an env var, pause, make all three edits, then resume, so it is a single rollout instead of three back-to-back ones. Second, a crude manual canary: pause after the new ReplicaSet has scaled up just one pod, observe it under real traffic, then resume to continue or undo to bail. A paused Deployment also will not self-heal template changes, so do not leave one paused and walk away.

Readiness probes: what makes a rollout actually safe

Here is the thing that separates a real rolling update from a hopeful one. By default, the only signal Kubernetes has that a new pod is "ready to take the next step" is that its container process started. But a process that has started is not a process that can serve requests - it might still be loading config, warming a cache, opening database connections, or JIT-compiling. If Kubernetes tears down old pods the instant new ones start, it will route traffic to pods that are not actually ready, and you get a wave of errors on every deploy even though nothing is technically crashing.

The readiness probe is what fixes this. As covered in the fundamentals guide, a readiness probe reports whether a pod can serve traffic; a failing readiness probe pulls the pod out of the Service's load balancing without restarting it. During a rollout, the readiness probe is the gate: the Deployment does not count a new pod toward "available" until its readiness probe passes, and maxUnavailable is enforced against available (Ready) pods, not merely started ones.

spec:
  template:
    spec:
      containers:
        - name: web
          image: myapp:1.5.0
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 5
            failureThreshold: 3

With that probe in place, the rollout self-throttles correctly: bring up a new pod, wait for /ready to pass, only then let the old pod go, respecting maxUnavailable the whole time. If the new version is broken and never becomes Ready, the rollout stalls rather than replacing every pod with a broken one - maxUnavailable is never satisfied because the new pods never go Ready, so old healthy pods are never removed. You keep serving traffic on the old version while the rollout sits stuck, which is exactly the failure mode you want: visible and non-destructive.

Without a readiness probe, a RollingUpdate degrades into a fast, blind swap. It will happily replace every pod with a broken new version because "the container started" is the only bar it has to clear. If you take one thing from this guide: a rolling update without a readiness probe is not a safe deploy. The probe is the mechanism.

minReadySeconds and progressDeadlineSeconds

Two more fields fine-tune the rollout's timing, and both earn their place.

minReadySeconds (default 0) is how long a new pod must be continuously Ready before the Deployment treats it as available and moves on to the next one. A pod that passes its readiness probe once but crashes two seconds later would, with minReadySeconds: 0, briefly count as available and let an old pod be retired before the new one proves stable. Setting minReadySeconds: 15 forces the pod to stay Ready for 15 seconds before the rollout advances, catching pods that flap right after startup.

spec:
  minReadySeconds: 15
  progressDeadlineSeconds: 300

progressDeadlineSeconds (default 600) is the rollout's patience. If the Deployment makes no progress - no new pods becoming available - for this many seconds, it marks the rollout as failed with reason: ProgressDeadlineExceeded in its status, and kubectl rollout status returns non-zero. This is what turns "stuck forever" into "failed deploy the pipeline can detect." Note it does not automatically roll back; it just reports failure. You (or your CI script reacting to the non-zero exit) decide to rollout undo. Tune it to slightly more than a healthy rollout's real duration so a genuinely stuck deploy fails fast instead of hanging your pipeline for ten minutes.

Together these give the rollout the right temperament: minReadySeconds makes it patient enough not to trust a pod that just barely came up, and progressDeadlineSeconds makes it impatient enough to declare failure rather than wait indefinitely.

Blue-green deployments

RollingUpdate always has both versions live at once for a while. Sometimes you want a hard cut instead: bring the new version up completely, verify it, then switch all traffic over at once, with the old version still standing by for an instant rollback. That is blue-green.

The concept: run two full environments, "blue" (current) and "green" (new). Both are fully deployed and healthy. A Service (or Ingress) points at blue. You deploy green alongside, test it directly, and when satisfied, flip the Service selector to green. Traffic cuts over in one step. If green misbehaves, flip back to blue - no rollout, no waiting, just a selector change.

Vanilla Kubernetes can approximate this with two Deployments and one Service whose selector you re-point:

# Two Deployments, distinguished by a version label
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-blue
spec:
  replicas: 5
  selector:
    matchLabels: { app: web, version: blue }
  template:
    metadata:
      labels: { app: web, version: blue }
    spec:
      containers:
        - name: web
          image: myapp:1.4.0
---
# web-green is identical but version: green and image: myapp:1.5.0
---
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  selector:
    app: web
    version: blue        # flip this to green to cut traffic over
  ports:
    - port: 80
      targetPort: 8080

The cutover is one command - kubectl patch service web -p '{"spec":{"selector":{"version":"green"}}}' - and rollback is the same patch back to blue. The upside is instant switch and instant rollback with a fully pre-warmed target. The downsides are cost (you run two full copies of the service during the transition) and that the flip is all-or-nothing: every user moves at once, so if green has a subtle bug it hits 100% of traffic immediately. Blue-green trades gradual exposure for a clean, reversible cut.

Canary deployments

A canary is the opposite bet from blue-green: instead of an all-at-once switch, you send a small slice of traffic to the new version, watch its error rate and latency, and only then widen it. The name is the canary in the coal mine - a small exposure that warns you before the whole flock is at risk. Canary is the best strategy when you want to limit the blast radius of a bad release.

You can approximate a canary in vanilla Kubernetes with replica math on two Deployments behind one Service. Because a Service load-balances evenly across all matching pods, the traffic split is roughly the pod ratio:

# Stable: 9 pods of the old version
# Canary: 1 pod of the new version
# Both carry label app: web, so the ONE Service spreads ~10% of traffic to canary
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-stable
spec:
  replicas: 9
  selector:
    matchLabels: { app: web, track: stable }
  template:
    metadata:
      labels: { app: web, track: stable }
    spec:
      containers:
        - name: web
          image: myapp:1.4.0
---
# web-canary: replicas 1, track: canary, image myapp:1.5.0, same app: web label

Scale the canary from 1 to 3 to 5 pods to widen exposure, then, once confident, roll the stable Deployment to the new image and scale the canary back to zero. This works, but notice its limits: the split is only as fine-grained as your pod ratio (getting 5% means 1 canary pod per 19 stable, which needs 20 pods), the routing is random rather than sticky, and - the big one - Kubernetes itself does not watch the canary's metrics. You are the control loop, eyeballing dashboards and scaling manually. It is a real canary in mechanism but a manual one in operation.

Where a tool takes over

The vanilla approximations above hit a wall in the same place: Kubernetes gives you the primitives (multiple ReplicaSets, label-based routing, readiness gating) but no automated, metric-driven progressive delivery. It cannot natively do "shift 5% of traffic, check the error rate over five minutes, if healthy shift to 25%, otherwise roll back automatically." That decision loop - analyze real metrics, advance or abort - is exactly what dedicated tools add.

  • Argo Rollouts replaces the Deployment with a Rollout resource that speaks canary and blue-green natively. You declare the steps (setWeight: 20, pause: {duration: 5m}, setWeight: 50, and so on), wire in AnalysisTemplates that query Prometheus, Datadog, or similar, and it automatically promotes or aborts each step based on whether the metrics stay within bounds. It also integrates with a service mesh or ingress for true percentage-based traffic splitting rather than pod-ratio approximation.
  • Flagger sits on top of your existing Deployments and a service mesh (Istio, Linkerd) or ingress (NGINX, Gateway API). It drives the canary by progressively shifting traffic weights at the mesh layer, running metric analysis and webhook checks between steps, and rolling back automatically on regression - all while you keep authoring a normal Deployment.

The line is clear. Native Kubernetes RollingUpdate, plus readiness probes and the timing fields, is enough for the everyday case and is what you should master first - it is genuinely safe and covers most services. Manual blue-green and canary with selectors and replica math are fine for occasional, supervised releases. But once you want repeatable, automated, metric-gated progressive delivery with automatic rollback, stop hand-rolling it and adopt Argo Rollouts or Flagger. Reimplementing their analysis loop with cron jobs and kubectl scale is a well-known way to build a fragile, half-working version of a solved problem.

The shape of it

A Deployment's real value is not "run N pods" - the ReplicaSet does that - it is managing change safely, and change is orchestrated by scaling a new ReplicaSet up while scaling the old one down. RollingUpdate governs that dance with maxSurge (extra pods for headroom and speed) and maxUnavailable (how much capacity you will give up), and the two trade capacity against speed against cluster room. Recreate is the deliberate all-at-once swap for versions that cannot coexist, at the price of downtime. The kubectl rollout commands - status to gate CI, history and undo for instant rollback off retained ReplicaSets, pause/resume to batch or hand-canary - are how you drive and recover a live deploy. Readiness probes are the load-bearing piece: they turn a blind swap into a gated, self-stalling rollout, and minReadySeconds plus progressDeadlineSeconds give it the right patience. Blue-green and canary are higher-level shapes you can approximate with selectors and replica math, but when you need automated, metric-driven rollouts with auto-rollback, that is where Argo Rollouts or Flagger earn their keep. When a rollout misbehaves, the debugging read is the same STATUS -> describe -> logs loop from the debugging guide, aimed at the new ReplicaSet's pods.