After an Outage, Mandate Manual Approval on Every Deploy?
After a bad deploy caused an outage, leadership wants a mandatory human sign-off on every production release. Defend a better way to prevent the next one.
CI/CD
ObservabilityYour decision
A bad deploy caused a two-hour outage. Leadership now wants a mandatory manual approval gate on every production deploy, where a lead signs off before anything ships. Do you agree, and what do you propose instead?
The situation
Yesterday a routine deploy shipped a change that took production down for about two hours. In the retro, leadership announced that from now on every deploy to production must be approved by a lead through a manual gate in the pipeline before it can proceed.
Context
Today the pipeline deploys straight to all production instances at once when tests pass. There is no canary or staged rollout, rollback is a manual redeploy of the previous version, and test coverage on the affected path was thin. The team currently ships several times a day. You are asked whether to implement the manual approval gate as directed.