How do you assess release reliability?

Direct answer

What a strong answer covers

How it works

Release reliability is measured, not felt. Change failure rate counts releases that caused a rollback, hotfix, or incident; rollback rate and time to rollback show how safe the process is; canary deltas in error rate and latency between new and old versions show the effect of each release before full rollout; SLO burn attributed to release windows shows what customers experienced; and time to detect and time to recover show how fast the loop closes. All of it needs release markers in telemetry and version labels on metrics so comparisons are between equivalent traffic on the two versions rather than before and after in time.

Example

A team publishes a monthly release scorecard: 62 releases, change failure rate 6 percent, median canary error delta 0.02 percent, one rollback taking 4 minutes, and 18 percent of the error budget consumed in the hour after releases. The scorecard showed that Friday afternoon releases accounted for most of the failures, which led to a change in the deploy window rather than a freeze.

Common mistakes

• Judging releases by whether anyone complained.
• Comparing metrics before and after a deploy without controlling for traffic changes.
• Measuring deploy frequency alone and calling it reliability.

A useful answer structure

Answer from your own work.

Do not memorise a script: use only examples you can explain honestly in follow-up questions.