Site Reliability Engineer interview question
How do you reduce alert fatigue?
Remove alerts that are not urgent or actionable, alert on user symptoms and SLO burn, deduplicate correlated failures, tune windows from incident evidence, route ownership correctly, and review noisy pages as defects.
Direct answer
What a strong answer covers
How it works
Alert fatigue comes from alerts that do not require action, fire for causes rather than symptoms, or duplicate each other during one failure. The cure is an audit: for each alert, ask what a responder did the last five times it fired; delete those with no action, convert those that need attention but not urgency into tickets, and rewrite the rest to trigger on user-facing symptoms or SLO burn rate. Group alerts that fire together into one notification, tune thresholds and windows from incident data rather than defaults, route each alert to the team that can fix it, and review every noisy page in the weekly ops meeting as a defect to be closed.
Example
An audit of 140 alert rules found 90 that had never led to an action; they were deleted or turned into dashboard panels. Eleven infrastructure alerts that fired together during any database failover were replaced by one SLO burn-rate alert on the affected API. Pages fell from 60 to 9 a week, and the on-call engineer began treating each remaining page as real.
Common mistakes
- Adding a new alert after every incident and never removing one.
- Alerting on infrastructure causes so one failure produces ten pages.
- Silencing noisy alerts indefinitely instead of fixing or deleting them.
A useful answer structure
Answer from your own work.
Do not memorise a script: use only examples you can explain honestly in follow-up questions.
Open the desktop workspace and rehearse a clear answer in your own words.