MTTR is often called a misleading metric. Why, and what should an SRE team measure about incident response instead?
Quick Answer
MTTR averages a fat-tailed distribution (one 8-hour incident swamps twenty 5-minute ones), conflates detection/triage/mitigation/repair into one number, and invites gaming (close fast, reopen quietly). Better: percentile-based phase timings (detect, engage, mitigate) per severity, plus distribution-aware views — and pair speed metrics with recurrence and postmortem-action completion so you're not just getting faster at having the same incident.
Detailed Answer
Three structural problems: (1) statistical — incident durations are log-normal-ish with heavy tails; the mean tracks your worst incident, not your typical one, so 'MTTR improved 30%' usually means 'we had fewer monsters this quarter', which is luck, not capability; (2) aggregation — time-to-detect (monitoring quality), time-to-engage (paging/escalation), time-to-mitigate (runbooks, rollback tooling), and time-to-full-repair are owned by different systems; one blended number can't tell you which to invest in; (3) incentive — a single headline number rewards closing tickets over restoring service. The better dashboard: per-severity percentiles (median and p90) of each phase; 'customer-first detection' count (should trend to zero); mitigation-vs-repair separated (mitigate fast, repair calmly); recurrence rate (same root-cause class within 90 days) as the counterweight — a team that mitigates in 5 minutes but has the same incident weekly is failing; and postmortem action-item completion within SLA as the learning-loop health check. Present distributions, not means, in ops reviews. The mature framing: incident speed metrics measure your response machinery; recurrence and action-completion measure whether the organization learns.
Code Example
incident scorecard (quarterly, per severity): detect_p50/p90: 2m / 11m (alert quality) engage_p50/p90: 3m / 9m (paging/escalation) mitigate_p50/p90: 14m / 62m (runbooks, rollback) customer_first_detections: 1 (target 0) recurrence_90d: 8% (learning loop) postmortem_actions_on_time: 74%
Interview Tip
Lead with the fat-tail statistics problem (means lie about skewed distributions), then decompose into phases with different owners — that two-step critique is what distinguishes metric literacy from metric recitation.