How do you implement SLO-based alerting with Prometheus using multi-window burn rate?

Think of a car's fuel gauge. A static threshold alert says 'warn at 25% fuel' — but that ignores whether you are on a highway burning fuel fast or parked with the engine off. Multi-window burn rate is like saying 'warn when fuel consumption over the last hour would empty the tank before you reach the next gas station, AND you are still burning fast right now.' This catches real problems while ignoring brief spikes.

SLO-based alerting starts with defining an error budget. If your SLO is 99.9% availability over 30 days, your error budget is 0.1% — about 43 minutes of downtime. The burn rate is how fast you are consuming this budget. A burn rate of 1x means you will exactly exhaust the budget by the end of the period. A burn rate of 14.4x means you will exhaust the 30-day budget in just 2 days.

Multi-window burn rate uses two windows to reduce false positives. The long window (typically 1 hour) detects sustained error rates that threaten the budget. The short window (typically 5 minutes) confirms the problem is still happening right now. Both conditions must be true for the alert to fire. This prevents alerting on brief spikes that self-resolve (short window would not fire) and on historical errors that have already been fixed (long window shows the past, short window confirms the present). Google's SRE book recommends multiple severity tiers: 14.4x burn rate over 1h/5m for critical (page), 6x over 6h/30m for warning (ticket).

At production scale, teams define recording rules that pre-compute error ratios for each SLI at multiple windows. The error ratio is calculated as rate(http_requests_total{status=~"5.."}[window]) / rate(http_requests_total[window]). Recording rules at 5m, 30m, 1h, and 6h windows avoid expensive queries at alert evaluation time. Grafana dashboards show the remaining error budget as a percentage, making it visual whether the team can ship features or must focus on reliability.

The non-obvious gotcha is that burn rate alerts assume a uniform error distribution, which rarely matches reality. A 5-minute outage that burns 10% of the monthly budget followed by 29 days of perfect operation is very different from a constant 0.1% error rate. Teams should complement burn rate alerts with absolute threshold alerts for catastrophic failures (error rate > 50% for 1 minute) that would cause immediate user impact regardless of the monthly budget.