How do you define SLOs and error budgets in practice, and what should actually happen operationally once an error budget is exhausted?
Quick Answer
Start with an SLI (Service Level Indicator — a directly measurable metric like the percentage of requests under 200ms) and set an SLO (Service Level Objective — a target for that metric, like 99.9% over 30 days); the error budget is simply 1 minus the SLO, expressed as an allowance of acceptable failure, roughly 43 minutes of downtime a month at 99.9%. Once that budget is exhausted, the agreed consequence is a reliability freeze — new feature launches pause and the team's priority shifts entirely to reliability work until the budget recovers, which only works if that consequence was agreed to and enforced before the crisis, not negotiated in the moment.
Detailed Answer
An error budget is like a household's monthly discretionary spending limit on top of fixed bills — you're not expected to spend zero on entertainment, you're allowed a specific, agreed amount, and the moment you blow through that amount, the sensible response is not to keep spending as normal and hope it works out, it's to stop discretionary spending until next month's budget resets. SLOs and error budgets formalize exactly this idea for reliability: perfection (100% uptime, zero errors) is neither achievable nor actually necessary for most services, so instead of chasing an impossible target, teams agree in advance on how much unreliability is acceptable, and then treat that allowance as a real, trackable budget rather than a vague aspiration.
This framework exists because, without it, reliability work and feature work compete for the same engineering time with no objective tiebreaker — every team believes their velocity matters and every team believes reliability matters, and without a quantified, pre-agreed threshold, that tension gets resolved by whoever argues loudest in a planning meeting rather than by data. Google's SRE model introduced error budgets specifically to convert 'how reliable is reliable enough' from a subjective argument into an objective, numeric answer that both the reliability-focused and feature-focused sides of an organization agree to up front, before there's a live incident to argue about.
Step by step: first, define the SLI — pick the specific, measurable signal that represents user-facing quality, commonly success rate (percentage of requests that don't return a 5xx) or latency (percentage of requests under some threshold, like p99 under 200ms). Second, set the SLO as a target percentage of that SLI over a rolling window, commonly 30 days — 99.9% is a common target implying roughly 43 minutes of acceptable downtime or SLI-violating time per month. Third, the error budget is simply the inverse: 1 minus 99.9% equals 0.1%, which converts into that same ~43 minutes over 30 days, or however many failed requests that percentage represents given your traffic volume. Fourth, track budget consumption continuously — not just after the fact — using recording rules that calculate a rolling burn rate (how fast you're consuming the budget relative to a sustainable pace), so the team knows in real time whether they're on track to exhaust the budget early or comfortably coast to the end of the window with room to spare.
At production scale, the actual mechanics matter as much as the definition: teams implement this with Prometheus recording rules that continuously calculate the SLI over rolling windows, feed that into a burn-rate calculation, and alert on two distinct thresholds — a fast-burn alert (consuming budget so quickly that at the current rate you'd exhaust it in hours, demanding immediate paging) and a slow-burn alert (a steadier, less urgent trend that would exhaust the budget in days, worth investigating but not necessarily waking anyone up at 3 AM). What actually happens when the budget hits zero is the part organizations most often get wrong in practice: the 'freeze new features until reliability improves' consequence only has teeth if it was agreed to by both engineering leadership and product stakeholders *before* the budget ran out — negotiating it in the moment, mid-crisis, means whoever has the most organizational leverage simply overrides it, and the entire framework collapses back into the same subjective argument it was designed to eliminate.
The gotcha: teams that only look at the error budget number at the end of the 30-day window, rather than tracking burn rate continuously, discover the budget is already gone with two weeks still left in the window and have no way to react proactively — by definition, a lagging, retrospective view of budget consumption can only tell you the crisis already happened, never that it's *about* to happen, which is exactly why burn-rate alerting (not just a raw budget-remaining dashboard) is the difference between an SRE practice that actually prevents outages and one that just narrates them after the fact.