How do you design Grafana dashboards following the RED method for microservices?

Think of monitoring a restaurant. Rate is how many customers you serve per hour. Errors is how many meals are sent back. Duration is how long customers wait for their food. If you track these three numbers for every section of the restaurant (bar, dining room, takeout), you know exactly where problems are and how severe they are. The RED method applies this same thinking to microservices.

The RED method was popularized by Tom Wilkie (Grafana Labs) as a simplification of Google's Four Golden Signals. While the Golden Signals include Latency, Traffic, Errors, and Saturation, RED focuses on the three most actionable signals for request-driven services. Rate tells you the traffic volume and whether it matches expectations. Errors tell you if something is broken. Duration tells you if something is slow. Together, they answer the question every on-call engineer asks: is the service healthy, and if not, what is wrong?

Implementation in Grafana uses Prometheus queries against standard HTTP metrics. Rate is computed as sum(rate(http_requests_total[5m])) by service. Error rate is sum(rate(http_requests_total{status=~"5.."}[5m])) divided by total rate. Duration uses histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m])) for p99 latency. Each service gets a dashboard row with three panels: a rate graph (requests/sec over time), an error percentage graph (% 5xx over time), and a latency heatmap or multi-quantile graph (p50, p95, p99 over time).

At production scale, dashboard design matters. Use template variables so one dashboard works for all services by selecting a service from a dropdown. Add annotations for deployments so you can visually correlate metric changes with code releases. Use the repeat-by-variable feature to automatically create rows for each service. Set meaningful Y-axis ranges — auto-scaling can hide problems when error rates go from 0.01% to 0.1% because the scale adjusts to make it look flat.

The non-obvious gotcha is that RED works well for request-driven services but not for batch jobs, queues, or databases. For those, the USE method (Utilization, Saturation, Errors) is more appropriate. A payments API gets RED; the Kafka consumer gets USE (CPU utilization, queue depth saturation, processing errors). Many teams make the mistake of applying RED to everything, then wondering why their database dashboard does not show useful information. Use RED for services that handle requests and USE for infrastructure components.