What are counters, gauges, histograms, and summaries in Prometheus?

Think of these four metric types like different measuring tools. A counter is like a car odometer -- it only goes up, and it resets to zero when the car is new (process restart). A gauge is like a speedometer -- it moves up and down to show the current state. A histogram is like a teacher sorting exam scores into grade buckets (A, B, C, D, F) -- you know how many scores fell into each range. A summary is like having a statistician who continuously calculates the median and 90th percentile of those scores on the fly.

Counters are the most common type. They track cumulative totals that only increase or reset to zero on restart. Examples include http_requests_total, errors_total, and bytes_sent_total. You almost always use rate() or increase() with counters because the raw number is meaningless on its own -- it depends on when the process started. Gauges show a snapshot of a current value that can move up or down: memory usage, temperature, active connections, queue depth. Unlike counters, gauges are directly useful as-is -- you can use max_over_time(), min_over_time(), or avg_over_time() to track how they change.

Histograms and summaries both measure distributions (usually latencies), but they work in very different ways. A histogram tells the application to count how many observations fall into preset buckets. For example, http_request_duration_seconds with buckets [0.01, 0.05, 0.1, 0.5, 1, 5] creates a counter for each bucket boundary using the _bucket metric with le (less than or equal) labels. It also creates _count (total observations) and _sum (sum of all observed values). You must set bucket boundaries when you write the code, and changing them requires a redeploy. A summary calculates percentiles (like p50, p90, p99) inside each application process using a streaming algorithm. It gives you ready-made percentile values, but you cannot combine them across instances -- averaging p99 values from different pods gives a statistically wrong result.

In production, histograms are almost always the better choice for latency measurement. The big advantage is that histogram_quantile() can combine data across all your instances -- you get the true overall p99 for your service, not just per-pod numbers. Summaries cannot do this. However, histograms need thoughtful bucket selection. If your API has a p99 of 200ms but your smallest bucket is 1 second, the calculated percentile will be wrong because Prometheus has to guess between wide bucket gaps. The default Prometheus client buckets (0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10) work well for typical HTTP latencies. Watch out for cardinality with histograms -- each label combination creates buckets_count + 2 time series.

A critical mistake is applying rate() to a gauge -- it gives nonsensical results because rate() assumes the value only goes up. Similarly, looking at a counter's raw value is almost never useful. Another common error is creating histograms with too many buckets or too many label dimensions, causing a cardinality explosion. With 20 buckets and 5 label dimensions each having 10 values, one histogram metric creates 20 x 10^5 = 2 million time series. Also remember that _bucket values are cumulative: le="0.5" counts ALL observations at or below 0.5 seconds, not just those between 0.25 and 0.5.