How does Prometheus's pull-based scraping model work, and why did Prometheus choose it over a push-based model?
Quick Answer
Prometheus periodically scrapes an HTTP endpoint (usually /metrics) exposed by each target, rather than having applications push metrics to it. This makes service health directly observable (a failed scrape is itself a signal), simplifies target lifecycle management via service discovery, and avoids a central ingestion bottleneck that a push model would require for backpressure and fan-in.
Detailed Answer
In the pull model, Prometheus's scrape config defines a set of targets (statically or via service discovery — Kubernetes, Consul, EC2, file_sd, etc.), and the server initiates an HTTP GET against each target's metrics endpoint on a fixed interval (commonly 15s-60s). The target's client library (e.g. client_golang, client_python) exposes current metric values in the Prometheus text exposition format or the newer OpenMetrics format.
This has several practical consequences interviewers probe for: (1) you can scrape from your laptop to debug a target directly with curl, since the target doesn't need to know Prometheus exists; (2) a target that's down simply fails to be scraped, which Prometheus surfaces as the synthetic up metric going to 0 — a push model needs a separate heartbeat/dead-man's-switch mechanism to detect this; (3) Prometheus, not the application, controls cardinality and scrape frequency, which caps the blast radius of a runaway metrics emitter; (4) short-lived batch jobs don't fit this model well, which is exactly why the Pushgateway exists as an explicit escape hatch — metrics are pushed to the gateway, which Prometheus then scrapes like any other target.
The tradeoff: pull doesn't work well through NAT/firewalls without extra plumbing (hence remote_write and Pushgateway), and very large fleets need service discovery to keep the target list current rather than static configs.
Interview Tip
If asked "why pull instead of push," lead with the `up` metric and target discoverability — that's the answer that shows you've actually operated Prometheus, not just read the docs.