Compare Datadog vs Prometheus+Grafana for monitoring a Kubernetes platform. When does each approach make sense?
Quick Answer
Datadog: SaaS, unified platform, lower ops burden, higher cost. Prometheus+Grafana: open-source, self-managed, highly customizable, lower cost at scale but requires dedicated team to operate.
Detailed Answer
Datadog Advantages
- Unified platform: metrics, traces, logs, synthetics, RUM in one tool - Zero infrastructure management — SaaS - Out-of-the-box integrations (700+) with auto-discovery - Built-in anomaly detection and forecasting (ML-powered) - Compliance features (audit logs, SSO, RBAC) - Faster time-to-value for teams without dedicated observability engineers
Prometheus+Grafana Advantages
- Open-source, no vendor lock-in - Cloud-native design, native Kubernetes integration (ServiceMonitor CRD) - PromQL is the industry standard for metrics querying - Full control over data retention and storage - Dramatically lower cost at scale (100+ nodes, 10M+ time series) - Extensible with Thanos/Mimir for long-term storage and multi-cluster
Decision Framework
- Choose Datadog when: team < 50 engineers, need unified observability fast, budget allows $20-30/host/month, compliance requirements favor SaaS - Choose Prometheus+Grafana when: team > 100 engineers, have SRE team to manage infrastructure, cost-sensitive at scale, need deep customization, multi-cloud/hybrid strategy
Hybrid Approach: Many organizations use Prometheus for metrics (cost-effective at scale) and Datadog for APM/tracing (harder to self-host). Or start with Datadog and migrate to self-hosted as the team matures.
Code Example
# Prometheus ServiceMonitor (Kubernetes)
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: order-service
spec:
selector:
matchLabels:
app: order-service
endpoints:
- port: metrics
interval: 15s
path: /metrics
# Equivalent Datadog annotation on pod
metadata:
annotations:
ad.datadoghq.com/order-service.checks: |
{
"openmetrics": {
"instances": [{
"prometheus_url": "http://%%host%%:8080/metrics",
"namespace": "order_service",
"metrics": ["*"]
}]
}
}
# Cost comparison at 100 nodes:
# Datadog: ~$3,000/month (infrastructure + APM + logs)
# Prometheus+Grafana: ~$500/month (compute for Prometheus/Mimir + storage)Interview Tip
Don't be dogmatic — show you can evaluate trade-offs. The hybrid approach (Prometheus for metrics, Datadog for APM) is often the most pragmatic answer and shows senior-level thinking.