What are Datadog tags and why is tag cardinality management critical? How do you design a tagging strategy?
Quick Answer
Tags are key:value pairs attached to all telemetry data for filtering and grouping. High-cardinality tags (user_id, request_id) create millions of unique time series, exploding costs and degrading performance.
Detailed Answer
Tags in Datadog
Tags are the universal correlation mechanism. Every metric, trace, and log can be tagged with key:value pairs. Tags enable: filtering dashboards by environment/service, grouping metrics in queries, scoping monitors to specific services, cost allocation per team.
Cardinality Problem
Each unique tag combination creates a new time series. A metric with tags {service, env, endpoint, status, host} across 100 services × 3 envs × 50 endpoints × 5 statuses × 20 hosts = 1.5M time series for ONE metric. Add user_id (100K users) and you get 150 BILLION series — system collapse.
Tagging Strategy
Required tags (enforce via Agent config)
- env: production/staging/development - service: service name (matches APM service) - team: owning team - version: deployment version
Recommended tags
- region, availability-zone, cluster - managed-by: terraform/helm/manual
Banned tags (never use on metrics)
- user_id, request_id, session_id — unbounded cardinality - timestamp, trace_id — already handled by Datadog - Any tag with >1000 unique values
Enforcement
- Datadog Metrics without Limits: control which tag combinations are queryable - Tag naming conventions documented and enforced via CI lint checks - Monthly cardinality reports per team
Code Example
# Datadog Agent: enforce global tags
# datadog.yaml
tags:
- env:production
- region:us-east-1
- team:platform
# Metrics without Limits: configure tag allowlist
# Only these tag combinations are queryable (reduces cost)
curl -X PUT "https://api.datadoghq.com/api/v2/metrics/http.requests" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-d '{
"data": {
"type": "metrics",
"id": "http.requests",
"attributes": {
"tags": ["service", "env", "status_code_class"]
}
}
}'
# Check cardinality per metric
# Datadog UI: Metrics Summary → sort by 'Distinct Tag Values'Interview Tip
Cardinality management is the most important cost control lever in Datadog. Show you understand why user_id as a tag is catastrophic and how Metrics without Limits helps control it.