Compare Elasticsearch vs Loki vs ClickHouse for log management. When would you choose each?
Quick Answer
Elasticsearch: full-text search, complex queries, rich ecosystem. Loki: cost-effective, label-based, Grafana-native. ClickHouse: SQL analytics, high compression, columnar storage for structured logs.
Detailed Answer
Elasticsearch
- Full-text indexing of all log content - Rich query language (KQL, Lucene, aggregations) - Mature ecosystem (Beats, Logstash, Kibana) - Best for: Teams needing full-text search across all log fields, complex aggregations, existing ELK investment - Drawback: Expensive — indexes every field, high memory/CPU/storage costs
Grafana Loki
- Only indexes labels (metadata), not log content - Log content is compressed and stored in object storage (S3/GCS) - Uses LogQL (PromQL-like) for querying - Best for: Cost-sensitive environments, Kubernetes-native logging, teams already using Grafana/Prometheus - Drawback: Grep-style search on content is slower than Elasticsearch's inverted index. Not suitable for complex analytics.
ClickHouse
- Columnar storage with extreme compression (10:1 to 50:1) - SQL interface for analytics - Blazing fast aggregation queries - Best for: Structured/semi-structured logs, analytics-heavy use cases, very high volume (billions of rows), teams comfortable with SQL - Drawback: Less mature log ecosystem, requires more custom pipeline work
Cost Comparison (1TB/day, 30-day retention)
- Elasticsearch: ~$15K-25K/month (hot-warm architecture) - Loki: ~$2K-5K/month (object storage + minimal compute) - ClickHouse: ~$3K-8K/month (high compression reduces storage)
Code Example
# Loki LogQL query (equivalent to ES full-text search)
{namespace="production", app="api-gateway"}
|= "error"
| json
| status_code >= 500
| line_format "{{.timestamp}} {{.message}}"
# ClickHouse log table
CREATE TABLE logs (
timestamp DateTime64(3),
service LowCardinality(String),
level LowCardinality(String),
message String,
trace_id String
) ENGINE = MergeTree()
PARTITION BY toYYYYMMDD(timestamp)
ORDER BY (service, timestamp)
TTL timestamp + INTERVAL 30 DAY;
# Elasticsearch equivalent query
GET logs-*/_search
{
"query": {
"bool": {
"must": [
{ "match": { "message": "error" }},
{ "range": { "status_code": { "gte": 500 }}}
]
}
}
}Interview Tip
Lead with the trade-off triangle: search capability vs cost vs query speed. Show you can make a recommendation based on the team's specific needs rather than just listing features.