How do consumer groups achieve scalable, exactly-once-ish processing?
⚡
Quick Answer
Partitions are distributed among consumers in a group so each partition is consumed by one member; adding consumers scales throughput up to the partition count.
Detailed Answer
Within a group, Kafka assigns each partition to exactly one consumer (rebalancing on membership changes), giving horizontal scaling and fault tolerance. Beyond the partition count, extra consumers sit idle. Offset commits plus idempotent producers/transactions enable effectively-once semantics.
💡
Interview Tip
Max useful consumers = partition count.
kafkaconsumer-groupsscaling