How do you run Kafka on Kubernetes using Strimzi, and what are the production challenges?
Quick Answer
Strimzi provides a Kubernetes Operator that manages Kafka clusters as custom resources. It handles broker deployment, topic management, user authentication, and rolling upgrades. Production challenges include persistent storage sizing, rack awareness for HA, and managing JVM heap tuning across broker pods.
Detailed Answer
Think of running a post office inside a shopping mall. The mall provides space, power, and security (Kubernetes), but the post office needs its own sorting machines, mailboxes, and delivery routes (Kafka brokers, topics, partitions). Strimzi is the contractor that builds and maintains the post office inside the mall, handling construction, repairs, and expansions without shutting down mail service.
Strimzi is a CNCF project that provides Kubernetes Operators for running Apache Kafka. Instead of manually deploying Kafka brokers as StatefulSets with complex configuration, you declare a Kafka custom resource that specifies replicas, storage, listeners, and authentication. The Strimzi Operator reconciles this into the actual Kubernetes resources: StatefulSets for brokers and ZooKeeper (or KRaft controllers), Services for client access, ConfigMaps for broker configuration, and PersistentVolumeClaims for data.
Internally, the Strimzi Operator watches for Kafka, KafkaTopic, KafkaUser, and KafkaConnect custom resources. When you create a Kafka CR with 3 replicas and 100Gi storage, the Operator creates a StatefulSet with 3 pods, each with a PVC for data and a PVC for logs. It configures broker IDs, advertised listeners with proper DNS names, inter-broker replication, and rack awareness using topology labels. For client access, Strimzi creates bootstrap Services and per-broker Services so clients can discover and connect to specific brokers.
At production scale, the key challenges are storage performance (Kafka is I/O intensive, requiring SSDs with provisioned IOPS), JVM tuning (brokers need careful heap sizing to avoid long GC pauses), rolling upgrades (Strimzi performs rolling restarts but you must ensure ISR counts stay healthy), and monitoring (JMX metrics exported via Prometheus JMX exporter). Teams should configure PodDisruptionBudgets to prevent multiple brokers from going down simultaneously during node maintenance.
The non-obvious gotcha is that Kafka's advertised listeners must be reachable by all clients. In Kubernetes, if you expose Kafka outside the cluster using NodePort or LoadBalancer listeners, each broker needs its own external address. Strimzi handles this with per-broker Services, but misconfigured DNS or security groups can cause clients to connect to the bootstrap but fail when redirected to individual brokers. Always test external client connectivity from outside the cluster before going live.