What is etcd's role, and why does quorum matter?
Quick Answer
etcd is the cluster's consistent key-value store holding all API objects. It uses the Raft consensus algorithm, so it needs a majority (quorum) of members available to accept writes — which is why you run an odd number, typically 3 or 5.
Detailed Answer
Every object you create — pods, secrets, configmaps, deployments — is persisted in etcd via the apiserver. etcd replicates data across members using Raft, which requires a majority to agree before committing a write. With 3 members you tolerate 1 failure (2 is still a majority); with 5 you tolerate 2. An even number buys you nothing: 4 members still only tolerate 1 failure but add cost and latency, so odd counts are standard. If quorum is lost, etcd goes read-only and the cluster can't accept changes — pods keep running but you can't schedule or update anything. Practical implications: back up etcd regularly (snapshot), keep it on fast low-latency disks, and never let latency between members grow (don't stretch a single etcd cluster across distant regions).
Code Example
# Snapshot etcd (run against the etcd endpoint with its certs) ETCDCTL_API=3 etcdctl snapshot save /backup/etcd-snapshot.db \ --endpoints=https://127.0.0.1:2379 \ --cacert=/etc/kubernetes/pki/etcd/ca.crt \ --cert=/etc/kubernetes/pki/etcd/server.crt \ --key=/etc/kubernetes/pki/etcd/server.key
Interview Tip
Explain *why* odd numbers: 'N members tolerate floor((N-1)/2) failures, and an even count adds a member without adding fault tolerance.' Then mention etcd backups — the follow-up is always 'how do you recover the cluster?'