Explain PersistentVolumeClaims with ReadWriteOnce (RWO) access mode and the scheduling implications for stateful workloads.
Quick Answer
A PVC requests storage from a StorageClass; RWO allows mount by a single node at a time. Pods using RWO volumes must schedule on the node where the volume is already attached, affecting failover speed and multi-replica Deployments.
Detailed Answer
PersistentVolumeClaims abstract storage consumption from underlying PersistentVolumes. A developer requests capacity, access mode, and StorageClass; the provisioner dynamically creates a PV (EBS, GCE PD, Ceph RBD, etc.) and binds it to the PVC. Access modes define concurrency: ReadWriteOnce (RWO) permits read-write mount by a single node, ReadOnlyMany (ROX) allows many nodes read-only, ReadWriteMany (RWX) allows many nodes read-write (requires NFS, EFS, or similar).
RWO is the default for block storage in cloud environments. Only one node may mount the volume in read-write mode at any time. If checkout-worker Pod using PVC checkout-data-pvc runs on worker-node-1, the EBS volume attaches to that node. Rescheduling the Pod to worker-node-2 requires detaching from node-1 and attaching to node-2—a process that can take 30–120 seconds on AWS, during which the Pod stays Pending or ContainerCreating.
This has profound design implications. You cannot run 3 replicas of a Deployment all mounting the same RWO PVC—they would fight for exclusive attachment. Stateful workloads needing shared storage require RWX storage classes or object storage. For per-replica durable state, StatefulSets with volumeClaimTemplates provision one RWO PVC per Pod, each bound exclusively to that Pod identity.
StorageClass parameters control performance (gp3 vs io2), encryption, reclaim policy (Retain vs Delete), and volume expansion (allowVolumeExpansion: true). Reclaim policy Retain preserves data when PVC is deleted—critical for databases.
Analogy: RWO is like a library book checkout system—only one borrower (node) may take the book home (mount read-write) at a time. Others must wait for return (detach) before checking it out elsewhere. RWX is like a reference copy that stays in the reading room for anyone.
Always align access mode with workload topology. Monitor attach/detach errors in events—they often indicate zone mismatches (volume in us-east-1a, Pod scheduled to us-east-1b without regional volume support).
Volume snapshots (VolumeSnapshot API) enable point-in-time backups without stopping checkout-worker. Bind PVCs eagerly with volumeBindingMode: WaitForFirstConsumer on the StorageClass to ensure the volume provisions in the same zone as the scheduled Pod, avoiding cross-AZ attach failures that plague multi-zone clusters.