At fleet scale, what is the operational impact of mixing decimal and binary memory units, and how do you prevent it organization-wide?
Quick Answer
Mixed units cause systematic under-provisioning, surprise OOMKills, inaccurate bin-packing and capacity forecasts, and noisy/incorrect alerting thresholds. Prevent it with policy-as-code admission control (Kyverno/OPA) mandating `Mi`/`Gi`, unit-safe defaults in shared Helm charts, and CI validation on every manifest.
Detailed Answer
Per pod the gap is a rounding error; across thousands of pods it becomes real capacity and reliability risk. Decimal limits silently give ~7% less memory than the Gi figure engineers reason about, so autoscaling headroom, node bin-packing math, and cost models drift from reality, and OOMKills cluster around load peaks. Alert rules written against 'GiB' thresholds misfire when the underlying limit was set in decimal.
Controls, defense in depth
- Admission policy (Kyverno/OPA Gatekeeper) that *rejects* any container memory request/limit not ending in a binary suffix — the durable backstop. - Golden Helm charts / base kustomizations that only expose Mi/Gi values, so app teams inherit the right units. - CI gates (conftest/kubeconform + policy) that fail PRs before merge. - A one-time audit + dashboard to find and migrate existing decimal usages.
The theme: make the correct thing the default and the incorrect thing impossible to ship.
Code Example
# Gatekeeper-style intent: deny decimal memory units at admission
# violation[{"msg": msg}] {
# c := input.review.object.spec.containers[_]
# mem := c.resources.limits.memory
# not endswith(mem, "i")
# msg := sprintf("memory limit %v must use binary units (Mi/Gi)", [mem])
# }Interview Tip
Frame it as capacity + reliability risk at scale, then give layered prevention (admission, chart defaults, CI) — architects think in guardrails, not one-off fixes.