How do you ensure one pod replica always runs on an on-demand node while remaining replicas run on spot instances?
Quick Answer
Use two deployments: one with replicas=1 and nodeSelector for on-demand nodes, another with remaining replicas and nodeSelector+tolerations for spot nodes.
Detailed Answer
This is a common cost optimization pattern. The two-deployment approach gives explicit control:
Deployment 1 (stable): replicas=1, nodeSelector: node-pool=on-demand. This pod is always available even during spot evictions. Deployment 2 (burst): replicas=N-1, nodeSelector: node-pool=spot, with tolerations for spot taints.
Alternatives considered
- Single deployment with topologySpreadConstraints: Cannot guarantee exactly 1 pod on on-demand. - Karpenter/Cluster Autoscaler priorities: Prefer spot but fall back to on-demand. Less explicit control.
Label node pools (node-pool=on-demand, node-pool=spot). Taint spot nodes. Add PodDisruptionBudgets to handle spot evictions gracefully. Both deployments should use the same service selector so traffic routes to all replicas.
Code Example
# Stable replica on on-demand
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-stable
spec:
replicas: 1
template:
spec:
nodeSelector:
node-pool: on-demand
containers:
- name: api
image: myapp:latest
---
# Burst replicas on spot
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-burst
spec:
replicas: 4
template:
spec:
nodeSelector:
node-pool: spot
tolerations:
- key: kubernetes.azure.com/scalesetpriority
operator: Equal
value: spot
effect: NoScheduleInterview Tip
Explain why two deployments is cleaner than one. Mention spot interruption handling and PodDisruptionBudgets. This tests scheduling knowledge AND cost awareness.