How do ArgoCD sync waves and hooks enable ordered multi-service deployments with dependency management, and what happens when a hook fails mid-wave?
Quick Answer
Sync waves assign integer ordering to resources so ArgoCD syncs lower-numbered waves first, waiting for each wave's resources to become healthy before proceeding. Hooks like PreSync, Sync, and PostSync run Jobs or Pods at specific lifecycle points. A hook failure in any wave halts the entire sync operation, leaving later waves unprocessed and requiring manual intervention or retry.
Detailed Answer
Think of a theater production with scene changes. The stagehands must set up the backdrop (wave 0) before the props (wave 1) before the actors enter (wave 2). If the backdrop collapses during setup, the director stops everything rather than sending actors onto a broken stage. ArgoCD sync waves work the same way: they enforce ordering so that infrastructure dependencies are ready before the services that depend on them.
Sync waves and hooks solve a fundamental problem in GitOps: not all resources should be applied simultaneously. A database migration must complete before the application deployment that depends on the new schema. A ConfigMap or Secret must exist before the Deployment that mounts it. A namespace must be created before resources are placed in it. Without ordering, ArgoCD would apply everything at once, leading to race conditions and transient failures.
Internally, ArgoCD processes sync waves by sorting all resources by their argocd.argoproj.io/sync-wave annotation (defaulting to 0). Within each wave, it applies resources and waits for them to reach a healthy state according to their resource health check. Hooks are special resources annotated with argocd.argoproj.io/hook, which can be PreSync (runs before any wave), Sync (runs during the sync phase), PostSync (runs after all waves complete), or SyncFail (runs when sync fails). Hook resources are typically Jobs or Pods. The hook-delete-policy annotation controls whether hooks are deleted before creation, after successful completion, or after failure.
At production scale, teams use wave 0 for namespaces and CRDs, wave 1 for ConfigMaps and Secrets, wave 2 for databases and stateful services, wave 3 for application Deployments, and wave 4 for ingress routes and monitoring. PreSync hooks commonly run database migrations, schema validators, or pre-deployment smoke tests. PostSync hooks run integration tests, cache warmers, or notification jobs. Monitoring should track sync duration per wave, hook execution time, hook pod resource consumption, and retry counts.
The non-obvious gotcha is partial sync state after a hook failure. If a PreSync migration Job fails at wave 1, ArgoCD stops the sync operation entirely. Wave 2 and beyond are never attempted, but wave 0 resources are already applied. The Application is now in a partially synced state that does not match either the previous git commit or the current one. Retrying the sync re-runs all hooks from the beginning, which means PreSync hooks must be idempotent. If a database migration is not idempotent and partially applied, retrying the sync can corrupt data. Architects must design every hook to be safely re-runnable and implement SyncFail hooks for cleanup or rollback notifications.
Code Example
# Deployment using sync waves and hooks for ordered rollout of payments platform
# Wave -1: PreSync hook runs database migration before any resources sync
apiVersion: batch/v1 # Kubernetes Job API for one-time execution
kind: Job # Runs the migration to completion then stops
metadata:
name: payments-db-migrate # Descriptive name for the migration Job
namespace: payments # Same namespace as the application
annotations:
argocd.argoproj.io/hook: PreSync # Runs before any sync wave begins
argocd.argoproj.io/hook-delete-policy: BeforeHookCreation # Deletes previous Job before creating new one
argocd.argoproj.io/sync-wave: "-1" # Ensures migration runs before other PreSync resources
spec:
backoffLimit: 2 # Retries the migration twice on failure before marking failed
activeDeadlineSeconds: 300 # Kills the migration if it runs longer than five minutes
template:
spec:
restartPolicy: Never # Jobs should not restart; ArgoCD handles retry logic
containers:
- name: migrate # Container running the migration tool
image: registry.company.com/payments-db-migrator:3.7.2 # Versioned migration image
command: ['./migrate', '--direction=up', '--env=production'] # Applies pending migrations
env:
- name: DATABASE_URL # Connection string for the payments database
valueFrom:
secretKeyRef:
name: payments-db-credentials # Secret containing database connection details
key: url # Key within the Secret holding the full connection string
---
# Wave 0: ConfigMaps and Secrets that Deployments depend on
apiVersion: v1 # Core Kubernetes API
kind: ConfigMap # Application configuration deployed before the service
metadata:
name: payments-api-config # Configuration for the payments API
namespace: payments # Matches the application namespace
annotations:
argocd.argoproj.io/sync-wave: "0" # Deploys in wave 0 after PreSync hooks
data:
MAX_CONNECTIONS: "50" # Database connection pool size for production
CACHE_TTL: "300" # Cache time-to-live in seconds for payment lookups
---
# Wave 1: Main application Deployment depends on ConfigMap from wave 0
apiVersion: apps/v1 # Deployment API for rolling updates
kind: Deployment # Long-running service managed by the ReplicaSet controller
metadata:
name: payments-api # Primary payments API service
namespace: payments # Application namespace
annotations:
argocd.argoproj.io/sync-wave: "1" # Deploys after ConfigMaps are ready
spec:
replicas: 5 # Five replicas for production traffic handling
selector:
matchLabels:
app: payments-api # Connects Deployment to its Pods
template:
metadata:
labels:
app: payments-api # Pod label for Service discovery and monitoring
spec:
containers:
- name: api # Main API container
image: registry.company.com/payments-api:4.12.0 # Production release image
ports:
- containerPort: 8080 # HTTP API port
envFrom:
- configMapRef:
name: payments-api-config # Loads all keys from the wave-0 ConfigMap
---
# Wave 2: PostSync hook runs integration tests after all resources are healthy
apiVersion: batch/v1 # Job API for test execution
kind: Job # One-time test run
metadata:
name: payments-api-smoke-test # Post-deployment verification
namespace: payments # Same namespace as the tested service
annotations:
argocd.argoproj.io/hook: PostSync # Runs after all sync waves complete successfully
argocd.argoproj.io/hook-delete-policy: HookSucceeded # Cleans up Job after success
spec:
backoffLimit: 1 # Allows one retry for transient network issues
template:
spec:
restartPolicy: Never # Let ArgoCD handle retry decisions
containers:
- name: smoke-test # Container running integration tests
image: registry.company.com/payments-smoke-tests:2.1.0 # Test suite image
command: ['./run-tests', '--endpoint=http://payments-api:8080', '--suite=critical'] # Tests critical payment flowsInterview Tip
A junior engineer typically answers that sync waves order resource creation, but for a senior/architect role, the interviewer is actually looking for failure handling and idempotency design. Explain how ArgoCD halts at a failed wave, what partial sync state looks like, why hooks must be idempotent for safe retry, and how hook-delete-policy affects re-execution. A mature answer describes the production pattern of PreSync for migrations, Sync for application resources, PostSync for validation, and SyncFail for alerting. Bonus points for mentioning that long-running hooks block the entire sync and consume cluster resources until activeDeadlineSeconds terminates them.