How does the Flux reconciliation loop work and what happens when drift is detected?

Imagine a meticulous gardener who walks through the garden every ten minutes with a blueprint showing exactly where each plant should be. If someone has moved a bush, added an unauthorized gnome, or a plant has wilted, the gardener restores everything to match the blueprint. This is what the Flux reconciliation loop does: it continuously ensures reality matches the plan, regardless of what happened in between checks.

The reconciliation loop is the heartbeat of FluxCD and the core mechanism that makes GitOps work in practice. Each Flux controller runs its own reconciliation loop for the resources it manages. The source-controller reconciles GitRepository, HelmRepository, and Bucket sources by periodically fetching content and producing artifacts. The kustomize-controller reconciles Kustomization resources by comparing the manifests in the latest source artifact with what is currently running in the cluster. The helm-controller reconciles HelmRelease resources by comparing the desired chart version and values with the installed release. Each controller operates independently, and the interval field on each resource determines how frequently the loop runs.

When the kustomize-controller detects drift during reconciliation, it takes corrective action using server-side apply. Drift can occur in several ways: a developer manually ran kubectl edit to change a resource, an autoscaler modified replica counts, or a third-party operator updated labels. Flux compares the desired state (manifests from Git) with the actual state (live resources in the cluster) and applies any differences. This means that manual changes to Flux-managed resources are automatically reverted on the next reconciliation cycle. The revert is not destructive; it uses server-side apply with the Flux field manager, so fields managed by other controllers (like the HPA managing replicas) can be excluded from Flux's ownership.

The reconciliation process follows a specific order of operations. First, the source-controller detects a new commit and produces a fresh artifact. Then, the kustomize-controller picks up the new artifact and begins reconciliation. It builds the Kustomize output, computes a diff against the live state, applies changes, and runs health checks. If health checks are configured, the controller watches the deployed resources until they reach a healthy state or the timeout expires. Only after health checks pass does the controller update the Kustomization status to indicate success. If health checks fail, the status reflects the failure, and any notification-controller alerts fire.

In production, teams tune reconciliation intervals based on the criticality and change frequency of each component. Infrastructure components like ingress controllers might reconcile every thirty minutes since they change infrequently, while active application deployments might reconcile every two minutes during deployment windows. The flux reconcile command allows on-demand triggering when you cannot wait for the next scheduled cycle. Additionally, Flux supports webhook receivers that trigger immediate reconciliation when a Git push event is received, reducing the delay between committing a change and seeing it applied. This webhook-triggered reconciliation is the preferred approach in production because it provides near-instant deployment while the interval-based loop serves as a safety net to catch any missed webhooks.

Drift detection behavior can be customized for advanced use cases. The force field on a Kustomization recreates resources that cannot be patched (useful for immutable fields). The targetNamespace field redirects all resources to a specific namespace regardless of what the manifests specify. Teams can also use the patches field to apply last-mile modifications to manifests without changing the source repository, which is useful when different clusters need slightly different configurations of the same base manifests.