How do you manage infrastructure drift detection at enterprise scale with scheduled plan runs?

Think of drift detection like a nightly security guard doing rounds. The guard has a checklist of how every door, window, and safe should look. If something has changed — a window left open, a safe combination altered, a new lock installed — the guard reports it. Drift detection in Terraform works the same way: scheduled plan runs compare what actually exists in AWS against what Terraform expects, and any discrepancy is flagged for investigation.

Infrastructure drift occurs when the actual state of cloud resources diverges from the Terraform-declared state. This happens through manual console changes (someone modifies a security group via the AWS console), changes by other tools (an automation script modifies a resource that Terraform also manages), auto-scaling events that modify resource attributes, and AWS service updates that change default behaviors. In a banking environment, drift is a compliance risk — if your Terraform code declares that an RDS instance has encryption enabled but someone disables it through the console, your compliance posture is degraded and your Terraform state does not reflect reality.

Terraform Enterprise enables scheduled plan-only runs on workspaces. You configure a workspace to run terraform plan automatically at a set interval — typically nightly for production workspaces and weekly for non-production. The plan compares the current Terraform configuration and state against the actual infrastructure via provider API calls. If the plan detects changes (resources to update, create, or destroy), it means drift has occurred. TFE marks the run as 'planned and finished' with a non-empty plan, and you can configure webhook notifications to alert your team via Slack, PagerDuty, or a custom drift-tracking system.

At enterprise scale, not all drift is equal. A changed tag is low-severity drift that might be auto-remediated. A modified security group rule is high-severity drift that requires immediate investigation — someone may have opened a port that violates PCI-DSS. A deleted resource is critical drift that needs urgent attention. Build a drift classification system: the webhook from TFE sends the plan summary to a Lambda function or custom service that parses the plan output, categorizes each change by resource type and attribute, assigns a severity level, and routes the notification appropriately. Low-severity drift creates a Jira ticket for the next sprint. High-severity drift pages the security team. Critical drift triggers an incident response.

For drift remediation, there are two approaches. Auto-remediation configures TFE to automatically apply the plan when drift is detected, restoring infrastructure to the declared state. This is appropriate for low-risk drifts like tag changes or description updates, but dangerous for high-risk resources — auto-applying a plan that wants to recreate an RDS instance would cause downtime. Selective auto-remediation uses Sentinel policies to evaluate the drift plan: if the only changes are to tags and descriptions, auto-apply; if the plan includes any destroy or replace actions, block and alert. Manual remediation requires a human to review the drift, determine whether the Terraform code or the infrastructure should be updated, and either apply the plan or update the code to match the new reality.

The biggest gotcha is drift detection generating noise that teams ignore. If your scheduled plans consistently show drift from resources that Terraform partially manages (like ASG instance counts that change with auto-scaling), the team learns to dismiss all drift alerts. Use lifecycle ignore_changes blocks in Terraform for attributes that are expected to drift (like ASG desired_count), and ensure your scheduled plans only flag genuine unauthorized changes. Another gotcha is the API rate limiting — running terraform plan across 200 workspaces simultaneously hammers the AWS API. Stagger your scheduled plans across the night, and use workspace tags to group and schedule them in batches. Finally, drift detection only catches drift in resources Terraform manages — resources created manually outside of Terraform are invisible. Complement TFE drift detection with AWS Config rules that detect unmanaged resources.