How do you integrate Azure DevOps Pipelines with Terraform for AWS infrastructure deployment?

Think of Terraform integration with Azure DevOps like an architecture firm using a structured blueprint approval process. The architect draws plans (terraform plan), submits them to the review board for approval (approval gate), and only after explicit sign-off does the construction crew begin building (terraform apply). The blueprints are stored in a secure vault (remote state) with a lock that prevents two construction crews from modifying the same building simultaneously. Azure DevOps orchestrates this entire workflow while Terraform handles the actual infrastructure provisioning.

The pipeline structure for Terraform deployments typically follows a three-stage pattern: Plan, Approve, and Apply. The Plan stage runs terraform init to configure the backend and download providers, then terraform plan to generate an execution plan showing what infrastructure changes will occur. The plan output is saved to a binary file and published as a pipeline artifact. This ensures the exact plan that was reviewed is the plan that gets applied, preventing any drift between review time and apply time. The Approve stage is simply an environment check that pauses for manual approval, giving infrastructure engineers time to review the plan output before any changes are made to production infrastructure.

Authentication to AWS from Azure DevOps uses the AWS Service Connection, which provides temporary credentials to Terraform through environment variables. The pipeline task exports AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from the Service Connection, or with OIDC federation, obtains temporary STS credentials. Terraform reads these environment variables automatically without any provider configuration in the HCL code. This separation means Terraform code remains portable across CI/CD systems while Azure DevOps handles credential management. The IAM role assumed by the pipeline should have permissions scoped to only the resources Terraform manages, following least privilege principles.

Remote state management is critical for team collaboration and pipeline safety. Terraform state for AWS infrastructure should be stored in an S3 bucket with versioning enabled, server-side encryption, and a DynamoDB table for state locking. The backend configuration in the Terraform code references the S3 bucket and DynamoDB table. When the pipeline runs terraform apply, Terraform acquires a lock in DynamoDB that prevents any other pipeline run or developer from applying changes simultaneously. This prevents the state corruption that occurs when two processes modify infrastructure concurrently. The S3 bucket versioning provides a backup of every previous state file for disaster recovery.

Pipeline variables and variable groups manage environment-specific configuration. Terraform variable values for different environments (dev, staging, production) are stored in Azure DevOps variable groups and passed to Terraform through -var flags or TF_VAR_ environment variables. Sensitive values like database passwords are stored as secret variables in the variable group, masked in pipeline logs, and optionally sourced from Azure Key Vault. This pattern allows the same Terraform code to deploy to multiple environments with different configurations without duplicating the pipeline definition.

The production gotcha is handling Terraform state drift and failed applies. If terraform apply partially succeeds before failing, the state file reflects the partial changes and subsequent runs attempt to reconcile. Teams must implement pipeline logic that detects partial failures and alerts operators rather than silently retrying. Another common issue is pipeline timeout: large infrastructure changes can exceed the default job timeout, causing the pipeline to kill the Terraform process mid-apply and leaving resources in an inconsistent state. Always set appropriate timeouts for infrastructure pipelines and implement cleanup logic for known failure modes.