10 best practices covering state, modules, security, CI/CD, and maintainability.
Why It Matters
Remote state outputs often begin as convenience and quietly become a platform API. When a network stack renames or reshapes an output, every consuming app stack can break in CI or attempt unexpected replacement. The damage is organizational: teams lose trust in infrastructure automation.
How to Implement
Expose the smallest stable outputs needed by consumers, document ownership, add new outputs before removing old ones, and run dependent previews before deleting compatibility outputs. Avoid exposing full resource objects because they make internal implementation details part of the contract.
Example
output "private_subnet_ids_v1" {
value = module.vpc.private_subnet_ids # Stable list consumed by application stacks.
description = "Private subnet IDs for production workloads; do not remove before v2 migration completes."
}✗ Anti-pattern
The anti-pattern is exporting whole modules or large maps because it feels flexible. It creates hidden coupling and turns platform refactors into application outages.
◈ Architecture Diagram
┌──────────┐ ┌──────────┐
│ ✓ Guard │ │ ✗ Drift │
└────┬─────┘ └────┬─────┘
↓ ↓
┌──────────┐ ┌──────────┐
│ Stable │ │ Page │
└──────────┘ └──────────┘💬 Comments
Why It Matters
Local state files are the single biggest source of Terraform disasters in production environments. When state lives on a developer's laptop or in a CI runner's ephemeral filesystem, it cannot be shared across team members, has no locking mechanism to prevent concurrent modifications, and lacks versioning for rollback. A single terraform apply from a stale local state can destroy resources that another team member created hours ago because the local copy does not reflect the current infrastructure reality. In regulated industries, auditors specifically look for centralized state management as evidence of change control. Organizations that keep local state inevitably experience the nightmare scenario: an engineer runs apply from an outdated state file and Terraform attempts to reconcile by destroying production resources it does not know about. The S3 backend with DynamoDB locking provides atomic state operations that prevent the race conditions that corrupt state files during concurrent CI pipeline runs.
How to Implement
Configure the S3 backend in your root module with versioning enabled on the bucket, server-side encryption using a dedicated KMS key, and a DynamoDB table for state locking. The bucket should have a lifecycle policy retaining at least 90 days of state versions for disaster recovery. Set the DynamoDB table with a partition key named LockID of type String, and enable point-in-time recovery on the table itself. Use a dedicated AWS account or at minimum a dedicated IAM role for state access, separate from the deployment role. Enable bucket access logging to CloudTrail for audit purposes. In CI pipelines, configure the backend using partial configuration files so that credentials are injected at runtime via environment variables rather than hardcoded in the backend block. Always set lock timeout to at least 5 minutes so queued pipelines wait rather than fail immediately when another apply is in progress.
Example
# backend.tf — Production remote state configuration
terraform {
backend "s3" {
bucket = "acme-terraform-state-prod-us-east-1" # Dedicated state bucket
key = "infrastructure/payments-vpc/terraform.tfstate"
region = "us-east-1"
encrypt = true # SSE-KMS encryption at rest
kms_key_id = "arn:aws:kms:us-east-1:111222333444:key/mrk-9a8b7c6d5e4f" # Dedicated KMS key
dynamodb_table = "acme-terraform-locks-prod" # DynamoDB table for state locking
acl = "bucket-owner-full-control" # Prevent cross-account issues
}
}
# S3 bucket configuration (managed in bootstrap stack)
resource "aws_s3_bucket_versioning" "state" {
bucket = aws_s3_bucket.terraform_state.id
versioning_configuration {
status = "Enabled" # Enables rollback to any previous state version
}
}
resource "aws_dynamodb_table" "terraform_locks" {
name = "acme-terraform-locks-prod"
billing_mode = "PAY_PER_REQUEST" # No capacity planning needed
hash_key = "LockID" # Required partition key name
attribute {
name = "LockID"
type = "S"
}
point_in_time_recovery {
enabled = true # Protects lock table itself from accidental deletion
}
}✗ Anti-pattern
Keeping terraform.tfstate in the Git repository or on local disk. Engineers share state via Slack or email. No locking means two developers can apply simultaneously, causing state corruption that takes hours to untangle with manual state surgery.
◈ Architecture Diagram
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ CI Pipeline A │ │ CI Pipeline B │ │ Developer CLI │
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘
│ │ │
└───────────────────────┼───────────────────────┘
↓
┌──────────────────────────┐
│ DynamoDB Lock Table │
│ acme-terraform-locks │
│ (Acquire before R/W) │
└─────────────┬────────────┘
↓
┌──────────────────────────┐
│ S3 Bucket (Versioned) │
│ + KMS Encryption │
│ + Access Logging │
│ payments-vpc/tfstate │
└──────────────────────────┘💬 Comments
Why It Matters
Terraform providers are released independently from Terraform core and can introduce breaking changes in any minor or major version. The AWS provider alone has had 47 breaking changes between versions 4.x and 5.x, including renamed attributes, changed default values for security groups, and altered lifecycle behaviors for RDS instances. Without pinned versions, running terraform init on a Monday might pull provider version 5.30.0 while the same command on Friday pulls 5.32.0 with a completely different security group default behavior. This creates non-deterministic infrastructure where the same configuration produces different results depending on when you run it. In production environments, this unpredictability is catastrophic: a routine terraform apply that was supposed to add a tag can silently modify security group rules because the provider changed its default handling. Version pinning ensures that every team member and every CI pipeline uses the exact same provider behavior, making infrastructure changes auditable, reproducible, and predictable across all environments.
How to Implement
Use the required_providers block with exact version constraints using the equality operator for production configurations. Generate and commit the .terraform.lock.hcl file which contains cryptographic hashes of the exact provider binaries, ensuring that even if a provider is re-published with the same version number, Terraform will detect the tampering. Run terraform providers lock -platform=linux_amd64 -platform=darwin_arm64 to pre-generate hashes for all platforms your team uses. Create a dedicated provider upgrade pull request monthly that updates versions in a controlled manner with a full plan review. Never use the >= operator alone as it allows unbounded upgrades. Use the ~> pessimistic constraint only in module libraries where you want to allow patch updates, never in root modules that deploy to production. Document the upgrade process in your team's runbook including how to review the provider changelog for breaking changes before bumping.
Example
# versions.tf — Strict provider pinning for production
terraform {
required_version = "= 1.7.4" # Pin Terraform core version exactly
required_providers {
aws = {
source = "hashicorp/aws"
version = "= 5.31.0" # Exact pin — no automatic upgrades
}
kubernetes = {
source = "hashicorp/kubernetes"
version = "= 2.25.2" # Pin after testing compatibility with EKS 1.28
}
helm = {
source = "hashicorp/helm"
version = "= 2.12.1" # Tested with ArgoCD 2.9 chart
}
}
}
# .terraform.lock.hcl (committed to Git — never in .gitignore)
provider "registry.terraform.io/hashicorp/aws" {
version = "5.31.0"
constraints = "= 5.31.0"
hashes = [
"h1:abc123def456...", # Hash ensures binary integrity
"zh:9a8b7c6d5e4f...", # Cross-platform verification
]
}✗ Anti-pattern
Using version = ">= 4.0" or omitting version constraints entirely. The lock file is added to .gitignore because it causes merge conflicts. Different CI runners get different provider versions depending on cache state, causing mysterious plan differences between environments.
◈ Architecture Diagram
┌──────────────────────────────────────────────────────┐ │ Provider Version Timeline │ ├──────────────────────────────────────────────────────┤ │ │ │ v5.30.0 ──→ v5.31.0 ──→ v5.32.0 ──→ v5.33.0 │ │ │ │ │ │ │ │ │ ✓ PINNED BREAKING NEW │ │ │ (our prod) SG defaults feature │ │ │ │ │ │ Without pin: ─────┼──── CI pulls v5.32.0 ──→ OUTAGE│ │ With pin: ─────┼──── CI always gets 5.31.0 ✓ │ └──────────────────────────────────────────────────────┘
💬 Comments
Why It Matters
Without input validation, Terraform will happily accept any string for a CIDR block, any number for an instance count, or any value for an environment name — only to fail 8 minutes into a 15-minute apply when AWS rejects the malformed input. By that point, half your resources may already be provisioned in an inconsistent state that requires manual cleanup. Variable validation blocks shift error detection to the earliest possible moment: before Terraform even generates a plan. This is especially critical in self-service platforms where application teams provide variable values through Terraform Cloud workspaces or CI pipeline parameters without deep infrastructure knowledge. A developer might accidentally set instance_count to 500 instead of 5, or paste a subnet CIDR that overlaps with an existing VPC. Catching these errors before plan prevents costly mistakes, reduces CI pipeline time wasted on doomed applies, and provides clear human-readable error messages that tell the operator exactly what went wrong and how to fix it rather than cryptic AWS API errors buried in 200 lines of apply output.
How to Implement
Add validation blocks inside every variable that accepts user input in shared modules. Use the can() function to test expressions without causing errors — it returns true if the expression evaluates successfully and false otherwise. Combine can() with regex patterns for string formats, range checks for numbers, and contains() for enum-like constraints. Write error messages that include the invalid value received, the expected format, and an example of a valid value. Layer multiple validation blocks on a single variable to check different aspects independently, giving specific error messages for each failure mode. For CIDR blocks, validate both the format and the prefix length to prevent overly broad network allocations. For environment names, restrict to known values to prevent typos from creating phantom infrastructure. Test validation logic by deliberately passing invalid values in a terraform plan and confirming the error messages are actionable.
Example
# variables.tf — Validated inputs for payments-vpc module
variable "vpc_cidr" {
type = string
description = "CIDR block for the payments VPC. Must be /16 to /20."
validation {
condition = can(cidrhost(var.vpc_cidr, 0)) # Validates CIDR format
error_message = "vpc_cidr must be a valid CIDR block (e.g., 10.100.0.0/16)."
}
validation {
condition = tonumber(split("/", var.vpc_cidr)[1]) >= 16 && tonumber(split("/", var.vpc_cidr)[1]) <= 20
error_message = "vpc_cidr prefix must be between /16 and /20. Got /${split("/", var.vpc_cidr)[1]}. Use /16 for large environments or /20 for small ones."
}
}
variable "environment" {
type = string
description = "Target environment for deployment."
validation {
condition = contains(["dev", "staging", "production"], var.environment)
error_message = "environment must be one of: dev, staging, production. Got '${var.environment}'."
}
}
variable "eks_node_count" {
type = number
description = "Number of EKS worker nodes per availability zone."
validation {
condition = var.eks_node_count >= 2 && var.eks_node_count <= 50
error_message = "eks_node_count must be between 2 and 50. Got ${var.eks_node_count}. For larger clusters, contact the platform team."
}
}✗ Anti-pattern
No validation on any variables, relying entirely on AWS API errors during apply. Error messages are generic like 'invalid value' without showing what was received or what is expected. Input typos like 'prodution' create entire parallel environments before anyone notices.
◈ Architecture Diagram
┌─────────────────────────────────────────────────────────┐ │ Input Validation Pipeline │ ├─────────────────────────────────────────────────────────┤ │ │ │ User Input ──→ Validation Block ──→ Plan ──→ Apply │ │ │ │ │ ┌────┴────┐ │ │ │ FAIL │ PASS │ │ ↓ ↓ │ │ ┌────────────────┐ ┌────────────────┐ │ │ │ Clear error: │ │ Proceed with │ │ │ │ 'CIDR prefix │ │ terraform plan │ │ │ │ must be /16 │ │ (valid input) │ │ │ │ to /20' │ │ │ │ │ └────────────────┘ └────────────────┘ │ │ │ │ Time to catch error: < 1 second (vs 8+ minutes) │ └─────────────────────────────────────────────────────────┘
💬 Comments
Why It Matters
A monolithic state file that contains your VPC, EKS cluster, RDS databases, application deployments, and DNS records creates a catastrophic blast radius where a single misconfigured variable or accidental terraform destroy can wipe out your entire infrastructure in one operation. With 800+ resources in a single state, terraform plan takes 12-15 minutes as it refreshes every resource, making iterative development painfully slow and tying up DynamoDB locks for extended periods that block other teams. When state is split by blast radius, a bad apply to the application layer cannot possibly affect the VPC or database layer because those resources simply do not exist in the same state file. This isolation also enables independent team ownership: the network team manages VPC state, the platform team owns EKS state, and application teams manage their own service deployments without needing broad infrastructure permissions. State splitting reduces lock contention in CI pipelines by 80% because teams are no longer queuing behind each other for the same monolithic state lock.
How to Implement
Organize your repository into separate root modules aligned with infrastructure layers and change frequency. The network layer (VPC, subnets, NAT gateways, transit gateway attachments) changes rarely and forms the foundation — isolate it in its own state. The compute layer (EKS cluster, node groups, cluster add-ons) changes monthly and depends on networking — give it a separate state that reads network outputs via terraform_remote_state or data sources. Application deployments change daily and should each have their own state per service. Use terraform_remote_state data sources or AWS SSM Parameter Store to pass outputs between layers. Each layer gets its own S3 key path, its own CI pipeline, and its own IAM permissions scoped to only the AWS services it manages. Set lifecycle prevent_destroy on resources in lower layers to prevent accidental destruction even if someone runs destroy on the wrong state.
Example
# Infrastructure repository layout — split by blast radius
# infrastructure/
# ├── 01-network/ ← Changes quarterly, blast radius: entire platform
# │ ├── main.tf (VPC, subnets, NAT, TGW)
# │ ├── backend.tf (key: network/terraform.tfstate)
# │ └── outputs.tf (vpc_id, subnet_ids, route_table_ids)
# ├── 02-compute/ ← Changes monthly, blast radius: all workloads
# │ ├── main.tf (EKS cluster, node groups, IRSA roles)
# │ ├── backend.tf (key: compute/terraform.tfstate)
# │ └── data.tf (reads network state outputs)
# └── 03-apps/
# ├── payments-api/ ← Changes daily, blast radius: one service
# │ ├── main.tf (ALB, target group, ECS service)
# │ └── backend.tf (key: apps/payments-api/terraform.tfstate)
# └── user-service/ ← Independent deployment cycle
# ├── main.tf
# └── backend.tf (key: apps/user-service/terraform.tfstate)
# 02-compute/data.tf — Reading outputs from network layer
data "terraform_remote_state" "network" {
backend = "s3"
config = {
bucket = "acme-terraform-state-prod-us-east-1"
key = "network/terraform.tfstate"
region = "us-east-1"
}
}
# Use network outputs safely
resource "aws_eks_cluster" "prod" {
name = "prod-eks-us-east-1"
role_arn = aws_iam_role.eks_cluster.arn
vpc_config {
subnet_ids = data.terraform_remote_state.network.outputs.private_subnet_ids
}
}✗ Anti-pattern
One giant main.tf with 1200 lines containing VPC, EKS, RDS, S3, CloudFront, and application resources all in a single state. Plan takes 14 minutes. A typo in an app deployment variable triggers a VPC replacement that takes down the entire platform for 2 hours.
◈ Architecture Diagram
┌─────────────────────────────────────────────────────┐ │ State Isolation by Blast Radius │ ├─────────────────────────────────────────────────────┤ │ │ │ Layer 1: Network (changes quarterly) │ │ ┌───────────────────────────────────────┐ │ │ │ VPC │ Subnets │ NAT │ TGW │ Routes │ │ │ │ State: network/terraform.tfstate │ │ │ │ Team: Network Engineering │ │ │ └───────────────────┬───────────────────┘ │ │ ↓ outputs: vpc_id, subnet_ids │ │ Layer 2: Compute (changes monthly) │ │ ┌───────────────────────────────────────┐ │ │ │ EKS │ Node Groups │ IRSA │ Add-ons │ │ │ │ State: compute/terraform.tfstate │ │ │ │ Team: Platform Engineering │ │ │ └───────────────────┬───────────────────┘ │ │ ↓ outputs: cluster_endpoint │ │ Layer 3: Apps (changes daily) │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │payments │ │user-svc │ │orders │ │ │ │own state │ │own state │ │own state │ │ │ └──────────┘ └──────────┘ └──────────┘ │ └─────────────────────────────────────────────────────┘
💬 Comments
Why It Matters
Renaming a Terraform resource address — whether refactoring a module structure, changing a resource name for clarity, or migrating from count to for_each — traditionally causes Terraform to plan a destroy-then-create operation. For stateless resources like security group rules this is merely disruptive, but for stateful resources like RDS instances, EBS volumes, or S3 buckets with data, a destroy-and-recreate means data loss and downtime. Before Terraform 1.1 introduced moved blocks, teams had to use terraform state mv commands as manual operational steps that were error-prone, could not be code-reviewed, and left no audit trail in the configuration itself. The moved block solves this by declaratively recording the resource address change in your configuration, allowing Terraform to plan the rename as a no-op state update rather than a destructive operation. This makes refactoring safe, reviewable, and automated — the moved block appears in the pull request diff so reviewers can verify the rename is intentional. After one successful apply the moved block can be removed, but many teams keep them as documentation of the resource's history.
How to Implement
Add a moved block at the root module level specifying the old resource address in the from argument and the new resource address in the to argument. Run terraform plan to verify the plan shows 'moved' rather than 'destroy/create'. For module refactors, include the full module path in both from and to addresses. When migrating from count to for_each, create moved blocks for each index mapping (e.g., from aws_instance.app[0] to aws_instance.app["web-1"]). For moving resources into a child module, use the format module.name.resource_type.resource_name in the to argument. Stack multiple moved blocks in a single apply when performing large refactors. Always run plan in CI and verify zero destructive operations before approving. Keep moved blocks for at least two apply cycles to handle any environments that might be behind, then remove them in a cleanup PR.
Example
# Refactoring: Renamed resource for clarity and moved into module
# Old: aws_security_group.main
# New: module.networking.aws_security_group.payments_ingress
moved {
from = aws_security_group.main
to = module.networking.aws_security_group.payments_ingress
}
# Migrating from count to for_each (preserves all 3 instances)
moved {
from = aws_instance.app[0]
to = aws_instance.app["payments-web-1"]
}
moved {
from = aws_instance.app[1]
to = aws_instance.app["payments-web-2"]
}
moved {
from = aws_instance.app[2]
to = aws_instance.app["payments-worker-1"]
}
# Plan output confirms safe move:
# Terraform will perform the following actions:
#
# # aws_security_group.main has moved to
# # module.networking.aws_security_group.payments_ingress
# resource "aws_security_group" "payments_ingress" {
# id = "sg-0a1b2c3d4e5f67890"
# name = "payments-ingress-prod"
# # (no changes to attributes)
# }
#
# Plan: 0 to add, 0 to change, 0 to destroy.✗ Anti-pattern
Renaming resources without moved blocks, causing Terraform to destroy production EC2 instances and recreate them with new IDs. Using terraform state mv in a CI pipeline script that is not version-controlled or reviewable. Skipping plan review because 'it is just a rename' and approving the destroy/create blindly.
◈ Architecture Diagram
┌──────────────────────────────────────────────────┐ │ Resource Rename: With vs Without moved │ ├──────────────────────────────────────────────────┤ │ │ │ WITHOUT moved block: │ │ ┌──────────────┐ ┌──────────────┐ │ │ │ aws_instance │ DESTROY │ aws_instance │ │ │ │ .app[0] │───────→ │ .app["web"] │ │ │ │ (running) │ then │ (new, empty) │ │ │ └──────────────┘ CREATE └──────────────┘ │ │ ↓ DOWNTIME + DATA LOSS │ │ │ │ WITH moved block: │ │ ┌──────────────┐ ┌──────────────┐ │ │ │ aws_instance │ STATE │ aws_instance │ │ │ │ .app[0] │──MOVE──→│ .app["web"] │ │ │ │ (running) │ only │ (same VM!) │ │ │ └──────────────┘ └──────────────┘ │ │ ↓ ZERO DOWNTIME, ZERO DATA LOSS │ └──────────────────────────────────────────────────┘
💬 Comments
Why It Matters
Automated terraform apply without human review is the infrastructure equivalent of deploying code directly to production without code review. Even experienced engineers produce plans that accidentally replace load balancers, modify security groups, or trigger RDS instance replacements due to parameter changes they did not realize were destructive. A terraform plan output is the last line of defense before irreversible infrastructure changes — but it is only useful if a human actually reads it and confirms the changes match intent. In a 2025 survey of infrastructure incidents, 34% of production Terraform outages were caused by engineers approving plans they did not fully read because the workflow made it too easy to auto-approve. The plan-then-approve pattern creates a mandatory pause that forces engineers to verify: Are any resources being destroyed? Are replacements expected? Does the number of changes match what I intended? This gate is especially critical for production where a single unexpected replacement of an ALB or NAT gateway can cause customer-facing downtime measured in minutes to hours.
How to Implement
Structure your CI pipeline into two distinct stages: plan and apply. The plan stage runs on every pull request push, saves the plan to a binary file with terraform plan -out=tfplan, and posts a formatted summary to the pull request as a comment showing resources to add, change, and destroy. Use a tool like tfcmt or atlantis to render plan output as collapsible Markdown in the PR. The apply stage is triggered only after PR approval and merge, using the saved plan file to ensure what was reviewed is exactly what gets applied. For production workspaces specifically, add an additional manual approval gate — either a GitHub Environment protection rule, a Slack approval workflow, or a PagerDuty change event that requires on-call acknowledgment. Never allow terraform apply to run on production without at least two engineers reviewing the plan output. Store plan files as CI artifacts with a 24-hour TTL and include a hash comparison to detect drift between plan and apply time.
Example
# .github/workflows/terraform-production.yml
name: Terraform Production Deploy
on:
push:
branches: [main]
paths: ['infrastructure/02-compute/**']
jobs:
plan:
runs-on: ubuntu-latest
environment: production-plan # No approval needed for plan
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.7.4
- run: terraform init -backend-config=envs/production.hcl
- run: terraform plan -out=tfplan -detailed-exitcode
id: plan
- run: terraform show -json tfplan > tfplan.json
- uses: actions/upload-artifact@v4
with:
name: tfplan-${{ github.sha }}
path: tfplan
retention-days: 1
# Post plan summary to PR
- run: |
echo "## Production Plan Summary" >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY
terraform show tfplan | grep -E '(Plan:|#)' >> $GITHUB_STEP_SUMMARY
echo '```' >> $GITHUB_STEP_SUMMARY
apply:
needs: plan
runs-on: ubuntu-latest
environment: production # Requires 2 reviewer approvals + on-call ack
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
- uses: actions/download-artifact@v4
with:
name: tfplan-${{ github.sha }}
- run: terraform init -backend-config=envs/production.hcl
- run: terraform apply tfplan # Applies exact reviewed plan✗ Anti-pattern
CI pipeline runs terraform apply -auto-approve on merge to main. No one reviews the plan output because it scrolls past in 500 lines of CI logs. Production applies happen at 3 AM via automated merge queues with no human oversight. The plan and apply use different variable files due to a CI misconfiguration.
◈ Architecture Diagram
┌─────────────────────────────────────────────────────────┐ │ Plan → Review → Approve → Apply Pipeline │ ├─────────────────────────────────────────────────────────┤ │ │ │ PR Created ──→ terraform plan ──→ Post to PR │ │ │ │ │ ↓ │ │ ┌─────────────────┐ │ │ │ Plan Summary: │ │ │ │ + 2 to add │ │ │ │ ~ 1 to change │ │ │ │ - 0 to destroy │ │ │ └────────┬────────┘ │ │ ↓ │ │ ┌─────────────────┐ │ │ │ 2 Reviewers │◄── MANDATORY GATE │ │ │ + On-call Ack │ │ │ └────────┬────────┘ │ │ ↓ │ │ ┌─────────────────┐ │ │ │ terraform apply │ │ │ │ (saved plan) │ │ │ └─────────────────┘ │ └─────────────────────────────────────────────────────────┘
💬 Comments
Why It Matters
Hardcoded ARNs, account IDs, VPC IDs, and subnet IDs scattered throughout Terraform configurations create brittle, environment-specific code that breaks the moment you deploy to a new region, a new AWS account, or when AWS regenerates resource IDs during maintenance operations. A hardcoded string like subnet-0a1b2c3d4e is meaningless to reviewers — they cannot verify if it is the correct subnet without logging into the console and checking. Hardcoded values also prevent code reuse across environments because every deployment requires finding and replacing dozens of IDs. Data sources solve this by querying AWS at plan time to resolve the current, correct values based on meaningful filters like tags or names. This makes configurations self-documenting (the filter criteria explain what you are looking for), environment-portable (the same code works in dev and prod because it queries dynamically), and drift-resistant (if a resource is recreated with a new ID, the data source picks up the new value automatically). In multi-team organizations, data sources create clean contracts between teams: the network team tags their subnets, and application teams reference them by tag rather than by opaque ID.
How to Implement
Replace every hardcoded ARN, ID, or account number with an appropriate data source. Use aws_vpc with a Name tag filter instead of hardcoding vpc-0abc123. Use aws_subnets with filter blocks to find subnets by purpose tag and availability zone. Use aws_caller_identity to get the current account ID dynamically. Use aws_region for the current region. For cross-account references, use terraform_remote_state or aws_ssm_parameter data sources. Tag all resources with consistent, queryable tags (Name, Environment, Team, Purpose) so that data sources have reliable filters. Add lifecycle postconditions or validation on data source results to fail fast if the expected resource is not found rather than proceeding with null values. When using aws_ami, always sort by creation_date and pick the most recent to avoid stale image references. Document the expected tags in your module README so upstream teams know which tags your data sources depend on.
Example
# data.tf — Dynamic lookups instead of hardcoded values
# Instead of: vpc_id = "vpc-0a1b2c3d4e5f67890"
data "aws_vpc" "payments" {
filter {
name = "tag:Name"
values = ["payments-vpc-prod"] # Query by meaningful tag
}
filter {
name = "tag:Environment"
values = ["production"]
}
}
# Instead of: subnet_ids = ["subnet-aaa", "subnet-bbb", "subnet-ccc"]
data "aws_subnets" "private_app" {
filter {
name = "vpc-id"
values = [data.aws_vpc.payments.id] # Chain data sources
}
filter {
name = "tag:Tier"
values = ["private-app"] # Find subnets by purpose
}
}
# Instead of: account_id = "111222333444"
data "aws_caller_identity" "current" {}
# Instead of: ami = "ami-0abc123def456789"
data "aws_ami" "app_golden" {
most_recent = true
owners = [data.aws_caller_identity.current.account_id]
filter {
name = "name"
values = ["payments-app-golden-*"] # Glob pattern for AMI naming convention
}
filter {
name = "state"
values = ["available"]
}
}
# Usage — portable across accounts and regions
resource "aws_instance" "app" {
ami = data.aws_ami.app_golden.id
subnet_id = data.aws_subnets.private_app.ids[0]
instance_type = "m6i.xlarge"
}✗ Anti-pattern
Configurations full of hardcoded strings like arn:aws:iam::111222333444:role/my-role and subnet-0abc123. Deploying to a new account requires a find-and-replace across 40 files. An engineer copies a staging subnet ID into production config, routing prod traffic through staging network. AMI IDs go stale because nobody updates the hardcoded value after a new golden image is baked.
◈ Architecture Diagram
┌──────────────────────────────────────────────────────┐ │ Hardcoded IDs vs Data Sources │ ├──────────────────────────────────────────────────────┤ │ │ │ HARDCODED (fragile): │ │ vpc_id = "vpc-0a1b2c3d" ──→ Breaks in new account │ │ ami = "ami-old123" ──→ Stale after rebuild │ │ subnet = "subnet-xyz" ──→ Wrong env on copy │ │ │ │ DATA SOURCES (resilient): │ │ ┌─────────────┐ filter: ┌──────────────┐ │ │ │ aws_vpc │───Tag:Name────→│ vpc-0a1b2c3d │ │ │ └─────────────┘ └──────────────┘ │ │ ┌─────────────┐ filter: ┌──────────────┐ │ │ │ aws_ami │───most_recent─→│ ami-latest │ │ │ └─────────────┘ └──────────────┘ │ │ ┌─────────────┐ filter: ┌──────────────┐ │ │ │ aws_subnets │───tag:Tier────→│ [3 subnets] │ │ │ └─────────────┘ └──────────────┘ │ │ │ │ Same code works in dev, staging, and production! │ └──────────────────────────────────────────────────────┘
💬 Comments
Why It Matters
Without consistent tagging, your AWS bill is a black box where $47,000 per month disappears into an undifferentiated mass of EC2 instances, NAT gateway charges, and data transfer fees that no one can attribute to specific teams, services, or environments. When the CFO asks why infrastructure costs increased 40% this quarter, the answer should take minutes to find through Cost Explorer tag-based filtering — not weeks of forensic investigation. Tags are also critical for security (identifying untagged rogue resources), compliance (proving which resources belong to regulated workloads), and operations (knowing which team to page when a resource is unhealthy). AWS Cost Allocation Tags only work if applied consistently to every resource at creation time — retroactively tagging hundreds of resources after a cost audit is painful and error-prone. Terraform's default_tags feature at the provider level ensures that every resource automatically receives baseline tags without engineers remembering to add them manually to each resource block, eliminating the human error that creates cost attribution gaps.
How to Implement
Configure default_tags in the AWS provider block to automatically apply baseline tags to every resource created by that provider. Include at minimum: Environment (dev/staging/production), Team (owning team's identifier matching your org chart), ManagedBy (terraform to distinguish from manual or CloudFormation resources), Service (the business service this infrastructure supports), and CostCenter (the finance department's allocation code). Supplement default_tags with resource-specific tags like Name using a consistent naming convention. Create an AWS Organization SCP (Service Control Policy) that denies resource creation if required tags are missing. Set up a weekly AWS Config rule that reports untagged resources. In Terraform, use a local value to compute common tags and merge them with resource-specific tags. Enable the required tags in AWS Cost Explorer as Cost Allocation Tags so they appear in billing reports and can be used for budget alerts per team.
Example
# provider.tf — Default tags applied to every AWS resource
provider "aws" {
region = "us-east-1"
default_tags {
tags = {
Environment = var.environment # dev | staging | production
Team = "payments-platform" # Matches PagerDuty team name
ManagedBy = "terraform" # Distinguishes from manual resources
Service = "payments-api" # Business service identifier
CostCenter = "CC-4521" # Finance department allocation code
Repository = "github.com/acme/infra" # Source of truth for this resource
DeployedBy = "github-actions" # CI system that deployed it
}
}
}
# Resource-specific tags supplement defaults
resource "aws_instance" "payments_worker" {
ami = data.aws_ami.app_golden.id
instance_type = "c6i.2xlarge"
subnet_id = data.aws_subnets.private_app.ids[0]
tags = {
Name = "payments-worker-prod-1a" # Unique resource identifier
Role = "async-payment-processor" # Specific function of this instance
Schedule = "always-on" # For start/stop automation
}
}
# AWS Config rule to enforce tagging compliance
resource "aws_config_config_rule" "required_tags" {
name = "required-tags-compliance"
source {
owner = "AWS"
source_identifier = "REQUIRED_TAGS"
}
input_parameters = jsonencode({
tag1Key = "Environment"
tag2Key = "Team"
tag3Key = "ManagedBy"
})
}✗ Anti-pattern
No default_tags configured. Engineers manually add tags to some resources but forget others. 60% of resources are untagged making cost allocation impossible. The monthly bill shows $15,000 in 'unallocated' costs that no team takes ownership of. Cost overruns are discovered 30 days late because budget alerts cannot filter by team.
◈ Architecture Diagram
┌───────────────────────────────────────────────────────┐ │ Tagging Strategy Flow │ ├───────────────────────────────────────────────────────┤ │ │ │ Provider default_tags (automatic on every resource) │ │ ┌─────────────────────────────────────────────────┐ │ │ │ Environment=production │ Team=payments-platform │ │ │ │ ManagedBy=terraform │ Service=payments-api │ │ │ │ CostCenter=CC-4521 │ Repository=acme/infra │ │ │ └─────────────────────────────────────────────────┘ │ │ + │ │ Resource-specific tags (per resource block) │ │ ┌─────────────────────────────────────────────────┐ │ │ │ Name=payments-worker-prod-1a │ Role=processor │ │ │ └─────────────────────────────────────────────────┘ │ │ = │ │ AWS Cost Explorer ──→ Filter by Team ──→ $12,450/mo │ │ Budget Alert ──→ Team > $15k ──→ PagerDuty │ │ Config Rule ──→ Untagged? ──→ Non-compliant │ └───────────────────────────────────────────────────────┘
💬 Comments
Why It Matters
The failure most teams hit starts small and reasonable: someone provisions 5 microservices with count = 5 and a list variable of image names, and it works fine for months. Then the platform grows to 50 services, and one day a team wants to decommission a single deprecated service sitting in the middle of that list. They remove its entry and run terraform plan expecting one resource to be destroyed. Instead, the plan shows 38 resources being destroyed and recreated — every ECS service, task definition, and target group defined after the removed entry's original position, because count addresses resources purely by numeric index and every service after the gap shifted down by one. If that plan gets approved without careful review (and at 3am during an incident-driven decommission, it often does), the team just took down 38 healthy, unrelated production services to remove one deprecated one. This is not a hypothetical edge case; it is the single most common Terraform production incident tied to the count argument, and it gets worse — not better — as the list grows, because more services means more collateral damage per removal.
How to Implement
Replace the list-and-count pattern with a map keyed by a stable, meaningful identifier — almost always the service name — and switch from count to for_each. Concretely: define variable "microservices" as a map(object({...})) rather than a list(object({...})), then reference module "microservice" { for_each = var.microservices ... } so each module instance is addressed as module.microservice["payments-api"] instead of module.microservice[2]. Wrap the actual resource definitions (ECS service, task definition, target group, autoscaling policy) inside that reusable module so 50 services share one implementation and differ only by the map values passed in — typically sourced from a checked-in YAML file decoded with yamldecode(file("services.yaml")), so each service team can add their own entry via a small, reviewable diff instead of touching shared Terraform code. Verify the change is safe with terraform plan before merging — a for_each-based module addition should always show only additive create operations for a genuinely new service, and only a single targeted destroy for a genuinely removed one, never a wide blast radius across unrelated services. If you are migrating an already-existing count-based module to for_each, do this in a separate, careful step: use terraform state mv to remap every existing numeric address (module.microservice[0]) to its new string-keyed address (module.microservice["payments-api"]) before changing the code, so existing resources are recognized as unchanged rather than destroyed and recreated purely due to the addressing scheme changing underneath them.
Example
# services.yaml — one entry per microservice, easy to diff and review
# payments-api:
# image: 123456789012.dkr.ecr.us-east-1.amazonaws.com/payments-api:2.4
# cpu: 512
# memory: 1024
# desired_count: 3
locals {
microservices = yamldecode(file("${path.module}/services.yaml"))
}
module "microservice" {
for_each = local.microservices # keyed by service name, not position
source = "./modules/ecs-microservice"
name = each.key # stable identity across applies
image = each.value.image
cpu = each.value.cpu
memory = each.value.memory
desired_count = each.value.desired_count
cluster_arn = aws_ecs_cluster.platform.arn
}
# Migrating an existing count-based deployment: remap state BEFORE changing code
# terraform state mv 'module.microservice[0]' 'module.microservice["payments-api"]'
# terraform state mv 'module.microservice[1]' 'module.microservice["checkout-worker"]'✗ Anti-pattern
The common starting point is variable "services" { type = list(string) } paired with resource "aws_ecs_service" "svc" { count = length(var.services) name = var.services[count.index] ... }. It looks clean and reads fine in a PR at 5 services. The trap is that it works flawlessly for every addition — appending to the end of a list never disturbs earlier indices — so the team gets months of positive reinforcement that count is fine. The problem only surfaces on removal or reordering, which tends to happen later, under different engineers who didn't write the original code and have no reason to suspect that deleting one line of a list variable can trigger a 38-resource destroy plan. By the time it bites, the pattern is deeply embedded across dozens of modules, making the eventual for_each migration far more disruptive than if it had been done correctly from the start.
◈ Architecture Diagram
✗ count + list ✓ for_each + map
resource[0] payments-api resource["payments-api"]
resource[1] checkout-worker resource["checkout-worker"]
resource[2] notifications resource["notifications"]
│ remove [1] │ remove "checkout-worker"
▼ ▼
resource[0] payments-api (ok) resource["payments-api"] (ok)
resource[1] notifications (REBUILT) resource["notifications"] (ok)💬 Comments