You need to recover from a corrupted Terraform state file where the state shows resources that no longer exist in AWS, and AWS has resources not tracked in state. Walk through the state surgery process.
Quick Answer
Use 'terraform state list' to inventory, 'terraform state rm' for phantom resources, 'terraform import' for untracked resources, and 'terraform state pull/push' for manual JSON editing. Always backup state before any operation.
Detailed Answer
State Corruption Scenarios
1. Phantom resources: State references resources deleted outside Terraform (manual AWS console deletion, another tool) 2. Orphaned resources: AWS resources exist but aren't in Terraform state (failed import, partial apply crash) 3. Duplicate entries: Same resource appears twice in state (rare, usually from concurrent applies) 4. Wrong resource attributes: State has stale attributes that don't match reality
Recovery Procedure
Step 1: Backup Before any state surgery, create a backup. With remote backends, download the state file. With S3 backend, versioning provides automatic backups.
Step 2: Inventory Run terraform state list to see all tracked resources. Run terraform plan to see what Terraform thinks needs to change. Resources showing 'destroy' that shouldn't be destroyed are phantom entries. Resources not in the plan but existing in AWS are orphans.
Step 3: Remove phantom resources Use terraform state rm <resource_address> for each phantom resource. This only removes from state, doesn't touch real infrastructure.
Step 4: Import orphaned resources Use terraform import <resource_address> <cloud_resource_id> to bring untracked resources into state. Since Terraform 1.5+, use import blocks in HCL for repeatable imports.
Step 5: Reconcile attributes Run terraform refresh (or terraform apply -refresh-only) to update state attributes from the real cloud state. Review the changes before approving.
Step 6: Validate Run terraform plan and verify it shows minimal or no changes. Any remaining diff should be reviewed carefully.
Code Example
# Step 1: Backup state
terraform state pull > state-backup-$(date +%Y%m%d-%H%M%S).json
# Step 2: List all resources in state
terraform state list
# Step 3: Remove phantom resources (deleted outside TF)
terraform state rm 'module.vpc.aws_subnet.private[2]'
terraform state rm 'aws_instance.old_bastion'
# Step 4: Import orphaned resources
# Method 1: CLI import
terraform import aws_instance.web i-0123456789abcdef0
terraform import 'module.rds.aws_db_instance.main' mydb-instance-id
# Method 2: Import blocks (Terraform 1.5+, repeatable)
import {
to = aws_instance.web
id = "i-0123456789abcdef0"
}
import {
to = aws_s3_bucket.data
id = "my-data-bucket-prod"
}
# Then run: terraform plan -generate-config-out=generated.tf
# Step 5: Refresh state to match reality
terraform apply -refresh-only
# Step 6: Advanced - manual state JSON editing
terraform state pull > state.json
# Edit state.json carefully (fix serial number, resource attributes)
terraform state push state.json
# Step 7: Move resources between state files (during refactoring)
terraform state mv -state-out=../new-project/terraform.tfstate \
'module.vpc.aws_vpc.main' 'aws_vpc.main'
# Verify clean state
terraform plan # Should show no changes
# S3 backend state recovery from versioning
aws s3api list-object-versions \
--bucket my-tf-state \
--prefix prod/terraform.tfstate \
--query 'Versions[0:5].{VersionId:VersionId,LastModified:LastModified,Size:Size}'Interview Tip
State surgery questions are extremely common in senior Terraform interviews. Always start with 'backup the state file' - this shows production awareness. Know the difference between state rm (remove from tracking) and destroy (delete actual resource). Mention import blocks as the modern approach over CLI import.