How do Terraform workspaces work and when should you use them vs separate directories?

Terraform workspaces are a built-in mechanism for managing multiple instances of the same infrastructure configuration. Think of workspaces like branches in a photo editing app — you have the same source image but can apply different filters and adjustments to each branch independently. Each workspace gets its own state file, so resources in the 'dev' workspace are completely isolated from resources in the 'prod' workspace, even though they share the same Terraform code.

Internally, workspaces work by modifying the state file path. When using the default local backend, Terraform stores state files in a terraform.tfstate.d/ directory with subdirectories for each workspace. With an S3 backend, workspace state files are stored under the key prefix with the workspace name appended — for example, payments-platform/dev/terraform.tfstate and payments-platform/prod/terraform.tfstate. The terraform.workspace variable is available in your HCL code, letting you conditionally set values based on the active workspace.

Workspaces shine when your environments are structurally identical but differ in scale or configuration. If dev, staging, and production all need the same VPC, RDS cluster, ECS service, and ALB, but dev uses smaller instance types and fewer replicas, workspaces with conditional expressions or workspace-specific tfvars files work beautifully. You can use terraform.workspace in locals to set instance sizes, replica counts, and CIDR ranges per environment.

However, workspaces have significant limitations that push many teams toward separate directories. First, all environments share the same provider configuration — you cannot easily use different AWS accounts per workspace without complex provider alias tricks. Second, if your production environment has additional resources that dev does not need (WAF rules, CloudFront distributions, compliance monitoring), you end up with count = terraform.workspace == "prod" ? 1 : 0 scattered throughout your code, which becomes unreadable. Third, workspaces provide no protection against accidentally running apply in the wrong workspace. A sleep-deprived engineer who forgets to run terraform workspace select prod before applying a production hotfix could accidentally modify the dev environment.

Separate directories (often called the directory-per-environment pattern) give you complete isolation. Each environment has its own directory with its own backend configuration, provider configuration, and state. This means prod can use a different AWS account, different provider version constraints, and completely different resource compositions. The tradeoff is code duplication — you need to keep common modules in sync across directories.

The modern consensus in the Terraform community is to use workspaces for lightweight environment differentiation within a single account and team, and separate directories (or separate Terraform Cloud workspaces with VCS integration) for production-grade multi-account setups. Many teams use a hybrid approach: modules contain the shared logic, and each environment directory calls those modules with environment-specific variables.