Think of a hospital records system. If every department writes to one giant patient file simultaneously, records get corrupted and the wrong medication gets administered. Splitting records by department, locking each file during edits, and storing everything in a central secure archive prevents these disasters. Terraform state management works the same way — it is the record of what infrastructure exists, and mismanaging it causes outages.
Terraform state is a JSON file that maps every resource in your configuration to a real cloud object. When terraform plan runs, it reads the state to determine what exists, compares it to the desired configuration, and calculates the diff. If two engineers run terraform apply simultaneously against the same state, one overwrites the other's changes, causing state corruption where Terraform's view of the world no longer matches reality. Remote backends solve storage and collaboration: S3 stores the state file durably, DynamoDB provides a lock table so only one operation can modify state at a time, and versioning on the S3 bucket enables recovery from bad applies.
Internally, when terraform apply starts, it sends a Lock request to the backend. For S3+DynamoDB, this writes a lock record to the DynamoDB table with a unique ID, the user's identity, and a timestamp. If another process already holds the lock, Terraform exits with an error. After the apply completes, Terraform writes the updated state to S3 and releases the lock. If a process crashes mid-apply, the lock remains until it expires or is manually force-unlocked with terraform force-unlock. Terraform Cloud handles locking internally and adds run queues so multiple plans can exist but only one apply executes at a time per workspace.
At production scale, the critical architectural decision is state splitting. A monolithic state file containing the VPC, databases, Kubernetes clusters, DNS records, and application services means a single terraform apply can accidentally destroy the database while updating a DNS record. The recommended pattern is splitting state by blast radius: network foundations in one state, data layer in another, compute in another, and application configurations in their own states. Each state has its own backend configuration and can use terraform_remote_state data sources or outputs stored in SSM Parameter Store to share values. Workspaces can further separate environments (dev, staging, production) within the same configuration, but they should not be used as a substitute for proper state splitting — all workspaces in a configuration share the same codebase, backend, and permissions.
The non-obvious gotcha is that terraform_remote_state creates a hard coupling between states, and if the upstream state is corrupted or the output names change, downstream plans break. Many mature teams replace terraform_remote_state with data sources that look up infrastructure by tags or names, or they store shared values in AWS SSM Parameter Store or HashiCorp Consul, which decouples state files completely. Another trap is that S3 bucket versioning does not protect against state file deletion — teams must also enable MFA Delete or use S3 Object Lock for regulatory environments.
TF Interview Questions