What is Terraform state and why is remote state critical?
Quick Answer
Terraform state is a JSON file that maps your HCL configuration to real-world infrastructure resources. Remote state stores this file in a shared backend like S3 or Terraform Cloud, enabling team collaboration, state locking, and disaster recovery.
Detailed Answer
Terraform state is the backbone of how Terraform understands what infrastructure it manages. Think of it like a warehouse inventory ledger: without it, workers would walk into the warehouse every day not knowing what is already on the shelves, what was ordered, or what needs restocking. The state file (terraform.tfstate) is that ledger — it records every resource Terraform has created, its current attributes, metadata about dependencies, and the mapping between your HCL resource blocks and the actual cloud API objects.
Internally, the state file is a JSON document containing a version number, a serial counter that increments on every write, a lineage UUID that uniquely identifies a state chain, and an array of resource objects. Each resource entry stores the provider, the resource type, the resource name, the mode (managed or data), and the full set of attributes returned by the provider API after creation. When you run terraform plan, Terraform reads this state, calls the cloud APIs to refresh the actual status of each resource, and then computes the diff between desired (your HCL) and actual (the refreshed state). Without state, Terraform would have no way to know that aws_rds_instance.payments_db already exists and would try to create a duplicate every time.
Local state works fine for a solo developer experimenting, but it becomes dangerous in production for several reasons. First, if two engineers run terraform apply simultaneously against local state files, they can create conflicting resources or corrupt state entirely — there is no locking mechanism. Second, if your laptop dies or someone accidentally deletes the state file, you lose the mapping between code and infrastructure, making it extremely difficult to recover. Third, local state may contain sensitive outputs like database passwords or API keys stored in plaintext on a developer workstation.
Remote state solves all of these problems. When you configure a backend like S3 with DynamoDB locking, the state file lives in a durable, versioned object store. DynamoDB provides a distributed lock so that only one terraform apply can run at a time, preventing race conditions. S3 versioning gives you automatic backup of every state revision, so you can roll back if something goes wrong. Additionally, remote state enables the terraform_remote_state data source, which lets one Terraform project read outputs from another — for example, a networking project can export VPC IDs that an application project consumes.
In production, teams typically enforce remote state from day one using a backend configuration block, require encryption at rest and in transit, restrict access via IAM policies, and enable state locking. Ignoring remote state is one of the most common causes of Terraform disasters in growing organizations.