You run terraform apply inside a Jenkins pipeline, but the build gets abruptly aborted halfway through execution. Now, the next pipeline run keeps failing with a 'State Locked' error. How do you resolve this safely?
Quick Answer
The lock is a DynamoDB (or equivalent backend) entry that Terraform writes before it starts mutating state, and it's meant to survive process death so two applies never corrupt state simultaneously — Jenkins killing the build mid-apply left that lock entry behind uncleared. You resolve it safely by first confirming no other apply is actually still running, then releasing the lock with the recorded Lock ID using terraform force-unlock, followed by a plan (not apply) to check whether the interrupted run left resources half-created.
Detailed Answer
Picture a public restroom with a single occupancy sign that someone flips to 'occupied' before going in and flips back to 'vacant' on the way out. If that person is dragged out through a side exit during a fire drill, the sign stays on 'occupied' forever, even though nobody is inside. Everyone else lines up outside, unable to get in, even though the room is actually empty.
Terraform's state locking exists specifically to prevent two people from running apply against the same state file at once, which could otherwise interleave writes and corrupt the state beyond repair. When you use a remote backend like S3 with a DynamoDB lock table, Terraform writes a lock record (containing a Lock ID, the operation, who's holding it, and a timestamp) to DynamoDB before touching the S3 state object, and deletes that record when the operation finishes normally.
When Jenkins aborts a build — whether from a timeout, a manual cancel, or the agent pod being evicted — the terraform apply process is killed with a signal it cannot trap and clean up after. It has no chance to run its deferred unlock logic, so the DynamoDB lock item is orphaned. Every subsequent plan or apply from any pipeline run then fails immediately with 'Error acquiring the state lock,' because Terraform is correctly doing its job: refusing to proceed while a lock exists, since it has no way to know whether that lock represents a still-running process or a dead one.
The safe resolution has three parts. First, verify there is genuinely no other apply in flight — check Jenkins for concurrent builds and check with your team, because force-unlocking while a real apply is running is exactly the corruption scenario locking was built to prevent. Second, run terraform force-unlock with the Lock ID shown in the error message, which deletes just that DynamoDB record. Third — and this is the step teams skip — run terraform plan afterward, not apply, because the aborted run may have created some resources via the AWS API before it died, and the state file may or may not reflect them depending on exactly when the SIGKILL landed. That plan output tells you if you have to import a stray resource or if everything is consistent.
At production scale, the actual fix isn't the manual unlock, it's pipeline hygiene: set a Jenkins build timeout longer than any expected apply, use terraform apply with -lock-timeout so Terraform retries briefly instead of failing instantly on transient contention, and consider wrapping applies in a step that traps SIGTERM to attempt a clean terraform state cleanup on cancellation. The gotcha experienced engineers hit: force-unlock does not roll back partially-applied infrastructure — it only removes the traffic light, it does not un-crash the cars that already collided.