Terraform State Management: Commands, Drift and Locking

TLDR: Terraform records every object it manages in the state (terraform.tfstate by default). Read it with terraform state list, terraform state show and terraform show, never by editing the JSON. Rename resources with moved blocks, adopt existing objects with import blocks, stop managing objects with removed blocks, and accept changes made outside Terraform with terraform apply -refresh-only (the old terraform refresh is deprecated). A state lock stops two runs from writing at the same time; force-unlock is for locks left behind by a crashed run. State holds secrets in plain text, so protect it like a secret.

Most Terraform problems that keep people up at night are state problems: a resource Terraform "forgot", a rename that destroyed a database, a lock nobody can release. This part shows what the state contains and walks through every common state operation on a real project, with the output each command prints in Terraform 1.17.

Prerequisites

• Terraform installed (Part 2) and the resources from Part 4.
• No cloud account: the examples use the local and random providers.

What State Is and Why Terraform Needs It

Your configuration says what should exist; it does not say which real object is which. The state is the mapping between the two: for each resource address it stores the object's ID and the attributes the provider returned. Terraform uses it to:

• Map code to real objects. aws_instance.web in your code is i-0b5c... in AWS only because the state says so.
• Plan updates and deletions. Without a record, Terraform could not know that it created something you have since removed from the code.
• Order deletions. The state stores each object's dependencies, so Terraform can destroy objects in the right order even after you deleted their blocks.
• Plan faster. Large configurations can skip some API reads with -refresh=false because the last known values are in the state.

Create a small project to look at:

terraform {
  required_providers {
    local = {
      source  = "hashicorp/local"
      version = "~> 2.9"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.9"
    }
  }
}

resource "random_pet" "server" {
  length = 2
}

resource "random_password" "db" {
  length  = 20
  special = false
}

resource "local_file" "config" {
  filename = "${path.module}/app.conf"
  content  = "server=${random_pet.server.id}\n"
}

Inspect the State

Use Terraform's commands to read state. They work the same for a local file and for state in S3 or HCP Terraform:

• terraform state list prints every resource address. Add an address prefix to filter: terraform state list module.network or terraform state list 'local_file.by_key'.
• terraform state show ADDRESS prints one object. It is the quickest way to find an ID or attribute you need. Sensitive attributes, such as the generated password, show as (sensitive value).
• terraform show prints every object in the same format, followed by the outputs.

For scripts, use JSON. terraform show -json prints the whole state; since Terraform 1.16, terraform state show -json also works for one object:

Inside terraform.tfstate

The file itself is JSON. Its top level looks like this:

• version is the format of the file (4 since Terraform 0.12). terraform_version is the version that last wrote it.
• serial goes up by one on every write. lineage is a random ID set when the state was first created. Together they stop Terraform from overwriting a newer state with an older copy, or with the state of a different project.
• resources holds each object with its attributes and dependencies.

Now the part that surprises people:

terraform show hid the password, but the state file stores it in plain text. The same happens with database passwords, private keys and access tokens that resources return. sensitive = true only hides values in CLI output. So:

• Never commit state to Git, and keep it in a backend with encryption at rest and tight access control (Part 9).
• Prefer ephemeral resources and write-only arguments, which keep secrets out of state entirely (Part 10).
• Treat anyone who can read the state as someone who can read every secret in it.

Do not edit the file by hand. If you must repair state, pull it (terraform state pull > backup.tfstate), use the commands below, and push a fixed copy only as a last resort (terraform state push), because a mistake there corrupts the record of everything you manage.

Drift: Changes Made Outside Terraform

Drift is the difference between the state and the real object, usually because someone changed it in a console or another tool. Simulate it by editing the managed file by hand:

On every plan, Terraform first refreshes: it asks the provider to read each object again. The local provider reports a file whose content no longer matches as gone, so the plan writes it again with the content from the code. A cloud provider usually reports the changed attributes instead, and the plan shows an update (~) that sets them back. Either way, the code is the source of truth, and apply undoes the manual change. That is what you want most of the time.

Sometimes the manual change is correct (an emergency fix in the console, a size someone raised during an incident). Then you update the code to match and, if the provider cannot express the change as a plan, record the new reality in the state without touching any object:

apply -refresh-only lists everything that changed outside Terraform (here, the file counts as deleted) and updates only the state. It replaces the old terraform refresh command, which still exists but is deprecated because it writes the state without showing you anything first. A plain terraform plan -refresh-only shows the drift without saving it, which makes it a good scheduled drift check. Here, the provider reported the edited file as gone, so the refresh removed it from the state, and the next plan creates it again:

Two flags change how much Terraform refreshes. -refresh=false skips the refresh entirely (faster, but blind to drift), and -minimal-refresh, new in Terraform 1.17, only refreshes resources that have changes in your configuration, which speeds up plans of large configurations. The post What Does Terraform Refresh Really Do? goes deeper.

Refactor Without Destroying Anything

When you rename a resource in code, Terraform sees one address disappear and a new one appear: by default it destroys the old object and creates a new one. For a server or a database, that is an outage. Tell Terraform about the rename instead.

Rename With a moved Block

Rename random_pet.server to random_pet.web, update the reference, and add a moved block. This version also splits the code into files: main.tf keeps only the terraform block, and the resources move to pet.tf and app.tf:

resource "random_pet" "web" {
  length = 2
}

moved {
  from = random_pet.server
  to   = random_pet.web
}

resource "random_password" "db" {
  length  = 20
  special = false
}

resource "local_file" "config" {
  filename = "${path.module}/app.conf"
  content  = "server=${random_pet.web.id}\n"
}

The plan moves the object in the state and changes nothing else. moved blocks live in code, so they go through code review, they work in CI where nobody runs state commands by hand, and they apply for everyone who uses the module. Keep them for a while (until every copy of the state has been applied), then delete them. They also handle moves into and out of modules (from = aws_instance.web, to = module.web.aws_instance.this) and from count to for_each (from = aws_instance.web[0], to = aws_instance.web["blog"]).

Apply it to record the move in the state:

The older command-line way is terraform state mv. It changes the state immediately, without a plan, so use it for one-off fixes on your own state:

Stop Managing an Object Without Deleting It

To hand an object over to another configuration, or to leave it alone for good, delete its resource block and add a removed block with destroy = false:

removed {
  from = random_password.db

  lifecycle {
    destroy = false
  }
}

Terraform no longer tracks the object. For a resource that stands for something real, such as a server, a bucket or a file, that object stays where it is. (A random_password exists only in the state, so here the generated value is simply dropped.) Without the removed block, deleting the resource block would destroy the object. Once the change is applied everywhere, you can delete the removed block too. The command-line equivalent is terraform state rm ADDRESS; try it safely with -dry-run first:

Since Terraform 1.16, you can also put destroy = false in a resource's own lifecycle block. Then a replacement, a terraform destroy, or an instance that drops out of count or for_each forgets the old object instead of deleting it; the plan marks those objects with . and ./+ and warns that they will no longer be managed. It does not help when you delete the whole resource block, because the setting goes with it: that case needs the removed block above.

Import Existing Objects

The opposite problem: the object exists (someone created it by hand, or you are moving it from another configuration), and you want Terraform to manage it. An import block names the address and the object's ID. This example imports a random number; for a real resource the ID is what the provider's documentation says, such as an instance ID or a bucket name.

import {
  to = random_integer.port
  id = "8443,1024,65535" # result,min,max for this resource type
}

You do not even have to write the resource block. -generate-config-out writes it for you from the real object:

Review the generated code, move it where it belongs, then apply. The plan and apply summaries count imports separately:

For real cloud resources, the generated code often needs cleanup (computed values written as literals, defaults spelled out), so read it before you commit it. Import blocks accept for_each since Terraform 1.7, and work inside modules since 1.16:

variable "bucket_names" {
  type    = set(string)
  default = ["acme-logs", "acme-assets"]
}

import {
  for_each = var.bucket_names
  to       = aws_s3_bucket.this[each.key]
  id       = each.key # the S3 import ID is the bucket name
}

resource "aws_s3_bucket" "this" {
  for_each = var.bucket_names
  bucket   = each.key
}

The older terraform import ADDRESS ID command still works, but it changes the state immediately and needs the resource block to exist first. Import blocks are planned, reviewed and repeatable, so prefer them. Since Terraform 1.14, some providers also support terraform query with list blocks in .tfquery.hcl files, which finds existing objects in an account and can generate import blocks for all of them.

State Locking

Two apply runs writing the same state at the same time would corrupt it, so Terraform locks the state for every operation that may write it. The local backend uses a file lock; S3 uses a lock file next to the state, other backends use their own mechanism (Part 9). Here is what a second run sees while a slow apply holds the lock, in a new directory called locking with this main.tf (run terraform init first):

# Takes about 10 seconds to "create", so there is time to start a second run
resource "terraform_data" "slow" {
  provisioner "local-exec" {
    command = "sleep 10"
  }
}

The error names who holds the lock (user and host), the operation, and when it started. With -lock-timeout=60s, the second run waits for the lock instead of failing, and goes ahead once the first apply finishes. Use a timeout like that in CI, where two pipelines can overlap.

If a run crashed or was killed and left a lock behind on a remote backend, release it with the lock ID from the error message:

terraform force-unlock 3b6a1f8c-2c9e-4b8a-9d55-0f1e2a3b4c5d

Only do this after you have checked that the run that owns the lock is really gone (the CI job is finished, the colleague's laptop is not mid-apply). Unlocking a live run is how state gets corrupted. The posts Terraform Statefile is Locked: How to Unlock It and Fix Error Acquiring the State Lock: ConditionalCheckFailedException cover backend-specific cases.

Backups and Recovery

The local backend writes terraform.tfstate.backup (the previous version) on every change. With a remote backend, plan and apply do not write local backups; recovery relies on versioning in the storage (S3 object versions, Azure blob versions, GCS object versions), on the state history in HCP Terraform, or on database backups for pg. Turn that on before you need it. The terraform state subcommands that change state, such as mv and rm, are the exception: they always write a local backup file, also for remote state, and you cannot turn that off. Those files contain the full state, secrets included, so delete them or store them as carefully as the state itself (-backup=PATH chooses where they go).

Problem	What to do
A resource was deleted outside Terraform	Run a plan: the refresh notices it's gone and plans to create it again. If it should stay gone, delete the block or add a removed block
Terraform wants to create something that already exists	import it instead of creating a duplicate
You renamed a resource and the plan wants to destroy it	Add a moved block
The state file is corrupted or was overwritten	Restore the previous version from backend versioning (or terraform.tfstate.backup locally), then run a plan to see the difference
The state is lost completely	Recreate it with import blocks for every object, then plan until there are no changes

Upgrading Terraform and the State

Any newer Terraform 1.x can read and upgrade state written by an older 1.x. Going back to an older version is not guaranteed, because a newer release may store things an older one does not understand. In a team, upgrade CI and every laptop together, and pin the version in code so a forgotten old binary fails early:

terraform {
  required_version = "~> 1.17.0" # 1.17.x only
}

Use ">= 1.17" instead if you are happy for newer minor versions to run the code. Part 10 covers version pinning in more detail.

Common Mistakes

• Renaming resources without moved blocks. The plan shows a destroy and a create. Read it before you approve.
• Running state commands against shared state casually. state mv, state rm and state push take effect immediately and have no plan. Pull a backup first.
• Using terraform refresh. Use terraform apply -refresh-only, which shows the changes before it saves them.
• Force-unlocking a lock that is still in use. Find the owner first.
• Committing state or saved plan files to Git. Both contain secrets.
• One huge state for everything. Every plan reads every object, and every mistake can touch everything. Split state by environment and by component (Part 8).

Summary

• State maps your code to real objects and stores their attributes, including secrets in plain text.
• Read it with state list, state show and show; script against -json output.
• Refactor with moved, adopt with import, let go with removed: all planned and reviewed.
• apply -refresh-only accepts drift; a normal apply undoes it.
• Locks protect against concurrent writes; use -lock-timeout in CI and force-unlock only for dead runs.

In Part 6 you make configurations reusable with variables and outputs.

Keep Learning

• Terraform State: Remove, Move and Migrate Resources, and Set Up a Remote Backend
• Who Owns the State File, and Other Questions That Decide Your Week
• How to Fix Terraform Import "Index Value Required" Error for String Keys
• Interview question: Terraform state management
• Terraform puzzle game