Why does Terraform need a state file instead of querying the cloud each run?
State maps configuration addresses (`aws_instance.web`) to real resource IDs, tracks dependencies, and caches metadata. Without it, Terraform cannot know which cloud resources it manages vs. which were created elsewhere.
Solo / small team / no policy enforcement β OSS CLI with remote backend. Multiple teams / Sentinel/OPA policies / VCS-driven runs / dynamic credentials β HCP Terraform.
Why: Both run the same Terraform binary; HCP adds collaboration, governance, and remote-runner infrastructure.
Terraform provisions cloud resources (VMs, networks, IAM). Ansible/Chef/Puppet configure software inside the VMs. They're complementary, not competitors.
AWS-only shop debating CloudFormation vs Terraform.
CloudFormation has tighter AWS integration, native rollback, and zero state-file management. Terraform wins on multi-cloud, larger module ecosystem, and HCL ergonomics. Pick CFN if AWS-only and rollback matters most; Terraform otherwise.
Understand Terraform Basics
Pin provider source and version for reproducible installs.
Declare in `terraform { required_providers { aws = { source = "hashicorp/aws", version = "~> 5.0" } } }`.
Why: Without explicit `required_providers`, Terraform assumes legacy registry namespaces and may pick incompatible versions.
Downloads providers per `required_providers`, initializes the backend, downloads module sources, and writes `.terraform/` plus `.terraform.lock.hcl`. Required before `plan` or `apply`.
CI runs `terraform init` repeatedly across many projects, each downloading the same providers from scratch.
Set `TF_PLUGIN_CACHE_DIR` (or `plugin_cache_dir` in CLI config). Providers are downloaded once and symlinked into per-project `.terraform/` directories.
`<hostname>/<namespace>/<type>` - e.g. `registry.terraform.io/hashicorp/aws`. Hostname omitted defaults to public Terraform Registry. Namespace = vendor. Type = provider name.
Why: Static long-lived secrets in config are the leading cause of leaked-credential incidents.
Where do top-level settings live?
A `terraform { ... }` block holds `required_version`, `required_providers`, `backend`, `cloud`, and experiments. Multiple `terraform` blocks across files are merged.
Use Terraform Outside Core Workflow
Resource exists in state but should no longer be Terraform-managed; the cloud resource must keep running.
`terraform state rm <addr>`. Removes from state without destroying the cloud resource.
Existing manually-created S3 bucket needs to come under Terraform management without recreation.
Write the resource block, then `terraform import aws_s3_bucket.legacy legacy-bucket-name`. State now records the existing bucket; subsequent plans show only drift.
Need to import many resources reproducibly through CI rather than ad-hoc CLI commands.
Use the `import` block (Terraform 1.5+): `import { to = aws_s3_bucket.legacy, id = "legacy-bucket-name" }`. Imports happen during `apply`, are version-controlled, and work in plan output.
`terraform state list` shows all resource addresses. `terraform state show <addr>` shows attributes for one resource. For indexed resources use quotes: `terraform state show 'aws_instance.web[0]'`.
Local path β `./modules/vpc`. Public registry β `terraform-aws-modules/vpc/aws`. Git β `git::https://github.com/org/repo.git//path?ref=v1.0`. S3/HTTP/Mercurial also supported.
Two patterns: (1) Workspaces with one root config + per-workspace `*.tfvars`. (2) Per-environment root configs (`envs/dev/main.tf`, `envs/prod/main.tf`) each calling shared modules. Pattern 2 is more common for multi-team isolation.
Use the Core Terraform Workflow
Ensure CI applies exactly what was reviewed in plan, with no drift between steps.
`terraform plan -out=tfplan` saves the plan. Then `terraform apply tfplan` applies that exact plan. Refusing to re-plan eliminates time-of-check/time-of-use risk.
Decode plan output symbols: `+`, `-`, `~`, `-/+`, `<=`.
`+` create. `-` destroy. `~` update in place. `-/+` destroy then create (replace). `<=` read (data source). The replace symbol means a `forces replacement` attribute changed.
Quickly fix one broken resource without touching the rest of the config.
`terraform apply -target=aws_instance.web`. Use sparingly - bypasses dependency tracking and can leave state inconsistent. Document why each `-target` was used.
Why: HashiCorp explicitly says `-target` is for exceptional troubleshooting, not normal workflow.
Resource A must exist before Resource B, but B doesn't reference A's attributes.
Add `depends_on = [resource_a.name]` to B. Use only when implicit attribute references can't express the dependency (e.g. IAM policy must propagate before EC2 uses it).
Replace an EC2 instance whenever a related launch template changes (without modifying the EC2 directly).
`lifecycle { replace_triggered_by = [aws_launch_template.web.latest_version] }` (Terraform 1.2+). The instance is replaced when any listed value changes.
CI/CD pipeline applies after a successful plan job; no human at the keyboard.
`terraform apply -auto-approve` skips the interactive confirmation. Combine with a saved plan file (`terraform apply tfplan`) to make CI deterministic.
Cloud provider rate-limits Terraform; many resource creations are failing intermittently.
Resource address β cloud resource ID mapping, attribute snapshots, dependency graph metadata, and module/provider references. Used to plan diffs and detect drift.
Single engineer prototyping locally - is local state OK?
Yes for solo throwaway work. Terraform writes `terraform.tfstate` next to the config. Switch to a remote backend the moment a second person, CI runner, or production environment is involved.
Azure-hosted shared state with no static credentials in CI.
Backend type `azurerm`. Set `use_msi = true` (Managed Identity) or `use_oidc = true` so the runner authenticates via its identity. Native blob lease handles locking.
Two engineers run apply simultaneously against the same S3-backed state.
DynamoDB lock table prevents concurrent writes. The first acquires the lock, the second sees a `state lock failed` error and must wait or `force-unlock` if stale.
Why: Without DynamoDB, concurrent applies can corrupt the S3 state file.
Manage dev / staging / prod with one root configuration.
Workspaces - `terraform workspace new staging`, `terraform workspace select prod`. Each has its own state file. Reference via `terraform.workspace` in HCL.
Why: Lightweight option. For real isolation (different IAM, separate accounts), prefer per-environment root configs.
Security team asks how Terraform handles RDS passwords in state.
Sensitive values are stored **plaintext** in state. Mitigations: encrypt the backend at rest (S3 SSE-KMS, HCP native), restrict IAM access to the state bucket, and avoid putting secrets in Terraform when possible.
Why: The `sensitive = true` flag only hides values from CLI output, not state.
S3: enable SSE-KMS on the bucket. Azure: storage account encryption (default ON). GCS: customer-managed encryption keys. HCP Terraform: encrypted natively at rest.
Engineer ran `terraform destroy` against prod by mistake. State now empty.
Restore the previous version of the state file from S3 versioning, then run `terraform plan` to see what Terraform now thinks needs to be created/imported. Combine with cloud-side restore (snapshots, `aws backup`) for the destroyed resources.
State file looks wrong; tempted to edit it directly.
Don't. Use `terraform state` subcommands (`mv`, `rm`, `replace-provider`, `pull`, `push`). Manual JSON edits skip integrity checks and corrupt the lineage.
Inspect raw remote state JSON, or push a recovered state file.
`terraform state pull` writes current remote state to stdout. `terraform state push <file>` overwrites remote with the given file. Push is destructive - back up first.
Pass a temporary API token through Terraform without it landing in state.
Mark variable `ephemeral = true` (Terraform 1.10+). Ephemeral values are never persisted to state or plan files. To export through a module output, also mark the output `ephemeral = true`.
Why: `sensitive` is hidden in CLI but stored in state plaintext. `ephemeral` is genuinely never stored.
Difference between a `sensitive` argument and a write-only argument (e.g. `password_wo`).
`sensitive` is stored in state plaintext, only redacted in CLI. Write-only is **never** stored in state. To detect changes on write-only attrs, Terraform uses a companion version field (e.g. `password_wo_version`).
Quote the address: `terraform state show 'aws_instance.web[0]'`. Without quotes, the shell may interpret the brackets.
Different teams need to operate independently without stepping on each other's state.
One state file per team (or per environment per team). Separate backends/buckets. Use `terraform_remote_state` data source to read another state read-only.
Configuration written against Terraform 0.13; want to use 1.x.
State files are forward-compatible: 1.x reads 0.13 state. HashiCorp recommends incremental upgrades (0.13 β 0.14 β β¦ β 1.x) running each version against the state.
Someone modified a security group via console; want Terraform to either re-assert or accept the new state.
Re-assert: `terraform apply` - Terraform reverts to configuration. Accept: update HCL to match reality, then apply (no diff). Detect first via `terraform plan -refresh-only`.
Read, Generate, and Modify Configuration
Strongly-type a variable.
Primitives: `string`, `number`, `bool`. Collections: `list(<type>)`, `set(<type>)`, `map(<type>)`. Structural: `object({...})`, `tuple([...])`. Use `any` only when truly polymorphic.
Create N similar resources - pick `count` or `for_each`.
`for_each` (with map or set) when items have stable identity (region names, environment keys). `count` for "I need N copies, order doesn't matter, identity is just an index". Adding/removing in the middle of `count` causes destroy/recreate; `for_each` preserves identity.
Get a list of attributes across all instances of a `count` resource.
`aws_instance.web[*].id` returns a list of IDs. Works with `count` and `for_each` (but `for_each` produces an unordered map, so `values(aws_instance.web)[*].id`).
Read a map value with a default if the key is missing.
`lookup(var.config, "region", "us-east-1")`. Returns the default when the key isn't present. For deeply optional structures, prefer the optional `try()`.
`remote-exec` provisioner inside a resource. Last-resort tool - prefer cloud-init, user data, or configuration management. Provisioners aren't tracked in state and don't re-run on drift.
Continuously verify a runtime invariant (e.g. health endpoint returns 200) without blocking apply.
`check "endpoint" { data "http" "h" { url = "..." }; assert { condition = data.http.h.status_code == 200; error_message = "..." } }`. Runs at plan/apply; failure is a warning, not a hard error.
Why: `check` allows scoped data sources usable only inside the check. `precondition`/`postcondition` are hard errors at plan/apply.
plan β cost estimation (if enabled) β policy check (Sentinel/OPA) β manual or auto-apply. Mandatory policy failures or run-task failures halt the pipeline.
Enforce that all S3 buckets are encrypted, blocking apply if violated.
Sentinel policy with hard-mandatory enforcement. Inspects the plan; failure blocks apply. Soft-mandatory allows admin override. Advisory logs but never blocks.
Integrate a third-party security scanner into the HCP Terraform run pipeline.
Run task at the post-plan stage. HCP POSTs the plan to your endpoint; the endpoint replies pass/fail. Mandatory enforcement blocks apply on failure; advisory only warns.
A built-in project that exists in every organization and cannot be deleted (renaming is allowed). All workspaces belong to it unless explicitly assigned to another project.
Apply downstream workspace whenever an upstream workspace finishes a successful apply.
Run triggers. Configure source workspace(s) on the dependent workspace; HCP queues a run on the dependent after each successful upstream apply.