Infrastructure you can review, test and change safely: the HCL language, providers and state, refactoring with moved, import and removed blocks, modules, tests, plan risk review, policy as code with Checkov and Trivy, repository layouts per environment, and CI/CD on AWS — with a lab for every chapter.
Use ← → / Space to navigate · I contents · O overview · T theme · F fullscreen
Click any line to jump there · press I to return to this page
Why infrastructure as code, what this deck builds, the versions it covers, and how to study it.
The console is fast for one change and hopeless for the hundredth. Code makes every change repeatable, reviewable and reversible.
0.0.0.0/0.terraform plan shows the exact diff against reality.apply can delete production faster than a human can.Read left to right. Each box is where this deck spends its time; the bottom row is the lab repository you build.
Highlighted boxes are the focus topics of this deck: modules, state, moved/import, plan risk review, policy as code and repository layout per environment.
Terraform ships a minor release every few months. When a slide and the official docs disagree, the docs win.
-minimal-refresh.on_failure. 1.15 (Apr 2026): variables and locals in module source/version.terraform query with list blocks (.tfquery.hcl), action blocks.import/check, 1.6 test, 1.7 mocks and removed, 1.10 ephemeral values, 1.11 write-only arguments and S3 use_lockfile.region, resource identity, list resources.use_lockfile = true; dynamodb_table locking is deprecated.prevent_destroy, enabled.trivy config; tfsec checks now live in Trivy).trivy-action tags were hijacked on 19 Mar 2026 → pin CI actions by SHA.required_version = ">= 1.14" and AWS ~> 6.0.Labs work on Terraform 1.14 or newer. Features from 1.15–1.17 are marked with their version on the slide that uses them.
OpenTofu forked from Terraform 1.5 after the license change. Most of this deck applies to both; differences are called out.
| Topic | Terraform (HashiCorp/IBM) | OpenTofu (Linux Foundation) |
|---|---|---|
| License | BUSL 1.1 (source-available) since 1.6 | MPL 2.0 (open source) |
| CLI and registry | terraform, registry.terraform.io | tofu, registry.opentofu.org (mirrors most providers) |
| State format | Compatible across both for the versions this deck uses; migrate one direction at a time | Same; adds optional client-side state encryption (1.7+) |
| Own features | terraform query/list blocks, actions, Terraform Policy, Stacks (HCP) | Dynamic prevent_destroy, enabled meta-argument, early variable evaluation in source |
| Managed service | HCP Terraform / Terraform Enterprise | Any TACOS (Spacelift, env0, Scalr, …) or plain CI |
This deck teaches with terraform. Chapter 13 shows a migration walkthrough and what to check before switching either way.
Labs use local-only providers or free AWS resources. The only standing cost is a few KB of state in S3.
tf-lab, region ap-northeast-1.<you>/tf-lab).terraform 1.14+ (hashicorp/tap/terraform), optional tofu.tflint, checkov, trivy, conftest.jq, gh, git, optional infracost, terraform-docs.tf-lab/
├── bootstrap/ # state bucket (Lab 0)
├── modules/
│ ├── network/ # VPC, subnets (Lab 6)
│ ├── app-bucket/ # S3 module (Lab 6)
│ └── ci-role/ # GitHub OIDC (Lab 11)
├── live/
│ ├── dev/
│ │ ├── network/ # 10.10.0.0/16
│ │ └── app/
│ └── prod/
│ ├── network/ # 10.20.0.0/16
│ └── app/
├── policies/ # Lab 9
├── scripts/ # Lab 8
└── .github/workflows # Lab 11Read a part, run its lab, then answer the self-check out loud without notes.
Order matters: state (ch 4) before refactoring (ch 5); modules (ch 6) before layout (ch 10); plan review and policy (ch 8–9) before CI/CD (ch 11).
What "infrastructure as code" buys you, how Terraform turns configuration into API calls, and the commands you will type every day.
Describe infrastructure in versioned text files, and let a tool make reality match them.
Clicking in a console works once. Code works the hundredth time, for the second environment, and for the person who joins next year.
apply.Terraform is declarative: you write the desired end state, and the tool computes the steps. A script is imperative: you write the steps.
# imperative: steps, and you handle every case
aws s3api create-bucket --bucket tflab-demo-123 \
--create-bucket-configuration LocationConstraint=ap-northeast-1
aws s3api put-bucket-versioning --bucket tflab-demo-123 \
--versioning-configuration Status=Enabled
# run it twice → BucketAlreadyOwnedByYou error
# rename the bucket → you must write the delete yourself# declarative: the end state
resource "aws_s3_bucket" "demo" {
bucket = "tflab-demo-123"
}
resource "aws_s3_bucket_versioning" "demo" {
bucket = aws_s3_bucket.demo.id
versioning_configuration {
status = "Enabled"
}
}
# apply twice → second run: "No changes."
# rename → plan shows destroy + createIdempotence: applying the same configuration again changes nothing. It is what makes apply safe to rerun after a failure, and what makes drift visible: any difference in a new plan means reality moved.
Drift is any difference between the code, the state file and the real resources. How you change things decides how much drift you get.
~ update when the API allows, -/+ replace when an argument cannot change (ch 3).terraform plan -refresh-only or a scheduled plan with -detailed-exitcode (ch 11).ManagedBy=terraform (ch 13).All of them describe desired state; they differ in language, state handling and scope.
| Tool | Language | State | Scope | Fits when |
|---|---|---|---|---|
| Terraform | HCL (+ JSON) | State file in a backend you choose | Any provider: AWS, GCP, Azure, GitHub, Datadog, … | Multi-cloud or multi-SaaS estates; huge provider and module ecosystem |
| OpenTofu | HCL (Terraform-compatible) | Same model; optional state encryption | Same providers (OpenTofu registry) | You want an MPL-licensed, Linux Foundation fork (ch 13) |
| AWS CloudFormation | YAML / JSON | Managed by AWS (stacks) | AWS only | AWS-only shops that want no state to manage; StackSets for org-wide baselines |
| AWS CDK | TypeScript, Python, Go, … | Synthesizes CloudFormation | AWS only | Teams that prefer general-purpose languages and loops over HCL |
| Pulumi | TypeScript, Python, Go, … | State in Pulumi Cloud or a bucket | Multi-cloud | Infrastructure written like application code |
| Crossplane | Kubernetes YAML (CRDs) | Kubernetes API (etcd) | Multi-cloud | Platform teams exposing infra through Kubernetes and GitOps |
This deck uses Terraform 1.17 with notes where OpenTofu differs. Everything in labs 0–13 also runs on OpenTofu unless a slide says otherwise.
Terraform is best at long-lived resources with an API. It is a poor fit for things that change every minute or need procedural steps.
ignore_changes at the boundary (ch 3).Core, providers, the graph and the state: the four ideas behind every plan.
The core is a single binary that knows HCL, the graph and state. Everything about AWS lives in the provider, a separate plugin process.
Providers are downloaded by terraform init from a registry (registry.terraform.io by default; OpenTofu uses registry.opentofu.org), recorded with checksums in .terraform.lock.hcl (ch 3) and talk to core over the plugin protocol (gRPC).
Every plan and apply runs the same pipeline. Knowing the stages tells you where an error comes from.
*.tf files in the directory (order of files does not matter).init.aws_s3_bucket.demo.id) and depends_on.plan stops after step 4 and shows the diff. apply without a plan file runs 1–4 again, asks for approval, then runs step 5. Validation errors come from step 1–2; API errors (AccessDenied, InvalidParameter) only appear in step 5.
Terraform cannot find "your" bucket by name alone. The state file records which real object each resource address manages.
{
"version": 4,
"terraform_version": "1.17.0",
"serial": 12,
"lineage": "3f2c…",
"resources": [
{
"mode": "managed",
"type": "aws_ssm_parameter",
"name": "hello",
"instances": [
{
"attributes": {
"id": "/tflab/dev/hello",
"arn": "arn:aws:ssm:…:parameter/tflab/dev/hello",
"value": "…"
}
}
]
}
]
}aws_ssm_parameter.hello ↔ /tflab/dev/hello.serial increases on every write; lineage identifies one state history.References create edges. Terraform creates independent resources in parallel (default -parallelism=10) and dependent ones in order.
vpc_id = aws_vpc.main.id); the route table uses both the VPC and the gateway.terraform graph | dot -Tsvg > graph.svg draws it (Graphviz). Since 1.7 the default output shows resources only; -type=plan shows the full plan graph.-parallelism when a provider hits API rate limits (ch 13).Some values exist only after creation. Terraform plans with placeholders and resolves them during apply.
# plan output (excerpt)
# aws_ssm_parameter.hello will be created
+ resource "aws_ssm_parameter" "hello" {
+ arn = (known after apply)
+ id = (known after apply)
+ name = "/tflab/dev/hello"
+ type = "String"
+ value = (sensitive value)
+ version = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.(known after apply): computed by the API (IDs, ARNs). Anything that depends on it is unknown too.count/for_each keys: "Invalid for_each argument" (ch 2).(sensitive value): hidden in output, still stored in state and plan files.-refresh=false skips it (faster, but blind to drift).-refresh-only plans only accept reality into state, no config changes (ch 5).init, plan, apply, destroy — and the small commands that keep you out of trouble.
Write → init → fmt/validate → plan → review → apply. Most mistakes come from skipping the review step.
| Command | What it does | Useful flags |
|---|---|---|
terraform init | Configure backend, download providers and modules, write the lock file | -upgrade, -backend-config=…, -reconfigure, -migrate-state |
terraform fmt | Rewrite files in canonical style | -check (CI), -recursive, -diff |
terraform validate | Check syntax, types and references (no API calls, no state) | -json |
terraform plan | Refresh and compute the changes | -out=tfplan, -var, -var-file, -detailed-exitcode, -refresh-only, -replace=ADDR |
terraform apply | Plan + approve + change; or apply a saved plan | tfplan (no prompt), -auto-approve, -parallelism=N |
terraform destroy | Plan and execute deletion of everything in state | alias for apply -destroy; plan -destroy to preview |
terraform output | Print root module outputs from state | -json, -raw NAME |
terraform show | Human or JSON view of state or a plan file | -json tfplan (ch 8) |
terraform console | Evaluate expressions interactively against config and state | pipe input: echo "1+1" | terraform console |
terraform init doesRun it first in every new working directory, and again after you change providers, modules or backend settings.
$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 6.0"...
- Finding latest version of hashicorp/random...
- Installing hashicorp/aws v6.x.y...
- Installed hashicorp/aws v6.x.y (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to
record the provider selections it made above.
Terraform has been successfully initialized!
$ tree -a -L 2 .
.
├── .terraform/
│ └── providers/ # downloaded plugins
├── .terraform.lock.hcl # COMMIT this file
├── main.tf
└── versions.tf.terraform/terraform.tfstate (not your real state).-upgrade allows newer versions and rewrites the lock file..terraform/modules/ (ch 6)..gitignore: .terraform/, *.tfstate*, *.tfplan, crash.log. Never ignore .terraform.lock.hcl.A saved plan file pins the reviewed changes. Applying it does not re-plan, and fails if the state changed since.
terraform plan -out=tfplan # write the plan
terraform show tfplan # human review
terraform show -json tfplan | jq '.resource_changes | length'
terraform apply tfplan # no prompt, no re-plan
# someone applied in between → state serial changed:
# Error: Saved plan is stale
# The given plan file can no longer be applied because
# the state was changed by another operation after the
# plan was created.
terraform plan -destroy -out=destroy.tfplan
terraform apply destroy.tfplanterraform apply without a file re-plans and asks yes: fine on a laptop, risky in automation (what you approve may not be what you reviewed).-auto-approve belongs only in pipelines that apply a saved, reviewed plan.-detailed-exitcode: plans that scripts can readBy default plan exits 0 whether or not there are changes. With -detailed-exitcode the exit code tells you what the plan found.
| Exit code | Meaning | Typical use |
|---|---|---|
0 | Succeeded, no changes | Drift check passes; nothing to apply |
1 | Error | Fail the job |
2 | Succeeded, changes present | Drift job opens an issue; PR shows "changes" |
set +e
terraform plan -detailed-exitcode -out=tfplan -input=false
rc=$?
set -e
case $rc in
0) echo "no changes" ;;
2) echo "changes: review tfplan" ;;
*) echo "plan failed"; exit 1 ;;
esacUse -input=false in automation: if a variable has no value, Terraform fails instead of waiting for keyboard input forever.
Know which files are code, which are generated, and which are secrets.
| Path | Created by | Commit to Git? | Notes |
|---|---|---|---|
*.tf, *.tfvars (non-secret) | You | Yes | All .tf files in one directory form one module |
.terraform.lock.hcl | init | Yes | Provider versions and hashes; makes CI use the same providers |
.terraform/ | init | No | Plugins, modules, backend settings; recreated by init |
terraform.tfstate, .backup | apply (local backend) | Never | Contains secrets; use a remote backend (ch 4) |
tfplan (any name) | plan -out | No | May contain secrets; short-lived CI artifact |
*.auto.tfvars | You | Only if not secret | Loaded automatically |
~/.terraformrc | You | No (per user) | CLI config: plugin_cache_dir, credentials, mirrors |
Speed up init across many directories with a shared plugin cache: export TF_PLUGIN_CACHE_DIR="$HOME/.terraform.d/plugin-cache" (the directory must exist).
Small commands, used constantly. They are also the first CI gate (ch 7).
$ terraform fmt -check -recursive
main.tf # files that need formatting; exit 3
$ terraform fmt -recursive # fix them
$ terraform validate
Success! The configuration is valid.
$ terraform validate
Error: Reference to undeclared resource
on main.tf line 12, in resource "aws_ssm_parameter" "hello":
12: value = random_pet.nmae.id
A managed resource "random_pet" "nmae" has not been declared
in the root module.$ terraform console
> cidrsubnet("10.10.0.0/16", 8, 1)
"10.10.1.0/24"
> random_pet.name.id # reads current state
"select-mollusk"
> exit
$ terraform output
param_name = "/tflab/dev/hello"
$ terraform output -raw param_name
/tflab/dev/hello
$ terraform output -json | jq -r .param_name.value
/tflab/dev/hellovalidate needs init first (it must know provider schemas), but no credentials and no state. That makes it cheap enough for every commit.
Each of these is a real incident pattern.
terraform.tfstate (with secrets) to Git..terraform.lock.hcl: CI silently uses a newer provider than your laptop.apply -auto-approve without a reviewed plan.apply that re-plans against changed state.plan exit 0 as "no changes" in scripts (use -detailed-exitcode).-refresh=false) by habit and missing drift.destroy in the wrong directory or with the wrong profile..gitignore from day one; remote state with locking (ch 4).aws sts get-caller-identity.plan -out → show → apply tfplan.fmt -check + validate before every commit (pre-commit hooks, ch 7).Answer out loud, with the command or file name.
for_each not use a value that is (known after apply)?destroy delete a VPC and its subnets, and why?init -upgrade change?plan -detailed-exitcode?.gitignore, and which must be committed?Practice: Lab 1 in Hands-on labs — a first root module with random, local and one SSM parameter: init, saved plans, exit codes, the graph, drift and destroy.
Blocks and types, variables and outputs, expressions and functions, count and for_each, and the values Terraform never stores.
Blocks, arguments and the type system that catches mistakes before any API call.
HCL has two constructs: blocks (type, labels, body) and arguments (name = expression). Everything else is expressions.
# block type label 1 (type) label 2 (local name)
resource "aws_ssm_parameter" "hello" {
name = "/tflab/dev/hello" # argument = expression
type = "String"
value = random_pet.name.id # reference to another resource
tags = { # map expression
Owner = "platform"
}
lifecycle { # nested block (no "=")
ignore_changes = [tags]
}
}
/* multi-line
comment */ // also a comment*.tf files in it are merged; file names and order do not matter.aws_ssm_parameter.hello. Local names are unique per type and module.=; nested blocks do not. Some providers accept both forms for the same setting — follow the provider docs.# (preferred), //, /* */. terraform fmt aligns = signs.terraform, provider, resource, data, variable, locals, output, module, moved, import, removed, check, ephemeral, action.Every value has a type. Type constraints on variables turn bad input into a clear error at plan time.
| Type | Example | Notes |
|---|---|---|
string | "tflab" | UTF-8; templates "${x}" |
number | 3, 0.5 | One numeric type for ints and floats |
bool | true | |
list(T) | ["a", "b"] | Ordered, index from 0 |
set(T) | toset(["b", "a"]) | Unordered, unique; no index |
map(T) | { a = 1, b = 2 } | String keys, values of one type |
object({…}) | { name = "x", size = 2 } | Named attributes with their own types |
tuple([…]) | ["x", 2, true] | Fixed length, per-position types |
any | — | Placeholder: Terraform infers a type |
null | null | Absent value: argument uses its default |
"5" → 5 where a number is expected, true → "true".["a", "b"] converts to list(string) or set(string) by context.tostring, tonumber, tolist, toset, tomap convert explicitly; convert(value, type) (1.15) converts to any type constraint.type(x) in terraform console shows the inferred type.[for x in set : …] iterates in sorted order for strings, but never index a set.optional() attributesOne typed object input is easier to use and document than ten loose variables. optional(type, default) fills in what callers omit.
variable "bucket" {
type = object({
name_suffix = string
versioning = optional(bool, true) # default true
expire_days = optional(number) # default null
lifecycle_rules = optional(list(object({
id = string
prefix = string
days = number
})), [])
})
}
# terraform.tfvars
bucket = {
name_suffix = "logs-123"
lifecycle_rules = [
{ id = "tmp", prefix = "tmp/", days = 7 }
]
}$ echo 'var.bucket' | terraform console
{
"expire_days" = tonumber(null)
"lifecycle_rules" = tolist([
{
"days" = 7
"id" = "tmp"
"prefix" = "tmp/"
},
])
"name_suffix" = "logs-123"
"versioning" = true
}
# missing required attribute:
# Error: Invalid value for input variable
# attribute "name_suffix" is required.
# unknown attribute (typo "versionning"):
# rejected as well — objects are strictoptional() works in variable type constraints (Terraform 1.3+). Use it for module inputs (ch 6): callers pass only what differs from your defaults.
Interpolation ${…} inserts values; directives %{…} add conditions and loops inside strings.
locals {
name = "tflab-${var.env}"
tier = "%{if var.env == "prod"}critical%{else}normal%{endif}"
user_data = <<-EOT
#!/bin/bash
echo "env=${var.env}" > /etc/tflab
EOT
# <<-EOT strips the common leading indentation
literal = "$${not_interpolated}" # → ${not_interpolated}
}# templates/params.tftpl
env: ${env}
%{ for k, v in params ~}
- ${k} = ${v}
%{ endfor ~}
# usage
templatefile("${path.module}/templates/params.tftpl",
{ env = var.env, params = var.params })
# result
env: dev
- greeting = hello
- owner = platform~ trims whitespace and the newline next to a directive. Build JSON and YAML with jsonencode() / yamlencode(), never with string templates: quoting and escaping are then always correct.
The interface of a module: what goes in, what is computed once, and what comes out.
Variables are the parameters of a module. Declare type and description always; add validation for anything a caller can get wrong.
variable "env" {
type = string
description = "Environment name (dev or prod)."
validation {
condition = contains(["dev", "prod"], var.env)
error_message = "env must be dev or prod."
}
}
variable "vpc_cidr" {
type = string
default = "10.10.0.0/16"
validation {
condition = can(cidrhost(var.vpc_cidr, 0))
error_message = "vpc_cidr must be a valid IPv4 CIDR."
}
}
variable "db_password" {
type = string
sensitive = true # hidden in CLI output
nullable = false # null is rejected
}type: constraint (section 04). Without it, -var values arrive as strings.default: makes the variable optional. No default = required.description: shows up in docs and terraform-docs output.validation: one or more condition + error_message; since 1.9 conditions may refer to other variables.sensitive: redacts the value in plan/apply output (still in state).nullable = false: passing null is an error instead of "use default".ephemeral: value is never written to state or plan (section 08).Later sources override earlier ones. Knowing the order explains "why is it still using the old value?"
| Order (low → high) | Source |
|---|---|
| 1 | default in the variable block |
| 2 | Environment: TF_VAR_env=dev |
| 3 | terraform.tfvars |
| 4 | terraform.tfvars.json |
| 5 | *.auto.tfvars / *.auto.tfvars.json, in lexical order of file names |
| 6 | -var and -var-file on the command line, in the order given |
$ terraform plan -var-file=dev.tfvars -var 'env=qa'
Error: Invalid value for variable
on variables.tf line 1:
1: variable "env" {
├────────────────
│ var.env is "qa"
env must be dev or prod.
This was checked by the validation rule at
variables.tf:5,3-13.
# required variable and -input=false:
# Error: No value for required variableA practical pattern: no terraform.tfvars in shared code; one explicit -var-file=env/dev.tfvars per environment (ch 10). Then every value in a plan has an obvious origin.
Locals are computed values inside a module. They keep naming and tagging consistent and remove repetition.
locals {
name = "tflab-${var.env}"
tags = {
Project = "tf-lab"
Env = var.env
ManagedBy = "terraform"
}
# derived maps for for_each
subnets = {
for i, az in var.azs : az => cidrsubnet(var.vpc_cidr, 8, i)
}
}
provider "aws" {
region = "ap-northeast-1"
default_tags {
tags = local.tags # applied to every taggable resource
}
}
resource "aws_s3_bucket" "logs" {
bucket = "${local.name}-logs-${var.account_id}"
}local.<name> (singular), declare in locals {} (plural).locals.tf if shared.default_tags (AWS) beats repeating tags = local.tags on every resource; per-resource tags merge on top.Root outputs are printed after apply and readable with terraform output; child module outputs are how modules pass data to each other.
output "param_names" {
description = "SSM parameter names by key."
value = { for k, p in aws_ssm_parameter.p : k => p.name }
}
output "bucket_arn" {
value = aws_s3_bucket.logs.arn
}
output "db_password" {
value = random_password.db.result
sensitive = true # required when the value is sensitive
}
# Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
# Outputs:
# bucket_arn = "arn:aws:s3:::tflab-dev-logs-123456789012"
# db_password = <sensitive>
# param_names = {
# "greeting" = "/tflab/dev/greeting"
# }sensitive = true, or plan fails.terraform output -json is the stable way for scripts to read them.precondition blocks inside an output check guarantees (ch 3).References, conditionals, for expressions, splats and the built-in function library.
Everything a configuration can refer to has a well-defined prefix.
| Reference | Meaning | Example |
|---|---|---|
<type>.<name>.<attr> | Managed resource attribute | aws_vpc.main.id |
data.<type>.<name>.<attr> | Data source attribute | data.aws_caller_identity.me.account_id |
var.<name> | Input variable | var.env |
local.<name> | Local value | local.tags |
module.<name>.<output> | Child module output | module.network.vpc_id |
each.key, each.value | Current element in for_each | each.value.cidr |
count.index | Current index in count | count.index |
path.module, path.root, path.cwd | Filesystem paths | "${path.module}/templates/x.tftpl" |
terraform.workspace | Current CLI workspace name | "default" (ch 4) |
self | The resource itself (provisioners, postconditions) | self.arn |
ephemeral.<type>.<name> | Ephemeral resource attribute | ephemeral.aws_secretsmanager_secret_version.db.secret_string |
These three cover most "logic" you need in HCL. Try each in terraform console.
# conditional: both results must have compatible types
instance_type = var.env == "prod" ? "m7g.large" : "t4g.small"
# for over a list → list
[for s in ["a", "b"] : upper(s)]
# ["A", "B"]
# for over a map → map, with a filter
{ for k, v in { a = 1, b = 2 } : k => v * 10 if v > 1 }
# { "b" = 20 }
# group values with "..." when keys repeat
{ for u in var.users : u.team => u.name... }
# { "core" = ["ann", "bo"], "web" = ["cy"] }
# splat: one attribute from every element
[{ n = "a" }, { n = "b" }][*].n
# ["a", "b"]
aws_subnet.private[*].id # with count
values(aws_subnet.private)[*].id # with for_each (a map)cond ? "a" : 1 fails: results need one type (string vs number).[*] does not work on a for_each resource (a map): use values(...)[*] or a for expression.for that builds a map fails on duplicate keys unless you use the grouping ... form.if inside for to filter, not a conditional that yields null elements.(known after apply) is unknown too — fine for arguments, not for count/for_each.There are no user-defined functions in HCL; the built-in library (plus provider-defined functions since 1.8) covers it. Results below are real terraform console output.
| Function | Example | Result |
|---|---|---|
merge | merge({a=1}, {a=2, b=3}) | { a = 2, b = 3 } (later wins) |
lookup | lookup({a=1}, "b", 9) | 9 (default for missing key) |
try | try(tonumber("x"), 0) | 0 (first expression without error) |
can | can(cidrhost("10.0.0.0/33", 1)) | false (great in validation) |
coalesce | coalesce("", "b") | "b" (first non-null, non-empty) |
one | one([]) | null (0 or 1 element → value; more → error) |
cidrsubnet | cidrsubnet("10.10.0.0/16", 8, 1) | "10.10.1.0/24" |
cidrsubnets | cidrsubnets("10.10.0.0/16", 8, 8, 4) | ["10.10.0.0/24", "10.10.1.0/24", "10.10.16.0/20"] |
format | format("%s-%03d", "tflab", 7) | "tflab-007" |
jsonencode | jsonencode({a=[1,2]}) | "{\"a\":[1,2]}" |
element | element(["a","b"], 3) | "b" (wraps around — usually a bug) |
terraform consoleThe console evaluates expressions against your configuration and current state. It is the fastest way to debug a for expression.
$ terraform console -var-file=dev.tfvars
> keys(var.params)
tolist([
"greeting",
"owner",
])
> { for k, v in var.params : upper(k) => v }
{
"GREETING" = "hello"
"OWNER" = "platform"
}
> var.params["missing"]
Error: Invalid index
The given key does not identify an element in this
collection value.
> try(var.params["missing"], "n/a")
"n/a"
> type(var.bucket.lifecycle_rules)
list(
object({
days: number,
id: string,
prefix: string,
}),
)echo 'cidrsubnet("10.0.0.0/16", 4, 2)' | terraform console.aws_vpc.main.id shows the real ID after apply.-var-file you plan with, or required variables will be prompted.provider::<name>::<fn>(…), e.g. provider::aws::arn_parse(arn) with AWS provider 5.40+.Create many instances from one block, and keep their addresses stable.
count and for_eachBoth meta-arguments create several instances of one resource or module. They differ in how instances are addressed.
# count: instances are numbered
resource "aws_ssm_parameter" "by_index" {
count = length(var.names)
name = "/tflab/dev/${var.names[count.index]}"
type = "String"
value = "x"
}
# aws_ssm_parameter.by_index[0], [1], [2]
# for_each: instances are keyed
resource "aws_ssm_parameter" "by_key" {
for_each = var.params # map or set of strings
name = "/tflab/dev/${each.key}"
type = "String"
value = each.value
}
# aws_ssm_parameter.by_key["greeting"], ["owner"]
# a list of objects → map keyed by a unique attribute
for_each = { for s in var.subnets : s.name => s }for_each** for anything with an identity (names, AZs, users): adding or removing one item touches only that item.count** for "N identical copies" or an on/off switch: count = var.enabled ? 1 : 0.for_each accepts a map or a set of strings; convert lists with toset() or a for expression.aws_x.y[0].id (count) vs aws_x.y["key"].id (for_each); one switched-on instance: one(aws_x.y[*].id).count → for_each later moves addresses: use moved blocks (ch 5).count over a list hurts: a real planRemove the first element of ["a", "b", "c"]. With count every later index shifts; with for_each only the removed key goes.
variable "names" {
type = list(string)
default = ["a", "b", "c"]
}
resource "random_pet" "c" { # count
count = length(var.names)
prefix = var.names[count.index]
}
resource "random_pet" "f" { # for_each
for_each = toset(var.names)
prefix = each.key
}
$ terraform plan -var 'names=["b","c"]' # random_pet.c[0] must be replaced
~ prefix = "a" -> "b" # forces replacement
# random_pet.c[1] must be replaced
~ prefix = "b" -> "c" # forces replacement
# random_pet.c[2] will be destroyed
# (because index [2] is out of range for count)
- prefix = "c" -> null
# random_pet.f["a"] will be destroyed
# (because key ["a"] is not in for_each map)
- prefix = "a" -> null
Plan: 2 to add, 0 to change, 4 to destroy.With real resources (subnets, users, buckets) the count version replaces two healthy objects. Also note the type = list(string): without it, -var 'names=[…]' arrives as a plain string and both blocks fail.
Terraform must know every instance key during plan. Keys derived from values computed at apply time cannot work.
resource "random_pet" "a" {}
resource "random_pet" "b" {
for_each = toset([random_pet.a.id]) # unknown at plan
}
$ terraform plan
Error: Invalid for_each argument
on main.tf line 9, in resource "random_pet" "b":
9: for_each = toset([random_pet.a.id])
├────────────────
│ random_pet.a.id is a string, known only after apply
The "for_each" set includes values derived from resource
attributes that cannot be determined until apply, ...for_each = { for n in var.names : n => aws_x.y[n].id } is fine because the keys are known.-target to create the dependency first works as a one-off, not as a design (ch 8).count: the number must be known at plan time.count/for_each either (section 08).dynamic blocks: repeat a nested blockSome resources take repeated nested blocks (lifecycle rules, ingress rules). dynamic generates them from a collection.
resource "aws_s3_bucket_lifecycle_configuration" "b" {
count = length(var.bucket.lifecycle_rules) > 0 ? 1 : 0
bucket = aws_s3_bucket.b.id
dynamic "rule" { # block type to generate
for_each = var.bucket.lifecycle_rules
content {
id = rule.value.id # iterator = block name
status = "Enabled"
filter {
prefix = rule.value.prefix
}
expiration {
days = rule.value.days
}
}
}
}rule.key, rule.value); rename with iterator = r.aws_vpc_security_group_ingress_rule instead of ingress {} blocks inside aws_security_group.count guard above.Keep secrets out of state and plan files, not just out of the terminal.
sensitive only hides outputBefore Terraform 1.10, any secret that touched a resource argument was written to state in plain text. sensitive = true only redacts the CLI display.
sensitive(sensitive value).terraform show -json or by anyone who can read the state bucket.nonsensitive() removes the mark (use only for values you know are not secret, such as a hash).ephemeral (1.10+)locals derived from them, and ephemeral resources.*_wo accept ephemeral values and are not stored.*_wo_version number tells it when to send a new value.aws_ssm_parameter.value_wo, aws_db_instance.password_wo, aws_secretsmanager_secret_version.secret_string_wo.Pass a secret from the environment, write it to SSM, and keep it out of state. Validated with Terraform 1.15 and AWS provider 6.x.
variable "db_password" {
type = string
sensitive = true
ephemeral = true # never persisted
}
variable "manage_secret" { type = bool }
resource "aws_ssm_parameter" "db_password" {
count = var.manage_secret ? 1 : 0
name = "/tflab/dev/db-password"
type = "SecureString"
value_wo = var.db_password
value_wo_version = 1 # bump to 2 to push a new value
}
# export TF_VAR_db_password="$(openssl rand -base64 24)"
# terraform apply
# state: value_wo = null, value_wo_version = 1# WRONG: count from an ephemeral value
resource "aws_ssm_parameter" "db_password" {
count = var.db_password == null ? 0 : 1 ...
$ terraform validate
Error: Invalid count argument
on main.tf line 14, in resource "aws_ssm_parameter"
"db_password":
14: count = var.db_password == null ? 0 : 1
The given "count" is derived from an ephemeral value,
which means that Terraform cannot persist it between
plan/apply rounds. Use only non-ephemeral values to
specify the number of resource instances.Ephemeral resources go one step further: ephemeral "aws_secretsmanager_secret_version" "db" { … } reads a secret at run time without storing it. Chapter 12 builds the full secret flow with these pieces.
Use this table when deciding how a secret flows through a configuration.
| Mechanism | CLI output | Plan file | State | Typical use |
|---|---|---|---|---|
| Plain value | Shown | Stored | Stored | Non-secret settings |
sensitive = true | Redacted | Stored | Stored | Values that are not secret enough to avoid state, e.g. generated names |
ephemeral = true variable | Redacted if also sensitive | Not stored | Not stored | Credentials passed in at run time |
ephemeral resource | — | Not stored | Not stored | Read a secret from Secrets Manager / generate a temporary token |
Write-only argument *_wo | Redacted | Not stored | Stored as null; only *_wo_version kept | Set a password on a resource without persisting it |
Provider-managed secret (manage_master_user_password = true on RDS) | — | — | Only the secret ARN | Let AWS generate and rotate it (ch 12) |
OpenTofu 1.11 added ephemeral values and write-only attributes as well; OpenTofu also offers client-side state encryption (ch 4), which Terraform does not have.
Most of these pass validate and only show up in a plan — or after apply.
-var values arrive as strings.map(any) inputs that accept typos silently; use object() with optional().terraform.tfvars committed to Git.sensitive to keep a secret out of state.count over a list of named things → mass replacement when the list changes.for_each keys from computed values → "Invalid for_each argument".element() wrapping around silently instead of an index error.dynamic blocks no reviewer can read.for_each over maps keyed by stable names.jsonencode() for policies, never string templates.terraform console before committing.Write the expression or block from memory, then check it in terraform console.
optional(number) default to, and how do you set another default?-var 'names=["a"]'?az => cidr from a list of AZs with cidrsubnet.[*] fail on a for_each resource, and what do you use instead?... suffix do in a for expression?try and can differ?count vs for_each.for_each = toset([random_pet.a.id])?sensitive value stored? An ephemeral one?*_wo_version for?Practice: Lab 2 in Hands-on labs — HCL drills in terraform console, then a config with validated variables, optional() objects, for_each, a dynamic block, templatefile and shared tags.
How Terraform talks to AWS, how resources and data sources are addressed and ordered, and how to control create, update and replace.
Plugins that translate resource blocks into API calls, pinned by version and by hash.
Terraform core knows nothing about AWS. It builds the graph and the plan; the provider plugin knows the resource schemas and calls the APIs.
hashicorp/aws = registry.terraform.io/hashicorp/aws (hostname/namespace/type).aws_vpc), data source, function and (1.14+) list resource and action.terraform providers schema -json | jq.Every root and child module declares which providers it needs. Root modules pin tight; reusable modules state the minimum they support.
# versions.tf
terraform {
required_version = ">= 1.14"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 6.0" # >= 6.0, < 7.0
}
random = {
source = "hashicorp/random"
version = "~> 3.6" # >= 3.6, < 4.0
}
}
}
# terraform version
# Terraform v1.17.0
# + provider registry.terraform.io/hashicorp/aws v6.72.0| Constraint | Allows |
|---|---|
= 6.2.0 or 6.2.0 | Exactly 6.2.0 |
>= 6.0 | 6.0 and anything newer, including 7.x |
~> 6.0 | ≥ 6.0 and < 7.0 (rightmost part may grow) |
~> 6.2.0 | ≥ 6.2.0 and < 6.3.0 (patches only) |
>= 6.10, < 6.30 | A range; combine with commas |
!= 6.15.0 | Exclude one known-bad release |
Reusable modules: >= 6.0 (a floor, no ceiling) so callers can upgrade. Root modules: ~> 6.0 plus the lock file, which pins the exact version.
Constraints say what is allowed; the lock file records what was chosen, with checksums. Commit it.
# .terraform.lock.hcl (written by terraform init)
provider "registry.terraform.io/hashicorp/aws" {
version = "6.72.0"
constraints = "~> 6.0"
hashes = [
"h1:mOeaj5DLQjxSyaVbYiilZh4htt9Yo2y0jngHnRucJtw=",
"zh:2ffb6b2caaf320b61b406cff192d6e4303b64e32...",
"zh:33cd70e1b3d0f80bfdf07480139142f81a3daec2...",
# ... one zh: per platform package in the release
]
}init installs the locked version even if newer ones match the constraint.init -upgrade picks the newest allowed version and rewrites the file.h1: = hash of the unpacked package for platforms you installed on; zh: = zip hashes from the registry for every platform.init: tampering or a wrong mirror.terraform init -lockfile=readonly fails instead of silently changing the file.version or a Git ref, ch 6).Laptop on darwin_arm64, CI on linux_amd64? Pre-record both: terraform providers lock -platform=darwin_arm64 -platform=linux_amd64 (Lab 3).
Provider blocks belong in root modules. Credentials come from the environment, never from code.
# providers.tf (root module live/dev/network)
provider "aws" {
region = var.region # ap-northeast-1
default_tags { # added to every taggable resource
tags = {
Project = "tf-lab"
Env = var.env
ManagedBy = "terraform"
}
}
ignore_tags { # tags someone else manages
key_prefixes = ["kubernetes.io/"]
}
# assume_role { # cross-account (ch 10)
# role_arn = "arn:aws:iam::<prod-acct>:role/tflab-deploy"
# session_name = "tf-lab"
# }
}access_key… — avoid).AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY/AWS_SESSION_TOKEN, AWS_PROFILE.aws sts get-caller-identity before every apply.default_tags merge with resource tags; on a key conflict the resource tag wins. Read tags_all in state to see the merged result.
Before v6, every extra region needed its own aliased provider. AWS provider v6 adds a region argument to most resources.
# classic: an aliased provider configuration
provider "aws" {
alias = "use1"
region = "us-east-1"
}
resource "aws_s3_bucket" "logs_use1" {
provider = aws.use1 # pick the configuration
bucket = "tflab-logs-use1-<acct>"
}
# AWS provider v6: region per resource, same provider
resource "aws_s3_bucket" "logs_v6" {
region = "us-east-1" # defaults to the provider region
bucket = "tflab-logs-v6-<acct>"
}
# modules receive aliased providers explicitly
module "dr" {
source = "../../modules/app-bucket"
providers = { aws = aws.use1 }
}region argument; fewer provider blocks and simpler modules.assume_role.region argument.region on an existing resource forces replacement: it is a different resource in a different place.Addresses, computed attributes, data sources and the dependency graph.
A resource block declares one or more remote objects. Its address is how state, plans, -target, moved and import refer to it.
resource "aws_subnet" "public" { # TYPE "NAME"
for_each = { for i, az in local.azs : az => i } # meta-argument
vpc_id = aws_vpc.this.id # argument (reference)
availability_zone = each.key
cidr_block = cidrsubnet(var.vpc_cidr, 8, each.value)
}
# attributes you can read after creation:
# aws_subnet.public["ap-northeast-1a"].id (computed)
# aws_subnet.public["ap-northeast-1a"].arn (computed)
# aws_subnet.public["ap-northeast-1a"].cidr_block| Address | Means |
|---|---|
aws_vpc.this | Single instance |
aws_subnet.public["ap-northeast-1a"] | One for_each instance (key) |
aws_instance.web[0] | One count instance (index) |
aws_subnet.public | All instances of the resource |
module.network.aws_vpc.this | Inside a child module |
module.app["dev"].aws_s3_bucket.this | Inside a for_each module |
data.aws_caller_identity.current | A data source |
In a shell, quote addresses with brackets: terraform state show 'aws_subnet.public["ap-northeast-1a"]'.
Arguments are what you set; attributes are what the provider reports. Some attributes only exist once the API has created the object.
cidr_block).id, arn, owner_id).enable_dns_support).password_wo) are never stored at all (ch 2, ch 12).$ terraform plan
# aws_vpc.this will be created
+ resource "aws_vpc" "this" {
+ arn = (known after apply)
+ cidr_block = "10.10.0.0/16"
+ enable_dns_hostnames = true
+ id = (known after apply)
+ tags_all = {
+ "Env" = "dev"
+ "ManagedBy" = "terraform"
+ "Name" = "tflab-dev"
+ "Project" = "tf-lab"
}
}
# aws_subnet.public["ap-northeast-1a"] will be created
+ resource "aws_subnet" "public" {
+ vpc_id = (known after apply)Unknown values propagate: anything derived from an unknown attribute is also unknown at plan time. That limits count/for_each keys (ch 2) and is a common review blind spot (ch 8).
A data block looks something up — in AWS, in another state, or computed locally — and exposes attributes like a resource does.
data "aws_availability_zones" "available" {
state = "available"
}
data "aws_caller_identity" "current" {}
data "aws_ssm_parameter" "vpc_id" { # from network (ch 4)
name = "/tflab/dev/network/vpc_id"
}
data "aws_iam_policy_document" "read" { # local, no API call
statement {
actions = ["s3:GetObject"]
resources = ["${aws_s3_bucket.app.arn}/*"]
}
}
locals {
azs = slice(data.aws_availability_zones.available.names, 0, 2)
acct = data.aws_caller_identity.current.account_id
vpc_id = data.aws_ssm_parameter.vpc_id.insecure_value
}depends_on), the read is deferred to apply: the plan shows <= and "(known after apply)".aws_ssm_parameter.value is always marked sensitive; use insecure_value for plain String parameters.terraform_remote_state across teams (ch 4).Terraform walks a graph. An expression that references another object adds an edge; independent nodes run in parallel (10 by default).
aws_iam_role_policy_attachment.x.policy_arn).depends_on makes everything in it wait: use sparingly.The same ordering problem, solved badly and well.
# Hidden dependency: the Lambda needs the policy to be
# ATTACHED before it runs, but only references the role.
resource "aws_iam_role_policy_attachment" "logs" {
role = aws_iam_role.fn.name
policy_arn = local.basic_execution_policy_arn
}
resource "aws_lambda_function" "fn" {
function_name = "tflab-fn"
role = aws_iam_role.fn.arn # implicit: role only
# ...
depends_on = [aws_iam_role_policy_attachment.logs]
}# Unnecessary depends_on: the reference already
# creates the edge, depends_on only adds noise.
resource "aws_subnet" "a" {
vpc_id = aws_vpc.this.id
depends_on = [aws_vpc.this] # remove this
}
# Wrong tool: ordering between root modules /
# separate states. depends_on cannot cross states;
# order the applies in CI instead (ch 10, ch 11).Rule of thumb: if you add depends_on, write a comment saying which hidden dependency it encodes. If you cannot name one, delete it.
Create, update in place, replace — and the meta-arguments that change the default behavior.
For each resource the provider decides whether a change can be applied in place or needs a new object.
| Symbol | Action | Typical cause |
|---|---|---|
+ | create | New block, new for_each key |
~ | update in place | Tags, description of some resources, most settings |
-/+ | destroy, then create | An argument that "forces replacement" (ForceNew) |
+/- | create, then destroy | Same, with create_before_destroy |
- | destroy | Block removed, key removed, terraform destroy |
<= | read (data source) | Data source deferred to apply |
# aws_security_group.web must be replaced
+/- resource "aws_security_group" "web" {
~ description = "HTTPS from the internet"
-> "HTTPS and HTTP" # forces replacement
~ id = "sg-0a1b..." -> (known after apply)
name_prefix = "tflab-dev-web-"
# (6 unchanged attributes hidden)
}
Plan: 1 to add, 0 to change, 1 to destroy.Which arguments force replacement is provider knowledge (marked in the docs and shown as "# forces replacement"). Always read it in the plan, never assume (ch 8).
Two guards for replacements: avoid downtime, and refuse to destroy what must never go away.
resource "aws_security_group" "web" {
name_prefix = "tflab-dev-web-" # unique names: old and
vpc_id = aws_vpc.this.id # new must coexist
description = "HTTPS from the internet"
lifecycle {
create_before_destroy = true
}
}
resource "aws_s3_bucket" "data" {
bucket = "tflab-dev-data-<acct>"
lifecycle {
prevent_destroy = true
}
}
# $ terraform destroy
# Error: Instance cannot be destroyed
# Resource aws_s3_bucket.data has lifecycle.prevent_destroy
# set, but the plan calls for this resource to be destroyed.create_before_destroy needs names that can coexist: use name_prefix or a random suffix, or the create fails with "already exists".prevent_destroy only works while the block is in the configuration: delete the block (or the module call) and the guard disappears with it.prevent_destroy (ch 13).Tell Terraform which differences to ignore, and which changes elsewhere should force a fresh object.
resource "aws_security_group" "web" {
# ...
lifecycle {
# another team adds an Owner tag in the console
ignore_changes = [tags["Owner"]]
}
}
# replace a resource when something else changes
resource "terraform_data" "config_rev" {
input = var.config_revision # plain value → resource
}
resource "aws_cloudwatch_log_group" "app" {
name = "/tflab/dev/app"
lifecycle {
replace_triggered_by = [terraform_data.config_rev]
}
}ignore_changes takes attribute paths (desired_capacity of an autoscaled group, a tag key), or all (rarely right: no updates at all).replace_triggered_by accepts references to managed resources or their attributes, not variables — wrap values in terraform_data.terraform apply -replace='aws_security_group.web' (replaces the old taint).ignore_changes hides drift: every entry needs a reason in a comment.Long-running resources, plain values that need a lifecycle, and the CLI flags that change one apply.
resource "aws_db_instance" "main" {
# (not in labs: costs money)
timeouts { # keys vary by resource
create = "60m"
update = "80m"
delete = "60m"
}
}
resource "terraform_data" "bootstrap" {
input = var.schema_version
triggers_replace = [aws_vpc.this.id]
}null_resource.input → output attribute: a value that follows the resource lifecycle.triggers_replace: replace when these values change.provisioner blocks (last resort) and a target for replace_triggered_by.-replace=ADDR: force replacement of one instance.-target=ADDR: limit the run to some resources — emergencies only; the plan is partial (ch 8).-parallelism=N: concurrent operations (default 10).-refresh=false: skip reading real state — faster, blinder.-lock-timeout=60s: wait for the state lock (ch 4).Assumptions written as code: validate inputs, guard resources, and keep checking after apply.
Reject bad input before anything is planned. Since Terraform 1.9 a validation may also reference other variables and objects.
variable "env" {
type = string
default = "dev"
validation {
condition = contains(["dev", "prod"], var.env)
error_message = "env must be dev or prod."
}
}
variable "vpc_cidr" {
type = string
default = "10.10.0.0/16"
validation {
condition = can(cidrnetmask(var.vpc_cidr))
error_message = "vpc_cidr must be a valid IPv4 CIDR."
}
}
# $ terraform plan -var vpc_cidr=10.10.0.0/33
# Error: Invalid value for variable
# vpc_cidr must be a valid IPv4 CIDR.contains([...], var.x).can(regex("^[a-z0-9-]+$", var.name)), can(cidrnetmask(...)).var.retention_days >= 1 && var.retention_days <= 3653.validation blocks → multiple specific messages.can() turns an error into false; try() returns a fallback value.Guards on a resource, data source or output: preconditions check assumptions before the change; postconditions check guarantees after it.
locals {
azs = slice(data.aws_availability_zones.available.names,
0, min(2, length(
data.aws_availability_zones.available.names)))
}
resource "aws_vpc" "this" {
cidr_block = var.vpc_cidr
enable_dns_support = true
enable_dns_hostnames = true
lifecycle {
precondition { # evaluated before plan of this resource
condition = length(local.azs) == 2
error_message = "Need 2 available AZs in ${var.region}."
}
postcondition { # evaluated after apply (self = result)
condition = self.enable_dns_hostnames
error_message = "DNS hostnames must be enabled."
}
}
}self exists only in postconditions. output blocks accept a precondition too.A check block (1.5+) runs at the end of every plan and apply. A failed assertion is a warning: it never blocks a change.
check "two_public_subnets" {
assert {
condition = length(aws_subnet.public) == 2
error_message = "Expected 2 public subnets."
}
}
# scoped data source: only visible inside the check,
# and its errors become warnings too
check "vpc_dns" {
data "aws_vpc" "check" {
id = aws_vpc.this.id
}
assert {
condition = data.aws_vpc.check.enable_dns_support
error_message = "DNS support is off on ${aws_vpc.this.id}."
}
}
# │ Warning: Check block assertion failed
# │ on main.tf line 80, in check "two_public_subnets":
# │ Expected 2 public subnets.plan in CI shows them (ch 11).Pick the weakest tool that still protects you: warnings for signals, errors for invariants.
| Guard | Lives in | Evaluated | On failure | Typical use |
|---|---|---|---|---|
validation | variable block | Before planning (as soon as the value is known) | Error | Allowed values, formats, ranges |
precondition | lifecycle of resource / data, or output | Before the object is planned or changed | Error | Assumptions about inputs and other objects |
postcondition | lifecycle of resource / data | After the object is read or applied | Error (dependents skipped) | Guarantees about the result (self) |
check / assert | Top-level check block | End of every plan and apply | Warning | Health, expiry, drift signals |
assert in tests | run block in .tftest.hcl | terraform test only | Test failure | Module behavior (ch 7) |
| Policy (Checkov, Trivy, conftest, Terraform Policy) | Outside the module | In CI on code or plan JSON | Pipeline failure | Organization rules (ch 9) |
Each of these has deleted something real or blocked a release.
for_each or count.provider "aws" arguments or tfvars.create_before_destroy with a fixed name → "already exists" on replacement.prevent_destroy protects data after the block was removed.ignore_changes = all to silence a diff nobody understood.depends_on everywhere "to be safe" → slower, vaguer plans..terraform.lock.hcl; init -lockfile=readonly in CI.name_prefix for anything that may be replaced.must be replaced before approving (ch 8).Answer out loud with the exact argument, symbol or command.
~> 6.2.0 allow? And ~> 6.0?h1: and a zh: hash?init behave with and without -upgrade?ap-northeast-1c instance of aws_subnet.public inside module.network.depends_on on a module expensive?create_before_destroy need name_prefix?prevent_destroy can be bypassed.check block do that a normal one cannot?Practice: Lab 3 in Hands-on labs — build live/dev/network with provider v6 and default_tags, lock two platforms, then provoke replacements, ignored tags, prevent_destroy and a failing check.
What Terraform remembers between runs, where to keep it safely, how to inspect and repair it, and how to split it.
The map between your configuration and the real objects in AWS.
Configuration says what should exist; AWS says what does exist. State is the record that binds each resource address to one real object.
aws_vpc.this is vpc-0a1b2c3d. Without it Terraform would create a second VPC.terraform output, remote state).JSON, format version 4. Read it to understand it; never edit it by hand.
{
"version": 4,
"terraform_version": "1.17.0",
"serial": 12,
"lineage": "3f9c2b1e-8d7a-4c55-9e0f-2a6b7c8d9e10",
"outputs": {
"vpc_id": {"value": "vpc-0a1b2c3d", "type": "string"}
},
"resources": [
{
"mode": "managed",
"type": "aws_vpc",
"name": "this",
"provider": "provider[\"registry…/hashicorp/aws\"]",
"instances": [
{
"schema_version": 1,
"attributes": {
"id": "vpc-0a1b2c3d",
"cidr_block": "10.10.0.0/16"
},
"sensitive_attributes": []
}
]
}
]
}count/for_each instances ("ap-northeast-1a").terraform state push refuses a lower serial or another lineage unless -force (ch 4.3).sensitive = true hides values in CLI output. It does not encrypt them in state.
random_password.result).aws_ssm_parameter of type SecureString, aws_secretsmanager_secret_version.secret_string.tls_private_key, access keys from aws_iam_access_key.data.aws_ssm_parameter.x.value.tfplan) contain the same values.terraform.tfstate or *.tfplan to Git (.gitignore).password_wo) never reach state (ch 2, ch 12).manage_master_user_password = true.Every plan refreshes; every apply writes. Knowing when helps you reason about locks, drift and partial failures.
-out=tfplan for review.force-unlock.Partial failure: resources that succeeded are already in state, failed ones are not (or are marked tainted). Fix the cause and run plan again; never "clean up" by editing state.
Every legitimate change to state has a command or a block. Hand edits break serial, lineage, schema versions and dependencies.
| You want to… | Use | Chapter |
|---|---|---|
| Rename or move a resource / into a module | moved block (or terraform state mv) | ch 5 |
| Adopt an existing AWS object | import block (or terraform import) | ch 5 |
| Stop managing an object without destroying it | removed block with destroy = false (or state rm) | ch 5 |
| Accept changes made outside Terraform into state | terraform apply -refresh-only | ch 4.3 |
| Switch a resource to another provider source | terraform state replace-provider | ch 13 |
| Recover a broken or deleted state | Restore an older S3 object version; state push as a last resort | ch 4.3 |
| Read values for scripts | terraform output -json, terraform show -json | ch 4.3 |
Shared, durable, encrypted state with a lock so two applies never interleave.
A local terraform.tfstate is fine for a tutorial and wrong for anything a team or CI touches.
bootstrap/ — the configuration that creates the state bucket (Lab 0).terraform { backend "…" {} } block of each root module.backend block cannot use variables or locals: values are literal or come from -backend-config.terraform init again.cloud block instead of backend "remote".Since Terraform 1.10, S3 can lock by itself (use_lockfile). The DynamoDB table is deprecated.
# backend.tf (live/dev/network)
terraform {
backend "s3" {
# bucket comes from -backend-config (one per account)
key = "dev/network/terraform.tfstate"
region = "ap-northeast-1"
encrypt = true
use_lockfile = true # default false: set it!
}
}
# terraform init -backend-config="bucket=$TF_STATE_BUCKET"
# objects: dev/network/terraform.tfstate
# dev/network/terraform.tfstate.tflock (while locked)| Argument | Meaning |
|---|---|
bucket, key, region | Where the state object lives |
encrypt | Server-side encryption of the state object |
kms_key_id | Use SSE-KMS with this key |
use_lockfile | Lock with a <key>.tflock object (default false) |
dynamodb_table | Legacy DynamoDB locking — deprecated |
assume_role { role_arn = … } | Read/write state with another role |
The lock object is created with an S3 conditional write, so only one writer wins. Permissions: get/put/delete on <key>.tflock, get/put on the state key, s3:ListBucket.
Both mechanisms can be enabled at once; a run then needs both locks. That makes the switch safe while old and new runners coexist.
# step 1: both locks (all runners on Terraform >= 1.11)
terraform {
backend "s3" {
bucket = "tflab-state-<acct>"
key = "dev/network/terraform.tfstate"
region = "ap-northeast-1"
encrypt = true
use_lockfile = true
dynamodb_table = "tflab-locks" # deprecated, transitional
}
}
# step 2: remove dynamodb_table once nothing older runs
# step 3: terraform init -reconfigure (state is unchanged)
# step 4: delete the DynamoDB table-reconfigure re-initializes the backend without migrating state; use it when only lock settings change.use_lockfile on its S3 backend (1.10+).Backend settings are resolved at init time and cached in .terraform/terraform.tfstate (the backend config, not your state).
# partial configuration: keep account-specific values out of code
cat > backend.dev.hcl <<'EOF'
bucket = "tflab-state-123456789012"
EOF
terraform init -backend-config=backend.dev.hcl
# or: -backend-config="bucket=$TF_STATE_BUCKET"
# local → S3 (or S3 key A → key B): copy the state
terraform init -migrate-state -backend-config=backend.dev.hcl
# Do you want to copy existing state to the new backend?
# Enter a value: yes
# non-interactive (CI) migration
terraform init -migrate-state -force-copy ...
# backend settings changed, state should NOT be copied
terraform init -reconfigure -backend-config=backend.dev.hclterraform.tfstate** (keep a backup first) so nobody applies against it.A lock error is a feature: someone (or something) is changing the same state right now.
$ terraform plan
│ Error: Error acquiring the state lock
│
│ Error message: ... (exact text varies by version)
│ Lock Info:
│ ID: 5b3c1a2e-7f6d-4e3b-9a1c-0d2e3f4a5b6c
│ Path: tflab-state-<acct>/dev/network/terraform.tfstate
│ Operation: OperationTypeApply
│ Who: alice@laptop
│ Version: 1.17.0
│ Created: 2026-11-09 03:12:45 +0000 UTC
# wait up to a minute for the lock instead of failing
terraform plan -lock-timeout=60s
# ONLY when the holder is certainly dead (crashed runner):
terraform force-unlock 5b3c1a2e-7f6d-4e3b-9a1c-0d2e3f4a5b6cforce-unlock a lock held by a live process: two writers = lost updates.-lock=false exists; do not use it on shared state.concurrency group (ch 11) so lock errors become rare.Same idea, different store. Pick the one your platform already secures and backs up.
| Backend | Locking | Notes |
|---|---|---|
s3 | use_lockfile (or legacy DynamoDB) | This deck; versioning for history |
cloud block (HCP Terraform / Terraform Enterprise) | Built in | Remote runs, UI, policies, private registry |
gcs | Built in | Google Cloud Storage |
azurerm | Blob lease | Azure Storage |
pg | Advisory locks | Postgres table; schema configurable |
kubernetes | Lease | Secret per state; 1 MiB size limit |
# HCP Terraform: cloud block instead of backend
terraform {
cloud {
organization = "my-org"
workspaces {
name = "tflab-dev-network"
}
}
}
# terraform login (stores an API token)
# terraform init (connects the workspace)
# plans/applies then run remotely by defaultThe state subcommands, JSON views, refresh-only runs and recovery from S3 versions.
Read-only commands are always safe. Start every investigation with them.
$ terraform state list
data.aws_availability_zones.available
aws_internet_gateway.this
aws_route_table.public
aws_route_table_association.public["ap-northeast-1a"]
aws_route_table_association.public["ap-northeast-1c"]
aws_security_group.web
aws_subnet.private["ap-northeast-1a"]
aws_subnet.private["ap-northeast-1c"]
aws_subnet.public["ap-northeast-1a"]
aws_subnet.public["ap-northeast-1c"]
aws_vpc.this
aws_vpc_security_group_ingress_rule.https
$ terraform state list 'aws_subnet.public' # filter by address
$ terraform state show aws_vpc.this
# aws_vpc.this:
resource "aws_vpc" "this" {
cidr_block = "10.10.0.0/16"
id = "vpc-0a1b2c3d"
...
}# outputs for scripts
terraform output -raw vpc_id
terraform output -json | jq -r '.public_subnet_ids.value[]'
# whole state as JSON (stable, documented format)
terraform show -json | jq -r '
.values.root_module.resources[]
| select(.type == "aws_subnet")
| "\(.address) \(.values.cidr_block)"'
# aws_subnet.private["ap-northeast-1a"] 10.10.10.0/24
# aws_subnet.public["ap-northeast-1a"] 10.10.0.0/24
# ...
# raw remote state (serial and lineage included)
terraform state pull | jq '{serial, lineage}'
# { "serial": 12, "lineage": "3f9c2b1e-..." }Prefer terraform show -json over parsing terraform.tfstate directly: its format is documented and stable across Terraform versions.
Imperative tools for emergencies. Each writes a backup, but on a remote backend run state pull > backup.tfstate yourself first.
terraform state pull > backup-$(date +%s).tfstate
# rename without destroy/create
terraform state mv aws_security_group.web aws_security_group.app
# Move "aws_security_group.web" to "aws_security_group.app"
# Successfully moved 1 object(s).
# → now rename the block in code too, or the next plan
# wants to destroy .app and create .web
# forget an object (it keeps running in AWS)
terraform state rm 'aws_subnet.private["ap-northeast-1c"]'
# overwrite remote state with a file (last resort)
terraform state push restored.tfstate
# refuses a different lineage or a higher remote serial
terraform state push -force restored.tfstatemoved instead of state mv: reviewed in a PR, applied by CI, works for every user of a module.removed instead of state rm: same benefits, explicit destroy = false.import blocks instead of terraform import: plannable.terraform plan right after: the target is No changes.Drift = AWS differs from state because someone changed it outside Terraform. A normal plan proposes to revert it; refresh-only accepts it.
# someone added a tag in the console
$ terraform plan -refresh-only
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of
Terraform since the last "terraform apply":
# aws_vpc.this has changed
~ resource "aws_vpc" "this" {
~ tags = {
+ "CostCenter" = "1234"
"Name" = "tflab-dev"
}
# (16 unchanged attributes hidden)
}
# accept the real values into state (no AWS change)
terraform apply -refresh-only
# a normal plan would instead propose:
# ~ tags = { - "CostCenter" = "1234" ... }apply): the code is right, the console change was wrong.apply -refresh-only): records reality but the next normal plan still reverts it unless code changes.ignore_changes: only when another system owns that attribute (ch 3).terraform refresh is the deprecated alias of apply -refresh-only -auto-approve: avoid it.With versioning on the state bucket, every write is a restorable snapshot. This is why Lab 0 turns versioning on.
B=$TF_STATE_BUCKET; K=dev/network/terraform.tfstate
# 1) list versions of the state object only
# (the prefix also matches terraform.tfstate.tflock)
aws s3api list-object-versions --bucket $B --prefix $K \
--query "Versions[?Key=='$K'].[VersionId,LastModified,IsLatest]" \
--output table
# 2) inspect a candidate before restoring
aws s3api get-object --bucket $B --key $K \
--version-id <id> /tmp/candidate.tfstate
jq '{serial, terraform_version}' /tmp/candidate.tfstate
# 3) restore = copy that version over the current object
aws s3api copy-object --bucket $B --key $K \
--copy-source "$B/$K?versionId=<id>"
# 4) compare with reality
terraform plan # expect: No changesimport it (ch 5).One state per component and environment, and clean ways to pass values between them.
Everything in one state is planned, locked and can be broken together. Split along ownership and change frequency.
The app component needs the VPC ID and subnet IDs from network. How it gets them decides coupling and permissions.
| Option | Consumer needs | Pros | Cons |
|---|---|---|---|
terraform_remote_state data source | Read access to the whole producer state | No extra resources; typed outputs | Exposes every secret in that state; couples to backend layout and producer's Terraform version |
| Published values: SSM Parameter Store (tf-lab) | ssm:GetParameter on /tflab/<env>/network/* | Narrow permissions; readable by scripts and other tools; explicit contract | Producer must write them; strings only (lists as StringList) |
| Provider data sources by tag or name | Normal AWS read permissions | No dependency on the other state at all | Lookups can match zero or many objects; needs good tagging |
HCP Terraform outputs (tfe_outputs) / Stacks | Workspace access in HCP Terraform | Only outputs are shared | Tied to HCP Terraform |
tf-lab uses SSM parameters: network publishes /tflab/dev/network/vpc_id and /tflab/dev/network/private_subnet_ids; app reads them (Lab 4, ch 10).
Producer and consumer agree on parameter names, not on each other's state.
# producer: live/dev/network/ssm.tf
resource "aws_ssm_parameter" "vpc_id" {
name = "/tflab/${var.env}/network/vpc_id"
type = "String"
value = aws_vpc.this.id
}
resource "aws_ssm_parameter" "private_subnet_ids" {
name = "/tflab/${var.env}/network/private_subnet_ids"
type = "StringList"
value = join(",", [for s in aws_subnet.private : s.id])
}# consumer: live/dev/app/network.tf
data "aws_ssm_parameter" "vpc_id" {
name = "/tflab/dev/network/vpc_id"
}
data "aws_ssm_parameter" "private_subnets" {
name = "/tflab/dev/network/private_subnet_ids"
}
locals {
# value is always marked sensitive; insecure_value is not
vpc_id = data.aws_ssm_parameter.vpc_id.insecure_value
subnet_ids = split(",",
data.aws_ssm_parameter.private_subnets.insecure_value)
}
# the same contract via remote state (needs full read):
# data "terraform_remote_state" "network" {
# backend = "s3"
# config = { bucket = "tflab-state-<acct>",
# key = "dev/network/terraform.tfstate",
# region = "ap-northeast-1" }
# }
# data.terraform_remote_state.network.outputs.vpc_idA workspace is another state under the same backend and code. Useful for short-lived copies; risky as the only separation between dev and prod.
terraform workspace list # * default
terraform workspace new feature-x
terraform workspace select default
terraform workspace show
# in code
locals {
name = "tflab-${terraform.workspace}"
}
# S3 keys used:
# default → dev/network/terraform.tfstate
# feature-x → env:/feature-x/dev/network/terraform.tfstate
# (workspace_key_prefix, default "env:")apply in the wrong one is a classic outage.OpenTofu 1.7+ can encrypt state and plan files before they reach the backend. Terraform has no equivalent; it relies on backend encryption.
# OpenTofu only — Terraform rejects this block
terraform {
encryption {
key_provider "aws_kms" "state" {
kms_key_id = "alias/tflab-state"
region = "ap-northeast-1"
key_spec = "AES_256"
}
method "aes_gcm" "main" {
keys = key_provider.aws_kms.state
}
state {
method = method.aes_gcm.main
enforced = true # refuse to write unencrypted state
}
plan {
method = method.aes_gcm.main
enforced = true
}
}
}pbkdf2 (passphrase), aws_kms, gcp_kms, OpenBao.fallback method during the switch; read the OpenTofu docs.Most Terraform incidents are state incidents.
terraform.tfstate committed to Git with secrets inside.use_lockfile forgotten: it defaults to false, so there is no lock.force-unlock while a CI apply is still running.state mv without changing the code → next plan destroys and recreates.apply in the wrong workspace.terraform.tfstate after migrating.terraform_remote_state everywhere: every consumer can read every producer secret.Explain with the exact field, argument or command.
sensitive = true encrypt anything? Where do the values end up?-migrate-state vs -reconfigure?force-unlock safe?apply -refresh-only change in AWS?Practice: Lab 4 in Hands-on labs — migrate live/dev/network to S3 with use_lockfile, watch the .tflock object, use the state commands, restore an older version, and publish outputs to SSM.
Change addresses without destroying anything, bring existing resources under Terraform, let go of resources without deleting them, and handle drift.
Rename, restructure and move resources into modules while the plan shows zero changes.
Every refactoring tool in this chapter changes exactly one of these links. Knowing which one is the whole trick.
The imperative equivalents (terraform state mv, terraform import, terraform state rm) change state immediately, with no plan and no review. Prefer the blocks; keep the commands for emergencies (ch 4).
Terraform tracks objects by address. Change the address and Terraform sees one object to delete and a new one to create.
# before
resource "aws_s3_bucket" "b" {
bucket = "tflab-dev-logs-123456789012"
}
# after: a clearer name, nothing else changed
resource "aws_s3_bucket" "logs" {
bucket = "tflab-dev-logs-123456789012"
}$ terraform plan
# aws_s3_bucket.b will be destroyed
# (because aws_s3_bucket.b is not in configuration)
- resource "aws_s3_bucket" "b" { ... }
# aws_s3_bucket.logs will be created
+ resource "aws_s3_bucket" "logs" { ... }
Plan: 1 to add, 0 to change, 1 to destroy.
# for a bucket with data: destroy fails (BucketNotEmpty)
# or, with force_destroy = true, deletes your dataFor an S3 bucket the create would even fail: the name is still taken by the bucket being destroyed in the same run. For a database the result is data loss. Never accept a destroy/create pair caused only by a rename.
A moved block tells Terraform that the object at from now lives at to. The plan updates the address in state and touches nothing in AWS.
resource "aws_s3_bucket" "logs" {
bucket = "tflab-dev-logs-123456789012"
}
moved {
from = aws_s3_bucket.b
to = aws_s3_bucket.logs
}$ terraform plan
# aws_s3_bucket.b has moved to aws_s3_bucket.logs
resource "aws_s3_bucket" "logs" {
id = "tflab-dev-logs-123456789012"
# (12 unchanged attributes hidden)
}
Plan: 0 to add, 0 to change, 0 to destroy.
$ terraform apply # records the new address
$ terraform state list
aws_s3_bucket.logsIf the plan shows a move and ~ update lines, the move worked but the code also differs from reality. Split the two: ship the pure move first (zero changes), then the real change.
Addresses can be whole resources, single instances, module calls or module instances. Since Terraform 1.8, a provider can also allow moves between resource types.
| Refactoring | from | to |
|---|---|---|
| Rename a resource | aws_vpc.main | aws_vpc.this |
count → for_each (one block per instance) | aws_subnet.public[0] | aws_subnet.public["a"] |
Single resource → count/for_each | aws_s3_bucket.logs | aws_s3_bucket.logs["dev"] |
| Resource into a module | aws_vpc.this | module.network.aws_vpc.this |
| Rename a module call (all its resources) | module.vpc | module.network |
Module call → for_each module | module.network | module.network["dev"] |
| Change resource type (provider must support it) | null_resource.x | terraform_data.x |
Adding count to a single resource is an implied move (x → x[0], no block needed); adding for_each is not: without a moved block the plan destroys x and creates x["dev"]. Type changes work only where the provider implements them; otherwise use removed + import.
Indexes are positions: delete the first subnet and every later one shifts and gets replaced. Keys are names. Moving once to for_each ends the shifting for good (ch 2).
# before: positional
resource "aws_subnet" "public" {
count = 2
vpc_id = aws_vpc.this.id
cidr_block = cidrsubnet("10.10.0.0/16", 8, count.index)
availability_zone = ["ap-northeast-1a", "ap-northeast-1c"][count.index]
}
# after: keyed by AZ suffix
locals {
public_subnets = { a = "10.10.0.0/24", c = "10.10.1.0/24" }
}
resource "aws_subnet" "public" {
for_each = local.public_subnets
vpc_id = aws_vpc.this.id
cidr_block = each.value
availability_zone = "ap-northeast-1${each.key}"
}moved {
from = aws_subnet.public[0]
to = aws_subnet.public["a"]
}
moved {
from = aws_subnet.public[1]
to = aws_subnet.public["c"]
}
$ terraform plan
# aws_subnet.public[0] has moved to aws_subnet.public["a"]
# aws_subnet.public[1] has moved to aws_subnet.public["c"]
Plan: 0 to add, 0 to change, 0 to destroy.
# also move every resource that was indexed the same
# way (route table associations, NACL associations…)Check that each key maps to the instance with the same arguments: [0] was 10.10.0.0/24 in 1a, so it must become "a". A wrong mapping shows up as -/+ forces replacement on cidr_block.
Moves are evaluated before planning, in order, and only inside one configuration and one state.
moved block in a module may only refer to addresses inside that module (and its children).removed + import for that (section 20).to address must exist in configuration; the from address must not.a → b and later b → c. Keep both blocks; Terraform follows the chain.moved blocks in shared modules for at least one major version (ch 6).state mv edits state now: no plan, no review, no record in Git.state pull).Bring an existing resource under Terraform through a reviewed plan instead of a one-off command.
Since Terraform 1.5, import is a block: it says which real object (id or identity) should be bound to which address (to). The plan shows the import before anything happens.
import {
to = aws_s3_bucket.reports
id = "tflab-clickops-reports-123456789012"
}
resource "aws_s3_bucket" "reports" {
bucket = "tflab-clickops-reports-123456789012"
}$ terraform plan
# aws_s3_bucket.reports will be imported
resource "aws_s3_bucket" "reports" {
arn = "arn:aws:s3:::tflab-clickops-reports-…"
bucket = "tflab-clickops-reports-123456789012"
...
}
Plan: 1 to import, 0 to add, 0 to change, 0 to destroy.
$ terraform apply
Apply complete! Resources: 1 imported, 0 added,
0 changed, 0 destroyed.Goal of every import: "1 to import, 0 to change". If the plan also shows ~ update, your resource block does not match reality yet; Terraform would change the real resource on apply. Fix the code until only the import remains (unless the change is intended).
The import id format is provider- and resource-specific (see "Import" at the bottom of each registry page). Resource identity (Terraform 1.12+) replaces the opaque string with named attributes.
# identity: named, provider-defined attributes
import {
to = aws_s3_bucket.reports
identity = {
bucket = "tflab-clickops-reports-123456789012"
# optional: account_id, region
}
}
# many objects at once (Terraform 1.7+)
locals {
params = {
feature_flag = "/tflab/dev/clickops/feature-flag"
owner = "/tflab/dev/clickops/owner"
}
}
import {
for_each = local.params
to = aws_ssm_parameter.clickops[each.key]
id = each.value
}id and identity are mutually exclusive in one block.aws_s3_bucket needs bucket; aws_ssm_parameter needs name; aws_vpc needs id (account_id, region optional).id/identity may use variables and locals, but must be known at plan time.to can point into a child module (module.network.aws_vpc.this).provider = aws.other to import through an aliased provider.With an import block and no matching resource block, terraform plan -generate-config-out=FILE writes HCL from the real object's attributes.
# only the import block exists
$ terraform plan -generate-config-out=generated.tf
...
Terraform has generated configuration and written it to
generated.tf. Please review the configuration and edit
it as necessary before adding it to version control.
Plan: 1 to import, 0 to add, 0 to change, 0 to destroy.
# generated.tf (excerpt)
# __generated__ by Terraform
resource "aws_s3_bucket" "reports" {
bucket = "tflab-clickops-reports-123456789012"
bucket_prefix = null
force_destroy = false
object_lock_enabled = false
tags = { Owner = "clickops" }
}null and default-valued arguments; replace literals with references (aws_vpc.this.id, local.tags).plan tells you which to delete.moved.terraform import ADDRESS ID predates the block. It writes state immediately; the resource block must already exist.
# imperative: state changes now, no plan
terraform import aws_s3_bucket.reports \
tflab-clickops-reports-123456789012
terraform import 'aws_ssm_parameter.clickops["owner"]' \
/tflab/dev/clickops/owner # quote the address
terraform plan # check for drift afterwardsimport block | terraform import | |
|---|---|---|
| Shown in a plan before it happens | Yes | No |
| Reviewed in a pull request, runs in CI | Yes | Only by hand |
Many objects (for_each) | Yes | One per call |
| Can generate configuration | Yes (-generate-config-out) | No |
| Works in older Terraform (< 1.5) | No | Yes |
With the command, a typo in the ID or address lands straight in shared state. If you must use it, terraform state pull > backup.tfstate first.
Search an account for unmanaged resources with list blocks and terraform query, then import them in bulk.
Terraform 1.14 added list blocks in *.tfquery.hcl files. terraform query asks the provider for existing objects of a type and prints their identities.
# search.tfquery.hcl (next to your .tf files)
list "aws_s3_bucket" "all" {
provider = aws
limit = 50 # default: 100 per list block
}
list "aws_ssm_parameter" "all" {
provider = aws
include_resource = true # full attributes, slower
config {
region = "ap-northeast-1"
}
}list block has a type and a name, like a resource: list.aws_s3_bucket.all.provider is required; config { } holds provider-specific filters (for aws_s3_bucket only region; aws_instance also accepts filter blocks).include_resource = true also fetches attributes.terraform providers schema -json | jq '.provider_schemas[].list_resource_schemas | keys'.terraform validate -query also validates the .tfquery.hcl files.In AWS provider 6.7x, list resources exist for S3 buckets, SSM parameters, VPCs, subnets, security groups, IAM roles, CloudWatch log groups and a few hundred more types. The list grows with each release.
terraform query -generate-config-out=FILE writes an import block (with identity) and a resource block for every result.
$ terraform query
list.aws_s3_bucket.all bucket=tflab-state-123456789012 …
list.aws_s3_bucket.all bucket=tflab-clickops-reports-… …
list.aws_ssm_parameter.all name=/tflab/dev/clickops/… …
# (exact columns depend on the provider)
$ terraform query -json | jq … # machine-readable
$ terraform query -generate-config-out=found.tf
# found.tf: import { to = … identity = { … } }
# resource "aws_s3_bucket" "all_0" { … }all_0) to meaningful names before the first apply: renaming later needs moved.plan -generate-config-out.terraform plan: expect only "N to import".-policies, ch 9).Bulk import is easy; deciding ownership is the real work. Treat adoption as a small project, one component at a time.
terraform query per type, or AWS Resource Explorer / Config.-generate-config-out.Import first, refactor second. Mixing "adopt" and "improve" in one plan hides real changes among dozens of imports. Ship the import with zero changes, then make the improvements visible in their own plans.
Most surprises come from resources that AWS models as one object but the provider splits into several.
aws_s3_bucket_versioning, …).aws_vpc_security_group_ingress_rule objects are imported one per rule ID.default_tags adds tags to every adopted resource: expect ~ tags_all updates on the first plan, which is usually fine and desired.ignore_changes or write-only arguments.lifecycle { prevent_destroy = true } to precious adopted resources (ch 3).import blocks in the PR history as documentation.Stop managing a resource without deleting it, and decide what to do when reality changes behind your back.
Since Terraform 1.7, a removed block drops an object from state through a normal plan. With destroy = false the real resource stays untouched.
# 1. delete the resource block from the code
# 2. add:
removed {
from = aws_s3_bucket.legacy
lifecycle {
destroy = false
}
}$ terraform plan
# aws_s3_bucket.legacy will no longer be managed by
# Terraform, but will not be destroyed
# (destroy = false is set in the configuration)
Plan: 0 to add, 0 to change, 0 to destroy.
Warning: Some objects will no longer be managed by Terraform
If you apply this plan, Terraform will discard its tracking
information for the following objects, but it will not
delete them:
- aws_s3_bucket.legacyWithout a removed block, deleting the resource block means destroy. destroy = true (or omitting the lifecycle) makes the intent explicit and lets you attach destroy-time provisioners. from names a whole resource or module, never a single instance key.
moved cannot cross state files. To hand a resource from one root module to another, forget it in the old one and import it in the new one.
# live/dev/network (old owner)
removed {
from = aws_security_group.app
lifecycle {
destroy = false
}
}
# live/dev/app (new owner)
import {
to = aws_security_group.app
id = "sg-0abc1234def567890"
}
resource "aws_security_group" "app" {
name = "tflab-dev-app"
vpc_id = data.aws_ssm_parameter.vpc_id.insecure_value
}destroy = true by mistake: plan text is the safeguard.A refresh-only plan compares state with reality and proposes to update state only. It never changes infrastructure.
# someone adds a tag in the console
$ terraform plan -refresh-only
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of
Terraform since the last "terraform apply":
# aws_s3_bucket.reports has changed
~ resource "aws_s3_bucket" "reports" {
~ tags = {
+ "Owner" = "alice"
}
# (11 unchanged attributes hidden)
}
This is a refresh-only plan, so Terraform will not take
any actions to undo these. …plan -refresh-only: what changed in reality since the last apply?plan: what will Terraform change to make reality match the code (refresh included)?apply -refresh-only: accept reality into state without touching AWS.terraform refresh is a deprecated alias of apply -refresh-only -auto-approve: no review, avoid it.plan -detailed-exitcode returns 2 when there are changes: the basis of nightly drift jobs (ch 11).Drift is a decision, not a bug report. Either the manual change was right and the code must learn it, or it was wrong and the next apply must undo it.
plan shows No changes.lifecycle { ignore_changes = [...] } (ch 3).plan: it proposes to undo the manual change.apply -refresh-only updates state, nothing else.Do not "fix" drift by running apply blindly: a plan that undoes a colleague's emergency change at 3 am can cause the next incident. Read the plan, then decide.
Each of these turns a harmless rename or adoption into an outage.
moved block → destroy + create.count indexes to the wrong for_each keys → -/+ replacements.moved blocks from a shared module → consumers on older versions see destroys.~ update: the import "fixes" reality to match sloppy code.-generate-config-out output unreviewed, full of literals and nulls.terraform refresh or apply -refresh-only without reading the diff.Explain each with the exact block, flag or plan line.
moved block?aws_subnet.public[0] into aws_subnet.public["a"] safely?moved block not move an object into another root module?moved blocks from a shared module?import block, and can you use both?plan -generate-config-out refuse to do if the file exists?list blocks, and what does include_resource change?removed block?plan, plan -refresh-only and apply -refresh-only?Practice: Lab 5 in Hands-on labs — adopt a ClickOps bucket and SSM parameters with import blocks and terraform query, rename with moved, let go with removed, then detect drift with a refresh-only plan.
Package infrastructure into reusable building blocks with a small, typed interface, compose them per environment, and version them so upgrades are deliberate.
Every directory of .tf files is a module; a module block calls one with inputs and reads its outputs.
A module is a directory of .tf files. The directory you run Terraform in is the root module; every module block calls a child module.
tf-lab/
├── modules/ # reusable child modules
│ ├── network/
│ │ ├── main.tf # resources
│ │ ├── variables.tf # inputs
│ │ ├── outputs.tf # outputs
│ │ ├── versions.tf # required_providers
│ │ ├── README.md
│ │ ├── examples/basic/ # a runnable root module
│ │ └── tests/ # *.tftest.hcl (ch 7)
│ └── app-bucket/
└── live/ # root modules: one state each
└── dev/
├── network/ # backend, provider, module "network"
└── app/variable, output, required_providers. No backend, ideally no provider blocks.backend, provider configuration, module calls, environment values.*.tf in the directory.module.network.aws_vpc.this.The module block passes inputs as arguments and exposes outputs as module.<name>.<output>. Changing source or version requires terraform init.
# live/dev/network/main.tf
module "network" {
source = "../../../modules/network"
name = "tflab-dev"
cidr = "10.10.0.0/16"
public_subnets = {
a = "10.10.0.0/24"
c = "10.10.1.0/24"
}
private_subnets = {
a = "10.10.10.0/24"
c = "10.10.11.0/24"
}
}
output "vpc_id" {
value = module.network.vpc_id
}source (required), version (registry sources only).count / for_each: several instances, module.app["dev"].providers: which provider configurations the module uses.depends_on: whole-module ordering. Avoid it: it makes every data source inside wait and can turn known values into "known after apply".The source string decides where terraform init downloads the module from. Local paths are used in place; everything else is copied into .terraform/modules/.
| Source | Example | Versioning |
|---|---|---|
| Local path | ../../modules/network | Same commit as the caller |
| Public registry | terraform-aws-modules/vpc/aws | version = "~> 6.7" |
| Private registry | app.terraform.io/acme/network/aws | version = "1.4.0" |
| GitHub shorthand | github.com/acme/tf-modules//network | ?ref=network-v1.4.0 |
| Generic Git | git::https://example.com/tf.git//modules/network?ref=v1.4.0 | Tag, branch or commit SHA |
| Git over SSH | git::ssh://git@github.com/acme/tf.git//network?ref=… | Same |
| S3 bucket | s3::https://s3-ap-northeast-1.amazonaws.com/bucket/network.zip | Object key / version |
// separates the repository from a subdirectory. version works only with registry sources; for Git, pin with ?ref= — ideally a tag you never move, or a full commit SHA for strict supply-chain hygiene (ch 12).
Variables are the parameters, outputs are the return values. Typed objects with optional() keep calls short without losing validation.
# modules/network/variables.tf
variable "name" {
type = string # name prefix, e.g. tflab-dev
}
variable "cidr" {
type = string
validation {
condition = (
can(cidrnetmask(var.cidr)) && endswith(var.cidr, "/16")
)
error_message = "cidr must be an IPv4 /16."
}
}
variable "public_subnets" {
type = map(string) # AZ suffix => CIDR
}
variable "flow_logs" { # typed object, optional fields
type = object({
enabled = optional(bool, false)
retention_days = optional(number, 14)
})
default = {}
}# modules/network/outputs.tf
output "vpc_id" {
description = "ID of the VPC"
value = aws_vpc.this.id
}
output "public_subnet_ids" {
description = "AZ suffix => subnet ID"
value = { for k, s in aws_subnet.public : k => s.id }
}
output "private_subnet_ids" {
value = { for k, s in aws_subnet.private : k => s.id }
}
# Terraform 1.15+: declare the output type
output "subnet_count" {
type = number
value = length(aws_subnet.public) + length(aws_subnet.private)
}Return maps keyed by something stable (AZ suffix) rather than lists: callers can then for_each over them without index shifts.
A child module declares which providers it needs; the caller decides which configurations it gets. With AWS provider v6, many multi-region cases need no aliases at all.
# modules/app-bucket/versions.tf
terraform {
required_version = ">= 1.14"
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 6.0" # a floor, not a pin
}
}
}
# a module that needs two configurations
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
configuration_aliases = [aws.replica]
}
}
}
# caller
module "logs" {
source = "../../modules/replicated-bucket"
providers = { aws = aws, aws.replica = aws.osaka }
}>=); root modules pin with ~> and the lock file (ch 3).provider blocks in a shared module: such modules cannot be used with count, for_each or depends_on, and removing the module call breaks destroy.providers, the child inherits the default (un-aliased) configurations.region = "ap-northeast-3", so a module can take a region input instead of an aliased provider.default_tags set in the root provider apply to every resource in every child module.Small interfaces, sane defaults, and modules that encode decisions instead of wrapping resources one to one.
A good module encodes your organization's decisions (encryption on, public access off, tags, naming) behind a small interface.
examples/ directory and tests (ch 7).create_vpc, use_existing_vpc, legacy_mode).moved blocks (ch 5).The caller chooses a name and a few knobs. Encryption, versioning, public access block and tags are not optional.
# modules/app-bucket/main.tf
resource "aws_s3_bucket" "this" {
bucket = "tflab-${var.env}-${var.name}"
force_destroy = var.force_destroy
}
resource "aws_s3_bucket_public_access_block" "this" {
bucket = aws_s3_bucket.this.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
resource "aws_s3_bucket_versioning" "this" {
bucket = aws_s3_bucket.this.id
versioning_configuration {
status = var.versioning ? "Enabled" : "Suspended"
}
}
resource "aws_s3_bucket_server_side_encryption_configuration" "this" {
bucket = aws_s3_bucket.this.id
rule {
apply_server_side_encryption_by_default {
sse_algorithm = var.kms_key_arn == null ? "AES256" : "aws:kms"
kms_master_key_id = var.kms_key_arn
}
}
}# modules/app-bucket/variables.tf
variable "env" {
type = string
}
variable "name" { # bucket = tflab-<env>-<name>
type = string
validation {
condition = can(regex("^[a-z0-9-]{3,40}$", var.name))
error_message = "name: 3-40 chars, a-z 0-9 and -."
}
}
variable "versioning" {
type = bool
default = true
}
variable "kms_key_arn" {
type = string
default = null # null → SSE-S3 (AES256)
}
variable "force_destroy" {
type = bool
default = false # labs may set true
}
# outputs.tf
output "bucket_name" { value = aws_s3_bucket.this.bucket }
output "bucket_arn" { value = aws_s3_bucket.this.arn }The interface is a contract with every caller. Changing it is a breaking change; design it so most changes are additive.
object({...}) with optional(type, default) fields: new fields stay backward-compatible.validation for formats and ranges; error messages should say how to fix it.nullable = false when null would be meaningless.sensitive = true (or ephemeral = true, ch 2).description.type on outputs to catch accidental shape changes.deprecated = "Use x instead." (1.15+), remove it in the next major version.moved blocks in the module.Terraform 1.15 added deprecated to variable and output blocks: callers get a warning, not an error, while they migrate.
# modules/app-bucket/variables.tf (v1.3.0)
variable "enable_versioning" {
type = bool
default = null
deprecated = "Use versioning instead; removed in v2.0.0."
}
variable "versioning" {
type = bool
default = true
}
locals {
versioning = coalesce(var.enable_versioning, var.versioning)
}# a caller still on the old name
$ terraform validate
Warning: Deprecated variable got a value
on main.tf line 4, in module "logs":
4: enable_versioning = true
Use versioning instead; removed in v2.0.0.
Success! The configuration is valid, but there were
some validation warnings as shown above.Referencing a deprecated output prints "Deprecated value used". Older Terraform versions reject the deprecated argument, so a module that uses it needs required_version = ">= 1.15".
Wire small modules together in root modules, use public modules carefully, and stamp out copies with for_each.
Each root module is one component with one state. Components talk through published values (SSM parameters here), not through each other's internals.
Why SSM instead of terraform_remote_state: the app needs read access to one parameter, not to the whole network state (which may contain secrets). Chapter 4 compares the options; chapter 10 covers ordering between components.
The root module wires inputs from the environment, published values and module outputs together. It contains almost no resource logic of its own.
# live/dev/app/main.tf
data "aws_ssm_parameter" "vpc_id" {
name = "/tflab/dev/network/vpc_id"
}
module "assets" {
source = "../../../modules/app-bucket"
env = "dev"
name = "assets-${local.acct}" # S3 names are global
force_destroy = true # lab only
}
data "aws_caller_identity" "me" {}
locals {
acct = data.aws_caller_identity.me.account_id
}
resource "aws_security_group" "app" {
name = "tflab-dev-app"
description = "App tier (no rules yet)"
vpc_id = data.aws_ssm_parameter.vpc_id.insecure_value
}
output "bucket" { value = module.assets.bucket_name }data.aws_ssm_parameter…insecure_value is known at plan time (read during plan), so the security group plan is complete.A module call supports for_each like a resource. Each instance gets its own address prefix and its own resources.
locals {
buckets = {
uploads = { versioning = true }
exports = { versioning = false }
}
}
module "bucket" {
source = "../../../modules/app-bucket"
for_each = local.buckets
env = "dev"
name = "${each.key}-${local.acct}"
versioning = each.value.versioning
}
output "bucket_arns" {
value = { for k, m in module.bucket : k => m.bucket_arn }
}$ terraform plan
# module.bucket["exports"].aws_s3_bucket.this
# will be created
# module.bucket["exports"].aws_s3_bucket_versioning.this
# will be created
...
# module.bucket["uploads"].aws_s3_bucket.this
# will be created
Plan: 8 to add, 0 to change, 0 to destroy.
# turning a single call into for_each later:
moved {
from = module.assets
to = module.bucket["assets"]
}The keys become part of every address inside the module. Choose them like resource names: stable and meaningful (uploads), never list indexes or values that may change.
Community modules such as terraform-aws-modules/vpc/aws (6.7.x in Sep 2026, for AWS provider v6) save weeks. They are also third-party code with broad permissions over your account.
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "~> 6.7" # allow 6.x patches/minors, not 7.0
name = "tflab-dev"
cidr = "10.10.0.0/16"
azs = ["ap-northeast-1a", "ap-northeast-1c"]
public_subnets = ["10.10.0.0/24", "10.10.1.0/24"]
private_subnets = ["10.10.10.0/24", "10.10.11.0/24"]
enable_nat_gateway = false # NAT costs money; labs never create one
}moved blocks or state moves.Module sources are resolved by terraform init, before normal evaluation. Terraform 1.15 allows const variables (and locals built from them) there.
variable "modules_ref" {
type = string
default = "app-bucket-v1.1.0"
const = true # value must be known at init
}
locals {
repo = "git::https://github.com/<you>/tf-lab.git"
}
module "assets" {
source = "${local.repo}//modules/app-bucket?ref=${var.modules_ref}"
env = "dev"
name = "assets-123456789012"
}
# terraform init -var modules_ref=app-bucket-v1.2.0const = true, init fails: "Only literal values and const variables can be evaluated during init."terraform init again, like any source change.Semantic versions, Git tags or a registry, changelogs and upgrade paths that never surprise a caller.
A version number is a promise about the plan a caller will see after upgrading.
| Change in the module | Bump | What the caller's plan shows |
|---|---|---|
| Bug fix, documentation, new optional output | Patch 1.4.1 | No changes |
| New optional input with a default that keeps today's behavior | Minor 1.5.0 | No changes |
Internal rename shipped with moved blocks | Minor | Moves only, 0 to change |
| New resource created by default (e.g. a log group) | Minor, announce it | + creates only |
| Removed or renamed input/output, new required input | Major 2.0.0 | Errors until the caller changes code |
| Anything that replaces or destroys existing resources | Major + upgrade guide | -/+ or -: must be planned |
| Raising the minimum Terraform or provider version | Major (or minor with notice) | init may fail on old versions |
Callers constrain with ~> 1.4 (any 1.x ≥ 1.4) for registry modules, or pin a tag with ?ref=. In a monorepo with local paths, there is no version: every commit is a release, and tests must catch breaking changes (ch 7).
In a repository with several modules, prefix tags with the module name so each module has its own release line.
# release app-bucket 1.1.0
git switch main && git pull
git log --oneline -- modules/app-bucket # what changed?
git tag -a app-bucket-v1.1.0 \
-m "app-bucket: add kms_key_arn input"
git push origin app-bucket-v1.1.0
# consume it (GitHub shorthand = git over HTTPS)
module "assets" {
source = "github.com/<you>/tf-lab//modules/app-bucket?ref=app-bucket-v1.1.0"
env = "prod"
name = "assets-123456789012"
}
# strictest: a full commit SHA (tags can be moved)
# ?ref=4f1c2a9e0b…terraform-aws-network): required by the public registry naming rules; most overhead.version constraints and discovery.Treat a module upgrade like an application release: change dev first, read the plan, then promote the exact same version.
app-bucket-v1.2.0, CHANGELOG entry.ref in live/dev/app.terraform init -upgrade.live/prod/app.terraform init -upgrade re-downloads modules (and providers within constraints). Without it, a changed ?ref= is still fetched on init, but a floating constraint like ~> 1.4 keeps the version already installed in .terraform/modules.
Generate the reference part with terraform-docs; write the decisions part by hand.
# modules/app-bucket/README.md
## app-bucket
Private, encrypted, versioned S3 bucket for one app.
### Decisions (not configurable)
- Public access fully blocked
- Encryption: SSE-S3, or SSE-KMS when kms_key_arn is set
- Versioning on by default
### Usage
module "uploads" {
source = "../../modules/app-bucket"
env = "dev"
name = "uploads-123456789012"
}
<!-- BEGIN_TF_DOCS -->
(inputs, outputs, providers: generated)
<!-- END_TF_DOCS -->terraform-docs markdown table --output-file README.md --output-mode inject modules/app-bucket fills the marked block.examples/basic is a root module that CI can plan (or test) on every change.Most module pain shows up months later, during an upgrade or a second environment.
provider blocks inside shared modules.?ref=: every init gets the latest default branch.moved blocks → destroys for every caller.terraform init after changing source or version.depends_on to "fix" ordering, making plans show (known after apply) everywhere.version on a Git source (only registry sources support it).Answer with the exact block, argument or command.
version, and how do you pin a Git source?// mean in a module source?provider blocks?optional() object attributes help evolve an interface?terraform init -upgrade change?const?module.assets into module.bucket["assets"] with no changes?Practice: Lab 6 in Hands-on labs — extract modules/network and modules/app-bucket from the flat Lab 3 code with moved blocks (zero-change plan), build live/dev/app from SSM inputs, then tag a module version and consume it with ?ref=.
Catch mistakes before a plan reaches a reviewer: formatting and validation, TFLint, terraform test with mocks, and the slower tests that touch real infrastructure.
Seconds-fast checks that need no cloud credentials: run them on every save and every pull request.
Cheap checks run often and catch most mistakes; expensive checks run rarely and catch what only real cloud APIs reveal.
command = plan and mocks** (seconds): every PR. Provider schema only; no API calls.command = apply** against real AWS (minutes): nightly or on module release, in a sandbox account.fmt makes diffs readable; validate proves the configuration is internally consistent. Neither talks to AWS.
# formatting: fail CI if anything is not canonical
terraform fmt -check -recursive -diff
# exit 0 = formatted, 3 = files need formatting
# fix locally
terraform fmt -recursive
# validate needs providers, not a backend:
terraform init -backend=false
terraform validate
# Success! The configuration is valid.
# machine-readable result for CI
terraform validate -json | jq '.valid, .error_count'validation with values from tfvars (runs at plan time).TFLint adds rules that validate cannot know: invalid instance types, deprecated syntax, unused declarations, missing version constraints.
# .tflint.hcl (repo root)
plugin "terraform" {
enabled = true
preset = "recommended"
}
plugin "aws" {
enabled = true
version = "0.50.0" # pin; check the latest release
source = "github.com/terraform-linters/tflint-ruleset-aws"
}tflint --init # downloads the aws plugin
tflint --recursive # every directory with .tf files
# 3 issue(s) found:
# Warning: `unused` variable has no type
# (terraform_typed_variables)
# Warning: [Fixable] variable "unused" is declared
# but not used (terraform_unused_declarations)
# Error: "t3.mircro" is an invalid value as
# instance_type (aws_instance_invalid_type)
tflint --format compact # one line per issue (CI logs)
tflint --fix # auto-fix the [Fixable] ones
tflint --minimum-failure-severity=error # warnings passExit codes: 0 = no issues; 2 = issues found (warnings included unless you raise --minimum-failure-severity). The terraform ruleset is bundled; the aws ruleset is a plugin you pin and download with --init.
The recommended terraform preset plus the AWS ruleset cover most of what reviewers otherwise comment on by hand.
| Rule | Ruleset | Catches |
|---|---|---|
terraform_required_version | terraform | Root module without required_version |
terraform_required_providers | terraform | Provider used without a version constraint |
terraform_typed_variables | terraform | variable without type |
terraform_unused_declarations | terraform | Variables, locals, data sources nobody references |
terraform_deprecated_interpolation | terraform | "${var.x}" where var.x is enough |
aws_instance_invalid_type | aws | Typos in instance_type (t3.mircro) |
aws_*_invalid_* (many) | aws | Enum-like arguments with impossible values, checked against AWS API models |
Deep checking (deep_check = true) | aws | Calls AWS APIs (needs credentials) to check e.g. that an AMI or IAM profile exists |
Disable a rule for one line with a comment above it: # tflint-ignore: aws_instance_invalid_type. Prefer fixing; when you must ignore, say why in the same comment.
The same commands in a pre-commit hook and in CI: developers see failures in seconds, CI enforces them for everyone.
# .pre-commit-config.yaml
repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
rev: <latest tag> # pin a release tag or SHA
hooks:
- id: terraform_fmt
- id: terraform_validate
- id: terraform_tflint
args: ["--args=--config=__GIT_WORKING_DIR__/.tflint.hcl"]
- id: terraform_docs # keeps README inputs/outputs in sync
- id: terraform_trivy # IaC scan (ch 9)
# install once per clone
pipx install pre-commit
pre-commit install
pre-commit run --all-files--all-files in CI.terraform_validate runs init -backend=false per directory: cache providers with TF_PLUGIN_CACHE_DIR.terraform test with real apply) in CI only, never in the hook.--no-verify.The built-in test framework (1.6+): run plans or applies of a module and assert on the result, with mocked providers when you want no cloud at all.
Test files end in .tftest.hcl and live in tests/ (or next to the module). Each run block plans or applies the module and checks assertions.
# modules/app-bucket/tests/unit.tftest.hcl
variables { # defaults for every run block
env = "dev"
name = "logs"
}
run "name_follows_convention" {
command = plan # default is apply
assert {
condition = aws_s3_bucket.this.bucket == "tflab-dev-logs"
error_message = "bucket name must be tflab-<env>-<name>"
}
}
run "rejects_uppercase_name" {
command = plan
variables {
name = "Logs" # overrides the file-level value
}
expect_failures = [var.name] # validation MUST fail
}terraform test runs every *.tftest.hcl in the module dir and tests/.apply created.aws_s3_bucket.this.bucket.output.bucket_arn, var.env, run.setup.vpc_id.plan, values only known after apply cannot be asserted (unless mocked, next slides).A failing assertion shows the condition, the actual vs expected values, and your error message.
terraform init -backend=false
terraform test
# tests/unit.tftest.hcl... in progress
# run "name_follows_convention"... pass
# run "versioning_on_by_default"... pass
# run "rejects_uppercase_name"... pass
# run "rejects_unknown_env"... pass
# run "apply_with_mocks"... pass
# tests/unit.tftest.hcl... tearing down
# tests/unit.tftest.hcl... pass
#
# Success! 5 passed, 0 failed.
# useful flags
terraform test -filter=tests/unit.tftest.hcl
terraform test -verbose # print plan/state per run
terraform test -junit-xml=test.xml # CI test report
terraform test -var 'env=prod'# run "subnet_plan"... fail
#
# Error: Test assertion failed
#
# on tests/network.tftest.hcl line 17,
# in run "subnet_plan":
# 17: condition = aws_subnet.private[
# "ap-northeast-1a"].cidr_block
# == "10.10.10.0/24"
# ├────────────────
# │ Diff:
# │ --- actual
# │ +++ expected
# │ - "10.10.20.0/24"
# │ + "10.10.10.0/24"
#
# private subnet in 1a must be 10.10.10.0/24
#
# Failure! 2 passed, 1 failed.Exit code is non-zero when any run fails: CI can gate on it directly. Output above is from Terraform 1.15 with the AWS provider 6.x and mocks.
Everything a run block can set. Most tests need only command, variables and assert.
| Attribute / block | Purpose |
|---|---|
command = plan | apply | Plan only (fast, nothing created) or apply (default) |
variables { … } | Inputs for this run; override file-level variables |
assert { condition, error_message } | One or more checks; all are evaluated and reported |
expect_failures = [var.x, resource.y, check.z] | The run passes only if these checkable objects fail (validation, pre/postconditions, check blocks) |
module { source = "./tests/setup" } | Run a helper module instead of the module under test (create prerequisites, then reference run.<name>.<output>) |
plan_options { mode, refresh, replace, target } | Same as CLI flags: refresh-only mode, -refresh=false, -replace, -target |
providers = { aws = aws.tokyo } | Pass specific provider configurations to the run |
state_key = "x" | Choose which in-memory state the run uses (1.9+): runs with the same key share state |
parallel = true | Let independent run blocks execute concurrently |
A test { parallel = true } block at the top of a file sets the default for all its runs; a run can override it with parallel = false.
A mocked provider keeps the real schema (types, required arguments) but never calls an API. Computed attributes get fake values you can control.
mock_provider "aws" {
# optional: fixed values for computed attributes
mock_resource "aws_s3_bucket" {
defaults = {
arn = "arn:aws:s3:::tflab-dev-logs"
}
}
mock_data "aws_caller_identity" {
defaults = { account_id = "123456789012" }
}
}
run "apply_with_mocks" {
# command = apply (default) against the MOCK:
# nothing is created in AWS
assert {
condition = output.bucket_arn == "arn:aws:s3:::tflab-dev-logs"
error_message = "arn output comes from the mocked bucket"
}
}apply runs in seconds and costs nothing: computed values (id, arn) become known.defaults, Terraform generates values of the right type (random strings for id).terraform init (it reads the schema), but no credentials.for_each keys, outputs, conditional resources, validation.apply tests in a sandbox (section 27).override_* blocks replace the result of one specific object. They work with real or mocked providers, at file level or inside one run.
mock_provider "aws" {}
run "ids_from_overrides" {
override_resource {
target = aws_vpc.this
values = { id = "vpc-0123456789abcdef0" }
}
assert {
condition = output.vpc_id == "vpc-0123456789abcdef0"
error_message = "vpc_id output must come from aws_vpc.this"
}
assert {
condition = alltrue([for s in aws_subnet.private :
s.vpc_id == "vpc-0123456789abcdef0"])
error_message = "every private subnet must be in the VPC"
}
}
# also available:
# override_data { target = data.aws_availability_zones.this
# values = { names = ["ap-northeast-1a"] } }
# override_module { target = module.network
# outputs = { vpc_id = "vpc-123" } }override_during = plan (on mock_provider or an override_* block) makes them known at plan time, so command = plan runs can assert on IDs.output.vpc_id after a plan fails because the value is unknown.override_data over real data sources in unit tests: tests must not depend on what exists in an account.Choose per run block. Most unit tests should be plans; a few apply tests prove the module really works.
command = plan | command = apply + mocks | command = apply real | |
|---|---|---|---|
| Speed | Seconds | Seconds | Minutes (real API calls) |
| Credentials | None with mocks; read-only with real provider | None | Sandbox account role |
| Cost / risk | None | None | Creates and destroys real resources |
Computed values (id, arn) | Unknown (unless override_during = plan) | Known (mocked) | Known (real) |
| Proves | Logic: names, counts, conditions, validation | Wiring between resources and outputs | AWS accepts it; no perpetual diff |
| Run on | Every PR | Every PR | Nightly / before a module release |
After a real apply test, check the cleanup: if a destroy fails, Terraform prints the objects it could not remove. A sandbox account with a budget alarm and tag-based clean-up is the safety net.
A run can apply a helper module to create prerequisites; later runs read its outputs through run.<name>.
# modules/app-bucket/tests/integration.tftest.hcl
# REAL apply in the sandbox account (nightly job)
provider "aws" {
region = "ap-northeast-1"
}
run "setup" {
module { source = "./tests/setup" } # random suffix
}
run "create_bucket" {
variables {
env = "dev"
name = "it-${run.setup.suffix}" # globally unique
}
assert {
condition = startswith(output.bucket_name,
"tflab-dev-it-")
error_message = "unexpected bucket name"
}
}
# after the file: both runs destroyed, newest first# modules/app-bucket/tests/setup/main.tf
terraform {
required_providers {
random = { source = "hashicorp/random" }
}
}
resource "random_id" "suffix" {
byte_length = 3
}
output "suffix" {
value = random_id.suffix.hex # e.g. "a1b2c3"
}Real-apply tests need credentials and a unique name per run (S3 bucket names are global). Never point them at a shared or production account.
Integration tests, contract tests, upgrade tests and cost checks: the slower layer that catches what mocks cannot.
Apply the module for real, check the result through the AWS API, destroy. Slow, so run them on a schedule and before module releases.
random_id, run ID) and a Project=tf-lab-test tag.aws-nuke with a tag filter) for leftovers.terraform plan -detailed-exitcode → exit 0.aws s3api get-bucket-encryption), public access blocked.prevent_destroy or dependency surprises.terraform test with command = apply and a real provider: native, HCL only.terraform and the AWS SDK; good for HTTP checks, retries, complex assertions.A Go test that applies the module, asserts with the AWS SDK and always destroys. Use it when HCL assertions are not enough.
// test/app_bucket_test.go
package test
import (
"strings"
"testing"
"github.com/gruntwork-io/terratest/modules/aws"
"github.com/gruntwork-io/terratest/modules/random"
"github.com/gruntwork-io/terratest/modules/terraform"
)
func TestAppBucket(t *testing.T) {
t.Parallel()
// UniqueId can contain upper case; S3 names cannot
name := "it-" + strings.ToLower(random.UniqueId())
opts := &terraform.Options{
TerraformDir: "../modules/app-bucket/examples/basic",
Vars: map[string]any{"env": "dev", "name": name},
}
defer terraform.Destroy(t, opts) // always clean up
terraform.InitAndApply(t, opts)
bucket := terraform.Output(t, opts, "bucket_name")
aws.AssertS3BucketExists(t, "ap-northeast-1", bucket)
}defer terraform.Destroy runs even when an assertion fails.go test -v -timeout 30m ./test/; the default Go test timeout (10 min) is too short for most infrastructure.examples/ root module, not the module directly: it also documents usage.A module's outputs and inputs are an API. Contract tests fail when a change would break callers, before a new version is released.
# modules/network/tests/contract.tftest.hcl
mock_provider "aws" {
override_during = plan
mock_data "aws_availability_zones" {
defaults = { names = ["ap-northeast-1a", "ap-northeast-1c"] }
}
}
variables {
env = "dev"
cidr = "10.10.0.0/16"
}
run "outputs_contract" {
command = plan
assert {
condition = can(regex("^vpc-", output.vpc_id))
error_message = "vpc_id must stay a VPC ID"
}
assert {
condition = length(output.private_subnet_ids) == 2
error_message = "callers expect one private subnet per AZ"
}
}vpc_id breaks every caller).moved blocks or imports.moved blocks cover renamed addresses.Infracost reads Terraform code or a plan and prices it from public price lists. Useful in PRs, not a bill.
# one-time: free API key
infracost auth login
# price a directory
infracost breakdown --path live/dev/network
# show the cost difference a PR introduces
infracost breakdown --path . \
--format json --out-file base.json # on main
infracost diff --path . --compare-to base.json \
--format json --out-file diff.json # on the PR branch
# in CI: post the diff as a PR comment
infracost comment github --path diff.json ...aws_nat_gateway in dev (ch 9).Project and Env (ch 9) so the bill can be split.Each of these lets a broken module reach main looking green.
bucket == var.name) instead of the rule (tflab-<env>-<name>).override_during = plan.expect_failures on the wrong object: the run passes for a different failure.tflab-dev-test) → parallel runs collide; S3 names are global.fmt -check, validate, tflint and plan-mode terraform test with mocks on every PR (< 1 min).Answer out loud with the exact command, block or attribute.
terraform fmt -check return when files need formatting?terraform validate not catch instance_type = "t3.mircro"?tflint --init do?command of a run block?apply runs destroyed?output.vpc_id, and how does override_during = plan fix it?run.setup.suffix refer to?Practice: Lab 7 in Hands-on labs — TFLint with the AWS ruleset, then terraform test for modules/app-bucket and modules/network with plan asserts, expect_failures and mocked providers, all without touching AWS.
The plan is the last point where a mistake is free. Read it like a diff of production, turn it into data, know which changes are dangerous, and apply exactly what was reviewed.
Symbols, annotations and the summary line: what Terraform is telling you it will do, object by object.
Every object in a plan gets one action. The two replacement symbols differ in order: which copy exists during the change.
| Symbol | Action | JSON change.actions | Meaning |
|---|---|---|---|
+ | create | ["create"] | New object |
~ | update in-place | ["update"] | Same object, changed arguments (API update call) |
- | destroy | ["delete"] | Object removed from AWS and state |
-/+ | destroy then create | ["delete","create"] | Replacement; downtime / data loss window |
+/- | create then destroy | ["create","delete"] | Replacement with create_before_destroy (ch 3) |
<= | read | ["read"] | Data source read during apply (inputs unknown at plan) |
. | forget | ["forget"] | removed block with destroy = false: leaves state only (ch 5) |
| (none) | no-op / moved | ["no-op"] + previous_address | moved block: address changes, object does not |
-, -/+, +/-: which object, and why (the # forces replacement comment).~ on IAM, security groups, KMS, bucket policies: access changes.+: expected? names, tags, sizes.Output from Terraform 1.15 on a small configuration. The comment line above each object says what and why.
# terraform_data.cbd must be replaced
+/- resource "terraform_data" "cbd" {
~ id = "dc3f…" -> (known after apply)
~ triggers_replace = "a" -> "b"
}
# terraform_data.forgot will no longer be managed by
# Terraform, but will not be destroyed
. resource "terraform_data" "forgot" { … }
# terraform_data.gone will be destroyed
# (because terraform_data.gone is not in configuration)
- resource "terraform_data" "gone" {
- input = "x" -> null
}
# terraform_data.old_name has moved to
# terraform_data.new_name
resource "terraform_data" "new_name" { … }
# terraform_data.upd will be updated in-place
~ resource "terraform_data" "upd" {
~ input = "v1" -> "v2"
~ output = "v1" -> (known after apply)
}
Plan: 3 to add, 1 to change, 3 to destroy.must be replaced + an attribute line ending in # forces replacement (provider marks the argument as ForceNew).is not in configuration: someone deleted code, or a for_each key disappeared.has moved to: refactor, no infrastructure change.will be read during apply: a data source depends on unknown values.(known after apply): computed by AWS; not a change by itself.N to import; forgotten objects are listed in a warning below the plan.For real providers the plan names the argument that forces the replacement. That line is the one to discuss in review.
# module.bucket.aws_s3_bucket.this must be replaced
-/+ resource "aws_s3_bucket" "this" {
~ arn = "arn:aws:s3:::tflab-dev-assets"
-> (known after apply)
~ bucket = "tflab-dev-assets" -> "tflab-dev-static"
# forces replacement
~ id = "tflab-dev-assets" -> (known after apply)
# (… unchanged attributes hidden)
}
# module.network.aws_vpc.this must be replaced
-/+ resource "aws_vpc" "this" {
~ cidr_block = "10.10.0.0/16" -> "10.30.0.0/16"
# forces replacement
…
}aws_s3_bucket.bucket (the name).aws_vpc.cidr_block; aws_subnet.cidr_block, availability_zone, vpc_id.aws_security_group.name, description, vpc_id.aws_iam_role.name; aws_lambda_function.function_name.aws_db_instance.identifier, engine; aws_dynamodb_table.name, hash_key.subnet_id.Three plan sections people scroll past: what changed outside Terraform, what other components will read, and what Terraform hides.
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside
of Terraform since the last "terraform apply" which may
have affected this plan:
# aws_security_group.web has changed
~ resource "aws_security_group" "web" {
~ ingress = [
+ { cidr_blocks = ["0.0.0.0/0"], from_port = 22, … },
]
}
Unless you have made equivalent changes to your
configuration, or ignored the relevant attributes using
ignore_changes, the following plan may include actions
to undo or respond to these changes.
Changes to Outputs:
~ vpc_id = "vpc-0a…" -> (known after apply)
+ bucket = "tflab-dev-static"
~ password = (sensitive value)(sensitive value) hides display only; the plan file and its JSON still contain the value (ch 12).A saved plan converted to JSON: filter it with jq, summarize it for reviewers, and gate on it in CI.
terraform show -json turns a saved plan into a documented JSON format. Policies (ch 9) and scripts read this, never the text output.
terraform plan -out=tfplan # binary plan file
terraform show -json tfplan > tfplan.json
jq 'keys' tfplan.json
# [ "applyable", "complete", "configuration",
# "errored", "format_version", "planned_values",
# "prior_state", "resource_changes",
# "terraform_version", "timestamp" ]
# (+ "resource_drift", "output_changes",
# "relevant_attributes", "checks" when present)
jq -r '.format_version, .terraform_version' tfplan.json
# 1.2
# 1.15.5| Key | Contains |
|---|---|
resource_changes[] | One entry per resource instance: the planned action |
resource_drift[] | Changes found by refresh (outside Terraform) |
output_changes | Per-output actions, before, after |
planned_values | Full expected state after apply |
prior_state | State after refresh, before the change |
configuration | The parsed configuration (expressions, references) |
errored / applyable | Plan failed / plan can be applied |
The JSON format is versioned by format_version (minor bumps add fields, a major bump would break readers). Scripts should ignore unknown keys.
Everything a reviewer or a policy needs about one object is in this structure.
{
"address": "terraform_data.rep",
"previous_address": null,
"mode": "managed",
"type": "terraform_data",
"name": "rep",
"provider_name": "terraform.io/builtin/terraform",
"change": {
"actions": ["delete", "create"],
"before": { "triggers_replace": "a", "id": "c698…" },
"after": { "triggers_replace": "b" },
"after_unknown": { "id": true, "output": true },
"before_sensitive": {},
"after_sensitive": {},
"replace_paths": [["triggers_replace"]]
},
"action_reason": "replace_because_cannot_update"
}type; change.actions (always an array: replace = two actions).change.replace_paths: argument(s) that forced the replacement.previous_address (moved) and change.importing.id (import blocks).before/after are not redacted: *_sensitive only marks values. Protect plan JSON like state (ch 12).| Reason | When |
|---|---|
replace_because_cannot_update | ForceNew argument changed |
replace_by_request | -replace=ADDR |
replace_by_triggers | replace_triggered_by |
delete_because_no_resource_config | Code removed |
Copy these into your toolbox. All were run against a real Terraform 1.15 plan.
P=tfplan.json
# every non-trivial change, one line each
jq -r '.resource_changes[]
| select(.change.actions != ["no-op"])
| "\(.change.actions | join(","))\t\(.address)"' $P
# create,delete terraform_data.cbd
# forget terraform_data.forgot
# delete terraform_data.gone
# create terraform_data.new
# delete,create terraform_data.rep
# update terraform_data.upd
# count by action
jq -r '[.resource_changes[].change.actions | join(",")]
| group_by(.) | map("\(.[0]): \(length)") | .[]' $P
# anything that deletes (delete or replace)
jq -r '.resource_changes[]
| select(.change.actions | index("delete"))
| .address' $P# why is it replaced?
jq -c '.resource_changes[]
| select(.change.replace_paths)
| {address, paths: .change.replace_paths,
reason: .action_reason}' $P
# {"address":"terraform_data.rep",
# "paths":[["triggers_replace"]],
# "reason":"replace_because_cannot_update"}
# IAM and security group changes (access review)
jq -r '.resource_changes[]
| select(.type | test("^aws_(iam_|security_group)"))
| select(.change.actions != ["no-op"])
| .address' $P
# moved objects
jq -r '.resource_changes[]
| select(.previous_address)
| "\(.previous_address) -> \(.address)"' $P
# does this plan change anything? (CI)
jq -e '[.resource_changes[]
| select(.change.actions != ["no-op"])] | length > 0' $Pscripts/plan-summary.sh (Lab 8) prints a reviewer-friendly table and fails when a protected resource type would be deleted or replaced.
#!/usr/bin/env bash
# usage: scripts/plan-summary.sh tfplan.json
set -euo pipefail
plan=${1:?usage: plan-summary.sh <plan.json>}
re=${PROTECTED_TYPES:-'^aws_(s3_bucket|vpc|subnet|kms_key|iam_role)$'}
jq -r '
{"create": "CREATE", "update": "UPDATE",
"delete": "DELETE", "forget": "FORGET",
"delete,create": "REPLACE(-/+)",
"create,delete": "REPLACE(+/-)"} as $label
| .resource_changes[] as $r
| ($r.change.actions | join(",")) as $a
| ($label[$a] // (if $r.previous_address
then "MOVE" else empty end))
| [., $r.address, ($r.action_reason // "")]
| @tsv' "$plan" | column -t -s $'\t'blocked=$(jq -r --arg re "$re" '
.resource_changes[]
| select((.change.actions | index("delete"))
and (.type | test($re)))
| .address' "$plan")
if [[ -n "$blocked" && "${ALLOW_DESTROY:-}" != "true" ]]; then
echo "BLOCKED: deletes/replaces protected:" >&2
echo "$blocked" >&2
exit 1
fi
echo "OK: protected-type gate passed"
# REPLACE(+/-) terraform_data.cbd
# FORGET terraform_data.forgot
# DELETE terraform_data.gone
# CREATE terraform_data.new
# MOVE terraform_data.new_name
# REPLACE(-/+) terraform_data.rep
# UPDATE terraform_data.upd
# OK: protected-type gate passedBLOCKED is not "never": it means "needs a decision". In CI a reviewer adds the PR label allow-destroy, which the pipeline turns into ALLOW_DESTROY=true (Lab 8 runs it by hand).
CI needs three answers from a plan: did it fail, does it change anything, and what exactly. Use exit codes for the first two, JSON for the third.
terraform plan -input=false -lock-timeout=5m \
-detailed-exitcode -out=tfplan
rc=$?
# 0 = success, no changes
# 1 = error
# 2 = success, there are changes
case $rc in
0) echo "no changes" ;;
2) terraform show -json tfplan > tfplan.json
scripts/plan-summary.sh tfplan.json ;;
*) exit 1 ;;
esac
# human text for the PR comment (no colors)
terraform show -no-color tfplan > tfplan.txt-input=false: a prompt hangs the job.-detailed-exitcode is the drift detector of ch 11: exit 2 on a schedule = drift.tfplan as an artifact for the apply job; the JSON and text are for humans and policies.terraform plan -json prints machine-readable log events (one JSON object per line), not the plan format above.show -json on a saved plan for review and policy.The handful of change types that cause most Terraform outages, and a checklist that makes reviewers look at them every time.
Classify every plan before reading the details. The level decides who must approve and how the apply is run.
| Level | Plan contains | Examples | Process |
|---|---|---|---|
| Low | Only creates, tag changes, no-ops, moves | New SSM parameter, extra tag, moved block | One reviewer; apply from the pipeline |
| Medium | In-place updates of non-data resources | Lambda memory, SG rule description, log retention | One reviewer who knows the service |
| High | Access changes, deletes of stateless objects, replacements with +/- | IAM policy, SG ingress, route table, replace a launch template | Owner + security for IAM/SG; apply in a change window |
| Critical | Delete or -/+ replace of stateful or shared objects | S3 bucket with data, RDS, KMS key, VPC/subnet, IAM role used by other systems | Explicit override, backup/snapshot first, rollback plan, two approvers |
Automate the classification with the plan JSON (type + actions) and let CI put the level in the PR comment. Humans then spend their attention on High and Critical.
Replacing these loses data or breaks other systems, and Terraform cannot undo it.
force_destroy = true).identifier or engine change → new instance; final snapshot only if skip_final_snapshot = false.Guards: lifecycle { prevent_destroy = true } on the resource (ch 3), protected types in the plan gate (this chapter), deletion protection in AWS itself (deletion_protection = true on RDS), S3 versioning and backups.
Small diffs in the code that produce large plans. Recognize them in the PR before running the plan.
| Change in code | What the plan does | Safer way |
|---|---|---|
Remove an item from a list used with count | Shifts every later index: updates or replaces items after it | Use for_each with stable keys (ch 2); moved blocks to migrate |
| Rename a resource or module label | Destroy + create | moved block (ch 5) |
Change a for_each key format (e.g. AZ name → index) | Every instance is replaced | Keep keys stable, or moved per key |
Edit a value that feeds name or cidr_block | Replacement (ForceNew) | Check provider docs; create new + migrate |
for_each over values known only after apply | Plan error: "Invalid for_each argument" | Use keys known at plan (variables, static maps) |
| Bump a provider major version | New defaults, renamed attributes, sometimes replacements | Upgrade guide + a plan with no other changes (ch 13) |
| Move code between root modules | Destroy in one state, create in the other | removed + import blocks, or a state mv plan (ch 5) |
Flags that change what the plan covers. Each is legitimate in an incident and dangerous as a habit.
-target=ADDR-refresh=false and -refresh-only-refresh=false: faster, but plans against stale state; drift is invisible.-refresh-only (plan or apply): updates state to match reality, changes no infrastructure. Review it like any plan.-minimal-refresh (1.17): refreshes only what the plan needs; read its docs before using it in CI.-replace=ADDRaction_reason = replace_by_request).terraform taint workflow; shows in the plan, so it is reviewable.create_before_destroy to avoid a gap.Paste it into the PR template. Each line takes seconds with the plan summary in front of you.
live/<env>/<component>).-target).-, -/+, +/-: which argument forces it, is data lost, is there a backup?Project, Env, ManagedBy).Apply exactly what was reviewed, handle locks and partial failures, and know what "rollback" means in Terraform.
The reviewed artifact is the plan file. terraform apply tfplan executes it without re-planning; terraform apply alone plans again.
If anything changed in between (another apply, a lock, a drift fix), Terraform refuses the saved plan instead of applying a different change. That refusal is a feature: re-plan and review again.
Two applies on the same state cannot run at once (ch 4). The tools below decide what happens when they collide.
# a plan saved before someone else applied:
terraform apply a.tfplan
# Error: Saved plan is stale
#
# The given plan file can no longer be applied
# because the state was changed by another
# operation after the plan was created.
# wait for a lock instead of failing at once
terraform plan -lock-timeout=5m -out=tfplan
terraform apply -lock-timeout=5m tfplan
# a lock left by a killed job (check first!)
terraform force-unlock <LOCK_ID>concurrency group per state (one apply per live/<env>/<component> at a time, ch 11).-lock-timeout long enough for a normal plan of another job.-lock=false against shared state.An apply is not a transaction. When one resource fails, the ones already created stay, and state records them.
terraform untaint ADDR if you verified it.errored.tfstate in the working directory → terraform state push errored.tfstate after review.terraform rollback.git revert the change, plan, review, apply. It goes through the same gates.Normal changes go through the pipeline. Emergencies need a documented path that is still auditable.
gha-tf-apply), after merge (ch 11).prod with required reviewers gates the apply job.-refresh-only review.force-unlock without an incident ticket.All of these have caused real outages; most were visible in the plan.
# forces replacement on a renamed bucket or role.terraform apply (re-plan) instead of terraform apply tfplan.-auto-approve on a laptop against prod.-target as a routine, leaving state stale.force-unlock while another apply was still running.prevent_destroy and AWS deletion protection on stateful resources.Use the exact symbol, JSON field or flag in each answer.
-/+ and +/-, and which lifecycle setting produces the second?. mean in front of a resource in a plan?terraform plan -json?delete.replace_paths and action_reason?-target and of -refresh=false.Practice: Lab 8 in Hands-on labs — write scripts/plan-summary.sh, trigger a bucket rename and a CIDR change, watch the gate block them, then apply only a reviewed, saved plan.
Turn security and team rules into checks that run on every change: static scans of HCL, policies over the plan, and how to handle findings without drowning in them.
Guardrails that review every change the same way, before anything reaches AWS.
Code review catches design problems. Machines are better at "every bucket blocks public access" on the 400th pull request.
0.0.0.0/0 on SSH, IAM without *:*.#checkov:skip=…:reason).The earlier a check runs, the cheaper the fix. The later it runs, the more it knows (resolved values, the actual plan).
Run the same tools with the same config everywhere. A check that only runs in CI produces "works on my machine" arguments; a check that only runs locally is optional.
Scanning HCL is fast and points at a file and line. Scanning the plan JSON sees what Terraform will actually do.
| Static (HCL directory) | Plan (terraform show -json tfplan) | |
|---|---|---|
| Needs credentials / init | No | Yes: a real plan against the backend and provider |
Variables and default_tags | Partly resolved; provider default_tags not merged | Fully resolved: tags_all has every tag |
| References between resources | Resolved by the scanner from code | IDs are (known after apply) on create |
| Actions (create, delete, replace) | Unknown | Exact: resource_changes[].change.actions |
| Points to file and line | Yes | Only with enrichment (--repo-root-for-plan-enrichment in Checkov) |
| Best for | Misconfigurations inside modules | Tags, deletes, values that depend on variables |
Tested in Lab 9: a "required tags" check fails on HCL when tags come from provider default_tags, and passes on the plan. Trivy on plan JSON reports a missing public access block that the HCL scan correctly links to the bucket.
A policy program dies when findings pile up and everybody learns to click past red.
block_public_acls = true).#checkov:skip=CKV_AWS_144:single-region lab data.#trivy:ignore:AWS-0089 above the block, or an ID in .trivyignore.--create-baseline writes .checkov.baseline; --baseline applies it.Gate on severity where the tool has it: trivy config --severity HIGH,CRITICAL --exit-code 1. Checkov OSS built-in checks report no severity without a Prisma Cloud API key, so gate with --check/--skip-check lists instead.
Graph-aware static analysis with 1000+ built-in checks, custom YAML/Python checks, and plan scanning.
Checkov builds a graph of resources and their connections, so it knows a public access block belongs to a bucket.
python3 -m pip install checkov # or: brew install checkov
checkov --version # 3.x
checkov -d modules/app-bucket --compact --quiet
terraform scan results:
Passed checks: 10, Failed checks: 5, Skipped checks: 0
Check: CKV2_AWS_62: "Ensure S3 buckets should have
event notifications enabled"
FAILED for resource: aws_s3_bucket.this
Check: CKV2_AWS_61: "Ensure that an S3 bucket has a
lifecycle configuration"
FAILED for resource: aws_s3_bucket.this
Check: CKV_AWS_18: "Ensure the S3 bucket has access
logging enabled"
FAILED for resource: aws_s3_bucket.this
Check: CKV_AWS_144: "Ensure that S3 bucket has
cross-region replication enabled"
FAILED for resource: aws_s3_bucket.this
Check: CKV_AWS_145: "Ensure that S3 buckets are encrypted
with KMS by default"
FAILED for resource: aws_s3_bucket.thisCKV_AWS_*: checks on one resource's attributes.CKV2_AWS_*: graph checks across connected resources (bucket ↔ public access block, VPC ↔ flow log).--quiet shows only failures; --compact hides code snippets.--soft-fail always returns 0.app-bucket module (Lab 9).Same scan on modules/network. Each finding is a design decision to make, not a box to tick.
| Check | Finding | tf-lab decision |
|---|---|---|
CKV_AWS_130 | Subnets assign public IPs by default | Keep for public subnets only; suppress with reason there |
CKV2_AWS_11 | VPC flow logging not enabled | Fix: flow logs to a CloudWatch log group with short retention |
CKV2_AWS_12 | Default security group does not restrict all traffic | Fix: aws_default_security_group with no rules |
CKV_AWS_18 | S3 access logging (app-bucket) | Suppress in the lab; production: server access logs or CloudTrail data events |
CKV_AWS_145 | S3 not encrypted with KMS by default | Lab: SSE-S3 is enough (suppress); production: SSE-KMS with a CMK |
CKV_AWS_144, CKV2_AWS_62 | Replication, event notifications | Not needed: skip in config with a comment |
Write the decision down where the next engineer sees it: the suppression comment, not a wiki page.
Inline skips belong to one resource; config-level skips switch a check off for the whole scan.
# modules/app-bucket/main.tf
resource "aws_s3_bucket" "this" {
#checkov:skip=CKV_AWS_144:single-region lab data
#checkov:skip=CKV_AWS_18:audited via CloudTrail
bucket = var.name
}# .checkov.yaml (picked up from the current directory)
framework:
- terraform
skip-check:
- CKV2_AWS_62 # S3 event notifications: not used in tf-lab
- CKV2_AWS_61 # lifecycle rules: buckets are short-lived
compact: true
quiet: true#checkov:skip=<ID>:<reason> inside the resource block.Skipped checks: 1) and listed with the reason, so they stay visible.--; --config-file points to another path.--skip-path excludes directories (e.g. examples/), not individual findings.YAML checks need no Python: conditions on attributes, connections and resource types. This one requires three tags.
# policies/checkov/required_tags.yaml
metadata:
id: "CKV2_TFLAB_1"
name: "Ensure taggable resources carry Project, Env and ManagedBy tags"
category: "GENERAL_SECURITY"
severity: "MEDIUM"
definition:
and:
- cond_type: attribute
resource_types: [aws_s3_bucket, aws_vpc, aws_subnet, aws_internet_gateway]
attribute: tags_all.Project
operator: exists
- cond_type: attribute
resource_types: [aws_s3_bucket, aws_vpc, aws_subnet, aws_internet_gateway]
attribute: tags_all.Env
operator: exists
- cond_type: attribute
resource_types: [aws_s3_bucket, aws_vpc, aws_subnet, aws_internet_gateway]
attribute: tags_all.ManagedBy
operator: exists--external-checks-dir policies/checkov; select with --check CKV2_TFLAB_1.tags_all = resource tags merged with provider default_tags. It only exists in the plan.default_tags — run it on the plan JSON.equals, not_equals, contains, within, regex_match, is_true, …; cond_type: connection for graph rules.BaseResourceCheck) cover logic YAML cannot express.Give Checkov the JSON of a saved plan. Resolved variables, tags_all and module calls are all there.
cd live/dev/app
terraform plan -out=tfplan
terraform show -json tfplan > tfplan.json
checkov -f tfplan.json \
--external-checks-dir ../../../policies/checkov \
--check CKV2_TFLAB_1 --compact --quiet
terraform_plan scan results:
Passed checks: 5, Failed checks: 0, Skipped checks: 0
# all built-in checks on the plan, findings mapped
# back to files so inline skips are honoured
checkov -f tfplan.json \
--repo-root-for-plan-enrichment ../../.. \
--compact --quietterraform_plan; force it with --framework terraform_plan.#checkov:skip comments are not seen.tfplan.json contains every resolved value, including secrets: treat it like state (ch 12).plan-summary.sh and conftest (section 35).One command, machine-readable output, and a clear pass/fail.
# several outputs in one run
checkov -d modules \
-o cli -o sarif --output-file-path console,results.sarif
# adopt on a legacy repo: fail only on NEW findings
checkov -d modules --create-baseline # writes modules/.checkov.baseline
checkov -d modules --baseline modules/.checkov.baseline
# report but never fail (first weeks of adoption)
checkov -d modules --soft-fail-o values include cli, json, junitxml, sarif, github_failed_only.--soft-fail-on / --hard-fail-on take check IDs or severities (severities need the platform API key for built-in checks).pip install checkov==3.x.y or a pinned image digest): new releases add checks and can break a green build.One scanner for images, filesystems and IaC; trivy config carries the former tfsec checks.
trivy config on Terraformtfsec's checks now live in Trivy; tfsec itself is no longer developed separately. IDs look like AWS-0089 (older output and docs show AVD-AWS-0089).
trivy --version # 0.7x
trivy config modules
app-bucket/main.tf (terraform)
Tests: 2 (SUCCESSES: 0, FAILURES: 2)
Failures: 2 (LOW: 1, MEDIUM: 0, HIGH: 1, CRITICAL: 0)
AWS-0089 (LOW): Bucket has logging disabled
AWS-0132 (HIGH): Bucket does not encrypt data with
a customer managed key.
network/main.tf (terraform)
Tests: 3 (SUCCESSES: 0, FAILURES: 3)
Failures: 3 (LOW: 0, MEDIUM: 1, HIGH: 2, CRITICAL: 0)
AWS-0164 (HIGH): Subnet associates public IP address.
via network/main.tf:6-11 (aws_subnet.public[0])
AWS-0164 (HIGH): Subnet associates public IP address.
via network/main.tf:6-11 (aws_subnet.public[1])
AWS-0178 (MEDIUM): VPC does not have VPC Flow Logs enabled.AWS-0164 ≈ CKV_AWS_130, AWS-0178 ≈ CKV2_AWS_11.--skip-check-update uses the bundled set (offline, reproducible).Fail the build on what matters, and record every exception next to the code or in one reviewed file.
# CI gate: only HIGH and CRITICAL fail the build
trivy config --severity HIGH,CRITICAL --exit-code 1 modules
# inline: the comment goes directly above the block
#trivy:ignore:AWS-0089 -- access is audited with CloudTrail data events
resource "aws_s3_bucket" "this" {
bucket = var.name
}
# .trivyignore in the directory you run trivy from
# (read automatically; --ignorefile points elsewhere)
AWS-0132 # lab buckets use SSE-S3; prod module uses a CMK
# pass variables so conditionals and counts resolve
trivy config --tf-vars live/dev/app/dev.tfvars modules
# SARIF for GitHub code scanning
trivy config -f sarif -o trivy.sarif modules.trivyignore is picked up from the working directory: running from another directory silently drops it.#trivy:ignore:<ID> is free text — use it for the reason.--exit-code 1 is needed: by default Trivy exits 0 even with failures.--tf-exclude-downloaded-modules skips findings inside registry modules you cannot change.--skip-dirs excludes paths such as examples/.Trivy also reads plan JSON (terraformplan-json) and binary plan files. Unknown IDs at plan time can break the link between resources.
terraform show -json tfplan > tfplan.json
trivy config tfplan.json
main.tf (terraformplan)
Failures: 11 (LOW: 2, MEDIUM: 2, HIGH: 7, CRITICAL: 0)
AWS-0086 (HIGH): No public access block so not blocking public acls
AWS-0087 (HIGH): No public access block so not blocking public policies
AWS-0090 (MEDIUM): Bucket does not have versioning enabled
AWS-0091 (HIGH): No public access block so not blocking public acls
AWS-0093 (HIGH): No public access block so not restricting public buckets
AWS-0094 (LOW): Bucket does not have a corresponding public access block.
...aws_s3_bucket_public_access_block.bucket is (known after apply) in the plan.aws_s3_bucket.this.id is visible.Trivy evaluates Rego checks against a normalized cloud model (input.aws.s3.buckets, …), so one check works for Terraform, CloudFormation and plan files.
# policies/trivy/bucket_prefix.rego
# METADATA
# title: S3 bucket names must start with tflab-
# custom:
# id: TFLAB-0001
# severity: MEDIUM
# input:
# selector:
# - type: cloud
# subtypes:
# - service: s3
# provider: aws
package user.tflab.bucket_prefix
import rego.v1
deny contains res if {
some bucket in input.aws.s3.buckets
not startswith(bucket.name.value, "tflab-")
msg := sprintf("bucket %q does not start with tflab-",
[bucket.name.value])
res := result.new(msg, bucket.name)
}trivy config \
--config-check policies/trivy \
--check-namespaces user \
modules
TFLAB-0001 (MEDIUM): bucket "random-name"
does not start with tflab-# METADATA block gives ID, severity and which inputs the check applies to.result.new(msg, attr) attaches the finding to the attribute's file and line.--check-namespaces (here user).contains, if, in).On 19 Mar 2026, attackers (the TeamPCP campaign) force-pushed most version tags of aquasecurity/trivy-action and setup-trivy to malicious commits that stole CI secrets.
aquasecurity/trivy-action@<tag> ran attacker code with the job's secrets and tokens.checkov==x.y.z).The same reasoning applies to providers and modules: lock file hashes and pinned module versions (ch 12).
Your own rules over the plan JSON, and HashiCorp's native policy framework.
conftest runs OPA Rego policies against structured files. For Terraform, the input is terraform show -json tfplan.
{
"format_version": "1.2",
"terraform_version": "1.17.0",
"resource_changes": [
{
"address": "module.bucket.aws_s3_bucket.this",
"module_address": "module.bucket",
"type": "aws_s3_bucket",
"name": "this",
"change": {
"actions": ["delete"],
"before": { "bucket": "tflab-dev-app-…", "...": "..." },
"after": null
}
},
{
"address": "module.bucket.aws_s3_bucket_public_access_block.this",
"type": "aws_s3_bucket_public_access_block",
"change": {
"actions": ["update"],
"after": { "block_public_acls": false, "...": "..." }
}
}
]
}resource_changes[].type, .address, .change.actions.["create"], ["update"], ["delete"], ["no-op"], ["read"], and replacement as ["delete","create"] or ["create","delete"]..change.before / .change.after: full attribute values (null on create/delete side)..change.after_unknown: which values are known only after apply.configuration and prior_state are also present for advanced rules."delete" in actions catches deletes and replacements.Two rules: never delete or replace buckets or their public access block, and never weaken a public access block setting.
# policies/opa/s3.rego
package main
import rego.v1
protected := {"aws_s3_bucket", "aws_s3_bucket_public_access_block"}
pab_settings := [
"block_public_acls", "block_public_policy",
"ignore_public_acls", "restrict_public_buckets",
]
deny contains msg if {
some rc in input.resource_changes
rc.type in protected
"delete" in rc.change.actions
msg := sprintf("%s: %v is not allowed for %s",
[rc.address, rc.change.actions, rc.type])
}
deny contains msg if {
some rc in input.resource_changes
rc.type == "aws_s3_bucket_public_access_block"
some setting in pab_settings
rc.change.after[setting] != true
msg := sprintf("%s: %s must be true", [rc.address, setting])
}package main is the namespace conftest evaluates by default.deny rules fail the run; warn rules print but pass; violation rules can return structured results.deny contains msg if { … } adds one message per matching resource.change.after is null, so the second rule simply does not match; the first one already fails.contains, if, in): current conftest releases are built on OPA v1; import rego.v1 keeps it valid on older ones.The policy is code, so it gets unit tests with small fake plans.
conftest test --policy policies/opa tfplan.json
FAIL - tfplan.json - main - module.bucket.aws_s3_bucket.this:
["delete"] is not allowed for aws_s3_bucket
# summary: N tests, N passed, N warnings,
# N failures, N exceptions
# exit code: 1 on any failure → usable as a CI gate
echo $? # 1# policies/opa/s3_test.rego
package main
import rego.v1
test_bucket_delete_denied if {
count(deny) == 1 with input as {"resource_changes": [{
"address": "aws_s3_bucket.x",
"type": "aws_s3_bucket",
"change": {"actions": ["delete"], "after": null},
}]}
}
# run the tests (no plan needed)
# conftest verify --policy policies/opa--data policies/opa/exceptions.json) changed in the same PR.--all-namespaces runs every package, not just main.-o json, -o junit, -o github change the output format.HashiCorp's newer policy framework: policies written in HCL and evaluated by Terraform itself during plan/apply, locally or in HCP Terraform.
# policies/tfpolicy/s3.policy.hcl — shape only;
# check the Terraform Policy reference for current syntax
policy {
required_providers {
aws = { source = "hashicorp/aws" }
}
}
resource_policy "aws_s3_bucket_public_access_block" "all_blocked" {
enforcement_level = "mandatory"
enforce {
condition = attrs.block_public_acls == true
error_message = "Public ACLs must be blocked."
}
enforce {
condition = attrs.restrict_public_buckets == true
error_message = "Public bucket access must be restricted."
}
}-policies on plan, apply and init no longer needs -allow-experimental-features.resource_policy, provider_policy and module_policy, each with one or more enforce { condition, error_message } blocks; a file needs a policy { required_providers { … } } block.mandatory and mandatory_overridable.tfpolicy CLI validates and tests policies (tfpolicy validate, tfpolicy test).New and moving fast: treat the snippet as an illustration and copy real policies from developer.hashicorp.com/terraform/policy. The labs use Checkov, Trivy and conftest.
Most teams end up with two or three layers. Pick by where the rule needs its data, not by tool loyalty.
| Rule | Best tool | Why |
|---|---|---|
| Secure defaults inside modules (encryption, public access, flow logs) | Trivy and/or Checkov on HCL | Fast, file and line, no credentials |
Required tags with default_tags | Checkov YAML on plan, or conftest | Only the plan has tags_all |
| No deletes/replacements of stateful resources | conftest on plan JSON | Needs change.actions |
| Org-specific naming, regions, instance types | Checkov YAML, Trivy Rego, or conftest | Whichever your team already reads |
| Blocking applies in HCP Terraform / TFE | Terraform Policy, Sentinel, OPA policy sets | Enforced by the platform, not by CI scripts |
| Anything done outside Terraform | SCPs, AWS Config, IAM boundaries | Policies on code cannot see ClickOps |
tf-lab uses Trivy + Checkov on HCL, Checkov + conftest on the plan, all behind one make policy target (Lab 9) that CI runs unchanged (Lab 11).
Each one turns a gate into noise or into a false sense of safety.
live/ and never modules/ (or the reverse).--soft-fail left on after the adoption phase.skip-check used for one resource's exception..trivyignore nobody reviews..trivyignore silently.trivy-action tags were hijacked).Answer with the exact flag, field or ID where one exists.
default_tags?CKV_AWS_* and CKV2_AWS_* checks?.trivyignore live to be picked up?trivy-action on 19 Mar 2026, and which control would have prevented it?aws_s3_bucket.-policies?Practice: Lab 9 in Hands-on labs — scan modules and plans with Checkov and Trivy, fix or justify every finding, add a required-tags check and an S3 delete policy, and wire it all into make policy.
Where code, state and environments live: directories vs workspaces, promoting the same module from dev to prod, wiring components together, and when to reach for Terragrunt, Terramate or Stacks.
Three common ways to express environments, and the one tf-lab uses.
Every layout answers the same questions: where is the code, where is the state, and how different may dev and prod be?
| Directory per environment | CLI workspaces | One root + tfvars per env | |
|---|---|---|---|
| Shape | live/dev/network, live/prod/network | One directory, terraform workspace select prod | One directory, -var-file=prod.tfvars + -backend-config=prod.hcl |
| State | Separate key per directory | Same backend, key prefix env:/<workspace>/ | Separate key via backend config |
| Visible in the repo? | Yes: ls live/ shows every environment | No: the current workspace is local CLI state | Partly: tfvars files, but the active one is a flag |
| Different shape per env | Easy (prod has extra resources) | Only via terraform.workspace conditionals | Only via variables and count |
| Credentials per env | Natural: one role per directory | Hard: same provider config for all | Possible via variables |
| Risk | Some duplication (thin root modules) | Applying to the wrong workspace | Forgetting a flag, mixing state and tfvars |
HashiCorp's own guidance: CLI workspaces are not a good fit for environments that need separate credentials or access control. Use them for short-lived copies (a feature branch test), not for dev/prod.
Reusable logic lives in modules/. Each directory under live/ is a small root module: backend, provider, one module call, outputs.
tf-lab/
├── bootstrap/ # state bucket (Lab 0), local state
├── modules/
│ ├── network/ # VPC, subnets, IGW (no NAT)
│ ├── app-bucket/ # S3 + encryption + PAB
│ └── ci-role/ # GitHub OIDC role (Lab 11)
├── live/
│ ├── dev/
│ │ ├── network/ backend.tf main.tf terraform.tfvars
│ │ └── app/ backend.tf main.tf terraform.tfvars
│ └── prod/
│ ├── network/ backend.tf main.tf terraform.tfvars
│ └── app/ backend.tf main.tf terraform.tfvars
├── policies/ # checkov, trivy, opa (ch 9)
├── scripts/ # plan-summary.sh (Lab 8)
└── Makefile # fmt, test, policy, plan-all
state keys in tflab-state-<account_id>:
dev/network/terraform.tfstate prod/network/terraform.tfstate
dev/app/terraform.tfstate prod/app/terraform.tfstateBoth work. The question is how a module change reaches production.
live/ refers to modules by relative path: source = "../../../modules/network".source = "git::https://github.com/acme/tf-modules.git//network?ref=network-v1.4.0".ref in dev, then in prod: an explicit, reviewable diff.terraform-aws-modules/vpc/aws) are always pinned with version.source/version; keep them constant per environment so plans stay reviewable.Workspaces are real and useful — for the right job.
cd live/app
terraform workspace new prod
terraform workspace select prod
terraform plan -var-file=prod.tfvars
# state lands at (S3 backend):
# s3://tflab-state-<acct>/env:/prod/app/terraform.tfstate
# the selected workspace is stored locally in
# .terraform/environment
# in code
locals {
env = terraform.workspace # "default", "prod", ...
instance_count = terraform.workspace == "prod" ? 3 : 1
}apply in the wrong one is one forgotten command away.terraform.workspace == conditionals across the code.Good uses: a disposable copy per pull request or developer (pr-123), destroyed afterwards. HCP Terraform "workspaces" are a different concept: one state + settings, closer to one live/ directory here.
Two environments, two components each, four states, one module source.
Arrows from modules/ to prod are omitted for clarity: prod calls the same module sources. Blue arrows are data flowing through SSM parameters (section 38).
Same code, different inputs: what may differ between dev and prod, and how a change moves forward.
Only the inputs differ. The module call is identical, so prod runs exactly the code dev tested.
# live/prod/network/main.tf
terraform {
required_version = ">= 1.14"
required_providers {
aws = { source = "hashicorp/aws", version = "~> 6.0" }
}
}
provider "aws" {
region = "ap-northeast-1"
default_tags {
tags = { Project = "tf-lab", Env = var.env, ManagedBy = "terraform" }
}
}
variable "env" { type = string }
variable "vpc_cidr" { type = string }
module "network" {
source = "../../../modules/network"
env = var.env
cidr = var.vpc_cidr
}# live/prod/network/terraform.tfvars
# (auto-loaded: plain `terraform plan` in CI
# needs no -var-file flag)
env = "prod"
vpc_cidr = "10.20.0.0/16"
# live/prod/network/backend.tf
terraform {
backend "s3" {
key = "prod/network/terraform.tfstate"
region = "ap-northeast-1"
encrypt = true
use_lockfile = true
# bucket comes from -backend-config: it
# contains the account ID and backend
# blocks cannot use variables
}
}
terraform init \
-backend-config="bucket=$TF_STATE_BUCKET"
terraform plan -out=tfplanIf dev and prod differ in shape, dev stops predicting prod. Differ in size and safety settings, not in architecture.
Env).deletion_protection, longer backups in prod.make policy gate for every environment.diff live/{dev,prod}/network/terraform.tfvars.validation blocks, ch 2) so prod cannot get a dev-sized value by mistake.A change is proven in dev, then the identical code is applied to prod. How "identical" is guaranteed depends on the layout.
Avoid changing prod first. If an emergency forces it, make the change in code (never in the console), apply prod, then bring dev to the same commit the same day.
Separate AWS accounts give the strongest isolation: a dev role cannot touch prod, even by mistake.
# live/prod/network/main.tf — prod in its own account
provider "aws" {
region = "ap-northeast-1"
assume_role {
role_arn = "arn:aws:iam::<prod-acct>:role/tflab-deployer"
session_name = "tf-prod-network"
}
allowed_account_ids = ["<prod-acct>"] # refuse anything else
default_tags {
tags = { Project = "tf-lab", Env = "prod", ManagedBy = "terraform" }
}
}allowed_account_ids makes Terraform error out if the credentials point at another account — cheap insurance.How the app component learns the VPC ID without sharing a state file.
The app needs vpc_id from network. Choose how tightly the two states are coupled.
| Option | How | Pros | Cons |
|---|---|---|---|
| SSM Parameter Store (tf-lab) | network writes /tflab/dev/network/vpc_id; app reads it with data "aws_ssm_parameter" | Explicit contract; readers need only ssm:GetParameter; usable by non-Terraform tools | One more resource per output; values are strings |
terraform_remote_state | app reads network's state file from S3 | No extra resources; typed outputs | Reader needs read access to the whole state (all attributes, secrets) |
| Data source lookup | data "aws_vpc" filtered by tags | No coupling to Terraform at all | Fragile if tags are not unique; implicit contract |
| Orchestrator | Terragrunt dependency, Stacks component outputs | Typed, ordered, mocked in plan | Another tool or platform (section 39) |
Whatever you choose, treat published outputs as an API: add new ones freely, rename or remove them only with a migration (like module outputs, ch 6).
The producer owns the parameter names; the consumer depends only on those names.
# live/dev/network/outputs.tf — producer
resource "aws_ssm_parameter" "vpc_id" {
name = "/tflab/${var.env}/network/vpc_id"
type = "String"
value = module.network.vpc_id
}
resource "aws_ssm_parameter" "private_subnet_ids" {
name = "/tflab/${var.env}/network/private_subnet_ids"
type = "StringList"
value = join(",", module.network.private_subnet_ids)
}# live/dev/app/main.tf — consumer
data "aws_ssm_parameter" "vpc_id" {
name = "/tflab/${var.env}/network/vpc_id"
}
data "aws_ssm_parameter" "private_subnet_ids" {
name = "/tflab/${var.env}/network/private_subnet_ids"
}
locals {
# insecure_value: non-sensitive copy for String/StringList
# (value is always marked sensitive)
vpc_id = data.aws_ssm_parameter.vpc_id.insecure_value
private_subnet_ids = split(",",
data.aws_ssm_parameter.private_subnet_ids.insecure_value)
}
# app plan fails (ParameterNotFound) until network
# is applied: the dependency order made visibleStandard-tier SSM parameters are free. Do not put secrets here as String; secrets belong in Secrets Manager or SecureString and stay out of state where possible (ch 12).
terraform_remote_state and why tf-lab avoids itIt works and is common. Know the trade-off before you depend on it.
data "terraform_remote_state" "network" {
backend = "s3"
config = {
bucket = "tflab-state-<account_id>"
key = "dev/network/terraform.tfstate"
region = "ap-northeast-1"
}
}
locals {
vpc_id = data.terraform_remote_state.network.outputs.vpc_id
}s3:GetObject on the producer's state key: state access spreads across teams.Separate states mean Terraform no longer orders things for you. Your pipeline or script has to.
network → app.needs:, Terragrunt, Stacks) — not in people's heads.(known after apply) or fail until the producer is applied: plan in order too.app → network. Destroying network first fails (dependent ENIs, security groups) or orphans the app.prevent_destroy on resources that other states depend on (ch 3).Plain shell is enough for a handful of components. Lab 10 builds this as make plan-all.
#!/usr/bin/env bash
# scripts/plan-all.sh ENV — plan every component in order
set -euo pipefail
ENV=${1:?usage: plan-all.sh dev|prod}
ORDER=(network app) # producers first
for c in "${ORDER[@]}"; do
dir="live/$ENV/$c"
echo "== $dir"
terraform -chdir="$dir" init -input=false \
-backend-config="bucket=$TF_STATE_BUCKET" >/dev/null
set +e
terraform -chdir="$dir" plan -input=false \
-out=tfplan -detailed-exitcode
rc=$?
set -e
case $rc in
0) echo " no changes" ;;
2) echo " changes pending" ;;
*) echo " plan FAILED"; exit $rc ;;
esac
done-chdir runs Terraform in another directory without cd; plan files are written relative to that directory.-detailed-exitcode: 0 = no changes, 1 = error, 2 = changes present (ch 1). set -e must not treat 2 as failure.live/dev/app/tfplan) for review and for Lab 8's summary script.app reads SSM values that network has not applied yet, its plan fails — the script stops at the first error on purpose.Terragrunt, Terramate and HCP Terraform Stacks — what they add and what they cost.
A thin wrapper around Terraform/OpenTofu. Terragrunt 1.0 (Mar 2026) froze its CLI and HCL syntax for the 1.x line.
# live/root.hcl — shared by every unit
remote_state {
backend = "s3"
generate = {
path = "backend.tf"
if_exists = "overwrite_terragrunt"
}
config = {
bucket = "tflab-state-${get_aws_account_id()}"
key = "${path_relative_to_include()}/terraform.tfstate"
region = "ap-northeast-1"
encrypt = true
use_lockfile = true
}
}# live/dev/app/terragrunt.hcl — one unit
include "root" {
path = find_in_parent_folders("root.hcl")
}
terraform {
source = "../../../modules//app-bucket"
}
dependency "network" {
config_path = "../network"
mock_outputs = { vpc_id = "vpc-00000000" }
mock_outputs_allowed_terraform_commands = ["plan", "validate"]
}
inputs = {
env = "dev"
vpc_id = dependency.network.outputs.vpc_id
}
# from live/dev, in dependency order:
# terragrunt run --all -- planThe state key becomes dev/app/terraform.tfstate — the same layout as plain tf-lab, so you can adopt or drop Terragrunt without moving state. Verify remote_state keys such as use_lockfile against the Terragrunt version you pin.
Two other answers to "many roots, one change".
stack {} block, usually in stack.tm.hcl).*.tfcomponent.hcl (components = module calls) and *.tfdeploy.hcl (deployments = environments, regions, accounts)..tfstack.hcl; GA renamed it to .tfcomponent.hcl.terraform stacks … (Terraform 1.13+); runs happen in HCP Terraform.Add a tool when the pain is real, not because the repo might grow.
| Signal | Plain Terraform + Makefile/CI | Add Terragrunt / Terramate / Stacks |
|---|---|---|
| Number of root modules | Up to a few dozen | Hundreds, many near-identical |
| Environments / regions / accounts | 2–4 environments, one region | Many accounts × regions (fan-out) |
| Backend/provider duplication | A few lines per root, rarely changed | Copy-paste drift between roots is a real problem |
| Ordering between states | One or two chains (network → app) | A deep graph that people get wrong |
| Team skills and tooling | Everyone knows plain Terraform | Team willing to learn and pin another tool |
| Platform | Self-run CI (GitHub Actions) | Already on HCP Terraform → Stacks is native |
tf-lab stays plain on purpose: four roots, one chain, a Makefile. Lab 10 shows the same layout in Terragrunt as a read-only comparison so you can judge the trade-off yourself.
Most of these are cheap to avoid on day one and expensive to undo once state exists.
terraform.workspace == "prod" conditionals across modules.terraform_remote_state across teams: everyone can read everyone's state.?ref=main) instead of a tag.key — two environments, one state.Explain each with a path, a key or a command.
prod?allowed_account_ids protect against?terraform_remote_state for passing vpc_id.network and app, and why?-detailed-exitcode return for "changes present"?dependency block with mock outputs for plan..tfcomponent.hcl and .tfdeploy.hcl for?Practice: Lab 10 in Hands-on labs — build live/prod/{network,app} from the same modules as dev, give each root its own backend key and tfvars, and plan all components in dependency order with one command.
Every change goes through a pull request: checks and a plan per component on the PR, a human reviews the plan, and the pipeline applies exactly that plan after merge.
What runs on a pull request, what runs after merge, and which plan gets applied.
Running terraform apply from laptops works for one person. With a team it produces surprises nobody reviewed.
main reaches real infrastructure.Two triggers, two credentials. The pull request proves the change is safe to apply; the merge applies it.
"Apply the reviewed plan" means: the plan the approver saw is the file that terraform apply tfplan executes. If anything changed in between, Terraform refuses the stale plan.
Cheap, credential-free checks first; anything that needs AWS runs only after them.
| Stage | Commands | AWS access | Fails the PR when |
|---|---|---|---|
| Format | terraform fmt -check -recursive | none | any file is not formatted |
| Validate | terraform init -backend=false + terraform validate | none | syntax or type errors |
| Lint | tflint --recursive | none | deprecated syntax, invalid instance types, unused declarations |
| Unit tests | terraform test in each module (mocks, ch 7) | none | an assertion fails |
| Policy | make policy: Checkov, Trivy, conftest (ch 9) | none (static) / plan JSON | a HIGH finding without a justified skip |
| Plan | terraform plan -out=tfplan per changed component | read-only + state lock | plan errors; risky changes are flagged for review (ch 8) |
| Apply | terraform apply tfplan after merge | write, per environment | apply error → fix forward with a new PR |
Plan-based policy (conftest on tfplan.json, ch 9) runs after the plan step, in the same job, because it needs the plan file.
tf-lab has one root module per environment and component. A PR that touches live/dev/app should not lock or plan live/prod/network.
# which root modules does this change touch?
BASE=$(git merge-base origin/main HEAD)
git diff --name-only "$BASE" HEAD > changed.txt
if grep -q '^modules/' changed.txt; then
# a shared module changed: plan every component
ls -d live/dev/* live/prod/* > dirs.txt
else
grep -oE '^live/(dev|prod)/[^/]+' changed.txt \
| sort -u > dirs.txt
fi
# JSON matrix for GitHub Actions
jq -R -s -c 'split("\n") | map(select(length > 0))
| map({dir: ., env: (split("/")[1]),
name: (split("/")[1:] | join("-"))})' dirs.txt
# [{"dir":"live/dev/app","env":"dev","name":"dev-app"}]live/shared/ci (the CI roles) stays out of the matrix: CI must not be able to rewrite its own permissions.network before app (ch 10).A saved plan is a contract: "these exact actions against this exact state". Terraform enforces the second half.
terraform apply tfplan performs only the actions in the file; no new diff is computed.retention-days: 1), never commit it, never attach it to a public PR.main after merge, approve, apply that file in the same workflow run.Two roles, three jobs, one concurrency group per state.
The OIDC exchange is covered in the CI/CD deck (ch 4). Here the point is splitting who may plan from who may apply.
| Role | Trusted sub (repo created after 15 Jul 2026) | Permissions |
|---|---|---|
gha-tf-plan | repo:<you>@<oid>/tf-lab@<rid>:pull_request and …:ref:refs/heads/main | ReadOnlyAccess + state read + lock-file write |
gha-tf-apply | …:environment:dev and …:environment:prod only | what the components create + state read/write |
prod has required reviewers.sub format with numeric IDs (CI/CD deck, ch 4).One apply role per environment (or one AWS account per environment with its own role) is stronger still; the lab uses one role and relies on environment protection to keep the setup small.
With use_lockfile = true, even a plan writes: it creates and deletes <key>.tflock next to the state object.
{
"Version": "2012-10-17",
"Statement": [
{ "Sid": "ListStateBucket",
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::tflab-state-<acct>" },
{ "Sid": "ReadState",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::tflab-state-<acct>/*" },
{ "Sid": "LockFiles",
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:PutObject",
"s3:DeleteObject"],
"Resource":
"arn:aws:s3:::tflab-state-<acct>/*.tflock" }
]
}s3:GetObject, s3:PutObject and s3:DeleteObject on the lock file when use_lockfile is on.s3:DeleteObject is never needed on the state file itself: Terraform does not delete it.s3:PutObject on */terraform.tfstate.kms:Decrypt (plan) and kms:Encrypt/kms:GenerateDataKey (apply).-lock=false) avoids the write but lets a plan race an apply: keep the lock.One workflow file handles both triggers. Credential-free checks run before anything touches AWS.
# .github/workflows/terraform.yml
name: terraform
on:
pull_request: {paths: ["live/**", "modules/**", "policies/**"]}
push: {branches: [main], paths: ["live/**", "modules/**"]}
permissions:
contents: read
id-token: write # OIDC
pull-requests: write # plan comments
jobs:
detect:
runs-on: ubuntu-latest
outputs:
dirs: ${{ steps.d.outputs.dirs }}
steps:
- uses: actions/checkout@<sha>
with: {fetch-depth: 0}
- id: d
run: echo "dirs=$(scripts/changed-dirs.sh)" >> "$GITHUB_OUTPUT" checks:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@<sha>
- uses: hashicorp/setup-terraform@<sha>
with: {terraform_version: 1.17.0}
- uses: terraform-linters/setup-tflint@<sha>
- run: terraform fmt -check -recursive
- run: tflint --init && tflint --recursive
- name: module tests (mocked, no AWS)
run: |
for m in modules/*/; do
(cd "$m" && terraform init -backend=false \
&& terraform test)
done
- run: make policy # Lab 9scripts/changed-dirs.sh is the diff script from section 40; on push it compares github.event.before with github.sha instead of the merge base.
Same job for PRs and for main. On a PR it comments; on main it uploads the plan for the apply job.
plan:
needs: detect
if: >-
needs.detect.outputs.dirs != '[]' &&
(github.event_name == 'push' ||
github.event.pull_request.head.repo.full_name
== github.repository)
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
c: ${{ fromJSON(needs.detect.outputs.dirs) }}
concurrency: # one run per state at a time
group: tf-${{ matrix.c.dir }}
cancel-in-progress: false
defaults: {run: {working-directory: "${{ matrix.c.dir }}"}}
steps:
- uses: actions/checkout@<sha>
- uses: hashicorp/setup-terraform@<sha>
with: {terraform_version: 1.17.0, terraform_wrapper: false} - uses: aws-actions/configure-aws-credentials@<sha>
with:
role-to-assume: arn:aws:iam::<acct>:role/gha-tf-plan
aws-region: ap-northeast-1
- run: terraform init -input=false -lockfile=readonly
-backend-config="bucket=tflab-state-<acct>"
- run: terraform plan -input=false -out=tfplan
- run: terraform show -no-color tfplan > plan.txt
- name: comment on the PR
if: github.event_name == 'pull_request'
env: {GH_TOKEN: "${{ github.token }}"}
run: |
{ echo "### ${{ matrix.c.dir }}"; echo '```'
head -c 60000 plan.txt; echo '```'; } > c.md
gh pr comment ${{ github.event.number }} -F c.md
- if: github.event_name == 'push'
uses: actions/upload-artifact@<sha>
with:
name: tfplan-${{ matrix.c.name }}-${{ github.run_attempt }}
retention-days: 1
path: ${{ matrix.c.dir }}/tfplanterraform_wrapper: false: real exit codes and plain stdout. -lockfile=readonly: fail if the lock file would change (ch 12). PR comments max out at 65,536 characters.
The job waits for environment approval before it gets credentials. Then it applies the file uploaded by the plan job of the same run.
apply:
needs: [detect, plan]
if: github.event_name == 'push'
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
c: ${{ fromJSON(needs.detect.outputs.dirs) }}
environment: ${{ matrix.c.env }} # dev | prod
concurrency:
group: tf-${{ matrix.c.dir }}
cancel-in-progress: false
defaults: {run: {working-directory: "${{ matrix.c.dir }}"}}
steps:
- uses: actions/checkout@<sha>
- uses: hashicorp/setup-terraform@<sha>
with:
terraform_version: 1.17.0
terraform_wrapper: false - uses: aws-actions/configure-aws-credentials@<sha>
with:
role-to-assume: arn:aws:iam::<acct>:role/gha-tf-apply
aws-region: ap-northeast-1
role-session-name: apply-${{ github.run_id }}
- uses: actions/download-artifact@<sha>
with:
name: tfplan-${{ matrix.c.name }}-${{ github.run_attempt }}
path: ${{ matrix.c.dir }}
- run: terraform init -input=false -lockfile=readonly
-backend-config="bucket=tflab-state-<acct>"
- run: terraform show -no-color tfplan # what we apply
- run: terraform apply -input=false tfplan
# GitHub → Settings → Environments:
# dev : no reviewers, branch main only
# prod: required reviewers, branch main only,
# prevent self-reviewThe sub of this job ends with environment:dev or environment:prod, so only these jobs can assume gha-tf-apply. The prod approver reads the plan in the plan job log of the same run before approving.
A matrix applies components in parallel. That is fine within one layer and wrong across layers.
app reads network outputs from SSM (ch 10): if both change, network must apply first.apply-network, then apply-app with needs:).run --all, Terramate, Stacks).terraform force-unlock <ID> by a person, after checking nothing is running.concurrency with cancel-in-progress: false queues runs; GitHub keeps at most one pending run per group and cancels older pending ones.Atlantis, HCP Terraform and hosted platforms do the same job with different trade-offs.
A self-hosted server that receives PR webhooks, runs plan for changed projects and applies when someone comments atlantis apply.
# atlantis.yaml (repo root)
version: 3
projects:
- name: dev-network
dir: live/dev/network
autoplan:
when_modified: ["*.tf", "../../../modules/network/**"]
- name: prod-network
dir: live/prod/network
apply_requirements: [approved, mergeable]
# in a PR comment:
# atlantis plan -p prod-network
# atlantis apply -p prod-networkapply_requirements adds approval and mergeability checks (the server must allow repo-level overrides).A managed backend plus remote runs. The cloud block replaces the S3 backend.
terraform {
cloud {
organization = "tf-lab-org"
workspaces {
name = "dev-network"
# or: tags = ["tf-lab"] (select many)
}
}
}
# terraform login: API token for the CLI
# terraform plan: runs remotely,
# logs stream to your terminalterraform plan/apply on a laptop or CI, executed remotely.All of them run the same Terraform. The differences are where plans live, who holds credentials, and what you have to operate.
| GitHub Actions (this deck) | Atlantis | HCP Terraform | |
|---|---|---|---|
| Where plans live | CI artifacts (you manage retention) | Atlantis server disk | HCP run history |
| Credentials | OIDC roles per job | Server role or per-project roles | Dynamic provider credentials or variables |
| Apply timing | After merge (environment approval) | Before merge by default | After merge (VCS) or on confirm |
| Locking | S3 lock file + concurrency | Project locks per PR + backend lock | Workspace lock |
| Policy | Checkov, Trivy, conftest steps | Built-in conftest policy checks | Sentinel, OPA, Terraform Policy |
| You operate | Workflow YAML | A server with cloud credentials | Nothing (SaaS); pay per resource |
Pick one runner per state. Two systems applying the same state is how locks get broken and drift appears.
Find out that reality changed before the next PR does.
A scheduled plan answers "would applying main change anything?". A refresh-only plan answers "did reality move away from state?".
terraform plan -detailed-exitcodemain** with refreshed reality.0: no changes, 1: error, 2: changes present.terraform plan -refresh-only -detailed-exitcodemain legitimately contains unapplied changes.apply -refresh-only) updates state, not resources.main to revert.ignore_changes only with a written reason (ch 3).Runs with the read-only plan role on main. Exit code 2 becomes one issue per component, not one per night.
# .github/workflows/drift.yml
name: drift
on: {schedule: [{cron: "0 21 * * *"}], workflow_dispatch: {}}
permissions: {contents: read, id-token: write, issues: write}
jobs:
drift:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
dir: [live/dev/network, live/dev/app,
live/prod/network, live/prod/app]
concurrency: {group: "tf-${{ matrix.dir }}",
cancel-in-progress: false}
steps:
- uses: actions/checkout@<sha>
- uses: hashicorp/setup-terraform@<sha>
with: {terraform_version: 1.17.0, terraform_wrapper: false}
- uses: aws-actions/configure-aws-credentials@<sha>
with: {role-to-assume: "arn:aws:iam::<acct>:role/gha-tf-plan",
aws-region: ap-northeast-1} # cron 21:00 UTC = 06:00 JST - name: plan
id: p
working-directory: ${{ matrix.dir }}
run: |
terraform init -input=false -lockfile=readonly \
-backend-config="bucket=tflab-state-<acct>"
set +e; terraform plan -input=false -no-color \
-detailed-exitcode > plan.txt; code=$?; set -e
echo "code=$code" >> "$GITHUB_OUTPUT"
[ "$code" -ne 1 ] # 1 = real error
- name: open or update an issue
if: steps.p.outputs.code == '2'
env: {GH_TOKEN: "${{ github.token }}"}
run: |
t="Drift: ${{ matrix.dir }}"
n=$(gh issue list --state open --search "$t in:title" \
--json number --jq '.[0].number')
{ echo '```'; tail -c 60000 ${{ matrix.dir }}/plan.txt
echo '```'; } > body.md
if [ -n "$n" ]; then gh issue comment "$n" -F body.md
else gh issue create -t "$t" -F body.md; fiUse the same concurrency key as the plan/apply workflow for each state. GitHub disables scheduled workflows in public repos after 60 days without activity.
Pinned versions are safe until they are old. Bots turn upgrades into ordinary reviewed PRs.
# .github/dependabot.yml
version: 2
updates:
- package-ecosystem: terraform
directories: ["/live/*/*", "/modules/*"]
schedule: {interval: weekly}
groups:
aws-provider:
patterns: ["hashicorp/aws"]
- package-ecosystem: github-actions
directory: "/"
schedule: {interval: weekly}terraform ecosystem updates provider and module versions; Renovate's terraform manager also updates required_version and can maintain the lock file.Each of these has caused a production outage or a credential leak somewhere.
-target in CI to "speed things up" (ch 8).repo:org/tf-lab:*.TF_VAR_ in logs with set -x.live/shared/ci in the matrix).Answer with the exact flag, permission or setting.
s3:PutObject with use_lockfile?live/shared/ci stay out of the pipeline?sub values does gha-tf-plan trust, and which does gha-tf-apply trust?terraform_wrapper: false change?-detailed-exitcode?-refresh-only the better drift check?Practice: Lab 11 in Hands-on labs — push tf-lab to GitHub, create the OIDC roles with modules/ci-role, and build the PR plan, gated apply and nightly drift workflows.
Who can change infrastructure, where secrets end up, and how to trust the providers and modules Terraform downloads.
Terraform runs with whatever power you give it. Give it as little as the plan or apply needs.
The identity that runs apply can usually create IAM roles. Whoever controls that identity controls the account.
aws sso login), never long-lived access keys.assume_role in the provider block from a central identity.Read the arrows as "may assume" or "may write". Everything else is denied by the bucket policy (Lab 12).
gha-tf-apply must not be able to edit gha-tf-apply. Keeping the CI roles in a separately applied root module (live/shared/ci) is what makes that true.
One identity logs in, Terraform assumes a narrow role in the target account. The role ARN is configuration; the secret material is not.
provider "aws" {
region = "ap-northeast-1"
assume_role {
role_arn = "arn:aws:iam::${var.account_id}:role/tflab-deployer"
session_name = "tf-${var.env}"
# optional: external_id, duration, tags,
# policy_arns to narrow the session
}
allowed_account_ids = [var.account_id] # wrong profile? stop.
default_tags {
tags = {
Project = "tf-lab"
Env = var.env
ManagedBy = "terraform"
}
}
}assume_role keeps the base credentials (SSO or OIDC) out of the configuration and gives CloudTrail a session name.allowed_account_ids makes Terraform refuse to run if your credentials point at another account: cheap insurance against applying dev to prod.access_key / secret_key in provider blocks or .tfvars; use the default credential chain.State is the most sensitive file Terraform produces: resource IDs, outputs, and any secret that went through a normal argument.
aws:SecureTransport = false).aws:PrincipalArn).s3:DeleteObject on *.tfstate away from CI: Terraform never deletes state objects.aws:PrincipalArn also applies to you if you get the ARN wrong./aws-reserved/sso.amazonaws.com/…): copy the role ARN, not the session ARN.bootstrap/, never from a CI-applied root module.An apply role that may call iam:CreateRole can create a role more powerful than itself. A permission boundary caps what any created role can do.
{
"Sid": "CreateRolesOnlyWithBoundary",
"Effect": "Allow",
"Action": ["iam:CreateRole", "iam:PutRolePolicy",
"iam:AttachRolePolicy"],
"Resource": "arn:aws:iam::<acct>:role/tflab-*",
"Condition": {
"StringEquals": {
"iam:PermissionsBoundary":
"arn:aws:iam::<acct>:policy/tflab-boundary"
}
}
}tflab-* name and only with permissions_boundary = tflab-boundary.iam:DeleteRolePermissionsBoundary and iam:CreatePolicyVersion on the boundary policy itself.resource "aws_iam_role" "x" { permissions_boundary = local.boundary_arn … }.gha-tf-apply gets no IAM write permissions at all.Every Terraform API call is a CloudTrail event signed by the role session. Good session names turn CloudTrail into a deployment log.
# which CI run created or changed things today?
aws cloudtrail lookup-events \
--lookup-attributes \
AttributeKey=Username,AttributeValue=apply-1234567890 \
--max-results 20 \
--query 'Events[].[EventTime,EventName]' --output table
# who assumed the apply role, and from which run?
aws cloudtrail lookup-events \
--lookup-attributes \
AttributeKey=EventName,AttributeValue=AssumeRoleWithWebIdentity \
--query 'Events[].CloudTrailEvent' --output text \
| jq -r 'select(.requestParameters.roleArn
| endswith("role/gha-tf-apply"))
| [.eventTime, .requestParameters.roleSessionName]
| @tsv'role-session-name: apply-${{ github.run_id }} in the workflow (ch 11) puts the run ID into every event.Username is the session name.lookup-events covers 90 days of management events in one Region; use a trail to S3 or CloudTrail Lake for longer retention and queries across accounts.ManagedBy = "terraform" (default_tags): an untagged resource in a managed VPC is a ClickOps signal.Where a password ends up depends on which kind of argument carries it.
Terraform shows (sensitive value) on screen. That is a display setting, not protection.
| Place | Contains secrets? | Notes |
|---|---|---|
Configuration (.tf, .tfvars in Git) | Only if you put them there | Never commit secrets; use variables, data sources or ephemeral resources |
| CLI output and CI logs | Masked for sensitive values | TF_LOG=TRACE can print request bodies; never enable it in CI |
Saved plan file (tfplan) | Yes, in clear text | Includes variable values and planned attributes; treat it like the secret itself (ch 11) |
| State (local or S3) | Yes, in clear text | Every attribute of every resource and data source, including sensitive ones |
terraform show -json, output -json | Yes | JSON output prints sensitive values unredacted |
| Ephemeral values / write-only arguments | No | Never written to plan or state (Terraform 1.10 / 1.11+) |
Rule: if a secret must exist in Terraform at all, it should flow through an ephemeral resource or variable into a write-only argument, or be generated and stored by the service itself.
sensitive vs ephemeralTwo different promises. sensitive hides a value from output; ephemeral keeps it out of plan and state files.
variable "db_password" {
type = string
sensitive = true # masked in CLI output
} # ...but stored in plan + state
# wherever it is used
variable "api_token" {
type = string
ephemeral = true # never written to plan or state
} # (Terraform 1.10+)
output "token_for_child" {
value = var.api_token
ephemeral = true # allowed in child modules only;
} # root module outputs cannot be
# ephemeralprovider blocks, provisioner and connection blocks, and write-only arguments.ephemeralasnull(value) turns an ephemeral value into null where a normal value is required.sensitive is still useful for non-secret-but-private values such as internal hostnames.An ephemeral block reads or generates a value for this run only. Nothing is stored; it is fetched or generated again in every plan and apply.
# generate a password for this run only
ephemeral "random_password" "db" { # hashicorp/random 3.7+
length = 24
special = true
}
# read an existing secret without storing it
ephemeral "aws_secretsmanager_secret_version" "api" {
secret_id = "tflab/api-token"
}
# use them only in ephemeral contexts, e.g. a provider
provider "github" {
token = jsondecode(
ephemeral.aws_secretsmanager_secret_version.api
.secret_string)["token"]
}aws_secretsmanager_secret_version, aws_ssm_parameter, aws_kms_secrets, aws_eks_cluster_auth, aws_lambda_invocation.random_password changes on every run, pair it with a write-only argument and an explicit *_wo_version (next slide).A write-only argument is sent to the provider during apply and then forgotten. Its companion *_wo_version decides when to send it again.
ephemeral "random_password" "app" {
length = 32
}
resource "aws_ssm_parameter" "app_secret" {
name = "/tflab/dev/app/secret"
type = "SecureString"
value_wo = ephemeral.random_password.app.result
value_wo_version = 1 # bump to 2 to rotate
}
resource "aws_secretsmanager_secret_version" "db" {
secret_id = aws_secretsmanager_secret.db.id
secret_string_wo = ephemeral.random_password.app.result
secret_string_wo_version = 1
}
resource "aws_db_instance" "main" {
# ...
password_wo = ephemeral.random_password.app.result
password_wo_version = 1
}password_wo, secret_string_wo, value_wo, master_password_wo, auth_token_wo, …).value_wo = null and value_wo_version = 1. Plans show (write-only attribute).value_wo produces no diff: Terraform cannot compare a value it never kept. Bump *_wo_version to send it again.*_wo_version to match what you want, then expect one update.Before migrating, find out what is exposed today: in the repository, in current state, and in old state versions.
# 1) the repository (and its history)
gitleaks git . # or: trufflehog git file://.
# 2) attributes the provider marks as sensitive,
# in the current state of one root module
terraform show -json | jq -r '
.values.root_module.resources[]
| select(.sensitive_values | length > 0)
| "\(.address): \(.sensitive_values | keys | join(", "))"'
# random_password.app: result
# aws_ssm_parameter.app_secret: value
# 3) how many old copies exist in S3
aws s3api list-object-versions --bucket $TF_STATE_BUCKET \
--prefix dev/app/terraform.tfstate \
--query 'length(Versions)'sensitive_values lists the attributes that are sensitive (from the provider schema or sensitive = true); their values sit in state..values.root_module.child_modules[]: extend the filter for a full audit..tf files (its secrets framework).From best to worst. The best secret is one Terraform never sees.
| Pattern | Secret in state? | Example | Use when |
|---|---|---|---|
| Service-managed secret | No | RDS manage_master_user_password = true (RDS stores and rotates it in Secrets Manager) | The service supports it |
| Ephemeral + write-only | No | ephemeral "random_password" → password_wo / value_wo | Terraform must set the initial value |
| Reference by ARN/name only | No | Pass the secret ARN to ECS/Lambda; the workload reads it at runtime | Consumers can read Secrets Manager/SSM themselves |
Normal random_password + normal argument | Yes | password = random_password.db.result | Legacy code; migrate it |
Secret in .tfvars or TF_VAR_ | Yes, and often in Git or CI logs | db_password = "hunter2" | Never |
Migrating a legacy secret: switch to the write-only argument, apply (the old value is removed from state on the next write), then rotate the secret, because the old value still sits in old state versions in S3.
Providers and modules are code that runs with your cloud credentials.
A provider is an executable that receives your AWS credentials. A module is code that decides what gets created.
terraform init from the registry (registry.terraform.io/hashicorp/aws), a mirror or a cache.SHA256SUMS files are signed; Terraform checks the signature and the package hash..terraform.lock.hcl (next slide).version; not recorded in the lock file.?ref= a tag can move; ?ref=<full commit SHA> cannot.terraform-aws-modules) or your own (ch 6).setup-terraform, tflint, checkov, trivy, conftest all run with the plan role's credentials.aquasecurity/trivy-action and setup-trivy were force-pushed to malicious commits that stole CI secrets (TeamPCP campaign)..terraform.lock.hcl pins the exact provider version and the hashes Terraform will accept. Commit it.
provider "registry.terraform.io/hashicorp/aws" {
version = "6.72.0"
constraints = "~> 6.0"
hashes = [
"h1:Pq3...=", # hash of the unpacked package
"zh:0a1...", # hashes of the release zips,
"zh:1f2...", # one per platform, from the
# ... # signed SHA256SUMS
]
}
# record h1: hashes for every platform you run on
terraform providers lock \
-platform=darwin_arm64 \
-platform=linux_amd64 \
-platform=linux_arm64
# CI: fail instead of silently updating the file
terraform init -lockfile=readonlyinit with a checksum error.init records h1: for the platform it ran on plus zh: hashes from the registry's signed checksums.h1: hashes, so record them for all platforms with providers lock.terraform init -upgrade rewrites versions and hashes, and the diff shows up in review.init.Speed up init and control what can be installed, without giving up hash verification.
# ~/.terraformrc (or TF_CLI_CONFIG_FILE)
plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"
provider_installation {
network_mirror {
url = "https://tf-mirror.example.com/providers/"
include = ["registry.terraform.io/hashicorp/*"]
}
direct {
exclude = ["registry.terraform.io/hashicorp/*"]
}
}
# fill a mirror directory from your lock file
terraform providers mirror -platform=linux_amd64 ./mirrorplugin_cache_dir or TF_PLUGIN_CACHE_DIR): one download per version on a machine or runner; still verified against the lock file.-lockfile=readonly, only reviewed versions and hashes can be installed.Each one turns a reviewable change into an unreviewable risk.
.tfvars or repo secrets.allowed_account_ids: a dev profile applies to prod.sensitive = true keeps a value out of state..tfvars committed "just for the sandbox".value_wo and expecting an update without bumping value_wo_version.@v1, @0.35.0).Answer with the exact argument, permission or command.
iam:CreateRole?allowed_account_ids protect against?sensitive variable end up?value_wo alone produce no plan diff?h1: and zh: hashes cover?terraform providers lock -platform=…?-lockfile=readonly do in CI?Practice: Lab 12 in Hands-on labs — see a password in state, move it to an ephemeral value and a write-only argument, lock down the state bucket, and record lock-file hashes for three platforms.
Keeping a growing estate healthy: routine upgrades, fast plans on big states, governance that scales with teams, and a clear view of Terraform vs OpenTofu.
Terraform core, providers and modules all move every month. Small, regular upgrades are cheap; a three-year jump is a project.
Each has its own changelog, its own constraint in code, and its own failure mode when you skip it for too long.
required_version in the terraform block.setup-terraform terraform_version) and locally (tenv, tfenv, .terraform-version).required_providers (version = "~> 6.0")..terraform.lock.hcl — commit it.terraform init -upgrade is the only command that moves the lock file to newer versions.version on registry modules, ?ref= on Git sources.init.moved blocks (ch 5, ch 6).Rule of thumb: never combine a core upgrade, a provider major and a module major in one PR. Three small plans are reviewable; one combined plan is not.
Constraints say what is allowed; the lock file says what is used. Upgrades become a deliberate PR that changes the lock file.
# versions.tf in every root module
terraform {
required_version = ">= 1.14, < 2.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 6.0" # any 6.x, never 7.0
}
}
}
# reusable modules: express what they NEED,
# not what you happen to run
terraform {
required_version = ">= 1.11"
required_providers {
aws = { source = "hashicorp/aws", version = ">= 6.0" }
}
}| Constraint | Allows |
|---|---|
= 6.20.0 / 6.20.0 | Exactly that version |
~> 6.20 | >= 6.20, < 7.0 (rightmost component may grow) |
~> 6.20.0 | >= 6.20.0, < 6.21.0 (patches only) |
>= 6.0, < 7.0 | Explicit range, same as ~> 6.0 |
!= 6.31.0 | Exclude a known-bad release |
Root modules pin tightly (~> + lock file); shared modules use lower bounds (>=) so callers can choose. A module that pins = 6.20.0 blocks every caller from upgrading.
Same loop for core, providers and modules. The plan is the test: an upgrade that should be a no-op must produce "No changes".
Commit the lock file for every platform you run on: terraform providers lock -platform=linux_amd64 -platform=darwin_arm64. Otherwise a lock file holding only one platform’s h1: hashes (for example from a filesystem mirror, without the registry’s zh: hashes) fails on CI with checksum errors.
v6.0 (GA Jun 2025) is a major release: read the "Version 6 Upgrade Guide" in the Registry before touching the constraint.
region argument, so one provider configuration can manage several Regions without aliases.auto_minor_version_upgrade).data.aws_region.current.name → data.aws_region.current.region.# 0) start from a clean plan on 5.x
terraform plan # No changes.
# 1) bump the constraint
# version = "~> 5.0" → "~> 6.0"
# 2) download v6, rewrite the lock file
terraform init -upgrade
git diff .terraform.lock.hcl
# - version = "5.100.0"
# + version = "6.x.y"
# 3) removed arguments fail validate;
# deprecated attributes warn in plan
terraform validate
terraform plan
# Warning: Deprecated attribute
# The attribute "name" is deprecated. …Upgrade the last 5.x minor first (~> 5.0 + init -upgrade) and fix every deprecation warning there: most v6 removals were warned about in late 5.x.
Fix the code, then make the plan prove nothing real changes. Only then roll forward environment by environment.
# 4) fix the deprecation
- region = data.aws_region.current.name
+ region = data.aws_region.current.region
# 5) the real test
terraform plan -out=tfplan
# Plan: 0 to add, 0 to change, 0 to destroy.
# if a diff appears, classify it:
# ~ update in-place → default changed?
# -/+ replace → STOP, read the guide
terraform show -json tfplan | jq -r '
.resource_changes[]
| select(.change.actions != ["no-op"])
| "\(.change.actions|join("/")) \(.address)"'
# 6) dev: apply; prod: same lock file, later
terraform apply tfplanregion argument — but as a separate PR.-refresh-only apply first, so state records the provider's new attributes before you refactor.A warning today is an error after the next major. Track them like any other debt.
terraform plan (and validate for some cases): "Deprecated attribute / argument / resource".dynamodb_table → move to use_lockfile = true (ch 4).variable descriptions or check blocks that warn about inputs being retired.terraform plan -json output (one JSON message per line).validation or check warning for one minor, remove it in the next major.terraform plan -json \
| jq -r 'select(.type=="diagnostic"
and .diagnostic.severity=="warning")
| .diagnostic.summary' | sort | uniq -c
# 2 Deprecated attribute
# 1 Argument is deprecatedA module author who renames resources must ship moved blocks. Callers then upgrade with a no-op plan instead of a destroy/create.
# modules/network v2.0.0 — inside the MODULE
# v1 used count, v2 uses for_each by AZ
moved {
from = aws_subnet.public[0]
to = aws_subnet.public["ap-northeast-1a"]
}
moved {
from = aws_subnet.public[1]
to = aws_subnet.public["ap-northeast-1c"]
}
# caller: bump the ref, nothing else
module "network" {
source = "git::https://github.com/<you>/tf-modules.git//network?ref=v2.0.0"
...
}
# plan:
# module.network.aws_subnet.public[0] has moved to
# module.network.aws_subnet.public["ap-northeast-1a"]
# Plan: 0 to add, 0 to change, 0 to destroy.moved blocks for at least one major (callers skip versions).CHANGELOG.md: removed inputs, new required inputs, expected plan.terraform test run that applies v1 and plans v2 is the strongest proof (ch 7).moved (e.g. a resource type changed), document removed + import steps (ch 5).When plan takes ten minutes, people stop reading it. Measure where the time goes, then fix the cause.
A plan is: load config and providers, refresh every resource in state, walk the graph, diff. On large states, refresh dominates.
-parallelism (default 10 concurrent operations).time terraform plan with and without refresh (Lab 13).terraform state list | wc -l: resources per state.TF_LOG=DEBUG TF_LOG_PATH=plan.log terraform plan — look for throttling and retry messages.grep -c "^data " *.tf.TF_PLUGIN_CACHE_DIR).-parallelism only after the above — higher values increase throttling.Every option trades plan speed against how much drift you can see. Pick per use case, not once for the whole team.
| Mode | What is refreshed | Sees drift on… | Use for |
|---|---|---|---|
terraform plan (default) | Every resource in state | Everything | Prod applies, nightly drift job |
plan -minimal-refresh (1.17) | Only resources with proposed changes; others use cached state | Only resources you are changing | Fast PR plans on large states |
plan -refresh=false | Nothing | Nothing — plans against stale state | Emergencies, debugging config |
plan -refresh-only | Everything, but proposes only state updates | Everything (that is the point) | Reviewing and accepting drift (ch 5) |
plan -target=ADDR | Target and its dependencies | Only the targeted part | Recovery only; never routine (ch 8) |
-minimal-refresh is new and opt-in. A PR plan with it can say "No changes" while an untouched resource has drifted — keep a full-refresh plan before apply in prod and in the scheduled drift job.
Most "Terraform is slow" reports on AWS are throttling or plugin downloads.
provider "aws" {
region = "ap-northeast-1"
# SDK retries: "standard" (default) or "adaptive"
# (client-side rate limiting when throttled)
retry_mode = "adaptive"
max_retries = 25 # the default; raise if throttled
default_tags {
tags = { Project = "tf-lab", ManagedBy = "terraform" }
}
}
# CI: reuse downloaded providers between jobs
export TF_PLUGIN_CACHE_DIR="$HOME/.terraform.d/plugin-cache"
mkdir -p "$TF_PLUGIN_CACHE_DIR"
# cache that directory keyed on .terraform.lock.hclterraform providers mirror ./mirror downloads providers for air-gapped or rate-limited CI; point the CLI config at it with provider_installation { filesystem_mirror { … } }.region can replace one alias per Region.for_each over many accounts or Regions in one root is a design smell: split by account/Region.Split by lifecycle and ownership, not by resource type. A good split makes plans fast and reviews small.
-target to avoid waiting.removed { lifecycle { destroy = false } } in the old root and import blocks in the new one (ch 5).terraform state mv -state-out on pulled copies (older, error-prone).dev/network, dev/app, …).Guardrails that let many teams change infrastructure without one central bottleneck.
Git already has the tools: code owners, rulesets and required checks. Map them onto the repository layout (ch 10).
# .github/CODEOWNERS
/modules/network/ @<org>/platform-network
/modules/ci-role/ @<org>/platform-security
/live/prod/ @<org>/platform-leads
/live/*/app/ @<org>/team-shop
/policies/ @<org>/platform-security
*.tftest.hcl @<org>/platform
# ruleset on main (GitHub):
# - require PR, 1–2 approvals, code owner review
# - required checks: fmt, validate, tflint,
# test, checkov, trivy, conftest, plan
# - no force push, no bypass except break-glassgha-tf-apply) is used only by the pipeline, gated by the prod environment (ch 11).If it is not tagged, nobody owns it and nobody pays for it. Enforce tags in code, in policy and in the account.
default_tags: Project, Env, ManagedBy, Owner applied to every taggable resource.merge(var.tags, {…}).tags_all in state shows the effective tags.Owner (ch 9).Console changes create drift and invisible risk, but you still need a way to act fast when production is down.
ManagedBy=terraform: anyone touching such a resource in the console knows it will be reverted.plan shows "No changes".apply reverts it (review the plan first).ignore_changes — with a comment saying why (ch 3).apply -refresh-only (ch 5).Generate what can be generated; write down only decisions and procedures.
brew install terraform-docs
# README.md in each module gets an injected section
# between the markers:
# <!-- BEGIN_TF_DOCS -->
# <!-- END_TF_DOCS -->
terraform-docs markdown table \
--output-file README.md --output-mode inject \
modules/network
# CI: fail if docs are stale
terraform-docs markdown table \
--output-file README.md --output-mode inject \
modules/network
git diff --exit-code modules/network/README.mdexamples/** folders that CI actually plans, so examples cannot rot.Two compatible-but-diverging tools. Know the differences before a migration decides for you.
OpenTofu forked from Terraform 1.5 (MPL 2.0) when Terraform moved to BUSL 1.1. Configurations and state are largely compatible; new features diverge.
| Topic | Terraform (HashiCorp, IBM) | OpenTofu (Linux Foundation) |
|---|---|---|
| License | BUSL 1.1 (1.6+) | MPL 2.0 |
| Latest (Nov 2026) | 1.17 (Oct 2026) | 1.12 line (1.12.0 May 2026) |
| Registry | registry.terraform.io | registry.opentofu.org (mirrors most providers and modules) |
| Only in Terraform | HCP Terraform integration, Stacks, Terraform Policy (GA 1.17), action blocks, list blocks + terraform query (1.14) | — |
| Only in OpenTofu | — | State and plan encryption (1.7), early evaluation of variables in backends and module sources (1.8), provider for_each and -exclude (1.9), enabled meta-argument (1.11), dynamic prevent_destroy (1.12) |
| In both (different versions) | Ephemeral values (1.10), write-only arguments (1.11) | Ephemeral values and write-only attributes (1.11) |
Feature parity is a moving target: check both changelogs before you rely on a feature list, including this one.
The OpenTofu docs publish a migration guide per Terraform version. The safe pattern: back up, try on a copy, compare plans, then switch.
# 0) prerequisites: a clean plan, a state backup
terraform plan # No changes.
terraform state pull > backup.tfstate
# 1) install OpenTofu
brew install opentofu
tofu version
# 2) init with OpenTofu (same backend block works)
tofu init
# lock file gains registry.opentofu.org entries:
# provider "registry.opentofu.org/hashicorp/aws"
# 3) the real test
tofu plan # expect: No changes.
# 4) small, harmless change through tofu apply
# (e.g. a tag in dev), then decideaction, .tfquery.hcl, policies, Stacks), OpenTofu may refuse it. Follow the guide for your exact Terraform version..terraform.lock.hcl with different registry hostnames — do not let both write it in the same branch.tofu apply writes a state, go back with care (the guide has a rollback section).setup-terraform for opentofu/setup-opentofu, pinned by SHA (CI/CD deck).Lab 13 runs step 2–3 against a local copy of the state with a separate TF_DATA_DIR, so your real backend and .terraform/ are never touched.
Both are production-grade. The decision is about license, platform, features you need, and the cost of switching later.
terraform query + bulk import, or action blocks, today.enabled / dynamic prevent_destroy features.tenv manages both).Terraform 1.14–1.17 added several new top-level concepts. Learn what problem each solves before adopting it.
| Feature | Version | Problem it solves |
|---|---|---|
List blocks (.tfquery.hcl) + terraform query | 1.14 | Find existing resources and generate import + resource blocks in bulk (ch 5) |
action blocks + terraform apply -invoke | 1.14 | Provider operations outside CRUD (invoke a Lambda, invalidate a CDN cache) |
Action triggers with on_failure (halt, taint, continue) | 1.16 | Control what happens when a triggered action fails |
Variables and locals in module source / version | 1.15 | One place to set module versions per environment |
Terraform Policy (-policies), GA | 1.17 | HCL-based policy checks in the CLI and HCP Terraform (ch 9) |
-minimal-refresh | 1.17 | Faster plans on large states (refreshes only changed resources) |
| Stacks (HCP Terraform) | GA Sep 2025 | Deploy the same components across many environments/accounts |
Keeping up: read the release notes of every minor (github.com/hashicorp/terraform/releases), the AWS provider CHANGELOG, and the OpenTofu blog. Try new features in a scratch root, never first in prod.
None of these fail on day one. All of them fail on the worst possible day.
init -upgrade "to fix an error" and committing whatever lock file it produced.darwin_arm64 h1: hashes and no zh: lines: Linux CI fails.-target because full plans are slow.-refresh=false or -minimal-refresh in the drift job — drift never shows.-parallelism until AWS throttles every call.tofu apply on prod state "to see if it works".Answer with the exact command, flag or file.
~> 6.20 allow? And ~> 6.20.0?-/+?-minimal-refresh skip, and when must you not use it?retry_mode = "adaptive" and TF_PLUGIN_CACHE_DIR change?Practice: Lab 13 in Hands-on labs — a provider upgrade with a deprecation fix, refresh-mode timings and drift, OpenTofu on a state copy, terraform-docs, and the full teardown.
One repository, two environments, a pipeline that plans every PR and applies only what was reviewed, with policies and tests as gates.
Build it after Lab 12, from a fresh clone: if the README and the pipeline are enough to recreate everything, the capstone is done. Lab 13 tears it down.
Done means every box is ticked and proven by a command or a pipeline run, not by a screenshot.
bootstrap/ state bucket: versioned, encrypted, prevent_destroy (Lab 0).network and app-bucket modules with typed inputs and tests (Labs 2, 6, 7).live/dev and live/prod roots, one state key each, SSM outputs (Labs 4, 10).import and one moved refactor, both no-destroy plans (Lab 5).terraform plan is "No changes" in every root on main.gha-tf-* sessions.live/ folder and tfvars.terraform query.The plan file is the contract: plan with -out, review, apply exactly that file.
# set up and check
terraform init -backend-config="bucket=$TF_STATE_BUCKET"
terraform init -upgrade # newer providers within constraints
terraform init -reconfigure # backend changed, do not migrate
terraform init -migrate-state # backend changed, copy state
terraform fmt -recursive -check # CI; drop -check to rewrite
terraform validate
# plan and apply
terraform plan -out=tfplan
terraform plan -detailed-exitcode # 0 none, 1 error, 2 changes
terraform plan -refresh-only # drift only
terraform plan -replace=aws_s3_bucket.logs
terraform apply tfplan # exactly the reviewed plan
terraform destroy
# inspect
terraform show tfplan
terraform show -json tfplan > plan.json
terraform output -json
terraform console# state (read first, change last)
terraform state list
terraform state show 'module.network.aws_vpc.this'
terraform state pull > backup.tfstate
terraform state mv SRC DST # prefer moved blocks
terraform state rm ADDR # prefer removed blocks
terraform force-unlock LOCK_ID # only if the holder is dead
# providers and graph
terraform providers
terraform providers lock \
-platform=linux_amd64 -platform=darwin_arm64
terraform graph | dot -Tsvg > graph.svg
# tests and generation
terraform test
terraform test -filter=tests/defaults.tftest.hcl
terraform plan -generate-config-out=generated.tf
terraform query # 1.14+, .tfquery.hcl
# useful environment variables
TF_LOG=DEBUG TF_LOG_PATH=tf.log # debug logging
TF_VAR_env=dev # sets var.env
TF_PLUGIN_CACHE_DIR=~/.terraform.d/plugin-cache
TF_IN_AUTOMATION=1 # quieter CI outputTry any of these in terraform console before using them in code.
# types and inputs
variable "subnets" {
type = map(object({
cidr = string
public = optional(bool, false) # default
}))
}
variable "env" {
type = string
validation {
condition = contains(["dev", "prod"], var.env)
error_message = "env must be dev or prod."
}
}
# for expressions and splat
[for k, s in var.subnets : s.cidr if s.public]
{for k, s in var.subnets : k => s.cidr}
aws_subnet.public[*].id # count resources
values(aws_subnet.private)[*].id # for_each resources
# conditional and string template
local.is_prod ? 3 : 1
"tflab-${var.env}-%{if local.is_prod}ha%{else}single%{endif}"
# for_each from a list of names
for_each = toset(["a", "b"])| Call | Result |
|---|---|
cidrsubnet("10.10.0.0/16", 8, 1) | "10.10.1.0/24" |
merge({a=1}, {b=2}) | {a=1, b=2} (later keys win) |
lookup({a="1"}, "b", "none") | "none" |
try(var.cfg.timeout, 30) | 30 if the attribute is missing |
can(regex("^tflab-", var.name)) | true / false |
coalesce("", "b") | "b" (first non-null, non-empty) |
one(["x"]) | "x"; null for []; error if > 1 |
zipmap(["a","b"], [1,2]) | {a=1, b=2} |
flatten([[1],[2,3]]) | [1, 2, 3] |
format("%s-%s", "tflab", "dev") | "tflab-dev" |
jsonencode(...) / templatefile(path, vars) | Policies and user data from data, not string concatenation |
Blocks are reviewed in a PR and shown in the plan; state commands are not. Prefer blocks.
# rename / count → for_each / into a module
moved {
from = aws_s3_bucket.logs
to = module.logs.aws_s3_bucket.this
}
# adopt an existing resource
import {
to = aws_s3_bucket.legacy
id = "tflab-legacy-assets"
}
# many at once
import {
for_each = var.legacy_buckets # map: key => name
to = aws_s3_bucket.legacy[each.key]
id = each.value
}
# write the resource blocks for you, then review them
# terraform plan -generate-config-out=generated.tf
# stop managing, keep the real resource
removed {
from = aws_s3_bucket.old
lifecycle {
destroy = false
}
}# find.tfquery.hcl — search existing resources (1.14+)
# (only types with list support; config is
# provider-specific: see the provider docs)
list "aws_instance" "dev" {
provider = aws
limit = 50 # default 100
config {
filter {
name = "tag:Env"
values = ["dev"]
}
}
}
terraform query
terraform query -generate-config-out=generated.tf
# → import + resource blocks to review and commit
# plan annotations to look for
# # aws_s3_bucket.a has moved to aws_s3_bucket.b
# # aws_s3_bucket.legacy will be imported
# # aws_s3_bucket.old will no longer be managed
# by Terraform
# Plan: 1 to import, 0 to add, 0 to change,
# 0 to destroy.Machine-read the plan for reviews and gates: actions tells you what happens to each resource.
terraform plan -out=tfplan
terraform show -json tfplan > plan.json
# every resource that will be deleted (incl. replacements)
jq -r '.resource_changes[]
| select(.change.actions | index("delete"))
| .address' plan.json
# count changes by action
jq -r '[.resource_changes[].change.actions | join("/")]
| group_by(.) | map({key: .[0], value: length})
| from_entries' plan.json
# { "create": 2, "no-op": 14, "update": 1,
# "delete/create": 1 }
# why a replacement happens
jq -r '.resource_changes[]
| select(.action_reason != null)
| "\(.address): \(.action_reason)"' plan.json
# aws_s3_bucket.app: replace_because_cannot_updatechange.actions values| Actions | Meaning |
|---|---|
["no-op"] | Nothing to do |
["create"] | New resource |
["read"] | Data source read during apply |
["update"] | Update in place |
["delete", "create"] | Replace, destroy first (-/+) |
["create", "delete"] | Replace, create first (+/-, create_before_destroy) |
["delete"] | Destroy |
["forget"] | Drop from state, keep the real resource (removed with destroy = false) |
Plan files and plan JSON contain sensitive values in clear text. Treat them like the state: short-lived CI artifacts, never committed.
Static scans on HCL catch most mistakes fast; scans on plan JSON see resolved values and modules.
# Checkov — HCL and plan
checkov -d . --framework terraform
checkov -f plan.json --framework terraform_plan
checkov -d . --check CKV_AWS_18,CKV_AWS_21
checkov -d . --skip-check CKV_AWS_144
checkov -d . --external-checks-dir policies/checkov
checkov -d . -o sarif --output-file-path reports/
checkov -d . --create-baseline # accept today's state
checkov -d . --baseline .checkov.baseline
# inline, with a reason:
# #checkov:skip=CKV_AWS_144:single-region lab data
# Trivy — IaC misconfigurations
trivy config .
trivy config --severity HIGH,CRITICAL --exit-code 1 .
trivy config --tf-vars live/dev/app/dev.tfvars .
trivy config --ignorefile .trivyignore .
# inline: #trivy:ignore:AVD-AWS-0089
# pin the binary and action by version/SHA:
# v0.69.4–0.69.6 were malicious (Mar 2026)# conftest — Rego over plan JSON
conftest test plan.json -p policies/rego
# policies/rego/s3.rego
package main
import rego.v1
deny contains msg if {
rc := input.resource_changes[_]
rc.type == "aws_s3_bucket"
"delete" in rc.change.actions
msg := sprintf("%s would be deleted", [rc.address])
}
warn contains msg if {
rc := input.resource_changes[_]
"create" in rc.change.actions
not rc.change.after.tags.Owner
msg := sprintf("%s has no Owner tag", [rc.address])
}
# exit code 1 on any deny → CI failsInline tags may be empty when the tag comes from provider default_tags; check tags_all in the plan when your policy is about effective tags.
One bucket, one key per root, S3-native locking. CI authenticates with OIDC and never stores keys.
# backend.tf (bucket passed at init: partial config)
terraform {
backend "s3" {
key = "dev/network/terraform.tfstate"
region = "ap-northeast-1"
encrypt = true
use_lockfile = true # lock object next to the state
# dynamodb_table = "…" deprecated
}
}
terraform init -input=false \
-backend-config="bucket=$TF_STATE_BUCKET"
# share outputs without remote state (ch 4, ch 10)
resource "aws_ssm_parameter" "vpc_id" {
name = "/tflab/dev/network/vpc_id"
type = "String"
value = aws_vpc.this.id
}# .github/workflows/plan.yml (excerpt)
permissions: {contents: read, id-token: write}
jobs:
plan:
runs-on: ubuntu-latest
defaults: {run: {working-directory: live/dev/network}}
steps:
- uses: actions/checkout@<sha>
- uses: hashicorp/setup-terraform@<sha>
with: {terraform_version: 1.17.0,
terraform_wrapper: false}
- uses: aws-actions/configure-aws-credentials@<sha>
with:
role-to-assume: arn:aws:iam::<acct>:role/gha-tf-plan
aws-region: ap-northeast-1
- run: terraform init -input=false
-backend-config="bucket=tflab-state-<acct>"
- run: terraform plan -input=false -lock-timeout=60s
-detailed-exitcode -out=tfplan
# exit 2 = changes: handle it, do not fail the job<sha> = the full commit SHA of the action's release (CI/CD deck). The full pipeline with PR comments, gated apply and drift detection is in ch 11 and Lab 11.
All labs run from your laptop against an AWS sandbox account in ap-northeast-1; Lab 11 adds a public GitHub repo. Resources are free or near-free.
tf-lab; export AWS_PROFILE=tf-lab AWS_REGION=ap-northeast-1.tflab; tags Project=tf-lab, Env, ManagedBy=terraform through provider default_tags.tflab-state-<account_id>; one key per component: <env>/<component>/terraform.tfstate; use_lockfile = true.brew tap hashicorp/tap
brew install hashicorp/tap/terraform \
tflint checkov trivy conftest jq gh awscli
brew install opentofu terraform-docs infracost # optional
terraform version # Terraform v1.14+ (examples: 1.17)
tflint --version
checkov --version
trivy --version
conftest --version
aws --version # aws-cli/2.x
# a profile for the sandbox account (SSO or keys)
aws configure sso --profile tf-lab # or: aws configure
export AWS_PROFILE=tf-lab AWS_REGION=ap-northeast-1
aws sts get-caller-identity
# { "Account": "123456789012", "Arn": "arn:aws:...", ... }
export ACCT=$(aws sts get-caller-identity \
--query Account --output text)
export TF_STATE_BUCKET=tflab-state-$ACCTGoal: a Git repo with a safe .gitignore, a budget alarm before anything bills, and proof that the CLI talks to the sandbox account.
mkdir tf-lab && cd tf-lab && git init
cat > .gitignore <<'EOF'
# provider binaries and module cache
.terraform/
# state and its backups: never in Git
*.tfstate
*.tfstate.*
# saved plans can contain secrets
*.tfplan
tfplan
tfplan.json
crash.log
crash.*.log
# local overrides
override.tf
override.tf.json
*_override.tf
*_override.tf.json
.terraformrc
terraform.rc
EOF
# commit the lock file, NOT .terraform/
git add .gitignore && git commit -m "gitignore"
aws sts get-caller-identity --query Account --output text
# 123456789012 ← must be the SANDBOX account# budget: e-mail when actual spend passes 80% of $10
aws budgets create-budget --account-id $ACCT --budget \
'{"BudgetName":"tf-lab","TimeUnit":"MONTHLY",
"BudgetType":"COST",
"BudgetLimit":{"Amount":"10","Unit":"USD"}}' \
--notifications-with-subscribers '[{"Notification":{
"NotificationType":"ACTUAL",
"ComparisonOperator":"GREATER_THAN",
"Threshold":80},
"Subscribers":[{"SubscriptionType":"EMAIL",
"Address":"you@example.com"}]}]'
aws budgets describe-budgets --account-id $ACCT \
--query 'Budgets[].BudgetName'
# [ "tf-lab" ]
# shared provider plugin cache: faster init everywhere
mkdir -p ~/.terraform.d/plugin-cache
cat >> ~/.terraformrc <<'EOF'
plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"
EOFACCT and TF_STATE_BUCKET come from the Lab setup slide. Put the export lines in your shell profile so every lab terminal has them.
Goal: one versioned, encrypted, private bucket for every state file. It starts with local state: the bucket does not exist yet.
mkdir bootstrap && cd bootstrap && cat > main.tf <<'EOF'
terraform {
required_version = ">= 1.14"
required_providers {
aws = { source = "hashicorp/aws", version = "~> 6.0" }
}
}
provider "aws" {
region = "ap-northeast-1"
default_tags {
tags = { Project = "tf-lab", Env = "shared", ManagedBy = "terraform" }
}
}
data "aws_caller_identity" "current" {}
resource "aws_s3_bucket" "state" {
bucket = "tflab-state-${data.aws_caller_identity.current.account_id}"
lifecycle {
prevent_destroy = true # the one bucket you never lose
}
}cat >> main.tf <<'EOF'
resource "aws_s3_bucket_versioning" "state" {
bucket = aws_s3_bucket.state.id
versioning_configuration {
status = "Enabled" # every state write is a version
}
}
resource "aws_s3_bucket_server_side_encryption_configuration" "state" {
bucket = aws_s3_bucket.state.id
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "AES256"
}
}
}
resource "aws_s3_bucket_public_access_block" "state" {
bucket = aws_s3_bucket.state.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
output "state_bucket" {
value = aws_s3_bucket.state.bucket
}
EOFGoal: create the bucket with local state, then migrate that state into the bucket so no state file lives only on your laptop.
terraform init
# Initializing provider plugins...
# - Installing hashicorp/aws v6.x.y...
# Terraform has been successfully initialized!
terraform fmt -check && terraform validate
terraform plan -out=tfplan
# Plan: 4 to add, 0 to change, 0 to destroy.
terraform apply tfplan
# Outputs:
# state_bucket = "tflab-state-123456789012"
aws s3api get-bucket-versioning --bucket $TF_STATE_BUCKET
# { "Status": "Enabled" }
aws s3api get-public-access-block \
--bucket $TF_STATE_BUCKET \
--query 'PublicAccessBlockConfiguration'
# all four flags: true# partial backend config: the bucket name is passed at
# init time (backend blocks cannot use variables)
cat > backend.tf <<'EOF'
terraform {
backend "s3" {
key = "bootstrap/terraform.tfstate"
region = "ap-northeast-1"
encrypt = true
use_lockfile = true # S3-native lock, no DynamoDB
}
}
EOF
terraform init -migrate-state \
-backend-config="bucket=$TF_STATE_BUCKET"
# Do you want to copy existing state to the new backend?
# Enter a value: yes
aws s3 ls s3://$TF_STATE_BUCKET/bootstrap/
# ... terraform.tfstate
rm -f terraform.tfstate terraform.tfstate.backup tfplan
terraform plan # No changes. (now read from S3)
cd .. && git add bootstrap && git commit -m "bootstrap"
# includes .terraform.lock.hclEvery later root module uses the same pattern: a backend "s3" block with its own key and use_lockfile = true, initialized with -backend-config="bucket=$TF_STATE_BUCKET".
Everything in later labs stores its state here. Make sure it is versioned, private and hard to delete.
terraform plan in bootstrap/ says No changes and reads state from S3.aws s3api list-object-versions --bucket $TF_STATE_BUCKET --prefix bootstrap/ shows one version.git status is clean; git ls-files lists bootstrap/.terraform.lock.hcl but no .tfstate.terraform destroy in bootstrap/ → "Instance cannot be destroyed … lifecycle.prevent_destroy". Good.terraform plan in two terminals at once → the second fails at once with "Error acquiring the state lock" (the default -lock-timeout is 0s; -lock-timeout=60s makes it wait).terraform init -reconfigure without -backend-config → Terraform asks for bucket interactively. Why does that make CI hang without -input=false?# keep the bucket until the very
# end of the deck: every lab
# stores its state here.
# final teardown (Lab 13):
# 1. destroy every live/* root
# 2. bootstrap: migrate its state
# back to local, remove
# prevent_destroy, apply
# 3. empty the bucket, including
# all object versions
# 4. destroy bootstrap, then
# delete the budgetGoal: the full workflow on a tiny configuration with local state. One free AWS resource (a Standard-tier SSM parameter) makes the plan real.
cd tf-lab && mkdir -p labs/01-hello && cd labs/01-hello
aws sts get-caller-identity --query Account --output text
# must print your sandbox account ($ACCT)
cat > versions.tf <<'EOF'
terraform {
required_version = ">= 1.14"
required_providers {
aws = { source = "hashicorp/aws", version = "~> 6.0" }
random = { source = "hashicorp/random", version = "~> 3.6" }
local = { source = "hashicorp/local", version = "~> 2.5" }
}
}
provider "aws" {
region = "ap-northeast-1"
default_tags {
tags = { Project = "tf-lab", Env = "dev", ManagedBy = "terraform" }
}
}
EOFcat > main.tf <<'EOF'
resource "random_pet" "name" {
length = 2
}
resource "local_file" "hello" {
filename = "${path.module}/out/hello.txt"
content = "hello, ${random_pet.name.id}\n"
}
resource "aws_ssm_parameter" "hello" {
name = "/tflab/dev/hello"
type = "String"
value = random_pet.name.id
}
output "pet" {
value = random_pet.name.id
}
output "param_name" {
value = aws_ssm_parameter.hello.name
}
EOF
terraform init
terraform fmt -check && terraform validate
# Success! The configuration is valid.Goal: plan to a file, review it, apply exactly that plan, then prove the second plan is empty with -detailed-exitcode.
terraform plan -out=tfplan
# Plan: 3 to add, 0 to change, 0 to destroy.
terraform show tfplan | less # review
terraform apply tfplan
# Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
# Outputs:
# param_name = "/tflab/dev/hello"
# pet = "select-mollusk" (yours differs)
cat out/hello.txt
aws ssm get-parameter --name /tflab/dev/hello \
--query Parameter.Value --output text
# idempotence: a second plan finds nothing
terraform plan -detailed-exitcode -input=false
echo "exit=$?"
# No changes. Your infrastructure matches the configuration.
# exit=0# what is in state?
terraform state list
# aws_ssm_parameter.hello
# local_file.hello
# random_pet.name
terraform state show aws_ssm_parameter.hello | head
# the dependency graph (needs Graphviz: brew install graphviz)
terraform graph | dot -Tsvg > graph.svg && open graph.svg
# random_pet.name ← local_file.hello
# random_pet.name ← aws_ssm_parameter.hello
# outputs for scripts
terraform output -raw param_name
terraform output -json | jq -r .pet.value
# .gitignore for every Terraform directory in tf-lab
cat >> ../../.gitignore <<'EOF'
.terraform/
*.tfstate
*.tfstate.*
*.tfplan
tfplan
crash.log
labs/*/out/
EOFGoal: change reality behind Terraform's back, watch the plan detect it, then trigger a replacement and read the -/+ symbols.
# 1) drift in AWS: edit the parameter outside Terraform
aws ssm put-parameter --name /tflab/dev/hello \
--value "changed-by-hand" --overwrite
terraform plan -refresh-only
# ~ value = (sensitive value) ... "changed outside of Terraform"
terraform plan -detailed-exitcode; echo "exit=$?"
# ~ update in-place → exit=2
terraform apply # type yes: value restored
# 2) drift locally: delete the generated file
rm out/hello.txt
terraform plan
# local_file.hello will be created# 3) replacement: a new pet name forces new dependants
terraform plan -replace=random_pet.name
# random_pet.name must be replaced (-/+)
# aws_ssm_parameter.hello will be updated in-place (~)
# local_file.hello must be replaced (content forces it)
terraform apply -replace=random_pet.name
# 4) destroy (preview first)
terraform plan -destroy
# Plan: 0 to add, 0 to change, 3 to destroy.
terraform destroy
aws ssm get-parameter --name /tflab/dev/hello
# ParameterNotFoundThe SSM value is shown as (sensitive value) because the AWS provider marks it sensitive in the schema: hidden in output, still stored in terraform.tfstate (ch 2, section 08).
You have now used every command of the core workflow on real infrastructure.
terraform plan -detailed-exitcode returns 0 right after apply and 2 after the manual put-parameter..terraform.lock.hcl lists hashicorp/aws, random and local with h1: and zh: hashes; commit it.graph.svg shows both resources depending on random_pet.name.terraform.tfstate exists locally and is ignored by Git (git status does not list it).plan -out=tfplan, then apply once without the file, then apply tfplan → "Saved plan is stale".AWS_PROFILE=default terraform plan in another account → Terraform wants to create everything again (the state does not match that account). Always check sts get-caller-identity.random_pet.name to random_pet.nmae in the reference only → validate fails with "Reference to undeclared resource"..terraform/ away and run plan → "Required plugins are not installed"; terraform init restores the exact versions from the lock file.# after destroy nothing in AWS
# remains; remove local files
cd tf-lab/labs/01-hello
rm -rf .terraform out tfplan \
terraform.tfstate*
# keep versions.tf, main.tf and
# .terraform.lock.hcl in Git
git add . ../../.gitignore
git commit -m "lab 1: hello"Goal: build intuition for types and expressions before writing resources. Predict each result, then press Enter.
mkdir -p tf-lab/labs/02-hcl && cd tf-lab/labs/02-hcl
terraform console # works in an empty directory
> cidrsubnet("10.10.0.0/16", 8, 1)
"10.10.1.0/24"
> [for i, az in ["a", "c", "d"] :
cidrsubnet("10.10.0.0/16", 8, i)]
["10.10.0.0/24", "10.10.1.0/24", "10.10.2.0/24"]
> { for k, v in { a = 1, b = 2 } : k => v * 10 if v > 1 }
{ "b" = 20 }
> merge({ a = 1 }, { a = 2, b = 3 })
{ "a" = 2, "b" = 3 }
> try(tonumber("x"), 0)
0
> can(regex("^[a-z0-9-]+$", "Bad_Key"))
false
> toset(["b", "a", "b"])
toset(["a", "b"])
> type(tomap({ a = 1 }))
map(number)
> format("%s-%03d", "tflab", 7)
"tflab-007"
> exit# grouping with ... (keys repeat)
> { for u in [
{ team = "core", name = "ann" },
{ team = "core", name = "bo" },
{ team = "web", name = "cy" } ] : u.team => u.name... }
{ "core" = ["ann", "bo"], "web" = ["cy"] }
# splat
> [{ n = "a" }, { n = "b" }][*].n
["a", "b"]
# string directive
> "%{if 1 > 0}yes%{else}no%{endif}"
"yes"
# your turn (answers in ch 2):
# 1. all pairs of ["dev","prod"] x ["a","b"]
# 2. a map az => cidr for ["a","c","d"]
# 3. first non-empty of "", null, "x"
# 4. why does this fail?
# element(["a","b"], 3) … it does not! why is
# that dangerous?Console output is shown condensed here; real output prints one element per line.
Goal: typed, validated inputs and optional() objects. Copy versions.tf from Lab 1 and set default_tags { tags = local.tags }.
cat > variables.tf <<'EOF'
variable "env" {
type = string
validation {
condition = contains(["dev", "prod"], var.env)
error_message = "env must be dev or prod."
}
}
variable "params" {
type = map(string)
default = {}
validation {
condition = alltrue([for k in keys(var.params) :
can(regex("^[a-z0-9-]+$", k))])
error_message = "Parameter keys may contain only a-z, 0-9 and '-'."
}
}
variable "bucket" {
type = object({
name_suffix = string
versioning = optional(bool, true)
lifecycle_rules = optional(list(object({
id = string
prefix = string
days = number
})), [])
})
}
EOFcat > dev.tfvars <<'EOF'
env = "dev"
params = {
greeting = "hello"
owner = "platform"
}
bucket = {
name_suffix = "lab2"
lifecycle_rules = [
{ id = "tmp", prefix = "tmp/", days = 7 },
{ id = "logs", prefix = "logs/", days = 30 },
]
}
EOF
mkdir -p templates && cat > templates/summary.tftpl <<'EOF'
# tf-lab ${env}
bucket: ${bucket}
%{ for k, name in params ~}
- ${k}: ${name}
%{ endfor ~}
EOF
# check the inputs before writing resources
# (console needs init once versions.tf exists)
terraform init
echo 'var.bucket' | terraform console -var-file=dev.tfvars
# versioning = true filled in by optional(bool, true)Goal: parameters keyed by name, a bucket named from locals, and lifecycle rules generated from the optional() list.
cat > main.tf <<'EOF'
data "aws_caller_identity" "me" {}
locals {
name = "tflab-${var.env}"
tags = { Project = "tf-lab", Env = var.env, ManagedBy = "terraform" }
}
resource "aws_ssm_parameter" "p" {
for_each = var.params
name = "/tflab/${var.env}/${each.key}"
type = "String"
value = each.value
}
resource "aws_s3_bucket" "b" {
bucket = join("-", [local.name, var.bucket.name_suffix,
data.aws_caller_identity.me.account_id])
force_destroy = true # lab only
}
EOFcat >> main.tf <<'EOF'
resource "aws_s3_bucket_versioning" "b" {
bucket = aws_s3_bucket.b.id
versioning_configuration {
status = var.bucket.versioning ? "Enabled" : "Suspended"
}
}
resource "aws_s3_bucket_lifecycle_configuration" "b" {
count = length(var.bucket.lifecycle_rules) > 0 ? 1 : 0
bucket = aws_s3_bucket.b.id
dynamic "rule" {
for_each = var.bucket.lifecycle_rules
content {
id = rule.value.id
status = "Enabled"
filter { prefix = rule.value.prefix }
expiration { days = rule.value.days }
}
}
}
EOFNew S3 buckets already block public access and encrypt with SSE-S3 by default; the shared app-bucket module (ch 6) still sets both explicitly so the intent is reviewable and scanners pass (ch 9).
Goal: render a summary from resource attributes, apply a saved plan and read the instance addresses.
cat >> main.tf <<'EOF'
resource "local_file" "summary" {
filename = "${path.module}/out/summary.md"
content = templatefile("${path.module}/templates/summary.tftpl", {
env = var.env
bucket = aws_s3_bucket.b.bucket
params = { for k, p in aws_ssm_parameter.p : k => p.name }
})
}
output "param_names" {
value = { for k, p in aws_ssm_parameter.p : k => p.name }
}
EOF
terraform init
terraform fmt -check -recursive && terraform validateterraform plan -var-file=dev.tfvars -out=tfplan
# + aws_ssm_parameter.p["greeting"]
# + aws_ssm_parameter.p["owner"]
# + aws_s3_bucket.b
# + aws_s3_bucket_versioning.b
# + aws_s3_bucket_lifecycle_configuration.b[0]
# (2 rule blocks: tmp, logs)
# + local_file.summary
# Plan: 6 to add, 0 to change, 0 to destroy.
terraform apply tfplan
cat out/summary.md
# # tf-lab dev
# bucket: tflab-dev-lab2-123456789012
# - greeting: /tflab/dev/greeting
# - owner: /tflab/dev/ownerEvery input mistake now fails at plan time with your own error message.
terraform output param_names shows greeting and owner mapped to /tflab/dev/....aws s3api get-bucket-lifecycle-configuration --bucket <name> returns rules tmp and logs.out/summary.md lists the bucket and both parameters.Project, Env, ManagedBy without a tags argument on the resource (provider default_tags).-var 'env=qa' → "env must be dev or prod."; a params key Bad_Key → your regex message.greeting from dev.tfvars → plan destroys only aws_ssm_parameter.p["greeting"]. Rewrite p with count over keys(var.params) and repeat: predict the replacements first.lifecycle_rules = [] → the count guard removes the lifecycle resource instead of sending an empty rule list.for_each = toset([aws_s3_bucket.b.id]) on a new resource → "Invalid for_each argument".cd tf-lab/labs/02-hcl
terraform destroy \
-var-file=dev.tfvars
# Destroy complete!
# Resources: 6 destroyed.
rm -rf .terraform out tfplan \
terraform.tfstate*
git add . && git commit \
-m "lab 2: hcl"Goal: the dev network root module with AWS provider v6, default_tags and validated inputs. Local state for now; Lab 4 moves it to S3. Needs Lab 0.
cd tf-lab && mkdir -p live/dev/network
cd live/dev/network
cat > versions.tf <<'EOF'
terraform {
required_version = ">= 1.14"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 6.0"
}
}
}
EOF
cat > providers.tf <<'EOF'
provider "aws" {
region = var.region
default_tags {
tags = {
Project = "tf-lab"
Env = var.env
ManagedBy = "terraform"
}
}
}
EOFcat > variables.tf <<'EOF'
variable "region" {
type = string
default = "ap-northeast-1"
}
variable "env" {
type = string
default = "dev"
validation {
condition = contains(["dev", "prod"], var.env)
error_message = "env must be dev or prod."
}
}
variable "vpc_cidr" {
type = string
default = "10.10.0.0/16"
validation {
condition = can(cidrnetmask(var.vpc_cidr))
error_message = "vpc_cidr must be a valid IPv4 CIDR."
}
}
EOFGoal: a lock file that works on your laptop and in Linux CI, and a VPC protected by a precondition and a postcondition.
terraform init
# - Installing hashicorp/aws v6.x.y...
# Terraform has created a lock file .terraform.lock.hcl
terraform providers lock \
-platform=darwin_arm64 -platform=linux_amd64
grep -A3 'hashicorp/aws' .terraform.lock.hcl
# version = "6.x.y"
# constraints = "~> 6.0"
# hashes = [ "h1:…", "h1:…", "zh:…", … ]
git add versions.tf providers.tf variables.tf \
.terraform.lock.hcl
git commit -m "network: provider and inputs"
# .gitignore (Lab 0) must exclude:
# .terraform/ *.tfstate *.tfstate.* *.tfplancat > main.tf <<'EOF'
data "aws_availability_zones" "available" {
state = "available"
}
locals {
name = "tflab-${var.env}"
az_list = data.aws_availability_zones.available.names
azs = slice(local.az_list, 0, min(2, length(local.az_list)))
}
resource "aws_vpc" "this" {
cidr_block = var.vpc_cidr
enable_dns_support = true
enable_dns_hostnames = true
tags = { Name = local.name }
lifecycle {
precondition {
condition = length(local.azs) == 2
error_message = "Need 2 available AZs in ${var.region}."
}
postcondition {
condition = self.enable_dns_hostnames
error_message = "DNS hostnames must be enabled."
}
}
}
EOFGoal: 2 public and 2 private subnets (no NAT gateway), a public route table, and a check that only warns.
cat >> main.tf <<'EOF'
resource "aws_subnet" "public" {
for_each = { for i, az in local.azs : az => i }
vpc_id = aws_vpc.this.id
availability_zone = each.key
cidr_block = cidrsubnet(var.vpc_cidr, 8, each.value)
map_public_ip_on_launch = true
tags = { Name = "${local.name}-public-${each.key}" }
}
resource "aws_subnet" "private" {
for_each = { for i, az in local.azs : az => i }
vpc_id = aws_vpc.this.id
availability_zone = each.key
cidr_block = cidrsubnet(var.vpc_cidr, 8, each.value + 10)
tags = { Name = "${local.name}-private-${each.key}" }
}
check "two_public_subnets" {
assert {
condition = length(aws_subnet.public) == 2
error_message = "Expected 2 public subnets."
}
}
EOFcat >> main.tf <<'EOF'
resource "aws_internet_gateway" "this" {
vpc_id = aws_vpc.this.id
tags = { Name = local.name }
}
resource "aws_route_table" "public" {
vpc_id = aws_vpc.this.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.this.id
}
tags = { Name = "${local.name}-public" }
}
resource "aws_route_table_association" "public" {
for_each = aws_subnet.public
subnet_id = each.value.id
route_table_id = aws_route_table.public.id
}
EOF
terraform fmt -check && terraform validate
terraform plan -out=tfplan
# Plan: 9 to add, 0 to change, 0 to destroy.
terraform apply tfplanThe for_each keys are the AZ names your account returns, usually ap-northeast-1a and ap-northeast-1c in Tokyo.
Goal: watch replacement order, an ignored tag, prevent_destroy and a failing check change the plan.
cat > sg.tf <<'EOF'
resource "aws_security_group" "web" {
name_prefix = "${local.name}-web-"
description = "HTTPS from the internet"
vpc_id = aws_vpc.this.id
lifecycle {
create_before_destroy = true
ignore_changes = [tags["Owner"]]
}
}
resource "aws_vpc_security_group_ingress_rule" "https" {
security_group_id = aws_security_group.web.id
cidr_ipv4 = "0.0.0.0/0"
from_port = 443
to_port = 443
ip_protocol = "tcp"
}
EOF
terraform apply # 2 to add
# A) description forces replacement
sed -i.bak 's/from the internet/(public)/' sg.tf
terraform plan
# aws_security_group.web must be replaced
# +/- resource "aws_security_group" "web"
# Plan: 2 to add, 0 to change, 2 to destroy.
terraform apply # new SG first, old one last# B) an ignored tag vs a managed one
SG=$(echo 'aws_security_group.web.id' \
| terraform console | tr -d '"')
aws ec2 create-tags --resources $SG \
--tags Key=Owner,Value=alice
terraform plan # Owner ignored in tags; if tags_all
# still differs, see provider ignore_tags
aws ec2 create-tags --resources $SG \
--tags Key=Team,Value=web
terraform plan # ~ tags: - "Team" (reverts)
terraform apply
# C) in aws_vpc.this lifecycle add:
# prevent_destroy = true
terraform plan -destroy
# Error: Instance cannot be destroyed
# D) make the check fail: == 2 → == 3 in main.tf
terraform plan
# Warning: Check block assertion failed
# Expected 2 public subnets.
# (the plan still succeeds)
# revert C and D, then:
terraform plan # No changesYou have watched each lifecycle rule change a plan. Keep the network: Lab 4 moves its state to S3.
terraform version lists hashicorp/aws v6.x; the lock file has h1: and zh: hashes.aws ec2 describe-vpcs --filters Name=tag:Project,Values=tf-lab shows Env=dev, ManagedBy=terraform from default_tags.terraform state list shows 11 managed resources (VPC, 4 subnets, IGW, route table, 2 associations, SG, ingress rule) plus the data source.terraform plan → No changes.name_prefix with name = "${local.name}-web", apply, then change the description → the replacement fails: that group name already exists in the VPC.terraform plan -var vpc_cidr=10.10.0.0/33 → validation error before any API call.region = "us-east-1" to aws_internet_gateway.this → plan shows -/+. Applying would delete the working gateway, then fail (no such VPC in us-east-1). Remove the line.terraform init -upgrade → if the lock file changes, review the diff before committing.# keep everything for Lab 4
# housekeeping
rm -f *.bak tfplan
# full teardown (only at the very
# end of the deck, after Lab 4
# moved the state to S3):
# terraform destroyGoal: the Lab 3 state moves from a local file into tflab-state-<acct> under dev/network/terraform.tfstate, with native S3 locking. Needs Labs 0 and 3.
cd tf-lab/live/dev/network
echo "$TF_STATE_BUCKET" # tflab-state-<acct> (Lab 0)
cat > backend.tf <<'EOF'
terraform {
backend "s3" {
# bucket: from -backend-config (no variables here)
key = "dev/network/terraform.tfstate"
region = "ap-northeast-1"
encrypt = true
use_lockfile = true
}
}
EOF
cp terraform.tfstate /tmp/network-local-backup.tfstate
terraform init -migrate-state \
-backend-config="bucket=$TF_STATE_BUCKET"
# ... Do you want to copy existing state to the new
# backend? Enter a value: yes
# Successfully configured the backend "s3"!# state is now remote
aws s3 ls s3://$TF_STATE_BUCKET/dev/network/
# ... terraform.tfstate
terraform plan # No changes (same state)
# remove the stale local copy so nobody applies with it
rm -f terraform.tfstate terraform.tfstate.backup
# record the bucket for teammates and CI
echo "bucket = \"$TF_STATE_BUCKET\"" > backend.hcl
echo 'backend.hcl' >> ../../../.gitignore # account-specific
# next time, on any machine:
# terraform init -backend-config=backend.hclWithout -migrate-state Terraform asks anyway; with -reconfigure it would start with an empty state in S3 and plan to create everything again.
Goal: see the .tflock object appear while an apply waits for approval, and a second run fail fast.
# terminal A: start an apply and leave it at the prompt
terraform apply -replace=aws_security_group.web
# Do you want to perform these actions?
# Enter a value: <- do not answer yet
# (-replace forces a change, so the prompt appears)
# terminal B: the lock object exists now
aws s3 ls s3://$TF_STATE_BUCKET/dev/network/
# ... terraform.tfstate
# ... terraform.tfstate.tflock
aws s3 cp \
s3://$TF_STATE_BUCKET/dev/network/terraform.tfstate.tflock - \
| jq '{ID, Operation, Who, Created}'# terminal B: a second run cannot take the lock
terraform plan
# Error: Error acquiring the state lock
# Lock Info:
# ID: <uuid>
# Operation: OperationTypeApply
# Who: you@laptop
terraform plan -lock-timeout=2m & # waits instead
# terminal A: answer "no"
# Apply cancelled.
# → lock released, the waiting plan in B continues
aws s3 ls s3://$TF_STATE_BUCKET/dev/network/
# ... terraform.tfstate (.tflock is gone)In a versioned bucket every lock leaves noncurrent versions and delete markers behind. Check for a rule that expires noncurrent versions (aws s3api get-bucket-lifecycle-configuration --bucket $TF_STATE_BUCKET); if there is none, add one to bootstrap/ (e.g. 90 days).
Goal: rename with state mv (and see why code must follow), then break the state on purpose and restore the previous version.
terraform state list | wc -l # 12 (11 + data)
terraform state show aws_vpc.this | head -5
terraform state pull | jq '{serial, lineage}'
# rename web → app: state AND code
terraform state mv aws_security_group.web \
aws_security_group.app
terraform plan
# - destroy aws_security_group.app
# + create aws_security_group.web ← code not renamed!
sed -i.bak 's/aws_security_group" "web"/aws_security_group" "app"/;
s/aws_security_group.web.id/aws_security_group.app.id/' sg.tf
terraform plan # No changes
# (ch 5: a moved block does this reviewably)
# break it: forget the internet gateway
terraform state rm aws_internet_gateway.this
terraform plan
# + aws_internet_gateway.this will be created
# ~ aws_route_table.public ...
# (applying would try to attach a 2nd gateway)B=$TF_STATE_BUCKET; K=dev/network/terraform.tfstate
aws s3api list-object-versions --bucket $B --prefix $K \
--query "Versions[?Key=='$K'].[VersionId,LastModified,IsLatest]" \
--output table
# pick the version just BEFORE the state rm (2nd newest)
V=<version-id>
aws s3api get-object --bucket $B --key $K \
--version-id $V /tmp/candidate.tfstate >/dev/null
jq '[.resources[] | select(.type=="aws_internet_gateway")]
| length' /tmp/candidate.tfstate # 1
aws s3api copy-object --bucket $B --key $K \
--copy-source "$B/$K?versionId=$V" >/dev/null
terraform plan # No changesstate rm and state mv each wrote a new version. If you restore a version from before the state mv, plan will want to rename back: pick the version right before the mistake.
Goal: a narrow, explicit contract: live/dev/app (ch 6, ch 10) reads two SSM parameters instead of the whole network state.
cat > outputs.tf <<'EOF'
output "vpc_id" {
value = aws_vpc.this.id
}
output "public_subnet_ids" {
value = [for s in aws_subnet.public : s.id]
}
output "private_subnet_ids" {
value = [for s in aws_subnet.private : s.id]
}
EOF
cat > ssm.tf <<'EOF'
resource "aws_ssm_parameter" "vpc_id" {
name = "/tflab/${var.env}/network/vpc_id"
type = "String"
value = aws_vpc.this.id
}
resource "aws_ssm_parameter" "private_subnet_ids" {
name = "/tflab/${var.env}/network/private_subnet_ids"
type = "StringList"
value = join(",", [for s in aws_subnet.private : s.id])
}
EOF
terraform apply # 2 to add (outputs change too)terraform output
# private_subnet_ids = ["subnet-0aa…", "subnet-0bb…"]
# public_subnet_ids = ["subnet-0cc…", "subnet-0dd…"]
# vpc_id = "vpc-0a1b2c3d…"
aws ssm get-parameters-by-path --path /tflab/dev/network \
--query 'Parameters[].[Name,Type,Value]' --output table
# /tflab/dev/network/private_subnet_ids StringList subnet-…,subnet-…
# /tflab/dev/network/vpc_id String vpc-0a1b…
# a consumer reads them like this (live/dev/app, later):
# data "aws_ssm_parameter" "vpc_id" {
# name = "/tflab/dev/network/vpc_id"
# }
# local.vpc_id = data.aws_ssm_parameter.vpc_id.insecure_value
git add backend.tf outputs.tf ssm.tf sg.tf
git commit -m "network: S3 backend, outputs, SSM contract" Standard-tier SSM parameters are free. Use SecureString only for secrets — and then keep them out of Terraform state where you can (ch 12).
The network state now lives in S3 with a lock, you have restored it once, and other components can consume it without reading it.
aws s3 ls s3://$TF_STATE_BUCKET/dev/network/ shows only terraform.tfstate when idle.terraform.tfstate left in live/dev/network/; terraform plan → No changes.terraform state pull | jq .serial grew with each write; lineage never changed.aws ssm get-parameter --name /tflab/dev/network/vpc_id --query Parameter.Value equals terraform output -raw vpc_id.use_lockfile = false, terraform init -reconfigure -backend-config=backend.hcl, and run two plans at once: both start. Restore true and reconfigure again.key to dev/network2/terraform.tfstate and run init -reconfigure → plan wants to create 13 resources (empty state). Do not apply; restore the key, init -reconfigure.terraform apply, then force-unlock <ID> from another terminal, and explain what could happen if both runs wrote. Then cancel A.# keep network + state for later labs
rm -f sg.tf.bak /tmp/candidate.tfstate
rm -f /tmp/network-local-backup.tfstate
# full teardown (end of the deck):
# terraform destroy
# (removes VPC, subnets, SG, SSM
# parameters; the state object
# stays, versioned, in S3)Goal: resources created by hand, found again with terraform query. A new root module live/dev/adopt with its own state key. Needs Lab 0 (ACCT, TF_STATE_BUCKET) and Lab 3 (files copied from network).
# 1) "ClickOps": made by hand, outside Terraform
B=tflab-clickops-$ACCT; aws s3api create-bucket --bucket $B \
--create-bucket-configuration LocationConstraint=ap-northeast-1
aws ssm put-parameter --type String \
--name /tflab/dev/clickops/feature-flag --value on
aws ssm put-parameter --type String \
--name /tflab/dev/clickops/owner --value alice
# 2) a root module for the adoption
mkdir -p tf-lab/live/dev/adopt && cd tf-lab/live/dev/adopt
cp ../network/versions.tf ../network/providers.tf \
../network/variables.tf .
cat > backend.tf <<'EOF'
terraform {
backend "s3" {
key = "dev/adopt/terraform.tfstate"
region = "ap-northeast-1"
encrypt = true
use_lockfile = true
}
}
EOF
terraform init -backend-config="bucket=$TF_STATE_BUCKET" # S3# 3) search the account (Terraform 1.14+)
cat > search.tfquery.hcl <<'EOF'
list "aws_s3_bucket" "all" {
provider = aws
}
list "aws_ssm_parameter" "all" {
provider = aws
config {
region = "ap-northeast-1"
}
}
EOF
terraform validate -query
terraform query
# list.aws_s3_bucket.all … tflab-clickops-123456789012 …
# list.aws_s3_bucket.all … tflab-state-123456789012 …
# list.aws_ssm_parameter.all … /tflab/dev/clickops/owner …
# (one line per object; columns depend on the provider)
# which types can be listed at all?
terraform providers schema -json | jq -r \
'.provider_schemas[].list_resource_schemas | keys[]' \
| grep -E 'aws_(s3_bucket|ssm_parameter)$'list blocks return identities of every matching object: here also your state bucket and anything else in the account. Choosing what to adopt is your job, not Terraform's.
Goal: an import plan that says "3 to import" and changes nothing except the tags that default_tags adds on purpose.
cat > imports.tf <<EOF
import {
to = aws_s3_bucket.reports
identity = {
bucket = "tflab-clickops-$ACCT" # shell expands ACCT
}
}
locals {
clickops = {
feature_flag = "/tflab/dev/clickops/feature-flag"
owner = "/tflab/dev/clickops/owner"
}
}
import {
for_each = local.clickops
to = aws_ssm_parameter.clickops[each.key]
id = each.value
}
EOF# the SSM resource by hand (it uses for_each)
cat > ssm.tf <<'EOF'
resource "aws_ssm_parameter" "clickops" {
for_each = local.clickops
name = each.value
type = "String"
value = each.key == "owner" ? "alice" : "on"
}
EOF
# let Terraform draft the bucket (single import)
terraform plan -generate-config-out=generated.tf
# Terraform has generated configuration and written it
# to generated.tf. Please review the configuration …
# review: delete null/default arguments, keep `bucket`
mv generated.tf bucket.tf && vi bucket.tf
terraform plan -out=tfplan
# # aws_s3_bucket.reports will be updated in-place
# # (imported from "tflab-clickops-123456789012")
# ~ tags_all = { + "Env" = "dev" + "ManagedBy" = … }
# Plan: 3 to import, 0 to add, 3 to change, 0 to destroy.
terraform apply tfplanThe 3 changes are only tags_all (from default_tags). Any other ~ line means the code disagrees with reality: fix the code first. Public access block, encryption and versioning are separate resources: unmanaged unless imported too.
Goal: replace positional count addresses with named keys, with a plan that shows moves and nothing else.
cat > extra.tf <<'EOF'
locals {
extra_names = [
"/tflab/dev/adopt/color",
"/tflab/dev/adopt/size",
]
}
resource "aws_ssm_parameter" "extra" { # positional
count = length(local.extra_names)
name = local.extra_names[count.index]
type = "String"
value = "v${count.index}"
}
EOF
terraform apply # 2 to add
terraform state list | grep extra
# aws_ssm_parameter.extra[0]
# aws_ssm_parameter.extra[1]# keyed version + one moved block per instance
cat > extra.tf <<'EOF'
locals {
extra = {
color = { name = "/tflab/dev/adopt/color", value = "v0" }
size = { name = "/tflab/dev/adopt/size", value = "v1" }
}
}
resource "aws_ssm_parameter" "extra" {
for_each = local.extra
name = each.value.name
type = "String"
value = each.value.value
}
moved {
from = aws_ssm_parameter.extra[0]
to = aws_ssm_parameter.extra["color"]
}
moved {
from = aws_ssm_parameter.extra[1]
to = aws_ssm_parameter.extra["size"]
}
EOF
terraform plan
# aws_ssm_parameter.extra[0] has moved to …extra["color"]
# aws_ssm_parameter.extra[1] has moved to …extra["size"]
# Plan: 0 to add, 0 to change, 0 to destroy.
terraform applyGoal: hand the bucket back without deleting it, then see how a console change shows up in a refresh-only plan.
# A) stop managing the bucket, keep the bucket
rm bucket.tf
cat > removed.tf <<'EOF'
removed {
from = aws_s3_bucket.reports
lifecycle {
destroy = false
}
}
EOF
# the import block for the bucket must go as well:
# edit imports.tf, delete the first import { } block
terraform plan
# # aws_s3_bucket.reports will no longer be managed by
# # Terraform, but will not be destroyed
# Plan: 0 to add, 0 to change, 0 to destroy.
# Warning: Some objects will no longer be managed …
terraform apply
aws s3api head-bucket --bucket tflab-clickops-$ACCT \
&& echo still-there # the bucket survived# B) drift: someone changes a managed parameter
aws ssm put-parameter --overwrite \
--name /tflab/dev/clickops/owner --value bob
terraform plan -refresh-only
# Note: Objects have changed outside of Terraform
# # aws_ssm_parameter.clickops["owner"] has changed
# ~ resource "aws_ssm_parameter" "clickops" {
# ~ value = (sensitive value)
# ~ version = 1 -> 2
# This is a refresh-only plan, so Terraform will not
# take any actions to undo these. …
terraform plan
# ~ value = (sensitive value)
# Plan: 0 to add, 1 to change, 0 to destroy.
# → decide: revert (apply this) or code it
# (change "alice" to "bob" in ssm.tf)
terraform plan -detailed-exitcode; echo "exit=$?"
# exit=2 → changes present (what CI drift jobs use)value of aws_ssm_parameter is sensitive, so plans show (sensitive value) instead of alice/bob. You still see that it changed, and the parameter version makes it obvious.
Every step should have produced a plan you could read and approve before anything changed.
terraform state list shows aws_ssm_parameter.clickops["feature_flag"], ["owner"] and aws_ssm_parameter.extra["color"], ["size"]; no aws_s3_bucket.reports.dev/adopt/terraform.tfstate exists in $TF_STATE_BUCKET.removed apply.Project, Env and ManagedBy tags.terraform plan says No changes.moved blocks commented out → 2 to add, 2 to destroy. Do not apply.id to /tflab/dev/clickops/nope → "Cannot import non-existent remote object".extra.tf away → plan wants to destroy both extra parameters. Do not apply; restore the file.ssm.tf had said value = "off" in step 2, the plan would show the import and ~ value. What would applying it do to the real parameter?cd tf-lab/live/dev/adopt
terraform destroy
# 4 parameters destroyed
# the bucket is no longer managed:
aws s3 rb s3://tflab-clickops-$ACCT \
--force
rm -f search.tfquery.hcl tfplan
git add . && git commit \
-m "lab5: adoption and refactor"
# the state object stays in S3
# until final teardown (Lab 13)Goal: move the resources into a child module without recreating anything. Step 1: make the module out of the existing files. Needs Labs 3–4 (live/dev/network on the S3 backend).
cd tf-lab/live/dev/network && terraform plan # "No changes"
mkdir -p ../../../modules/network
git mv main.tf sg.tf ../../../modules/network/
cd ../../../modules/network
sed -i.bak -e 's/ in ${var.region}//' \
-e 's/var\.vpc_cidr/var.cidr/g' main.tf && rm main.tf.bak
cat > variables.tf <<'EOF'
variable "env" {
type = string
validation {
condition = contains(["dev", "prod"], var.env)
error_message = "env must be dev or prod."
}
}
variable "cidr" {
type = string
validation {
condition = can(cidrnetmask(var.cidr)) && endswith(var.cidr, "/16")
error_message = "cidr must be an IPv4 /16."
}
}
EOFcat > outputs.tf <<'EOF'
output "vpc_id" { value = aws_vpc.this.id }
output "app_sg_id" { value = aws_security_group.app.id }
output "public_subnet_ids" {
value = { for k, s in aws_subnet.public : k => s.id }
}
output "private_subnet_ids" {
value = { for k, s in aws_subnet.private : k => s.id }
}
EOF
cat > versions.tf <<'EOF'
terraform {
required_version = ">= 1.14"
required_providers {
aws = { source = "hashicorp/aws", version = ">= 6.0" }
}
}
EOF
terraform fmt -check # names stay as in Lab 3:
# every move is X → module.network.XKeeping the resource names unchanged inside the module is what makes the moves mechanical. Rename resources later, in a separate change, with their own moved blocks inside the module.
Goal: the root module calls module.network; the plan shows only "has moved to" lines and 0 to add, 0 to change, 0 to destroy.
cd ../../live/dev/network
cat > main.tf <<'EOF'
module "network" {
source = "../../../modules/network"
env = var.env
cidr = var.vpc_cidr
}
EOF
# fix references left in the root (Lab 4's SSM
# parameters and outputs still point at aws_vpc.this …)
grep -n 'aws_vpc\.\|aws_subnet\.\|aws_security_group\.' *.tf
# → aws_vpc.this.id becomes module.network.vpc_id
# [for s in aws_subnet.public : s.id] becomes
# [for id in module.network.public_subnet_ids : id]
terraform init # installs the local module
terraform plan | tail -1
# Plan: 11 to add, 0 to change, 11 to destroy. ← NO!# one moved block per resource (all instances move)
terraform state list \
| grep -v -e '^data\.' -e '^aws_ssm_parameter\.' \
| sed 's/\[.*//' | sort -u \
| while read a; do
printf 'moved {\n from = %s\n to = module.network.%s\n}\n' \
"$a" "$a"
done > moved.tf
cat moved.tf
# moved {
# from = aws_internet_gateway.this
# to = module.network.aws_internet_gateway.this
# } … one per resource
terraform plan
# # aws_vpc.this has moved to module.network.aws_vpc.this
# # aws_subnet.public["ap-northeast-1a"] has moved to
# # module.network.aws_subnet.public["ap-northeast-1a"]
# ...
# Plan: 0 to add, 0 to change, 0 to destroy.
terraform apply
cd ../../.. && git add . && git commit -m "network module"The SSM parameters stay in the root (they publish the module's outputs), so they are excluded from the moves. If the plan still shows ~ lines, a reference in the root changed a value: compare it with the old expression.
Goal: a second module with decisions built in (private, encrypted, versioned), validated input names, and outputs for callers.
mkdir -p modules/app-bucket && cd modules/app-bucket
cp ../network/versions.tf . && cat > main.tf <<'EOF'
resource "aws_s3_bucket" "this" {
bucket = "tflab-${var.env}-${var.name}"
force_destroy = var.force_destroy
}
resource "aws_s3_bucket_public_access_block" "this" {
bucket = aws_s3_bucket.this.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
resource "aws_s3_bucket_versioning" "this" {
bucket = aws_s3_bucket.this.id
versioning_configuration {
status = "Enabled" # a decision, not an input
}
}
EOF
cat > outputs.tf <<'EOF'
output "bucket_name" { value = aws_s3_bucket.this.bucket }
output "bucket_arn" { value = aws_s3_bucket.this.arn }
EOFcat >> main.tf <<'EOF'
resource "aws_s3_bucket_server_side_encryption_configuration" "this" {
bucket = aws_s3_bucket.this.id
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "AES256"
}
}
}
EOF
cat > variables.tf <<'EOF'
variable "env" {
type = string
}
variable "name" { # bucket = tflab-<env>-<name>
type = string
validation {
condition = can(regex("^[a-z0-9-]{3,40}$", var.name))
error_message = "name: 3-40 chars, a-z 0-9 and -."
}
}
variable "force_destroy" {
type = bool
default = false
}
EOF
terraform fmt -check && cd ../..Goal: a second component that reads the network through SSM, and a module consumed by version instead of by path.
mkdir -p live/dev/app && cd live/dev/app
cp ../network/{versions,providers,variables}.tf .
sed 's#dev/network/#dev/app/#' ../network/backend.tf \
> backend.tf # key = dev/app/terraform.tfstate
cat > main.tf <<'EOF'
data "aws_caller_identity" "me" {}
data "aws_ssm_parameter" "vpc_id" {
name = "/tflab/dev/network/vpc_id" # Lab 4's name
}
module "assets" { # S3 names are global: add ACCT
source = "../../../modules/app-bucket"
env = "dev"
name = "assets-${data.aws_caller_identity.me.account_id}"
force_destroy = true # lab only
}
resource "aws_security_group" "app" {
name = "tflab-dev-app"
description = "App tier"
vpc_id = data.aws_ssm_parameter.vpc_id.insecure_value
}
output "bucket_name" { value = module.assets.bucket_name }
EOF
terraform init -backend-config="bucket=$TF_STATE_BUCKET"
terraform apply # Plan: 5 to add, 0 to change, 0 to destroy.# release the module: commit + annotated tag
cd ../../.. && git add . && git commit -m "app-bucket"
git tag -a app-bucket-v1.0.0 -m "app-bucket 1.0.0"
# consume the tag instead of the path
cd live/dev/app
REPO=$(git rev-parse --show-toplevel)
SRC="git::file://$REPO//modules/app-bucket?ref=app-bucket-v1.0.0"
sed -i.bak "s#\"../../../modules/app-bucket\"#\"$SRC\"#" main.tf
rm main.tf.bak && grep source main.tf
terraform init
# Downloading git::file:///…/tf-lab?ref=app-bucket-v1.0.0
# for assets...
terraform plan # No changes: same code, new source
# main moves on; dev stays on v1.0.0 until you bump it
cat >> ../../../modules/app-bucket/outputs.tf <<'EOF'
output "bucket_domain_name" {
value = aws_s3_bucket.this.bucket_regional_domain_name
}
EOF
terraform init && terraform plan # still No changesgit::file:// works for a local repository. After Lab 11 pushes the repo, the source becomes github.com/<you>/tf-lab//modules/app-bucket?ref=app-bucket-v1.0.0. Commit the new output, tag app-bucket-v1.1.0, bump ref, terraform init: only then does dev see it.
The proof of a good refactor is a boring plan.
terraform state list in live/dev/network shows only module.network.*, data.* and the SSM parameters.0 added, 0 changed, 0 destroyed; the VPC ID in the console did not change.live/dev/app state has module.assets.aws_s3_bucket.this (bucket tflab-dev-assets-<ACCT>) and aws_security_group.app in the dev VPC..terraform/modules/modules.json in live/dev/app shows the git::file://…?ref=app-bucket-v1.0.0 source.git tag -l lists app-bucket-v1.0.0.moved.tf to moved.tf.off and plan → every network resource is destroyed and recreated. Restore it.to = module.network.aws_subnet.private for the public subnets → Terraform reports an error instead of guessing. Read it, restore.name = "Assets" (upper case) → the module's validation rejects it at plan time; set the network cidr to a /20 → rejected too.git tag -f app-bucket-v1.0.0 on a newer commit → dev silently gets new code on the next init. Why do some teams pin full commit SHAs?# keep network and app for Labs 7+
# (both cost nothing)
# keep moved.tf: it documents the
# refactor; delete it once every
# environment that uses this code
# has applied it (prod: Lab 10)
# app bucket (force_destroy = true):
# cd live/dev/app
# terraform destroy
git status # cleanGoal: one lint config for the whole repo, run recursively, with the AWS ruleset pinned. No AWS credentials needed.
cd tf-lab
cat > .tflint.hcl <<'EOF'
plugin "terraform" {
enabled = true
preset = "recommended"
}
plugin "aws" {
enabled = true
version = "0.50.0" # latest release when written: pin yours
source = "github.com/terraform-linters/tflint-ruleset-aws"
}
EOF
tflint --init # installs the aws plugin
tflint --recursive \
--config "$PWD/.tflint.hcl" # every dir with .tf files
echo "exit=$?" # 0 = clean, 2 = issues
# plant two mistakes to see the rules fire
cat > modules/app-bucket/lintbait.tf <<'EOF'
variable "unused" {}
EOFtflint --chdir=modules/app-bucket \
--config "$PWD/.tflint.hcl" --format compact
# 2 issue(s) found:
# modules/app-bucket/lintbait.tf:1:1: Warning -
# `unused` variable has no type
# (terraform_typed_variables)
# modules/app-bucket/lintbait.tf:1:1: Warning -
# variable "unused" is declared but not used
# (terraform_unused_declarations)
echo "exit=$?" # 2
# warnings only → still fail? decide per repo:
tflint --chdir=modules/app-bucket \
--config "$PWD/.tflint.hcl" \
--minimum-failure-severity=error; echo "exit=$?" # 0
rm modules/app-bucket/lintbait.tf
# fix what the clean run reported (typical first-run
# findings: missing `type`, unused variables, missing
# required_version), then re-run until exit=0
git add .tflint.hcl && git commit -m "lint: tflint + aws ruleset" Check the latest tflint-ruleset-aws release on GitHub before pinning. --config "$PWD/.tflint.hcl" makes the root config apply when TFLint changes into subdirectories.
Goal: plan-mode asserts, validation tests with expect_failures, and one apply against a mocked provider. Seconds, no credentials.
# assumes Lab 6: vars env (dev|prod), name; bucket
# tflab-<env>-<name>; resources "this"; output bucket_arn
cd modules/app-bucket && mkdir -p tests
cat > tests/unit.tftest.hcl <<'EOF'
mock_provider "aws" {
mock_resource "aws_s3_bucket" {
defaults = { arn = "arn:aws:s3:::tflab-dev-logs" }
}
}
variables {
env = "dev"
name = "logs"
}
run "name_follows_convention" {
command = plan
assert {
condition = aws_s3_bucket.this.bucket == "tflab-dev-logs"
error_message = "bucket name must be tflab-<env>-<name>"
}
}
# (continued on the right)run "versioning_on_by_default" {
command = plan
assert {
condition = (aws_s3_bucket_versioning.this
.versioning_configuration[0].status == "Enabled")
error_message = "versioning must default to Enabled"
}
}
run "rejects_uppercase_name" {
command = plan
variables { name = "Logs" }
expect_failures = [var.name]
}
run "rejects_unknown_env" {
command = plan
variables { env = "staging" }
expect_failures = [var.env]
}
run "apply_with_mocks" { # apply against the MOCK only
assert {
condition = output.bucket_arn == "arn:aws:s3:::tflab-dev-logs"
error_message = "arn output comes from the mocked bucket"
}
}
EOF
unset AWS_PROFILE # prove no credentials are used
terraform init -backend=false && terraform test
# … Success! 5 passed, 0 failed.Goal: assert subnet math at plan time, reject bad CIDRs, wire IDs with overrides, and run every module's tests in one command.
cd ../network && mkdir -p tests
cat > tests/network.tftest.hcl <<'EOF'
mock_provider "aws" {
mock_data "aws_availability_zones" {
defaults = { names = ["ap-northeast-1a", "ap-northeast-1c"] }
} # else names = []
}
variables {
env = "dev"
cidr = "10.10.0.0/16"
}
run "subnet_plan" {
command = plan
assert {
condition = (length(aws_subnet.public) == 2
&& length(aws_subnet.private) == 2)
error_message = "expected 2 public and 2 private subnets"
}
assert {
condition = (aws_subnet.private["ap-northeast-1a"]
.cidr_block == "10.10.10.0/24")
error_message = "private subnet in 1a must be 10.10.10.0/24"
}
}
run "rejects_bad_cidr" {
command = plan
variables { cidr = "10.10.0.0/24" }
expect_failures = [var.cidr]
}cat >> tests/network.tftest.hcl <<'EOF'
run "ids_from_overrides" {
override_resource {
target = aws_vpc.this
values = { id = "vpc-0123456789abcdef0" }
}
assert {
condition = alltrue([for s in aws_subnet.private :
s.vpc_id == "vpc-0123456789abcdef0"])
error_message = "every private subnet must be in the VPC"
}
}
EOF
terraform init -backend=false && terraform test
# … 3 passed, 0 failed.
# all modules at once (from the repo root)
cd ../..
for m in modules/*/; do
(cd "$m" && terraform init -backend=false -input=false \
>/dev/null && terraform test -junit-xml=test.xml) \
|| exit 1
done
printf 'test.xml\n' >> .gitignore
git add modules .gitignore
git commit -m "tests: unit tests with mocked aws provider" A test suite is only useful if it fails when the module breaks. Break the module on purpose and watch which test catches it.
tflint --recursive exits 0 on the whole repo.terraform test passes in every module with AWS_PROFILE unset.test.xml files exist (JUnit format, for CI test reports in ch 11).terraform test -verbose prints the planned values for each run.i + 10 to i + 20 in modules/network → subnet_plan fails with a Diff (10.10.20.0/24 vs 10.10.10.0/24).app-bucket → rejects_uppercase_name fails: the run expected a failure that did not happen.output.bucket_arn → "Unknown condition value"; fix it with override_during = plan on the mock.required_version aside → TFLint reports terraform_required_version.# nothing was created in AWS:
# mocks only. Revert the breaks:
git checkout -- modules
# keep the tests and .tflint.hcl;
# Lab 11 runs them in CI
# provider cache (optional):
export TF_PLUGIN_CACHE_DIR=\
"$HOME/.terraform.d/plugin-cache"
mkdir -p "$TF_PLUGIN_CACHE_DIR"Goal: scripts/plan-summary.sh turns any saved plan into a short table and exits 1 when a protected resource type would be deleted or replaced.
cd tf-lab && mkdir -p scripts
# no-op/read map to nothing (skipped), except a no-op
# with previous_address = a moved object
cat > scripts/plan-summary.sh <<'EOF'
#!/usr/bin/env bash
# usage: scripts/plan-summary.sh tfplan.json
set -euo pipefail
plan=${1:?usage: plan-summary.sh <plan.json>}
re=${PROTECTED_TYPES:-'^aws_(s3_bucket|vpc|subnet|kms_key|iam_role)$'}
jq -r '
{"create": "CREATE", "update": "UPDATE",
"delete": "DELETE", "forget": "FORGET",
"delete,create": "REPLACE(-/+)",
"create,delete": "REPLACE(+/-)"} as $label
| .resource_changes[] as $r
| ($r.change.actions | join(",")) as $a
| ($label[$a] // (if $r.previous_address
then "MOVE" else empty end))
| [., $r.address, ($r.action_reason // "")]
| @tsv' "$plan" | column -t -s $'\t'
EOFcat >> scripts/plan-summary.sh <<'EOF'
blocked=$(jq -r --arg re "$re" '
.resource_changes[]
| select((.change.actions | index("delete"))
and (.type | test($re)))
| .address' "$plan")
if [[ -n "$blocked" && "${ALLOW_DESTROY:-}" != "true" ]]; then
echo "BLOCKED: deletes/replaces protected:" >&2
echo "$blocked" >&2
exit 1
fi
echo "OK: protected-type gate passed"
EOF
chmod +x scripts/plan-summary.sh
# baseline: no changes expected in dev/app
cd live/dev/app
terraform plan -input=false -out=tfplan -detailed-exitcode
echo "plan exit=$?" # 0 = nothing to do
terraform show -json tfplan > tfplan.json
../../../scripts/plan-summary.sh tfplan.json
# OK: protected-type gate passed (empty table)
printf 'tfplan\ntfplan.json\n' >> ../../../.gitignoreGoal: see a bucket rename and a CIDR change as reviewers should see them. Neither plan is applied.
# 1) rename the bucket: in live/dev/app change the
# module input (Lab 6): "assets-<ACCT>" -> "static-<ACCT>"
terraform plan -input=false -out=tfplan
# # module.assets.aws_s3_bucket.this must be replaced
# -/+ resource "aws_s3_bucket" "this" {
# ~ bucket = "tflab-dev-assets-<ACCT>"
# -> "tflab-dev-static-<ACCT>"
# # forces replacement
# Plan: 4 to add, 0 to change, 4 to destroy.
# (bucket + 3 companions that take `bucket`)
terraform show -json tfplan > tfplan.json
../../../scripts/plan-summary.sh tfplan.json
echo "gate exit=$?"
# REPLACE(-/+) module.assets.aws_s3_bucket.this
# replace_because_cannot_update
# REPLACE(-/+) module.assets.aws_s3_bucket_versioning.this …
# BLOCKED: deletes/replaces protected:
# module.assets.aws_s3_bucket.this
# gate exit=1
jq -c '.resource_changes[] | select(.change.replace_paths)
| {address, paths: .change.replace_paths}' tfplan.json
git checkout -- . # undo the rename# 2) change the dev VPC CIDR 10.10.0.0/16 -> 10.30.0.0/16
cd ../network # live/dev/network
F=$(grep -l '10.10.0.0/16' *.tf *.tfvars 2>/dev/null)
sed -i.bak 's#10.10.0.0/16#10.30.0.0/16#' $F && rm -f *.bak
terraform plan -input=false -out=tfplan
terraform show -json tfplan > tfplan.json
../../../scripts/plan-summary.sh tfplan.json
# REPLACE(-/+) module.network.aws_vpc.this
# REPLACE(-/+) module.network.aws_subnet.private["ap-northeast-1a"]
# … every subnet, route table association, IGW attachment
# UPDATE aws_ssm_parameter.vpc_id (new value)
# BLOCKED: deletes/replaces protected:
# module.network.aws_vpc.this
# module.network.aws_subnet.private["ap-northeast-1a"] …
# severity: Critical (ch 8) — everything in the VPC
# would be replaced, and dev/app reads these IDs
git checkout -- . # undo the CIDR change
# count actions for the PR description
jq -r '[.resource_changes[].change.actions | join(",")]
| group_by(.) | map("\(.[0]): \(length)") | .[]' \
tfplan.jsonAddresses depend on your Lab 6 module names (module.assets, module.network); the shape of the output does not. Exact counts depend on what your network module contains.
Goal: review a low-risk plan, apply exactly that plan, and watch Terraform refuse a plan that went stale.
cd ../app # live/dev/app
# low-risk change: one more default tag in the provider
# default_tags { tags = { …, Owner = "<you>" } }
terraform plan -input=false -out=tfplan
terraform show -json tfplan > tfplan.json
../../../scripts/plan-summary.sh tfplan.json
# UPDATE module.assets.aws_s3_bucket.this (tags_all)
# UPDATE aws_ssm_parameter.… (tags_all)
# OK: protected-type gate passed
# review the human-readable plan once more
terraform show -no-color tfplan | less
# apply EXACTLY the reviewed plan (no re-plan, no prompt)
terraform apply -input=false tfplan
# Apply complete! Resources: 0 added, N changed,
# 0 destroyed.# a stale plan: make two plans, apply the second
# (change Owner to another value first)
terraform plan -input=false -out=a.tfplan
terraform plan -input=false -out=b.tfplan
terraform apply -input=false b.tfplan
terraform apply -input=false a.tfplan
# Error: Saved plan is stale
#
# The given plan file can no longer be applied
# because the state was changed by another
# operation after the plan was created.
# the escape hatch for a reviewed deletion:
ALLOW_DESTROY=true \
../../../scripts/plan-summary.sh tfplan.json
# (Lab 11 sets it from a PR label "allow-destroy")
rm -f tfplan tfplan.json a.tfplan b.tfplan
git add ../../../scripts ../../../.gitignore .
git commit -m "plan-summary gate; Owner tag in dev/app" The script is only as good as its list of protected types and the habit of reading its output. Break both on purpose.
plan -detailed-exitcode exits 0 in live/dev/app and live/dev/network after the clean-up steps.# forces replacement on bucket, and the gate exits 1.tfplan; a new plan is empty.a.tfplan was refused as stale.PROTECTED_TYPES="^aws_kms_key$" on the rename plan → exit 0 with a bucket replacement in it. The list is policy: review changes to it.terraform apply (no file) after editing a tag → it plans again and asks; with -auto-approve nobody reviewed that plan.terraform plan -target=module.assets → "Resource targeting is in effect": the summary looks small because the plan is incomplete.force_destroy = true on the bucket and re-run the rename plan: the gate still blocks it, but an apply would delete the objects.# nothing risky was applied.
# make sure both roots are clean:
for c in app network; do
(cd live/dev/$c &&
terraform plan -input=false \
-detailed-exitcode >/dev/null;
echo "$c exit=$?") # expect 0
done
# remove the Owner tag again if
# you do not want it: edit, plan,
# summary, apply tfplan.
rm -f live/dev/*/tfplan*Goal: scan modules/ with both tools and list every finding. Needs Labs 6–8. Your list may be longer if your modules have more resources.
cd tf-lab
pipx install checkov # Checkov 3.x (Python)
brew install trivy conftest
checkov --version; trivy --version; conftest --version
mkdir -p policies/checkov policies/opa
checkov -d modules --compact --quiet
# Check: CKV2_AWS_11: "Ensure VPC flow logging is enabled …"
# FAILED for resource: aws_vpc.this
# Check: CKV2_AWS_12: "Ensure the default security group of
# every VPC restricts all traffic"
# Check: CKV_AWS_130: "Ensure VPC subnets do not assign
# public IP by default" (aws_subnet.public[…])
# Check: CKV_AWS_18 / CKV_AWS_144 / CKV_AWS_145 /
# CKV2_AWS_61 / CKV2_AWS_62 (aws_s3_bucket.this)
# … plus anything your own modules addtrivy config modules
# AWS-0089 (LOW) Bucket has logging disabled
# AWS-0132 (HIGH) … customer managed key
# AWS-0164 (HIGH) Subnet associates public IP address
# AWS-0178 (MEDIUM) VPC does not have VPC Flow Logs
# triage each finding:
# real risk in OUR use? → fix in the module
# accepted for this module? → inline skip + reason
# never applies to tf-lab? → global skip, reviewed
#
# rule: every exception says WHY, in the diff
# where a reviewer will see itGoal: one real fix, inline suppressions with reasons for accepted risks, and two small reviewed files for global decisions.
# 1) FIX: lock down the default security group (free)
cat >> modules/network/main.tf <<'EOF'
# no ingress/egress blocks = all rules removed
resource "aws_default_security_group" "this" {
vpc_id = aws_vpc.this.id
}
EOF
# 2) SUPPRESS WITH A REASON, next to the resource:
# modules/network/main.tf, inside aws_subnet.public
# #checkov:skip=CKV_AWS_130:public subnets by design
# and directly ABOVE resource "aws_subnet" "public"
# #trivy:ignore:AWS-0164 -- public subnets by design
#
# modules/app-bucket/main.tf, inside aws_s3_bucket.this
# #checkov:skip=CKV_AWS_18:lab data, no access logs
# #checkov:skip=CKV_AWS_145:SSE-S3 is enough for lab
# and above it
# #trivy:ignore:AWS-0089 -- lab data, no access logs# 3) GLOBAL decisions in one reviewed file each
cat > policies/checkov.yaml <<'EOF'
skip-check:
- CKV_AWS_144 # replication: single-region lab
- CKV2_AWS_61 # lifecycle rules: short-lived lab
- CKV2_AWS_62 # event notifications: unused
- CKV2_AWS_11 # flow logs: cost; enable in prod
compact: true
quiet: true
EOF
cat > policies/.trivyignore <<'EOF'
AWS-0132 # SSE-S3 in the lab; prod uses a CMK
AWS-0178 # flow logs: same decision as CKV2_AWS_11
EOFRun terraform fmt -recursive and the module tests from Lab 7 after editing: suppression comments change nothing, but the default security group resource removes the default egress rule.
Goal: every taggable resource carries Project, Env and ManagedBy — checked on the plan, where default_tags are merged into tags_all.
cat > policies/checkov/required_tags.yaml <<'EOF'
metadata:
id: "CKV2_TFLAB_1"
name: "Ensure taggable resources carry Project, Env and ManagedBy tags"
category: "GENERAL_SECURITY"
severity: "MEDIUM"
definition:
and:
- cond_type: attribute
resource_types: &taggable
[aws_s3_bucket, aws_vpc, aws_subnet, aws_internet_gateway]
attribute: tags_all.Project
operator: exists
- cond_type: attribute
resource_types: *taggable
attribute: tags_all.Env
operator: exists
- cond_type: attribute
resource_types: *taggable
attribute: tags_all.ManagedBy
operator: exists
EOFcd live/dev/app
terraform plan -out=tfplan && terraform show -json tfplan > tfplan.json
checkov -f tfplan.json --external-checks-dir ../../../policies/checkov \
--check CKV2_TFLAB_1 --compact --quiet
# terraform_plan scan results:
# Passed checks: 1, Failed checks: 0, Skipped checks: 0
cd ../../..
# a resource without ManagedBy would show:
# Check: CKV2_TFLAB_1: "Ensure taggable resources …"
# FAILED for resource: module.network.aws_vpc.thisYAML anchors (&taggable, *taggable) keep the three resource lists identical. The pass count depends on what your live/dev/app plan contains.
Goal: a Rego policy over the plan JSON, with unit tests that run without any plan.
cat > policies/opa/s3.rego <<'EOF'
package main
import rego.v1
protected := {"aws_s3_bucket", "aws_s3_bucket_public_access_block"}
pab_settings := [
"block_public_acls", "block_public_policy",
"ignore_public_acls", "restrict_public_buckets",
]
deny contains msg if {
some rc in input.resource_changes
rc.type in protected
"delete" in rc.change.actions
msg := sprintf("%s: %v is not allowed for %s",
[rc.address, rc.change.actions, rc.type])
}
deny contains msg if {
some rc in input.resource_changes
rc.type == "aws_s3_bucket_public_access_block"
some setting in pab_settings
rc.change.after[setting] != true
msg := sprintf("%s: %s must be true", [rc.address, setting])
}
EOFcat > policies/opa/s3_test.rego <<'EOF'
package main
import rego.v1
test_bucket_delete_denied if {
count(deny) == 1 with input as {"resource_changes": [{
"address": "aws_s3_bucket.x", "type": "aws_s3_bucket",
"change": {"actions": ["delete"], "after": null}}]}
}
test_create_allowed if {
count(deny) == 0 with input as {"resource_changes": [{
"address": "aws_s3_bucket.x", "type": "aws_s3_bucket",
"change": {"actions": ["create"], "after": {}}}]}
}
EOF
conftest verify --policy policies/opa # 2 tests pass
conftest test --policy policies/opa live/dev/app/tfplan.json
# 0 failures on a normal planGoal: a single command that runs every static and plan check, locally and later in CI (Lab 11). Any failure stops it with a non-zero exit code.
cat >> Makefile <<'EOF'
PLAN_DIRS ?= live/dev/network live/dev/app
.PHONY: policy policy-static policy-plan
policy: policy-static policy-plan
policy-static:
checkov -d modules --config-file policies/checkov.yaml
trivy config --severity HIGH,CRITICAL --exit-code 1 \
--ignorefile policies/.trivyignore modules
conftest verify --policy policies/opa
policy-plan:
@for d in $(PLAN_DIRS); do \
test -f $$d/tfplan.json || { \
echo "missing $$d/tfplan.json: plan first"; exit 1; }; \
echo "== $$d"; \
checkov -f $$d/tfplan.json --compact --quiet \
--external-checks-dir policies/checkov \
--check CKV2_TFLAB_1 || exit 1; \
conftest test --policy policies/opa \
$$d/tfplan.json || exit 1; \
done
EOF# plans for both dev roots (Lab 8 style)
for d in live/dev/network live/dev/app; do
terraform -chdir=$d plan -out=tfplan >/dev/null
terraform -chdir=$d show -json tfplan > $d/tfplan.json
done
make policy
# checkov: Failed checks: 0, Skipped checks: N
# trivy: no HIGH/CRITICAL findings left
# conftest verify: all tests pass
# == live/dev/network … 0 failures
# == live/dev/app … 0 failures
echo $? # 0
git add Makefile policies modules
git commit -m "policy: checkov, trivy, conftest gates"Recipe lines start with a tab (spaces → missing separator); $$d passes $d to the shell.
The gate is only useful if it fails for the right reasons. Prove it with plans that must be rejected.
make policy exits 0 on the current code.checkov -d modules lists each suppression under "Skipped checks" with its reason.trivy config modules (no ignore file) still shows AWS-0132; with --ignorefile policies/.trivyignore it is gone.checkov -d live/dev/app --external-checks-dir policies/checkov --check CKV2_TFLAB_1) fails — explain why (ch 9).git grep -n "checkov:skip\|trivy:ignore" gives the full list of exceptions to review.live/dev/app → plan shows -/+; make policy fails with ["delete","create"] is not allowed. Do not apply; revert.block_public_policy = false in modules/app-bucket → Trivy (HCL) and conftest (plan) both fail.ManagedBy from default_tags → CKV2_TFLAB_1 fails on the plan.#checkov:skip=CKV_AWS_18 without a reason → it still passes. Add a review rule for that (ch 13).modules/ without --ignorefile → ignores vanish.# nothing billable was created
# in this lab: only files
# remove generated plans (they
# contain resolved values)
find live -name 'tfplan*' -delete
# keep policies/, Makefile and
# the module fixes: Lab 11 runs
# make policy in CI
# scratch tools can stay;
# uninstall if you like:
# pipx uninstall checkov
# brew uninstall trivy conftestGoal: no literal "dev" left in live/dev/*/*.tf; every environment value comes from an auto-loaded terraform.tfvars. The dev plans must stay at "No changes". Needs Labs 4–9.
cd tf-lab
grep -n '"dev"\|/dev/\|-dev-' live/dev/*/*.tf | grep -v backend.tf
# live/dev/app/main.tf: name = "/tflab/dev/network/vpc_id"
# live/dev/app/main.tf: env = "dev"
# live/dev/app/main.tf: name = "tflab-dev-app"
# (your list may differ: fix every hit except backend keys)
cd live/dev/app
sed -i.bak \
-e 's#"/tflab/dev/network/#"/tflab/${var.env}/network/#' \
-e 's#env *= "dev"#env = var.env#' \
-e 's#"tflab-dev-app"#"tflab-${var.env}-app"#' \
main.tf && rm main.tf.bak
terraform fmt# environment values: auto-loaded by plain `terraform plan`
# (Lab 11's CI runs plan without -var-file)
cat > terraform.tfvars <<'EOF'
env = "dev"
EOF
cat > ../network/terraform.tfvars <<'EOF'
env = "dev"
vpc_cidr = "10.10.0.0/16"
EOF
terraform plan -detailed-exitcode; echo "exit=$?"
# No changes. Your infrastructure matches the configuration.
# exit=0
cd ../network
terraform plan -detailed-exitcode; echo "exit=$?"
# exit=0
# same values, now explicit: the refactor is invisible to AWS
git add -A && git commit -m "live: env values in terraform.tfvars"Exit 2 means something changed: read the diff, you replaced a value that was not identical. variables.tf keeps its defaults from Lab 3; the tfvars files make the environment explicit and win over defaults.
Goal: live/prod/network (10.20.0.0/16) and live/prod/app, each with its own state key. Copy only source files — never .terraform/, state or plans.
cd tf-lab
for c in network app; do
mkdir -p live/prod/$c
cp live/dev/$c/*.tf live/dev/$c/.terraform.lock.hcl \
live/prod/$c/ # same providers
sed -i.bak "s#dev/$c/terraform.tfstate#prod/$c/terraform.tfstate#" \
live/prod/$c/backend.tf && rm live/prod/$c/backend.tf.bak
done
cat > live/prod/network/terraform.tfvars <<'EOF'
env = "prod"
vpc_cidr = "10.20.0.0/16"
EOF
cat > live/prod/app/terraform.tfvars <<'EOF'
env = "prod"
EOF
# any "dev" left in prod is a bug (→ no output)
grep -n 'dev' live/prod/*/*.tf live/prod/*/*.tfvars
grep -h key live/prod/*/backend.tf
# key = "prod/network/terraform.tfstate"
# key = "prod/app/terraform.tfstate"# producer first
cd live/prod/network
terraform init -backend-config="bucket=$TF_STATE_BUCKET"
terraform plan -out=tfplan
# Plan: N to add, 0 to change, 0 to destroy.
# + cidr_block = "10.20.0.0/16"
# + name = "/tflab/prod/network/vpc_id"
terraform apply tfplan
# then the consumer
cd ../app
terraform init -backend-config="bucket=$TF_STATE_BUCKET"
terraform plan -out=tfplan
# + bucket = "tflab-prod-assets-<ACCT>"
# + name = "tflab-prod-app"
terraform apply tfplan
aws s3 ls s3://$TF_STATE_BUCKET --recursive | grep tfstate
# dev/app/terraform.tfstate
# dev/network/terraform.tfstate
# prod/app/terraform.tfstate
# prod/network/terraform.tfstateRead the prod plan before applying: only + lines, prod names, the 10.20.0.0/16 CIDR. Any - or ~ means prod is pointing at dev state — stop and check the backend key. All resources here are free (VPC, subnets, IGW, SG, S3, SSM Standard).
Goal: make plan-all ENV=prod plans network then app, stops at the first error, and leaves tfplan + tfplan.json next to each root for review and for make policy.
mkdir -p scripts && cat > scripts/plan-all.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
ENV=${1:?usage: plan-all.sh dev|prod}
ORDER=(network app) # producers first
changed=0
for c in "${ORDER[@]}"; do
dir="live/$ENV/$c"
echo "== $dir"
terraform -chdir="$dir" init -input=false \
-backend-config="bucket=$TF_STATE_BUCKET" >/dev/null
rc=0
terraform -chdir="$dir" plan -input=false -no-color \
-out=tfplan -detailed-exitcode > "$dir/plan.txt" || rc=$?
case $rc in
0) echo " no changes" ;; 2) echo " CHANGES"; changed=1 ;;
*) cat "$dir/plan.txt"; echo " FAILED"; exit 1 ;;
esac
terraform -chdir="$dir" show -json tfplan > "$dir/tfplan.json"
done
exit $(( changed ? 2 : 0 )) # like -detailed-exitcode
EOF
chmod +x scripts/plan-all.sh
printf 'plan.txt\n' >> .gitignorecat >> Makefile <<'EOF'
ENV ?= dev
.PHONY: plan-all
plan-all:
scripts/plan-all.sh $(ENV)
EOF
make plan-all ENV=prod
# == live/prod/network
# no changes
# == live/prod/app
# no changes
# policies over both environments (Lab 9 target)
make policy PLAN_DIRS="live/dev/network live/dev/app \
live/prod/network live/prod/app"
git add -A && git commit -m "live/prod + plan-all"Exit 2 (changes pending) makes make plan-all fail: usable as a drift check.
Goal: feel the trade-off of section 39 without touching real state. The comparison uses its own key prefix tg-compare/, so nothing it plans can affect tf-lab. Requires brew install terragrunt (1.x).
mkdir -p tg-compare/live/prod/network
cd tg-compare/live && cat > root.hcl <<'EOF'
remote_state {
backend = "s3"
generate = { path = "backend.tf", if_exists = "overwrite_terragrunt" }
config = {
bucket = "tflab-state-${get_aws_account_id()}"
key = "tg-compare/${path_relative_to_include()}/terraform.tfstate"
region = "ap-northeast-1"
encrypt = true
use_lockfile = true
}
}
generate "provider" {
path = "provider.tf"
if_exists = "overwrite_terragrunt"
contents = "provider \"aws\" { region = \"ap-northeast-1\" }"
}
EOFcat > prod/network/terragrunt.hcl <<'EOF'
include "root" {
path = find_in_parent_folders("root.hcl")
}
terraform {
source = "../../../../modules//network"
}
inputs = { env = "prod", cidr = "10.20.0.0/16" }
EOF
cd prod
terragrunt run --all -- plan
# Plan: N to add, 0 to change, 0 to destroy.
# (empty key tg-compare/prod/network/… → all "create")
wc -l network/terragrunt.hcl ../../../live/prod/network/*.tf
# do NOT apply: it would be a second prod VPC
cd ../../.. && rm -rf tg-compareTerragrunt generates backend.tf and provider.tf in .terragrunt-cache/. Check remote_state keys (e.g. use_lockfile) against your Terragrunt release.
Prod is a parameterized copy of dev, not a fork. Prove it, then break the layout in the ways that happen in real teams.
make plan-all ENV=dev and ENV=prod both print "no changes" (exit 0).aws ssm get-parameters-by-path --path /tflab/prod/network returns the prod VPC; its CIDR is 10.20.0.0/16 (aws ec2 describe-vpcs).diff -r live/dev live/prod -x .terraform -x "tfplan*" -x plan.txt shows only backend.tf keys and terraform.tfvars.source (?ref=app-bucket-v…) — promotion is a visible diff when you bump it.make policy passes for all four tfplan.json files.cp -r live/dev/network /tmp/x, edit its key to prod/…, run terraform plan without init → "Backend configuration changed". The dangerous answer is init -migrate-state: it would copy dev state into the prod key. Answer: init -reconfigure.dev/network/… in the prod backend → the plan wants to change the dev VPC CIDR (forces replacement). Never apply; this is why the grep step exists.app before network is applied fails with ParameterNotFound for /tflab/<env>/network/vpc_id. ORDER=(network app) in the script is what prevents it.# keep dev AND prod for Lab 11:
# CI plans and applies both
# at the end of the course,
# destroy consumers first:
for e in prod dev; do
for c in app network; do
terraform -chdir=live/$e/$c \
destroy
done
done
# remove generated files
find live -name 'tfplan*' \
-o -name plan.txt | xargs rm -fGoal: one small module that creates an IAM role trusted by GitHub OIDC for an exact list of token subjects.
# modules/ci-role/variables.tf
variable "name" { type = string }
variable "oidc_provider_arn" { type = string }
variable "subjects" {
description = "Exact GitHub OIDC sub claims"
type = list(string)
validation {
condition = alltrue([for s in var.subjects : !strcontains(s, "*")])
error_message = "No wildcards in subjects."
}
}
variable "managed_policy_arns" {
type = list(string)
default = []
}
variable "inline_policy_json" {
type = string
default = null
}
# modules/ci-role/outputs.tf
output "arn" { value = aws_iam_role.this.arn }# modules/ci-role/main.tf (1/2)
data "aws_iam_policy_document" "trust" {
statement {
actions = ["sts:AssumeRoleWithWebIdentity"]
principals {
type = "Federated"
identifiers = [var.oidc_provider_arn]
}
condition {
test = "StringEquals"
variable = "token.actions.githubusercontent.com:aud"
values = ["sts.amazonaws.com"]
}
condition {
test = "StringEquals"
variable = "token.actions.githubusercontent.com:sub"
values = var.subjects
}
}
}StringEquals with an explicit list: no StringLike, no wildcards, enforced by the validation block (ch 2) so nobody can loosen it by accident.
Goal: finish the module, then create the public repo and read the numeric IDs that the immutable sub claim contains.
# modules/ci-role/main.tf (2/2)
resource "aws_iam_role" "this" {
name = var.name
assume_role_policy = data.aws_iam_policy_document.trust.json
max_session_duration = 3600
}
resource "aws_iam_role_policy_attachment" "managed" {
for_each = toset(var.managed_policy_arns)
role = aws_iam_role.this.name
policy_arn = each.value
}
resource "aws_iam_role_policy" "inline" {
count = var.inline_policy_json == null ? 0 : 1
role = aws_iam_role.this.id
policy = var.inline_policy_json
}# create the repo (public: free runner minutes)
cd tf-lab
git rev-parse --git-dir >/dev/null 2>&1 || git init -b main
git add -A && git commit -qm "tf-lab"
gh repo create tf-lab --public --source . --push
gh api repos/<you>/tf-lab --jq '.owner.id, .id'
# 1234567 ← owner_id
# 8901234 ← repo_id
# repos created, renamed or transferred after
# 15 Jul 2026 get sub claims of the form
# repo:<you>@1234567/tf-lab@8901234:<context>
# older repos: repo:<you>/tf-lab:<context>
# (print a real token's claims: CI/CD deck, Lab 4)Goal: the OIDC provider and both roles, in a root module that the pipeline never applies.
# live/shared/ci/main.tf (backend key shared/ci/...)
locals {
repo = "repo:<you>@1234567/tf-lab@8901234"
st = "arn:aws:s3:::tflab-state-${data.aws_caller_identity.me.account_id}"
apply_policies = ["AmazonVPCFullAccess", "AmazonS3FullAccess",
"AmazonSSMFullAccess", "CloudWatchLogsFullAccess"]
}
data "aws_caller_identity" "me" {}
resource "aws_iam_openid_connect_provider" "gh" {
url = "https://token.actions.githubusercontent.com"
client_id_list = ["sts.amazonaws.com"]
}
module "apply" {
source = "../../../modules/ci-role"
name = "gha-tf-apply"
oidc_provider_arn = aws_iam_openid_connect_provider.gh.arn
subjects = [
"${local.repo}:environment:dev",
"${local.repo}:environment:prod",
]
managed_policy_arns = [for p in local.apply_policies :
"arn:aws:iam::aws:policy/${p}"]
}module "plan" {
source = "../../../modules/ci-role"
name = "gha-tf-plan"
oidc_provider_arn = aws_iam_openid_connect_provider.gh.arn
subjects = [
"${local.repo}:pull_request",
"${local.repo}:ref:refs/heads/main",
]
managed_policy_arns = ["arn:aws:iam::aws:policy/ReadOnlyAccess"]
inline_policy_json = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Action = ["s3:PutObject", "s3:DeleteObject"]
Resource = "${local.st}/*.tflock"
}]
})
}
# terraform init -backend-config="bucket=$TF_STATE_BUCKET" \
# -backend-config="key=shared/ci/terraform.tfstate"
# terraform applyReadOnlyAccess already allows s3:GetObject and s3:ListBucket, so the inline policy only adds lock-file writes. The managed write policies keep the lab short; in real use, scope apply to what each component manages and never give CI IAM write access.
Goal: environments dev and prod (prod with you as required reviewer), the changed-directories script, and the workflow from ch 11.
R=<you>/tf-lab
ME=$(gh api user --jq .id)
gh api -X PUT repos/$R/environments/dev
gh api -X PUT repos/$R/environments/prod --input - <<EOF
{"reviewers": [{"type": "User", "id": $ME}],
"deployment_branch_policy":
{"protected_branches": false,
"custom_branch_policies": true}}
EOF
gh api -X POST -f name=main \
repos/$R/environments/prod/deployment-branch-policies
mkdir -p scripts && cat > scripts/changed-dirs.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
git diff --name-only "$BASE" HEAD > /tmp/changed.txt
if grep -q '^modules/' /tmp/changed.txt; then
ls -d live/dev/* live/prod/*
else
grep -oE '^live/(dev|prod)/[^/]+' /tmp/changed.txt || true
fi | sort -u | jq -R -s -c 'split("\n")
| map(select(length > 0))
| map({dir: ., env: (split("/")[1]),
name: (split("/")[1:] | join("-"))})'
EOF
chmod +x scripts/changed-dirs.sh# .github/workflows/terraform.yml
# = the four jobs from ch 11, section 41:
# detect → checks (PR) → plan (matrix) → apply (push)
# the detect step passes the diff base:
- id: d
env:
BASE: ${{ github.event.pull_request.base.sha
|| github.event.before }}
run: echo "dirs=$(scripts/changed-dirs.sh)" \
>> "$GITHUB_OUTPUT"
# checks job: Lab 9's `make policy` needs its tools.
# Install pinned versions + verify checksums:
- run: pipx install checkov==<version>
# trivy + conftest: download the release tarball
# and its checksums file, then sha256sum -c
# every `terraform init` there needs
# -backend-config="bucket=tflab-state-<acct>"
# (partial backend config, Lab 0)
# .github/workflows/drift.yml = ch 11, section 43
# replace every <sha> with the full commit SHA of the
# action's current release, then:
git add -A && git commit -m "ci: terraform pipeline"
git pushIn this single-person lab, leave "Prevent self-review" off so you can approve your own prod deployment. In a team, turn it on.
Goal: watch a tag change go PR → plan comment → merge → dev apply → prod approval → prod apply, then catch a console edit with the drift job.
git switch -c owner-tag
# add Owner = "<you>" to default_tags in
# live/dev/app and live/prod/app
git commit -am "tags: add Owner" && git push -u origin HEAD
gh pr create --fill
gh pr checks --watch
# detect → checks → plan (dev-app, prod-app)
gh pr view --comments | head -40
# ### live/dev/app
# ~ update in-place ... tags_all ... + "Owner" = "<you>"
# Plan: 0 to add, N to change, 0 to destroy.
gh pr merge --squash --delete-branch
gh run watch # push run on main
# apply (dev-app) ✓ applied automatically
# apply (prod-app) ⏳ waiting for review# approve: Actions → the run → Review deployments
# → prod → Approve (read the plan job log first)
gh run watch # apply (prod-app) ✓
# drift: change a tag outside Terraform
B=$(cd live/dev/app && terraform output -raw bucket_name)
aws s3api put-bucket-tagging --bucket "$B" --tagging \
'TagSet=[{Key=Env,Value=hacked}]'
gh workflow run drift.yml && gh run watch
gh issue list
# #7 Drift: live/dev/app
# revert: re-run the whole push run, so it plans
# again and dev applies main (a re-run of only the
# apply job would hit "Saved plan is stale")
gh run rerun <push-run-id>
# ...or codify the change in a PRThe output name bucket_name is whatever live/dev/app exposes for its bucket; adjust it to your Lab 6 outputs. put-bucket-tagging replaces the whole tag set, so the plan will want to restore every tag.
The pipeline should make the safe path the only easy path.
apply (prod-app) job waits in "Waiting" until you approve; dev-app does not.AssumeRoleWithWebIdentity for gha-tf-plan on the PR, gha-tf-apply only in environment jobs.gh secret list shows no AWS keys.terraform apply to the plan job → AWS denies the writes (ReadOnlyAccess). Revert.sub:** remove environment: ${{ matrix.c.env }} from the apply job → "Not authorized to perform sts:AssumeRoleWithWebIdentity"..terraform.lock.hcl in a PR → init -lockfile=readonly fails.# keep the repo and roles for Lab 12;
# at the very end:
cd live/shared/ci
terraform destroy
gh repo delete <you>/tf-lab --yes
# disable the nightly drift job
# if you keep the repo:
gh workflow disable drift.ymlGoal: see with your own eyes that sensitive hides a value on screen but stores it in state, plan files and old S3 versions.
mkdir -p live/dev/secrets-demo && cd live/dev/secrets-demo
cat > main.tf <<'EOF'
terraform {
required_version = ">= 1.14"
required_providers {
aws = { source = "hashicorp/aws", version = "~> 6.0" }
random = { source = "hashicorp/random", version = "~> 3.7" }
}
backend "s3" {
key = "dev/secrets-demo/terraform.tfstate"
region = "ap-northeast-1"
use_lockfile = true
}
}
provider "aws" { region = "ap-northeast-1" }
resource "random_password" "app" {
length = 32
}
resource "aws_ssm_parameter" "app_secret" {
name = "/tflab/dev/secrets-demo/app-secret"
type = "SecureString"
value = random_password.app.result
}
EOFterraform init -backend-config="bucket=$TF_STATE_BUCKET"
terraform apply # value = (sensitive value) on screen
# ...but in the state, in clear text:
terraform state pull | jq -r '.resources[]
| select(.type == "random_password")
| .instances[0].attributes.result'
# Xk2… (32 characters)
terraform state pull | jq -r '.resources[]
| select(.type == "aws_ssm_parameter")
| .instances[0].attributes.value'
# Xk2… (the same value, a second time)
# and in every saved plan:
terraform plan -out=tfplan -replace=random_password.app
terraform show -json tfplan | jq -r \
'.prior_state.values.root_module.resources[]
| select(.type=="random_password") | .values.result'
# Xk2…State lives in tflab-state-<acct> with versioning on: each apply keeps a full copy of the old state, including this value. Delete tfplan now (rm tfplan).
Goal: the same SSM parameter, but the password exists only during the run. Terraform 1.11+ and hashicorp/random with the ephemeral random_password.
# replace the two resources in main.tf with:
ephemeral "random_password" "app" {
length = 32
}
resource "aws_ssm_parameter" "app_secret" {
name = "/tflab/dev/secrets-demo/app-secret"
type = "SecureString"
value_wo = ephemeral.random_password.app.result
value_wo_version = 1
}
terraform plan
# - random_password.app will be destroyed
# (only leaves state: it never existed in AWS)
# ~ aws_ssm_parameter.app_secret will be updated in-place
# + value_wo = (write-only attribute)
# + value_wo_version = 1
# (exact lines vary by provider version)
terraform applyterraform state pull | jq '.resources[]
| select(.type == "aws_ssm_parameter")
| .instances[0].attributes
| {value, value_wo, value_wo_version}'
# { "value": null, "value_wo": null, "value_wo_version": 1 }
# (an empty string for value is also "not stored")
# the secret still exists, in SSM only:
aws ssm get-parameter --with-decryption \
--name /tflab/dev/secrets-demo/app-secret \
--query Parameter.Value --output text
# 9fQ… (new value: the ephemeral password of the apply)
terraform plan # No changes. (a new ephemeral password is
# generated, but value_wo_version is still 1)
# rotate: set value_wo_version = 2, then
terraform apply # ~ update in-place; SSM gets a new valueThe exact rendering of value in state depends on the provider version; what matters is that neither the old nor the new password appears in it. Old state versions in S3 still contain the first password: rotating (version 2) makes that copy useless.
Goal: TLS only, and object access only for you and the two CI roles. Applied from bootstrap/, never by CI.
# bootstrap/state-policy.tf
# A bucket has ONE policy: if Lab 0 already created an
# aws_s3_bucket_policy, merge these statements into it.
variable "admin_role_arn" {
description = "IAM role ARN (not the STS session ARN)"
type = string
}
data "aws_caller_identity" "pol" {}
locals {
b = "tflab-state-${data.aws_caller_identity.pol.account_id}"
acct = data.aws_caller_identity.pol.account_id
}
resource "aws_s3_bucket_policy" "state" {
bucket = local.b
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{ Sid = "DenyInsecureTransport"
Effect = "Deny"
Principal = "*"
Action = "s3:*"
Resource = ["arn:aws:s3:::${local.b}",
"arn:aws:s3:::${local.b}/*"]
Condition = { Bool = { "aws:SecureTransport" = "false" } } }, { Sid = "OnlyAdminAndCI"
Effect = "Deny"
Principal = "*"
Action = ["s3:GetObject", "s3:PutObject",
"s3:DeleteObject"]
Resource = "arn:aws:s3:::${local.b}/*"
Condition = { ArnNotLike = { "aws:PrincipalArn" = [
var.admin_role_arn,
"arn:aws:iam::${local.acct}:role/gha-tf-plan",
"arn:aws:iam::${local.acct}:role/gha-tf-apply",
] } } },
]
})
}
# your role ARN: take the role name from the session ARN
aws sts get-caller-identity --query Arn --output text
# arn:aws:sts::<acct>:assumed-role/AWSReservedSSO_Admin…_abc/you
aws iam list-roles --query "Roles[?RoleName=='<that name>'].Arn"
# arn:aws:iam::<acct>:role/aws-reserved/sso.amazonaws.com/
# ap-northeast-1/AWSReservedSSO_Admin…_abc
cd bootstrap && terraform apply \
-var admin_role_arn='<the role ARN>'Run terraform plan in live/dev/app right after applying: if it fails with AccessDenied, your ARN is wrong. Fix it from bootstrap/ (local state, so you are not locked out of it). Worst case, the account root user can delete the bucket policy.
Goal: a lock file that verifies providers on your Mac and on Linux runners, and a CI init that refuses to change it.
cd live/dev/app
grep -c 'h1:' .terraform.lock.hcl # per provider:
# 1 h1: (your platform) + many zh: (registry checksums)
terraform providers lock \
-platform=darwin_arm64 \
-platform=linux_amd64 \
-platform=linux_arm64
git diff --stat .terraform.lock.hcl
# 1 file changed, 4 insertions(+) ← new h1: lines
# do the same in every root module, then commit
for d in ../../*/*/; do
(cd "$d" && terraform providers lock \
-platform=darwin_arm64 -platform=linux_amd64 \
-platform=linux_arm64)
done
git add -A && git commit -m "lock: hashes for 3 platforms"# what CI runs (Lab 11):
terraform init -input=false -lockfile=readonly \
-backend-config="bucket=$TF_STATE_BUCKET"
# readonly mode refuses changes to the lock file:
# use a provider that is not locked yet ...
echo 'resource "null_resource" "x" {}' > extra.tf
terraform init -lockfile=readonly \
-backend-config="bucket=$TF_STATE_BUCKET"
# - Installing hashicorp/null v3.x...
# Error: Provider dependency changes detected
# Changes to the required provider dependencies were
# detected, but the lock file is read-only. ...
rm extra.tf # undoModule versions are not in the lock file. Pin registry modules with an exact version and Git modules with ?ref=<full commit SHA> (ch 12).
Prove each control with a command, then break it on purpose.
terraform state pull | grep -c <first 6 chars of the SSM value> → 0.aws ssm get-parameter --with-decryption returns the value: AWS has it, Terraform does not.gha-tf-plan (re-run a PR check) still works with the new bucket policy.curl http://<bucket>.s3.ap-northeast-1.amazonaws.com/ style access (no TLS) is denied.h1: hashes for three platforms.length = 40 only → No changes. Bump value_wo_version → update.value = ephemeral.random_password.app.result → error: ephemeral values are not valid in that argument.aws s3 cp s3://<bucket>/dev/app/terraform.tfstate - → AccessDenied.null provider test above.cd live/dev/secrets-demo
terraform destroy
# deletes the SSM parameter
# optional: purge old state
# versions of this key only
aws s3api list-object-versions \
--bucket $TF_STATE_BUCKET \
--prefix dev/secrets-demo/ \
--query 'Versions[].VersionId'
# then delete-object --version-id
# for each (keep the bucket policy
# until the very end of the deck)Goal: feel a major provider upgrade without risking real infrastructure. The scratch root has only data sources and outputs, so applying it creates nothing in AWS.
mkdir -p scratch/upgrade && cd scratch/upgrade
cat > main.tf <<'EOF'
terraform {
required_version = ">= 1.14"
required_providers {
aws = { source = "hashicorp/aws", version = "~> 5.0" }
}
}
provider "aws" { region = "ap-northeast-1" }
data "aws_region" "current" {}
data "aws_caller_identity" "me" {}
output "where" {
value = format("%s in %s",
data.aws_caller_identity.me.account_id,
data.aws_region.current.name)
}
EOF
terraform init # local state: scratch only
grep -A1 'registry.terraform.io/hashicorp/aws' \
.terraform.lock.hcl # version = "5.x.y"
terraform apply -auto-approve
# where = "123456789012 in ap-northeast-1"# bump the major: "~> 5.0" → "~> 6.0"
sed -i.bak 's/~> 5.0/~> 6.0/' main.tf && rm main.tf.bak
terraform plan
# Error: Inconsistent dependency lock file
# … locked version selection 5.x.y doesn't match
# the updated version constraints
# → run "terraform init -upgrade"
terraform init -upgrade
grep -A1 'registry.terraform.io/hashicorp/aws' \
.terraform.lock.hcl # version = "6.x.y"
terraform validate # Success! (removed args fail here)
terraform plan
# Warning: Deprecated attribute
# The attribute "name" is deprecated. …
# fix: .name → .region (new attribute in v6)
sed -i.bak 's/current\.name/current.region/' main.tf
rm main.tf.bak
terraform plan # No changes. and no warningExact error and warning wording varies by Terraform and provider version; what matters is the sequence: constraint → init -upgrade → lock file diff → validate → a no-op plan without warnings. Deprecated attributes show up in plan, not always in validate.
Goal: upgrade a live root inside its constraint, then see which plan modes notice a console change.
cd live/dev/network
terraform plan # No changes. (baseline)
# newest 6.x allowed by "~> 6.0"
terraform init -upgrade
git diff --stat .terraform.lock.hcl
terraform plan # still No changes.
# hashes for every platform that runs this root
terraform providers lock \
-platform=linux_amd64 -platform=darwin_arm64
git add .terraform.lock.hcl
git commit -m "dev/network: aws provider 6.x.y"
# timings (run each twice; the first warms caches)
time terraform plan -refresh=false
time terraform plan
time terraform plan -minimal-refresh # 1.17+# ClickOps: add a tag in the "console"
VPC=$(aws ec2 describe-vpcs --filters \
Name=tag:Project,Values=tf-lab Name=tag:Env,Values=dev \
--query 'Vpcs[0].VpcId' --output text)
aws ec2 create-tags --resources $VPC \
--tags Key=Owner,Value=clickops
terraform plan -refresh=false # No changes. (blind)
terraform plan -minimal-refresh # No changes. (VPC not
# part of any change)
terraform plan # ~ update in-place
# ~ tags_all = {
# - "Owner" = "clickops" -> null …
terraform plan -refresh-only # shows the drift as a
# state update instead
# decide: we do not want it → revert through Terraform
terraform apply # review, then yes
terraform plan # No changes.Your timings depend on the root size: dev/network has only a handful of resources, so the differences are small. On a state with hundreds of resources, the refresh is most of the plan time.
Goal: run tofu init and tofu plan against the same code without touching the real backend, the real lock file or Terraform's .terraform/ directory.
brew install opentofu && tofu version
cd live/dev/network
# 1) a private copy of the state (contains real IDs:
# keep it out of Git, delete it afterwards)
terraform state pull > /tmp/dev-network.tfstate
# 2) point THIS directory at the copy, for tofu only:
# *_override.tf files replace the backend block
cat > backend_override.tf <<'EOF'
terraform {
backend "local" {
path = "/tmp/dev-network.tfstate"
}
}
EOF
# 3) separate data dir: Terraform's .terraform/ untouched
export TF_DATA_DIR=.terraform-tofu
tofu init
tofu plan# expected: No changes.
# or an error naming a feature / state version that
# OpenTofu does not support yet. Both are results:
# write them down for the ADR (ch 13).
# what changed in the lock file?
git diff .terraform.lock.hcl
# + provider "registry.opentofu.org/hashicorp/aws" {
# + version = "6.x.y"
# + hashes = [ … ]
# 4) undo everything
unset TF_DATA_DIR
rm -rf .terraform-tofu backend_override.tf
git checkout -- .terraform.lock.hcl
rm /tmp/dev-network.tfstate
git status # clean
terraform plan # No changes. (real backend)Never run tofu apply here: the copy is local, so an apply would only diverge the copy from reality. A real migration follows the OpenTofu guide for your exact Terraform version, one root at a time.
Goal: generated module READMEs, then destroy everything — dependents before dependencies, and the state bucket last because it holds every other state.
brew install terraform-docs
# add the markers once to each module README:
# <!-- BEGIN_TF_DOCS -->
# <!-- END_TF_DOCS -->
for m in network app-bucket ci-role; do
terraform-docs markdown table \
--output-file README.md --output-mode inject \
modules/$m
done
git diff --stat modules/ # Inputs/Outputs tables
# ---- teardown: app before network, prod then dev
for env in prod dev; do
for c in app network; do
(cd live/$env/$c && terraform destroy)
done
done
# BucketNotEmpty? empty its versions (right), retry
# Lab 12 demo and the CI roles from Lab 11
(cd live/dev/secrets-demo && terraform destroy)
(cd live/shared/ci && terraform destroy)
gh repo delete <you>/tf-lab # optional# ---- bootstrap LAST: its own state is in the bucket
cd bootstrap # ADMIN_ARN: same role ARN as Lab 12
rm backend.tf # 1) state back to local
terraform init -migrate-state # answer: yes
# 2) delete the prevent_destroy line in main.tf, then
terraform apply -var admin_role_arn=$ADMIN_ARN # (lifecycle)
# 3) empty every object version and delete marker
B=$TF_STATE_BUCKET
for kind in Versions DeleteMarkers; do
aws s3api delete-objects --bucket $B --delete "$(
aws s3api list-object-versions --bucket $B \
--query "{Objects: $kind[].{Key:Key,VersionId:VersionId}}" \
--output json)"
done
# (an empty list makes delete-objects fail: harmless)
terraform destroy -var admin_role_arn=$ADMIN_ARN # 4)
# 5) last: the budget
aws budgets delete-budget --account-id $ACCT \
--budget-name <name from Lab 0>delete-objects handles up to 1,000 keys per call — enough for this lab. Migrating bootstrap's state back to local before emptying the bucket matters: emptying first would delete the only copy of that state.
Each step has a check you can run. The teardown is complete only when the account has nothing tagged Project=tf-lab.
dev/network lock file commit lists h1: hashes for both linux_amd64 and darwin_arm64.Owner=clickops tag and why.tofu plan result recorded; git status clean afterwards.aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=tf-lab returns an empty list (it may lag a few minutes).init -upgrade:** bump the constraint and run plan → "Inconsistent dependency lock file". Why is that error a feature?darwin_arm64 h1: line (delete every zh: line) and push → the Lab 11 CI init fails with a checksum mismatch. With zh: hashes present, a registry install on Linux still verifies.terraform destroy in live/dev/network before app → read the error (dependent resources or SSM lookups) instead of forcing it.live/* state if the bucket went first. (Do not run it.)# verify nothing is left
aws resourcegroupstaggingapi \
get-resources --tag-filters \
Key=Project,Values=tf-lab \
--query \
'ResourceTagMappingList[].ResourceARN'
# []
aws s3api head-bucket \
--bucket $TF_STATE_BUCKET
# ... (404) Not Found
aws iam get-role \
--role-name gha-tf-apply
# NoSuchEntity
# local clean-up
rm -rf scratch/
# keep the tf-lab repo: it is your
# reference implementationThe labs use only free or near-free AWS resources. The real risks are leftovers, a forgotten resource type, and credentials — not the bill for a VPC.
default_tags with Project=tf-lab: one tag query finds every leftover.terraform plan before every destroy, and read what will go.live/prod then live/dev, each app before network.live/dev/secrets-demo (Lab 12) and live/shared/ci (Lab 11); optionally delete <you>/tf-lab.prevent_destroy, empty all object versions, destroy.aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=tf-lab is empty; then delete the budget.Short anchors; explain each in your own words.
| Term | Short meaning |
|---|---|
| Root / child module | Directory you run terraform in (own backend, state) / module called with a module block |
| Provider | Plugin that maps resources to an API (e.g. hashicorp/aws) |
| Resource / data source | Object Terraform manages / object it only reads |
| Resource address | module.network.aws_subnet.public["a"] — identity in state |
| State | Map from addresses to real IDs and attributes; holds secrets in clear text |
| Backend | Where state is stored and locked (s3, HCP Terraform, …) |
| State lock vs lock file | Run-time lock on a state (use_lockfile) vs .terraform.lock.hcl (provider versions, hashes) |
| Drift | Real infrastructure differs from state/config (often ClickOps) |
| Term | Short meaning |
|---|---|
| Plan / refresh | Proposed changes (-out file applied as reviewed) / reading real objects before diffing |
| Replacement (ForceNew) | Destroy + create because an argument cannot change in place |
moved / removed block | Rename an address without touching the resource / stop managing it (destroy = false keeps it) |
import block | Brings an existing resource under management through a plan |
List block / terraform query | Searches existing resources for bulk import (1.14+) |
| Ephemeral value | Value used during a run but never written to state or plan |
| Write-only argument | *_wo argument sent to the provider, never stored in state |
| Workspace (CLI) | Several states for one configuration; not an environment boundary |
Short anchors; explain each in your own words.
| Term | Short meaning |
|---|---|
| IaC | Infrastructure defined in reviewed, versioned code |
| OpenTofu | MPL-licensed fork of Terraform 1.5 under the Linux Foundation |
| BUSL 1.1 | Terraform's source-available license since 1.6 |
| HCP Terraform | HashiCorp's hosted runs, state, registry and policies |
| Stacks | HCP Terraform feature to deploy components across many deployments |
| Terragrunt / Terramate | Orchestration tools for many roots (DRY config, ordering) |
| Blast radius | How much one bad apply or one state can break |
action block | Provider operation outside CRUD, run with apply -invoke (1.14+) |
| Resource identity | Provider-defined identifying attributes, used by import and query |
| Term | Short meaning |
|---|---|
| Policy as code | Guardrails as versioned, tested rules run in CI |
| Checkov | Scanner with built-in policies for HCL and plan JSON |
Trivy config | IaC misconfiguration scanner (absorbed tfsec) |
| OPA / Rego / conftest | Policy engine, its language, and a CLI to test files with it |
| Terraform Policy | HCL-based policy as code, GA in Terraform 1.17 |
| Sentinel | HashiCorp's policy language for HCP Terraform / Enterprise |
| OIDC federation | CI gets short-lived cloud credentials without stored keys |
| Drift detection | Scheduled plan -detailed-exitcode; exit 2 = changes |
| Break-glass | Audited emergency access outside the normal pipeline |
Terraform and the AWS provider release often; when a slide and the docs disagree, the docs win.
developer.hashicorp.com/terraform/language, …/terraform/cligithub.com/hashicorp/terraform/releasesuse_lockfile) — developer.hashicorp.com/terraform/language/backend/s3moved — developer.hashicorp.com/terraform/language/modules/develop/refactoringterraform query — developer.hashicorp.com/terraform/language/import, …/language/import/bulkdeveloper.hashicorp.com/terraform/language/testsopentofu.org/docs, opentofu.org/docs/intro/migrationsupport.hashicorp.com/hc/en-us/articles/47520801288083registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-6-upgradegithub.com/hashicorp/terraform-provider-aws/blob/main/CHANGELOG.mdwww.checkov.io; Trivy misconfiguration scanning — trivy.devwww.aquasec.com/blog/trivy-supply-chain-attack/www.openpolicyagent.org/docs; conftest — www.conftest.devgithub.com/terraform-linters/tflint; terraform-docs — terraform-docs.ioterragrunt.gruntwork.io; Infracost — www.infracost.io/docsdocs.github.com/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services