terraform Documentation

1. Project Overview & Quickstart (hashicorp/terraform) Copy Chapter

File: README.md Copy Chapter

Terraform

• Website: https://developer.hashicorp.com/terraform
• Forums: HashiCorp Discuss
• Documentation: https://developer.hashicorp.com/terraform/docs
• Tutorials: HashiCorp's Learn Platform
• Certification Exam: HashiCorp Certified: Terraform Associate

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

The key features of Terraform are:

• Infrastructure as Code: Infrastructure is described using a high-level configuration syntax. This allows a blueprint of your datacenter to be versioned and treated as you would any other code. Additionally, infrastructure can be shared and re-used.

• Execution Plans: Terraform has a "planning" step where it generates an execution plan. The execution plan shows what Terraform will do when you call apply. This lets you avoid any surprises when Terraform manipulates infrastructure.

• Resource Graph: Terraform builds a graph of all your resources, and parallelizes the creation and modification of any non-dependent resources. Because of this, Terraform builds infrastructure as efficiently as possible, and operators get insight into dependencies in their infrastructure.

• Change Automation: Complex changesets can be applied to your infrastructure with minimal human interaction. With the previously mentioned execution plan and resource graph, you know exactly what Terraform will change and in what order, avoiding many possible human errors.

For more information, refer to the What is Terraform? page on the Terraform website.

Getting Started & Documentation Copy Chapter

Documentation is available on the Terraform website:

• Introduction
• Documentation

If you're new to Terraform and want to get started creating infrastructure, please check out our Getting Started guides on HashiCorp's learning platform. There are also additional guides to continue your learning.

Show off your Terraform knowledge by passing a certification exam. Visit the certification page for information about exams and find study materials on HashiCorp's learning platform.

Developing Terraform Copy Chapter

This repository contains only Terraform core, which includes the command line interface and the main graph engine. Providers are implemented as plugins, and Terraform can automatically download providers that are published on the Terraform Registry. HashiCorp develops some providers, and others are developed by other organizations. For more information, refer to Plugin development.

• To learn more about compiling Terraform and contributing suggested changes, refer to the contributing guide.

• To learn more about how we handle bug reports, refer to the bug triage guide.

• To learn how to contribute to the Terraform documentation, refer to the Web Unified Docs repository.

License Copy Chapter

Business Source License 1.1

File: docs/plugin-protocol/object-wire-format.md Copy Chapter

Wire Format for Terraform Objects and Associated Values

The provider wire protocol (as of major version 5) includes a protobuf message
type DynamicValue which Terraform uses to represent values from the Terraform
Language type system, which result from evaluating the content of resource,
data, and provider blocks, based on a schema defined by the corresponding
provider.

Because the structure of these values is determined at runtime, DynamicValue
uses one of two possible dynamic serialization formats for the values
themselves: MessagePack or JSON. Terraform most commonly uses MessagePack,
because it offers a compact binary representation of a value. However, a server
implementation of the provider protocol should fall back to JSON if the
MessagePack field is not populated, in order to support both formats.

The remainder of this document describes how Terraform translates from its own
type system into the type system of the two supported serialization formats.
A server implementation of the Terraform provider protocol can use this
information to decode DynamicValue values from incoming messages into
whatever representation is convenient for the provider implementation.

A server implementation must also be able to produce DynamicValue messages
as part of various response messages. When doing so, servers should always
use MessagePack encoding, because Terraform does not consistently support
JSON responses across all request types and all Terraform versions.

Both the MessagePack and JSON serializations are driven by information the
provider previously returned in a Schema message. Terraform will encode each
value depending on the type constraint given for it in the corresponding schema,
using the closest possible MessagePack or JSON type to the Terraform language
type. Therefore a server implementation can decode a serialized value using a
standard MessagePack or JSON library and assume it will conform to the
serialization rules described below.

MessagePack Serialization Rules Copy Chapter

The MessagePack types referenced in this section are those defined in
The MessagePack type system specification.

Note that MessagePack defines several possible serialization formats for each
type, and Terraform may choose any of the formats of a specified type.
The exact serialization chosen for a given value may vary between Terraform
versions, but the types given here are contractual.

Conversely, server implementations that are producing MessagePack-encoded
values are free to use any of the valid serialization formats for a particular
type. However, we recommend choosing the most compact format that can represent
the value without a loss of range.

`Schema.Block` Mapping Rules for MessagePack Copy Section

To represent the content of a block as MessagePack, Terraform constructs a
MessagePack map that contains one key-value pair per attribute and one
key-value pair per distinct nested block described in the Schema.Block message.

The key-value pairs representing attributes have values based on
the Schema.Attribute mapping rules.
The key-value pairs representing nested block types have values based on
the Schema.NestedBlock mapping rules.

`Schema.Attribute` Mapping Rules for MessagePack Copy Section

The MessagePack serialization of an attribute value depends on the value of the
type field of the corresponding Schema.Attribute message. The type field is
a compact JSON serialization of a
Terraform type constraint,
which consists either of a single
string value (for primitive types) or a two-element array giving a type kind
and a type argument.

The following table describes the type-specific mapping rules. Along with those
type-specific rules there are two special rules that override the mappings
in the table below, regardless of type:

• A null value is represented as a MessagePack nil value.
• An unknown value (that is, a placeholder for a value that will be decided
  only during the apply operation) is represented as a
  MessagePack extension
  value, described in more detail below.

Unknown values have two possible representations, both using
MessagePack extension
values.

The older encoding is for unrefined unknown values and uses an extension
code of zero, with the extension value payload completely ignored.

Newer Terraform versions can produce "refined" unknown values which carry some
additional information that constrains the possible range of the final value/
Refined unknown values have extension code 12 and then the extension object's
payload is a MessagePack-encoded map using integer keys to represent different
kinds of refinement:

• 1 represents "nullness", and the value of that key will be a boolean
  value that is true if the value is definitely null or false if it is
  definitely not null. If this key isn't present at all then the value may or
  may not be null. It's not actually useful to encode that an unknown value
  is null; use a known null value instead in that case, because there is only
  one null value of each type.
• 2 represents string prefix, and the value is a string that the final
  value is known to begin with. This is valid only for unknown values of string
  type.
• 3 and 4 represent the lower and upper bounds respectively of a number
  value, and the value of both is a two-element msgpack array whose
  first element is a valid encoding of a number (as in the table above)
  and whose second element is a boolean value that is true for an inclusive
  bound and false for an exclusive bound. This is valid only for unknown values
  of number type.
• 5 and 6 represent the lower and upper bounds respectively of the length
  of a collection value. The value of both is an integer representing an
  inclusive bound. This is valid only for unknown values of the three kinds of
  collection types: list, set, and map.

Unknown value refinements are an optional way to reduce the range of possible
values for situations where that makes it possible to produce a known result
for unknown inputs or where it allows detecting an error during the planning
phase that would otherwise be detected only during the apply phase. It's always
safe to ignore refinements and just treat an unknown value as wholly unknown,
but considering refinements may allow a more precise answer. A provider that
produces refined values in its planned new state (from PlanResourceChange)
must honor those refinements in the final state (from ApplyResourceChange).

Unmarshalling code should ignore refinement map keys that they don't know about,
because future versions of the protocol might define additional refinements.

When encoding an unknown value without any refinements, always use the older
format with extension code zero instead of using extension code 12 with an
empty refinement map. Any refined unknown value must have at least one
refinement map entry. This rule ensures backward compatibility with older
implementations that predate the value refinements concept.

A server implementation of the protocol should treat any MessagePack extension
code as representing an unknown value, but should ignore the payload of that
extension value entirely unless the extension code is 12 to indicate that
the body represents refinements. Future versions of this protocol may define
specific formats for other extension codes, but they will always represent
unknown values.

`Schema.NestedBlock` Mapping Rules for MessagePack Copy Section

The MessagePack serialization of a collection of blocks of a particular type
depends on the nesting field of the corresponding Schema.NestedBlock message.
The nesting field is a value from the Schema.NestingBlock.NestingMode
enumeration.

All nesting values cause the individual blocks of a type to be represented
by applying
the Schema.Block mapping rules
to the block's contents based on the block field, producing what we'll call
a block value in the table below.

The nesting value then in turn defines how Terraform will collect all of the
individual block values together to produce a single property value representing
the nested block type. For all nesting values other than MAP, blocks may
not have any labels. For the nesting value MAP, blocks must have exactly
one label, which is a string we'll call a block label in the table below.

For the LIST and SET nesting modes, Terraform guarantees that the
MessagePack array will have a number of elements between the min_items and
max_items values given in the schema, unless any of the block values contain
nested unknown values. When unknown values are present, Terraform considers
the value to be potentially incomplete and so Terraform defers validation of
the number of blocks. For example, if the configuration includes a dynamic
block whose for_each argument is unknown then the final number of blocks is
not predictable until the apply phase.

JSON Serialization Rules Copy Chapter

The JSON serialization is a secondary representation for DynamicValue, with
MessagePack preferred due to its ability to represent unknown values via an
extension.

The JSON encoding described in this section is also used for the json field
of the RawValue message that forms part of an UpgradeResourceState request.
However, in that case the data is serialized per the schema of the provider
version that created it, which won't necessarily match the schema of the
current version of that provider.

`Schema.Block` Mapping Rules for JSON Copy Section

To represent the content of a block as JSON, Terraform constructs a
JSON object that contains one property per attribute and one property per
distinct nested block described in the Schema.Block message.

The properties representing attributes have property values based on
the Schema.Attribute mapping rules.
The properties representing nested block types have property values based on
the Schema.NestedBlock mapping rules.

`Schema.Attribute` Mapping Rules for JSON Copy Section

The JSON serialization of an attribute value depends on the value of the type
field of the corresponding Schema.Attribute message. The type field is
a compact JSON serialization of a
Terraform type constraint,
which consists either of a single
string value (for primitive types) or a two-element array giving a type kind
and a type argument.

The following table describes the type-specific mapping rules. Along with those
type-specific rules there is one special rule that overrides the rules in the
table regardless of type:

• A null value is always represented as JSON null.

`Schema.NestedBlock` Mapping Rules for JSON Copy Section

The JSON serialization of a collection of blocks of a particular type depends
on the nesting field of the corresponding Schema.NestedBlock message.
The nesting field is a value from the Schema.NestingBlock.NestingMode
enumeration.

All nesting values cause the individual blocks of a type to be represented
by applying
the Schema.Block mapping rules
to the block's contents based on the block field, producing what we'll call
a block value in the table below.

The nesting value then in turn defines how Terraform will collect all of the
individual block values together to produce a single property value representing
the nested block type. For all nesting values other than MAP, blocks may
not have any labels. For the nesting value MAP, blocks must have exactly
one label, which is a string we'll call a block label in the table below.

For the LIST and SET nesting modes, Terraform guarantees that the JSON
array will have a number of elements between the min_items and max_items
values given in the schema.

File: docs/plugin-protocol/README.md Copy Chapter

Terraform Plugin Protocol

This directory contains documentation about the physical wire protocol that
Terraform Core uses to communicate with provider plugins.

Most providers are not written directly against this protocol. Instead, prefer
to use an SDK that implements this protocol and write the provider against
the SDK's API.

If you want to write a plugin for Terraform, please refer to
Plugin development instead.

This documentation is for those who are developing Terraform SDKs, rather
than those implementing plugins.

From Terraform v0.12.0 onwards, Terraform's plugin protocol is built on
gRPC. This directory contains .proto definitions of
different versions of Terraform's protocol.

Only .proto files published as part of Terraform release tags are actually
official protocol versions. If you are reading this directory on the main
branch or any other development branch then it may contain protocol definitions
that are not yet finalized and that may change before final release.

RPC Plugin Model Copy Chapter

Terraform plugins are normal executable programs that, when launched, expose
gRPC services on a server accessed via the loopback interface. Terraform Core
discovers and launches plugins, waits for a handshake to be printed on the
plugin's stdout, and then connects to the indicated port number as a
gRPC client.

For this reason, we commonly refer to Terraform Core itself as the plugin
"client" and the plugin program itself as the plugin "server". Both of these
processes run locally, with the server process appearing as a child process
of the client. Terraform Core controls the lifecycle of these server processes
and will terminate them when they are no longer required.

The startup and handshake protocol is not currently documented. We hope to
document it here or to link to external documentation on it in future.

Versioning Strategy Copy Chapter

The Plugin Protocol uses a versioning strategy that aims to allow gradual
enhancements to the protocol while retaining compatibility, but also to allow
more significant breaking changes from time to time while allowing old and
new plugins to be used together for some period.

The versioning strategy described below was introduced with protocol version
5.0 in Terraform v0.12. Prior versions of Terraform and prior protocol versions
do not follow this strategy.

The authoritative definition for each protocol version is in this directory
as a Protocol Buffers (protobuf) service definition. The files follow the
naming pattern tfpluginX.proto, where X is the major version.

Major and minor versioning Copy Section

The minor version increases for each change introducing optional new
functionality that can be ignored by implementations of prior versions. For
example, if a new field were added to an response message, it could be a minor
release as long as Terraform Core can provide some default behavior when that
field is not populated.

The major version increases for any significant change to the protocol where
compatibility is broken. However, Terraform Core and an SDK may both choose
to support multiple major versions at once: the plugin handshake includes a
negotiation step where client and server can work together to select a
mutually-supported major version.

The major version number is encoded into the protobuf package name: major
version 5 uses the package name tfplugin5, and one day major version 6
will switch to tfplugin6. This change of name allows a plugin server to
implement multiple major versions at once, by exporting multiple gRPC services.
Minor version differences rely instead on feature-detection mechanisms, so they
are not represented directly on the wire and exist primarily as a human
communication tool to help us easily talk about which software supports which
features.

Version compatibility for Core, SDK, and Providers Copy Chapter

A particular version of Terraform Core has both a minimum minor version it
requires and a maximum major version that it supports. A particular version of
Terraform Core may also be able to optionally use a newer minor version when
available, but fall back on older behavior when that functionality is not
available.

Likewise, each provider plugin release is compatible with a set of versions.
The compatible versions for a provider are a list of major and minor version
pairs, such as "4.0", "5.2", which indicates that the provider supports the
baseline features of major version 4 and supports major version 5 including
the enhancements from both minor versions 1 and 2. This provider would
therefore be compatible with a Terraform Core release that supports only
protocol version 5.0, since major version 5 is supported and the optional
5.1 and 5.2 enhancements will be ignored.

If Terraform Core and the plugin do not have at least one mutually-supported
major version, Terraform Core will return an error from terraform init
during plugin installation:

Provider "aws" v1.0.0 is not compatible with Terraform v0.12.0.

Provider version v2.0.0 is the earliest compatible version.
Select it with the following version constraint:

    version = "~> 2.0.0"

Provider "aws" v3.0.0 is not compatible with Terraform v0.12.0.
Provider version v2.34.0 is the latest compatible version. Select 
it with the following constraint:

    version = "~> 2.34.0"

Alternatively, upgrade to the latest version of Terraform for compatibility with newer provider releases.

The above messages are for plugins installed via terraform init from a
Terraform registry, where the registry API allows Terraform Core to recognize
the protocol compatibility for each provider release. For plugins that are
installed manually to a local plugin directory, Terraform Core has no way to
suggest specific versions to upgrade or downgrade to, and so the error message
is more generic:

The installed version of provider "example" is not compatible with Terraform v0.12.0.

This provider was loaded from:
     /usr/local/bin/terraform-provider-example_v0.1.0

Adding/removing major version support in SDK and Providers Copy Chapter

The set of supported major versions is decided by the SDK used by the plugin.
Over time, SDKs will add support for new major versions and phase out support
for older major versions.

In doing so, the SDK developer passes those capabilities and constraints on to
any provider using their SDK, and that will in turn affect the compatibility
of the plugin in ways that affect its semver-based version numbering:

• If an SDK upgrade adds support for a new provider protocol, that will usually
  be considered a new feature and thus warrant a new minor version.
• If an SDK upgrade removes support for an old provider protocol, that is
  always a breaking change and thus requires a major release of the provider.

For this reason, SDK developers must be clear in their release notes about
the addition and removal of support for major versions.

Terraform Core also makes an assumption about major version support when
it produces actionable error messages for users about incompatibilities:
a particular protocol major version is supported for a single consecutive
range of provider releases, with no "gaps".

Using the protobuf specifications in an SDK Copy Chapter

If you wish to build an SDK for Terraform plugins, an early step will be to
copy one or more .proto files from this directory into your own repository
(depending on which protocol versions you intend to support) and use the
protoc protocol buffers compiler (with gRPC extensions) to generate suitable
RPC stubs and types for your target language.

For example, if you happen to be targeting Python, you might generate the
stubs using a command like this:

protoc --python_out=. --grpc_python_out=. tfplugin5.1.proto

You can find out more about the tool usage for each target language in
the gRPC Quick Start guides.

The protobuf specification for a version is immutable after it has been
included in at least one Terraform release. Any changes will be documented in
a new .proto file establishing a new protocol version.

The protocol buffer compiler will produce some sort of library object appropriate
for the target language, which depending on the language might be called a
module, or a package, or something else. We recommend to include the protocol
major version in your module or package name so that you can potentially
support multiple versions concurrently in future. For example, if you are
targeting major version 5 you might call your package or module tfplugin5.

To upgrade to a newer minor protocol version, copy the new .proto file
from this directory into the same location as your previous version, delete
the previous version, and then run the protocol buffers compiler again
against the new .proto file. Because minor releases are backward-compatible,
you can simply update your previous stubs in-place rather than creating a
new set alongside.

To support a new major protocol version, create a new package or module
and copy the relevant .proto file into it, creating a separate set of stubs
that can in principle allow your SDK to support both major versions at the
same time. We recommend supporting both the previous and current major versions
together for a while across a major version upgrade so that users can avoid
having to upgrade both Terraform Core and all of their providers at the same
time, but you can delete the previous major version stubs once you remove
support for that version.

Note: Some of the .proto files contain statements about being updated
in-place for minor versions. This reflects an earlier version management
strategy which is no longer followed. The current process is to create a
new file in this directory for each new minor version and consider all
previously-tagged definitions as immutable. The outdated comments in those
files are retained in order to keep the promise of immutability, even though
it is now incorrect.

Updating the plugin protocol in Terraform core Copy Chapter

Note: This section's audience is contributors to Terraform, not developers creating a new SDK.

New features added to Terraform core often require an update to the plugin protocol. These changes are reflected in a new minor version of the plugin protocol.

A new minor release of Terraform should only introduce one new minor release of the plugin-protocol. The minor release numbers of the plugin protocol and Terraform do not match, but they should have a one-to-one relationship.

Add updates to a new plugin protocol minor version Copy Section

1. Edit the .proto files for Protocol 5 and 6 in docs/plugin-protocol/.
   • If this is the first bigger change after a Terraform release, you may want to increase the minor protocol version in the file header
2. Commit your changes.
3. Run make protobuf.
   • This will use the symlinks in the internal/tfplugin* directories to access the latest minor versions' .proto files.
   • You should see diffs in internal/tfplugin5/tfplugin5.pb.go and internal/tfplugin6/tfplugin6.pb.go.
4. Run make generate.
   • You should see diffs in internal/plugin/mock_proto/mock.go and internal/plugin6/mock_proto/mock.go.

File: docs/plugin-protocol/releasing-new-version.md Copy Chapter

Releasing a New Version of the Protocol

Terraform's plugin protocol is the contract between Terraform's plugins and
Terraform, and as such releasing a new version requires some coordination
between those pieces. This document is intended to be a checklist to consult
when adding a new major version of the protocol (X in X.Y) to ensure that
everything that needs to be is aware of it.

New Protobuf File Copy Chapter

The protocol is defined in protobuf files that live in the hashicorp/terraform
repository. Adding a new version of the protocol involves creating a new
.proto file in that directory. It is recommended that you copy the latest
protocol file, and modify it accordingly.

New terraform-plugin-go Package Copy Chapter

The
hashicorp/terraform-plugin-go
repository serves as the foundation for Terraform's plugin ecosystem. It needs
to know about the new major protocol version. Either open an issue in that repo
to have the Plugin SDK team add the new package, or if you would like to
contribute it yourself, open a PR. It is recommended that you copy the package
for the latest protocol version and modify it accordingly.

Update the Registry's List of Allowed Versions Copy Chapter

The Terraform Registry validates the protocol versions a provider advertises
support for when ingesting providers. Providers will not be able to advertise
support for the new protocol version until it is added to that list.

Update Terraform's Version Constraints Copy Chapter

Terraform only downloads providers that speak protocol versions it is
compatible with from the Registry during terraform init. When adding support
for a new protocol, you need to tell Terraform it knows that protocol version.
Modify the SupportedPluginProtocols variable in hashicorp/terraform's
internal/getproviders/registry_client.go file to include the new protocol.

Test Running a Provider With the Test Framework Copy Chapter

Use the provider test framework to test a provider written with the new
protocol. This end-to-end test ensures that providers written with the new
protocol work correctly with the test framework, especially in communicating
the protocol version between the test framework and Terraform.

Test Retrieving and Running a Provider From the Registry Copy Chapter

Publish a provider, either to the public registry or to the staging registry,
and test running terraform init and terraform apply, along with exercising
any of the new functionality the protocol version introduces. This end-to-end
test ensures that all the pieces needing to be updated before practitioners can
use providers built with the new protocol have been updated.

File: docs/architecture.md Copy Chapter

Terraform Core Architecture Summary

This document is a summary of the main components of Terraform Core and how
data and requests flow between these components. It's intended as a primer
to help navigate the codebase to dig into more details.

We assume some familiarity with user-facing Terraform concepts like
configuration, state, CLI workflow, etc. The Terraform website has
documentation on these ideas.

Links to Go documentation assume you are running pkgsite locally, with the default URL of localhost:8080.

Terraform Request Flow Copy Chapter

The following diagram shows an approximation of how a user command is
executed in Terraform:

Each of the different subsystems (solid boxes) in this diagram is described
in more detail in a corresponding section below.

CLI (`command` package) Copy Chapter

Each time a user runs the terraform program, aside from some initial
bootstrapping in the root package (not shown in the diagram) execution
transfers immediately into one of the "command" implementations in
the command package.
The mapping between the user-facing command names and
their corresponding command package types can be found in the commands.go
file in the root of the repository.

The full flow illustrated above does not actually apply to all commands,
but it applies to the main Terraform workflow commands terraform plan and
terraform apply, along with a few others.

For these commands, the role of the command implementation is to read and parse
any command line arguments, command line options, and environment variables
that are needed for the given command and use them to produce a
backendrun.Operation
object that describes an action to be taken.

An operation consists of:

• The action to be taken (e.g. "plan", "apply").
• The name of the workspace
  where the action will be taken.
• Root module input variables to use for the action.
• For the "plan" operation, a path to the directory containing the configuration's root module.
• For the "apply" operation, the plan to apply.
• Various other less-common options/settings such as -target addresses, the
  "force" flag, etc.

The operation is then passed to the currently-selected
backend. Each backend name
corresponds to an implementation of
backend.Backend, using a
mapping table in
the backend/init package.

Backends that are able to execute operations additionally implement
backendrun.OperationsBackend;
the command-handling code calls Operation with the operation it has
constructed, and then the backend is responsible for executing that action.

Backends that execute operations, however, do so as an architectural implementation detail and not a
general feature of backends. That is, the term 'backend' as a Terraform feature is used to refer to
a plugin that determines where Terraform stores its state snapshots - only the default local
backend and HCP Terraform's backends (remote, cloud) perform operations.

Thus, most backends do not implement this interface, and so the command package wraps these
backends in an instance of
local.Local,
causing the operation to be executed locally within the terraform process itself.

Backends Copy Chapter

A backend determines where Terraform should store its state snapshots.

As described above, the local backend also executes operations on behalf of most other
backends. It uses a state manager
(either
statemgr.Filesystem if the
local backend is being used directly, or an implementation provided by whatever
backend is being wrapped) to retrieve the current state for the workspace
specified in the operation, then uses the config loader to load and do
initial processing/validation of the configuration specified in the
operation. It then uses these, along with the other settings given in the
operation, to construct a
terraform.Context,
which is the main object that actually performs Terraform operations.

The local backend finally calls an appropriate method on that context to
begin execution of the relevant command, such as
Plan
or
Apply, which in turn constructs a graph using a graph builder,
described in a later section.

Configuration Loader Copy Chapter

The top-level configuration structure is represented by model types in
package configs.
A whole configuration (the root module plus all of its descendant modules)
is represented by
configs.Config.

The configs package contains some low-level functionality for constructing
configuration objects, but the main entry point is in the sub-package
configload,
via
configload.Loader.
A loader deals with all of the details of installing child modules
(during terraform init) and then locating those modules again when a
configuration is loaded by a backend. It takes the path to a root module
and recursively loads all of the child modules to produce a single
configs.Config
representing the entire configuration.

Terraform expects configuration files written in the Terraform language, which
is a DSL built on top of
HCL. Some parts of the configuration
cannot be interpreted until we build and walk the graph, since they depend
on the outcome of other parts of the configuration, and so these parts of
the configuration remain represented as the low-level HCL types
hcl.Body
and
hcl.Expression,
allowing Terraform to interpret them at a more appropriate time.

State Manager Copy Chapter

A state manager is responsible for storing and retrieving snapshots of the
Terraform state
for a particular workspace. Each manager is an implementation of
some combination of interfaces in
the statemgr package,
with most practical managers implementing the full set of operations
described by
statemgr.Full
provided by a backend. The smaller interfaces exist primarily for use in
other function signatures to be explicit about what actions the function might
take on the state manager; there is little reason to write a state manager
that does not implement all of statemgr.Full.

The implementation
statemgr.Filesystem is used
by default (by the local backend) and is responsible for the familiar
terraform.tfstate local file that most Terraform users start with, before
they switch to remote state.
Other implementations of statemgr.Full are used to implement remote state.
Each of these saves and retrieves state via a remote network service
appropriate to the backend that creates it.

A state manager accepts and returns a state snapshot as a
states.State
object. The state manager is responsible for exactly how that object is
serialized and stored, but all state managers at the time of writing use
the same JSON serialization format, storing the resulting JSON bytes in some
kind of arbitrary blob store.

Graph Builder Copy Chapter

A graph builder is called by a
terraform.Context
method (e.g. Plan or Apply) to produce the graph that will be used
to represent the necessary steps for that operation and the dependency
relationships between them.

In most cases, the
vertices of Terraform's
graphs each represent a specific object in the configuration, or something
derived from those configuration objects. For example, each resource block
in the configuration has one corresponding
GraphNodeConfigResource
vertex representing it in the "plan" graph. (Terraform Core uses terminology
inconsistently, describing graph vertices also as graph nodes in various
places. These both describe the same concept.)

The edges
in the graph represent "must happen after" relationships. These define the
order in which the vertices are evaluated, ensuring that e.g. one resource is
created before another resource that depends on it.

Each operation has its own graph builder, because the graph building process
is different for each. For example, a "plan" operation needs a graph built
directly from the configuration, but an "apply" operation instead builds its
graph from the set of changes described in the plan that is being applied.

The graph builders all work in terms of a sequence of transforms, which
are implementations of
terraform.GraphTransformer.
Implementations of this interface just take a graph and mutate it in any
way needed, and so the set of available transforms is quite varied. Some
important examples include:

• ConfigTransformer,
  which creates a graph vertex for each resource block in the configuration.

• StateTransformer,
  which creates a graph vertex for each resource instance currently tracked
  in the state.

• ReferenceTransformer,
  which analyses the configuration to find dependencies between resources and
  other objects and creates any necessary "happens after" edges for these.

• ProviderTransformer,
  which associates each resource or resource instance with exactly one
  provider configuration (implementing
  the inheritance rules)
  and then creates "happens after" edges to ensure that the providers are
  initialized before taking any actions with the resources that belong to
  them.

There are many more different graph transforms, which can be discovered
by reading the source code for the different graph builders. Each graph
builder uses a different subset of these depending on the needs of the
operation that is being performed.

The result of graph building is a
terraform.Graph, which
can then be processed using a graph walker.

Graph Walk Copy Chapter

The process of walking the graph visits each vertex of that graph in a way
which respects the "happens after" edges in the graph. The walk algorithm
itself is implemented in
the low-level dag package
(where "DAG" is short for Directed Acyclic Graph), in
AcyclicGraph.Walk.
However, the "interesting" Terraform walk functionality is implemented in
terraform.ContextGraphWalker,
which implements a small set of higher-level operations that are performed
during the graph walk:

• EnterPath is called once for each module in the configuration, taking a
  module address and returning a
  terraform.EvalContext
  that tracks objects within that module. terraform.Context is the global
  context for the entire operation, while terraform.EvalContext is a
  context for processing within a single module, and is the primary means
  by which the namespaces in each module are kept separate.

Each vertex in the graph is evaluated, in an order that guarantees that the
"happens after" edges will be respected. If possible, the graph walk algorithm
will evaluate multiple vertices concurrently. Vertex evaluation code must
therefore make careful use of concurrency primitives such as mutexes in order
to coordinate access to shared objects such as the states.State object.
In most cases, we use the helper wrapper
states.SyncState
to safely implement concurrent reads and writes from the shared state.

Vertex Evaluation Copy Chapter

The action taken for each vertex during the graph walk is called
execution. Execution runs a sequence of arbitrary actions that make sense
for a particular vertex type.

For example, evaluation of a vertex representing a resource instance during
a plan operation would include the following high-level steps:

• Retrieve the resource's associated provider from the EvalContext. This
  should already be initialized earlier by the provider's own graph vertex,
  due to the "happens after" edge between the resource node and the provider
  node.

• Retrieve from the state the portion relevant to the specific resource
  instance being evaluated.

• Evaluate the attribute expressions given for the resource in configuration.
  This often involves retrieving the state of other resource instances so
  that their values can be copied or transformed into the current instance's
  attributes, which is coordinated by the EvalContext.

• Pass the current instance state and the resource configuration to the
  provider, asking the provider to produce an instance diff representing the
  differences between the state and the configuration.

• Save the instance diff as part of the plan that is being constructed by
  this operation.

Each execution step for a vertex is an implementation of
terraform.GraphNodeExecutable.Execute.
As with graph transforms, the behavior of these implementations varies widely:
whereas graph transforms can take any action against the graph, an Execute
implementation can take any action against the EvalContext.

The implementation of terraform.EvalContext used in real processing
(as opposed to testing) is
terraform.BuiltinEvalContext.
It provides coordinated access to plugins, the current state, and the current
plan via the EvalContext interface methods.

In order to be executed, a vertex must implement
terraform.GraphNodeExecutable,
which has a single Execute method that handles. There are numerous Execute
implementations with different behaviors, but some prominent examples are:

• NodePlannableResource.Execute, which handles the plan operation.

• NodeApplyableResourceInstance.Execute, which handles the main apply operation.

• NodeDestroyResourceInstance.Execute, which handles the main destroy operation.

A vertex must complete successfully before the graph walk will begin evaluation
for other vertices that have "happens after" edges. Evaluation can fail with one
or more errors, in which case the graph walk is halted and the errors are
returned to the user.

Expression Evaluation Copy Section

An important part of vertex evaluation for most vertex types is evaluating
any expressions in the configuration block associated with the vertex. This
completes the processing of the portions of the configuration that were not
processed by the configuration loader.

The high-level process for expression evaluation is:

1. Analyze the configuration expressions to see which other objects they refer
   to. For example, the expression aws_instance.example[1] refers to one of
   the instances created by a resource "aws_instance" "example" block in
   configuration. This analysis is performed by
   lang.References,
   or more often one of the helper wrappers around it:
   lang.ReferencesInBlock
   or
   lang.ReferencesInExpr

2. Retrieve from the state the data for the objects that are referred to and
   create a lookup table of the values from these objects that the
   HCL evaluation code can refer to.

3. Prepare the table of built-in functions so that HCL evaluation can refer to
   them.

4. Ask HCL to evaluate each attribute's expression (a
   hcl.Expression
   object) against the data and function lookup tables.

In practice, steps 2 through 4 are usually run all together using one
of the methods on lang.Scope;
most commonly,
lang.EvalBlock
or
lang.EvalExpr.

Expression evaluation produces a dynamic value represented as a
cty.Value.
This Go type represents values from the Terraform language and such values
are eventually passed to provider plugins.

Sub-graphs Copy Section

Some vertices have a special additional behavior that happens after their
evaluation steps are complete, where the vertex implementation is given
the opportunity to build another separate graph which will be walked as part
of the evaluation of the vertex.

The main example of this is when a resource block has the count argument
set. In that case, the plan graph initially contains one vertex for each
resource block, but that graph then dynamically expands to have a sub-graph
containing one vertex for each instance requested by the count. That is, the
sub-graph of aws_instance.example might contain vertices for
aws_instance.example[0], aws_instance.example[1], etc. This is necessary
because the count argument may refer to other objects whose values are not
known when the main graph is constructed, but become known while evaluating
other vertices in the main graph.

This special behavior applies to vertex objects that implement
terraform.GraphNodeDynamicExpandable.
Such vertices have their own nested graph builder, graph walk,
and vertex evaluation steps, with the same behaviors as described in these
sections for the main graph. The difference is in which graph transforms
are used to construct the graph and in which evaluation steps apply to the
nodes in that sub-graph.

File: docs/debugging.md Copy Chapter

How to Debug Terraform

Contents:

• Debugging automated tests
  • Debugging automated tests in VSCode
• Debugging Terraform operations that use real Terraform configurations
  • Launch Terraform with the dlv CLI tool
  • Launch Terraform with VS Code's debugging tool

As Terraform is written in Go you may use Delve to debug it.

GoLand includes debugging features, and the Go extension for VS Code makes it easy to use Delve when debugging Go codebases in VS Code.

Debugging automated tests Copy Chapter

Debugging an automated test is often the most straightforward workflow for debugging a section of the codebase. For example, the Go extension for VS Code adds run test | debug test options above all tests in a *_test.go file. These allow debugging without any prior configuration.

Debugging automated tests in VSCode Copy Section

As described above, debugging tests in VS Code is easily achieved through the Go extension.

If you need more control over how tests are run while debugging, e.g. environment variable values, look at the example debugger launch configuration 'Run selected test'. You can adapt this example to create your own launch configuration file.

When using this launch configuration you must highlight a test's name before starting the debugger:

Debugging Terraform operations that use real Terraform configurations Copy Chapter

Launch Terraform with the `dlv` CLI tool Copy Section

In this workflow you:

• Build Terraform using compiler flags.
• Start a debug server with a command containing the terraform command you want to debug.
  • This command is run in the working directory that contains your Terraform configuration.
• Connect to the debug server to monitor progress through breakpoints.

1. Compile & Start Debug Server

One way to do it is to compile a binary with the appropriate compiler flags:

go install -gcflags="all=-N -l"

This enables you to then execute the compiled binary via Delve, pass any arguments and spin up a debug server which you can then connect to:

# Update the path to the terraform binary if your install directory is influenced by $GOPATH or $GOBIN
dlv exec $HOME/go/bin/terraform --headless --listen :2345 --log -- apply

2a. Connect via CLI

You may connect to the headless debug server via Delve CLI

dlv connect :2345

2b. Connect from VS Code

The repository provides an example 'Connect to dlv server' launch configuration, making it possible to use VS Code's native debugging integration via the Go extension for VS Code:

Launch Terraform with VS Code's debugging tool Copy Section

In this workflow you:

• Update the debugger's launch configuration to point at the directory containing your Terraform configuration.
• Start the debugger through VS Code and monitor progress through breakpoints.

1. Update the debugger's launch configuration

Look at the example debugger launch configuration 'Run Terraform in debug mode'. You can adapt this example to create your own launch configuration file.

To use this launch configuration:

• Prepare a local Terraform project.
• Get the absolute path to that directory.
• Update the launch configuration to use that path, either in the -chdir argument or as a cwd attribute in the launch configuration.
• Make sure the args array's element reflect the command you'd like to debug.
• Provide any required environment variables through the env or envFile attributes.

2. Run the launch configuration in VS Code

Navigate to the Run and Debug view in the left-side Activity Bar. After selecting the Run Terraform in debug mode configuration in the Run and Debug view from the left-side, press the green arrow.

This is equivalent to running a Terraform CLI command in the local Terraform project's directory. For example, if you run and debug a plan command that saves a plan file, that plan file will be created.

This workflow is useful if you need to set up a complicated prior state to replicate a bug or if you want to debug code behaviour given a specific configuration.

File: docs/dependency-upgrades.md Copy Chapter

Upgrading Terraform's Library Dependencies

This codebase depends on a variety of external Go modules. At the time of
writing, a Terraform CLI build includes the union of all dependencies required
by Terraform CLI and Core itself and each of the remote state backends.

Because the remote state backends have quite a different set of needs than
Terraform itself -- special client libraries, in particular -- we've declared
them as being separate Go Modules with their own go.mod and go.sum files,
even though we don't intend to ever publish them separately. The goal is just
to help track which dependencies are used by each of these components, and to
more easily determine which of the components are affected by a particular
dependency upgrade so that we can make sure to do the appropriate testing.

Dependency Synchronization Copy Chapter

Because all of these components ultimately link into the same executable, there
can be only one version of each distinct module and thus all of the modules
must agree on which version to use.

The makefile target syncdeps runs a script which synchronizes all of the
modules to declare compatible dependencies, selecting the newest version of
each external module selected across all of the internal modules:

make syncdeps

After running this, use git status to see what's been changed. If you've
changed the dependencies of any of the modules then that should typically
cause an update to the root module, because that one imports all of the others.

Upgrading a Dependency Copy Chapter

To select a newer version of one of the dependencies, use go get in the
root to specify which version to use:

go get example.com/foo/[email protected]

Or, if you just want to move to the latest stable release, you can use the
latest pseudo-version:

go get example.com/foo/bar@latest

Then run make syncdeps to update any of the child modules that also use
this dependency. The remote state backends use only a subset of the packages
in Terraform CLI/Core, so not all dependency updates will affect the remote
state backends, and an update might affect only a subset of the backends.

When you open the pull request for your change, our code owners rules will
automatically request review from the team that maintains any affected remote
state backend. The affected teams can judge whether the update seems likely
to affect their backend and run their acceptance tests if so, before approving
the pull request. As usual, these PRs should also be reviewed by at least
one member of the Terraform Core team since they are ultimately responsible
for the complete set of dependencies used in Terraform CLI releases.

Note: Currently our code owners rules are simplistic and will request
review for any change under a remote state backend module directory, but
in practice an update that only changes a backend's go.sum cannot affect
the runtime behavior of the backend, and so those review requests are not
strictly required. You should therefore remove the review requests for
any backend whose only diff is the go.sum file once you've opened the
pull request.

Dependabot Updates Copy Chapter

When Dependabot automatically opens a pull request to upgrade a dependency,
unfortunately it isn't smart enough to automatically synchronize the change
across the modules and so the code consistency checks for the change will
typically fail.

To apply the proposed change, you'll need to check out the branch that
Dependabot created on your development system, run make syncdeps, add
all of the files that get modified, and then amend Dependabot's commit using
git commit --amend.

After you've done this, use git push --force to replace Dependabot's original
commit with your new commit, and then wait for GitHub to re-run the PR
checks. The code consistency checks should now pass.

We've configured Dependabot to monitor only the root go.mod file for potential
upgrades, because that one aggregates the dependencies for all other child
modules. Therefore there should never be a Dependabot upgrade targeting a
module in a subdirectory. If one does get created somehow, you should close
it and perform the same upgrade at the root of the repository instead, using
the instructions in Upgrading a Dependency above.

Dependencies with Special Requirements Copy Chapter

Most of our dependencies can be treated generically, but a few have some
special constraints due to how Terraform uses them:

• HCL, cty, and their indirect dependencies golang.org/x/text and
  github.com/apparentlymart/go-textseg all include logic based on Unicode
  specifications, and so should be updated with care to make sure that
  Terraform's support for Unicode follows a consistent Unicode version
  throughout.

  Additionally, each time we adopt a new minor release of Go, we may need to
  upgrade some or all of these dependencies to match the Unicode version used
  by the Go standard library.

  For more information, refer to How Terraform Uses Unicode.

  (This concern does not apply if the new version we're upgrading to is built
  for the same version of Unicode that Terraform was already using.)

• github.com/hashicorp/go-getter represents a significant part of Terraform
  CLI's remote module installer, and is the final interpreter of Terraform's
  module source address syntax. Because the module source address syntax is
  protected by the Terraform v1.x Compatibility Promises, for each upgrade
  we must make sure that:

  • The upgrade doesn't expand the source address syntax in a way that is
    undesirable from a Terraform product standpoint or in a way that we would
    not feel comfortable supporting indefinitely under the compatibility
    promises.
  • The upgrade doesn't break any already-supported source address forms
    that would therefore cause the next Terraform version to break the
    v1.x compatibility promises.

  Terraform's use of go-getter is all encapsulated in internal/getmodules
  and is set up to try to minimize the possibility that a go-getter upgrade
  would immediately introduce new functionality, but that encapsulation cannot
  prevent adoption of changes made to pre-existing functionality that
  Terraform already exposes.

• github.com/hashicorp/go-tfe -- the client library for the HCP Terraform
  API -- includes various types corresponding to HCP Terraform API
  requests and responses. The internal package internal/cloud contains mock
  implementations of some of those types, which may need to be updated when
  the client library is upgraded.

  These upgrades should typically be done only in conjunction with a project
  that will somehow use the new features through the Cloud integration, so
  that the team working on that project can perform any needed updates to
  the mocks as part of their work.

• go.opentelemetry.io/otel and the other supporting OpenTelemetry modules
  should typically be upgraded together in lockstep, because some of the
  modules define interfaces that other modules implement, and strange behavior
  can emerge if one is upgraded without the other.

  The main modules affected by this rule are the ones under the
  go.opentelemetry.io/otel prefix. The "contrib" packages can be trickier
  to upgrade because they tend to have dependencies that overlap with ours
  and so might affect non-telemetry-related behavior, and so it's acceptable
  for those to lag slightly behind to reduce risk in routine upgrades.

File: docs/destroying.md Copy Chapter

Terraform Core Resource Destruction Notes

This document intends to describe some of the details and complications
involved in the destruction of resources. It covers the ordering defined for
related create and destroy operations, as well as changes to the lifecycle
ordering imposed by create_before_destroy. It is not intended to enumerate
all possible combinations of dependency ordering, only to outline the basics
and document some of the more complicated aspects of resource destruction.

The graph diagrams here will continue to use the inverted graph structure used
internally by Terraform, where edges represent dependencies rather than order
of operations.

Simple Resource Creation Copy Chapter

In order to describe resource destruction, we first need to create the
resources and define their order. The order of creation is that which fulfills
the dependencies for each resource. In this example, A has no dependencies,
B depends on A, and C depends on B, and transitively depends on A.

Order of operations:

1. A is created
2. B is created
3. C is created

Resource Updates Copy Chapter

An existing resource may be updated with references to a newly created
resource. The ordering here is exactly the same as one would expect for
creation.

Order of operations:

1. A is created
2. B is updated
3. C is updated

Simple Resource Destruction Copy Chapter

The order for destroying resource is exactly the inverse used to create them.
This example shows the graph for the destruction of the same nodes defined
above. While destroy nodes will not contain attribute references, we will
continue to use the inverted edges showing dependencies for destroy, so the
operational ordering is still opposite the flow of the arrows.

Order of operations:

1. C is destroyed
2. B is destroyed
3. A is destroyed

Resource Replacement Copy Chapter

Resource replacement is the logical combination of the above scenarios. Here we
will show the replacement steps involved when B depends on A.

In this first example, we simultaneously replace both A and B. Here B is
destroyed before A, then A is recreated before B.

Order of operations:

1. B is destroyed
2. A is destroyed
3. A is created
4. B is created

This second example replaces only A, while updating B. Resource B is only
updated once A has been destroyed and recreated.

Order of operations:

1. A is destroyed
2. A is created
3. B is updated

While the dependency edge from B update to A destroy isn't necessary in
these examples, it is shown here as an implementation detail which will be
mentioned later on.

A final example based on the replacement graph; starting with the above
configuration where B depends on A. The graph is reduced to an update of
A while only destroying B. The interesting feature here is the remaining
dependency of A update on B destroy. We can derive this ordering of
operations from the full replacement example above, by replacing A create
with A update and removing the unused nodes.

Create Before Destroy Copy Chapter

Currently, the only user-controllable method for changing the ordering of
create and destroy operations is with the create_before_destroy resource
lifecycle attribute. This has the obvious effect of causing a resource to be
created before it is destroyed when replacement is required, but has a couple
of other effects we will detail here.

Taking the previous replacement examples, we can change the behavior of A to
be that of create_before_destroy.

Order of operations:

1. B is destroyed
2. A is created
3. B is created
4. A is destroyed

Note that in this first example, the creation of B is inserted in between the
creation of A and the destruction of A. This becomes more important in the
update example below.

Order of operations:

1. A is created
2. B is updated
3. A is destroyed

Here we can see clearly how B is updated after the creation of A and before
the destruction of the deposed resource A. (The prior resource A is
sometimes referred to as "deposed" before it is destroyed, to disambiguate it
from the newly created A.) This ordering is important for resource that
"register" other resources, and require updating before the dependent resource
can be destroyed.

The transformation used to create these graphs is also where we use the extra
edges mentioned above connecting B to A destroy. The algorithm to change a
resource from the default ordering to create_before_destroy simply inverts
any incoming edges from other resources, which automatically creates the
necessary dependency ordering for dependent updates. This also ensures that
reduced versions of this example still adhere to the same ordering rules, such
as when the dependency is only being removed:

Order of operations:

1. B is updated
2. A is destroyed

Forced Create Before Destroy Copy Section

In the previous examples, only resource A was being used as is it were
create_before_destroy. The minimal graphs used show that it works in
isolation, but that is only when the create_before_destroy resource has no
dependencies of it own. When a create_before_resource depends on another
resource, that dependency is "infected" by the create_before_destroy
lifecycle attribute.

This example demonstrates why forcing create_before_destroy is necessary. B
has create_before_destroy while A does not. If we only invert the ordering
for B, we can see that results in a cycle.

In order to resolve these cycles, all resources that precede a resource
with create_before_destroy must in turn be handled in the same manner.
Reversing the incoming edges to A destroy resolves the problem:

Order of operations:

1. A is created
2. B is created
3. B is destroyed
4. A is destroyed

This also demonstrates why create_before_destroy cannot be overridden when
it is inherited; changing the behavior here isn't possible without removing
the initial reason for create_before_destroy; otherwise cycles are always
introduced into the graph.

File: docs/planning-behaviors.md Copy Chapter

Planning Behaviors

A key design tenet for Terraform is that any actions with externally-visible
side-effects should be carried out via the standard process of creating a
plan and then applying it. Any new features should typically fit within this
model.

There are also some historical exceptions to this rule, which we hope to
supplement with plan-and-apply-based equivalents over time.

This document describes the default planning behavior of Terraform in the
absence of any special instructions, and also describes the three main
design approaches we can choose from when modelling non-default behaviors that
require additional information from outside of Terraform Core.

This document focuses primarily on actions relating to resource instances,
because that is Terraform's main concern. However, these design principles can
potentially generalize to other externally-visible objects, if we can describe
their behaviors in a way comparable to the resource instance behaviors.

This is developer-oriented documentation rather than user-oriented
documentation. See
the main Terraform documentation for
information on existing planning behaviors and other behaviors as viewed from
an end-user perspective.

Default Planning Behavior Copy Chapter

When given no explicit information to the contrary, Terraform Core will
automatically propose taking the following actions in the appropriate
situations:

• Create, if either of the following are true:
  • There is a resource block in the configuration that has no corresponding
    managed resource in the prior state.
  • There is a resource block in the configuration that is recorded in the
    prior state but whose count or for_each argument (or lack thereof)
    describes an instance key that is not tracked in the prior state.
• Delete, if either of the following are true:
  • There is a managed resource tracked in the prior state which has no
    corresponding resource block in the configuration.
  • There is a managed resource tracked in the prior state which has a
    corresponding resource block in the configuration but its count
    or for_each argument (or lack thereof) lacks an instance key that is
    tracked in the prior state.
• Update, if there is a corresponding resource instance both declared in the
  configuration (in a resource block) and recorded in the prior state
  (unless it's marked as "tainted") but there are differences between the prior
  state and the configuration which the corresponding provider doesn't
  explicitly classify as just being normalization.
• Replace, if there is a corresponding resource instance both declared in
  the configuration (in a resource block) and recorded in the prior state
  marked as "tainted". The special "tainted" status means that the process
  of creating the object failed partway through and so the existing object does
  not necessarily match the configuration, so Terraform plans to replace it
  in order to ensure that the resulting object is complete.
• Read, if there is a data block in the configuration.
  • If possible, Terraform will eagerly perform this action during the planning
    phase, rather than waiting until the apply phase.
  • If the configuration contains at least one unknown value, or if the
    data resource directly depends on a managed resource that has any change
    proposed elsewhere in the plan, Terraform will instead delay this action
    to the apply phase so that it can react to the completion of modification
    actions on other objects.
• No-op, to explicitly represent that Terraform considered a particular
  resource instance but concluded that no action was required.

The Replace action described above is really a sort of "meta-action", which
Terraform expands into separate Create and Delete operations. There are
two possible orderings, and the first one is the default planning behavior
unless overridden by a special planning behavior as described later. The
two possible lowerings of Replace are:

1. Delete then Create: first delete the existing object bound to an
   instance, and then create a new object at the same address based on the
   current configuration.
2. Create then Delete: mark the existing object bound to an instance as
   "deposed" (still exists but not current), create a new current object at the
   same address based on the current configuration, and then delete the deposed
   object.

Special Planning Behaviors Copy Chapter

For the sake of this document, a "special" planning behavior is one where
Terraform Core will select a different action than the defaults above,
based on explicit instructions given either by a module author, an operator,
or a provider.

There are broadly three different design patterns for special planning
behaviors, and so each "special" use-case will typically be met by one or more
of the following depending on which stakeholder is activating the behavior:

• Configuration-driven Behaviors are
  activated by additional annotations given in the source code of a module.

  This design pattern is good for situations where the behavior relates to
  a particular module and so should be activated for anyone using that
  module. These behaviors are therefore specified by the module author, such
  that any caller of the module will automatically benefit with no additional
  work.

• Provider-driven Behaviors are activated by
  optional fields in a provider's response when asked to help plan one of the
  default actions given above.

  This design pattern is good for situations where the behavior relates to
  the behavior of the remote system that a provider is wrapping, and so from
  the perspective of a user of the provider the behavior should appear
  "automatic".

  Because these special behaviors are activated by values in the provider's
  response to the planning request from Terraform Core, behaviors of this
  sort will typically represent "tweaks" to or variants of the default
  planning behaviors, rather than entirely different behaviors.

• Single-run Behaviors are activated by explicitly
  setting additional "plan options" when calling Terraform Core's plan
  operation.

  This design pattern is good for situations where the direct operator of
  Terraform needs to do something exceptional or one-off, such as when the
  configuration is correct but the real system has become degraded or damaged
  in a way that Terraform cannot automatically understand.

  However, this design pattern has the disadvantage that each new single-run
  behavior type requires custom work in every wrapping UI or automaton around
  Terraform Core, in order provide the user of that wrapper some way
  to directly activate the special option, or to offer an "escape hatch" to
  use Terraform CLI directly and bypass the wrapping automation for a
  particular change.

We've also encountered use-cases that seem to call for a hybrid between these
different patterns. For example, a configuration construct might cause Terraform
Core to invite a provider to activate a special behavior, but let the
provider make the final call about whether to do it. Or conversely, a provider
might advertise the possibility of a special behavior but require the user to
specify something in the configuration to activate it. The above are just
broad categories to help us think through potential designs; some problems
will require more creative combinations of these patterns than others.

Configuration-driven Behaviors Copy Section

Within the space of configuration-driven behaviors, we've encountered two
main sub-categories:

• Resource-specific behaviors, whose effect is scoped to a particular resource.
  The configuration for these often lives inside the resource or data
  block that declares the resource.
• Global behaviors, whose effect can span across more than one resource and
  sometimes between resources in different modules. The configuration for
  these often lives in a separate location in a module, such as a separate
  top-level block which refers to other resources using the typical address
  syntax.

The following is a non-exhaustive list of existing examples of
configuration-driven behaviors, selected to illustrate some different variations
that might be useful inspiration for new designs:

• The ignore_changes argument inside resource block lifecycle blocks
  tells Terraform that if there is an existing object bound to a particular
  resource instance address then Terraform should ignore the configured value
  for a particular argument and use the corresponding value from the prior
  state instead.

  This can therefore potentially cause what would've been an Update to be
  a No-op instead.

• The replace_triggered_by argument inside resource block lifecycle
  blocks can use a proposed change elsewhere in a module to force Terraform
  to propose one of the two Replace variants for a particular resource.

• The create_before_destroy argument inside resource block lifecycle
  blocks only takes effect if a particular resource instance has a proposed
  Replace action. If not set or set to false, Terraform will decompose
  it to Destroy then Create, but if set to true Terraform will use
  the inverted ordering.

  Because Terraform Core will never select a Replace action automatically
  by itself, this is an example of a hybrid design where the config-driven
  create_before_destroy combines with any other behavior (config-driven or
  otherwise) that might cause Replace to customize exactly what that
  Replace will mean.

• Top-level moved blocks in a module activate a special behavior during the
  planning phase, where Terraform will first try to change the bindings of
  existing objects in the prior state to attach to new addresses before running
  the normal planning process. This therefore allows a module author to
  document certain kinds of refactoring so that Terraform can update the
  state automatically once users upgrade to a new version of the module.

  This special behavior is interesting because it doesn't directly change
  what actions Terraform will propose, but instead it adds an extra
  preparation step before the typical planning process which changes the
  addresses that the planning process will consider. It can therefore
  indirectly cause different proposed actions for affected resource
  instances, such as transforming what by default might've been a Delete
  of one instance and a Create of another into just a No-op or
  Update of the second instance.

  This one is an example of a "global behavior", because at minimum it
  affects two resource instance addresses and, if working with whole resource
  or whole module addresses, can potentially affect a large number of resource
  instances all at once.

Provider-driven Behaviors Copy Section

Providers get an opportunity to activate some special behaviors for a particular
resource instance when they respond to the PlanResourceChange function of
the provider plugin protocol.

When Terraform Core executes this RPC, it has already selected between
Create, Delete, or Update actions for the particular resource
instance, and so the special behaviors a provider may activate will typically
serve as modifiers or tweaks to that base action, and will not allow
the provider to select another base action altogether. The provider wire
protocol does not talk about the action types explicitly, and instead only
implies them via other content of the request and response, with Terraform Core
making the final decision about how to react to that information.

The following is a non-exhaustive list of existing examples of
provider-driven behaviors, selected to illustrate some different variations
that might be useful inspiration for new designs:

• When the base action is Update, a provider may optionally return one or
  more paths to attributes which have changes that the provider cannot
  implement as an in-place update due to limitations of the remote system.

  In that case, Terraform Core will replace the Update action with one of
  the two Replace variants, which means that from the provider's
  perspective the apply phase will really be two separate calls for the
  decomposed Create and Delete actions (in either order), rather
  than Update directly.

• When the base action is Update, a provider may optionally return a
  proposed new object where one or more of the arguments has its value set
  to what was in the prior state rather than what was set in the configuration.
  This represents any situation where a remote system supports multiple
  different serializations of the same value that are all equivalent, and
  so changing from one to another doesn't represent a real change in the
  remote system.

  If all of those taken together causes the new object to match the prior
  state, Terraform Core will treat the update as a No-op instead.

Of the three genres of special behaviors, provider-driven behaviors is the one
we've made the least use of historically but one that seems to have a lot of
opportunities for future exploration. Provider-driven behaviors can often be
ideal because their effects appear as if they are built in to Terraform so
that "it just works", with Terraform automatically deciding and explaining what
needs to happen and why, without any special effort on the user's part.

Single-run Behaviors Copy Section

Terraform Core's "plan" operation takes a set of arguments that we collectively
call "plan options", that can modify Terraform's planning behavior on a per-run
basis without any configuration changes or special provider behaviors.

As noted above, this particular genre of designs is the most burdensome to
implement because any wrapping software that can ask Terraform Core to create
a plan must ideally offer some way to set all of the available planning options,
or else some part of Terraform's functionality won't be available to anyone
using that wrapper.

However, we've seen various situations where single-run behaviors really are the
most appropriate way to handle a particular use-case, because the need for the
behavior originates in some process happening outside of the scope of any
particular Terraform module or provider.

The following is a non-exhaustive list of existing examples of
single-run behaviors, selected to illustrate some different variations
that might be useful inspiration for new designs:

• The "replace" planning option specifies zero or more resource instance
  addresses.

  For any resource instance specified, Terraform Core will transform any
  Update or No-op action for that instance into one of the
  Replace actions, thereby allowing an operator to respond to something
  having become degraded in a way that Terraform and providers cannot
  automatically detect and force Terraform to replace that object with
  a new one that will hopefully function correctly.

• The "refresh only" planning mode ("planning mode" is a single planning option
  that selects between a few mutually-exclusive behaviors) forces Terraform
  to treat every resource instance as No-op, regardless of what is bound
  to that address in state or present in the configuration.

Legacy Operations Copy Chapter

Some of the legacy operations Terraform CLI offers that aren't integrated
with the plan and apply flow could be thought of as various degenerate kinds
of single-run behaviors. Most don't offer any opportunity to preview an effect
before applying it, but do meet a similar set of use-cases where an operator
needs to take some action to respond to changes to the context Terraform is
in rather than to the Terraform configuration itself.

Most of these legacy operations could therefore most readily be translated to
single-run behaviors, but before doing so it's worth researching whether people
are using them as a workaround for missing configuration-driven and/or
provider-driven behaviors. A particular legacy operation might be better
replaced with a different sort of special behavior, or potentially by multiple
different special behaviors of different genres if it's currently serving as
a workaround for many different unmet needs.

File: docs/README.md Copy Chapter

Terraform Core Codebase Documentation

This directory contains some documentation about the Terraform Core codebase,
aimed at readers who are interested in making code contributions.

If you're looking for information on using Terraform, please instead refer
to the main Terraform CLI documentation.

Terraform Core Architecture Documents Copy Chapter

• Modules Runtime Architecture Summary: an overview of the
  main components of Terraform Core related to planning and applying modules.
  This is the best starting point if you are diving in to this codebase for the
  first time.

• Stacks Runtime Architecture Summary: an
  overview of the main components of Terraform Core related to planning and
  applying stack configurations.

• Resource Instance Change Lifecycle:
  a description of the steps in validating, planning, and applying a change
  to a resource instance, from the perspective of the provider plugin RPC
  operations. This may be useful for understanding the various expectations
  Terraform enforces about provider behavior, either if you intend to make
  changes to those behaviors or if you are implementing a new Terraform plugin
  SDK and so wish to conform to them.

  (If you are planning to write a new provider using the official SDK then
  please refer to the Plugin development documentation
  instead; it presents similar information from the perspective of the SDK
  API, rather than the plugin wire protocol.)

• Plugin Protocol: gRPC/protobuf definitions for the
  plugin wire protocol and information about its versioning strategy.

  This documentation is for SDK developers, and is not necessary reading for
  those implementing a provider using the official SDK.

• Terraform Core RPC API: an integration point
  for external software that needs to integrate Terraform Core functionality.

• Upgrading Terraform's Dependencies: guidance on
  some special details that arise when we upgrade Go Module dependencies, due
  to this codebase containing Terraform CLI, Terraform Core, and the various
  remote state backends which all have some overlapping dependencies.

• How Terraform Uses Unicode: an overview of the various
  features of Terraform that rely on Unicode and how to change those features
  to adopt new versions of Unicode.

• Deadlock-free Promises: a utility package
  used by the Terraform Stacks runtime to coordinate its concurrent work.

Contribution Guides Copy Chapter

• Contributing to Terraform: a complete guideline for those who want to contribute to this project.

2. Official Technical Reference & Guides (hashicorp/web-unified-docs) Copy Chapter

Web Unified Docs

[!IMPORTANT]
This README is for developers working on the documentation website. If you
want to contribute docs content, refer to the Contribute to HashiCorp
documentation guide.

The project in this repository, hashicorp/web-unified-docs, aims to implement DEVDOT-023: Unified Product Documentation Repository. The RFC for this project was intentionally light on implementation details, in order to foster consensus on the broad direction.

• PR previews: Show broken links in comments for awareness (informational only, don't block PRs)
• Production monitoring: Weekly scans create GitHub issues and send critical alerts to Datadog when users are affected

The weekly broken-link-check-full workflow generates comprehensive broken link reports with prioritization guidance. When contributors create PRs that modify content, the link checker shows any broken links in PR comments with actionable guidance without blocking development.

Quick tips for contributors:

• Fix internal HashiCorp links (high priority)
• Check external docs/API links (medium priority)
• Consider removing unreliable external links (low priority)

For detailed information about the monitoring system, see Broken Link
Monitoring Documentation. The EDU-053: Unified Education Style Guide
for this project was intentionally light on implementation details, in order
to foster consensus on the broad direction.

The existing API (content.hashicorp.com) has endpoints that serve documentation content. You can find the source code in hashicorp/mktg-content-workflows.

The goal of the unified docs API is to host all of HashiCorp's product documentation. The unified docs API will eventually replace the existing content API.

Local development Copy Chapter

Requirements Copy Section

• Docker and Docker Compose (for managing containers)

Quick start Copy Section

To get a migration preview running, run make from the root of this repo. The make command starts the unified-docs Docker profile that spins up a local instance of unified-devdot-api and dev-portal.

Once this command completes, you can access the following endpoints:

• http://localhost:3000 - An instance of the dev-portal container configured to pull from the experimental docs API (this repo). This image depends on the unified docs API (unified-devdot-api).

• http://localhost:8080 - An instance of the unified docs API container (this repo - unified-devdot-api) that serves content from the content directory. On startup, this container processes the content and assets in /content into public/assets and public/content. In addition, the container also generates app/api/docsPaths.json and app/api/versionMetadata.json from the contents within /content.

  Use the following example to test this endpoint: http://localhost:8080/api/content/terraform-plugin-framework/doc/latest/plugin/framework

[!NOTE]
The unified docs API container takes time to process the content and assets. You must wait for both the unified-devdot-api and dev-portal containers to complete before you can successfully test content in the dev-portal preview environment (localhost:3000). Visit http://localhost:8080/api/all-docs-paths to verify the unified-devdot-api container is complete.

To spin this down gracefully, run make clean in a separate terminal.

If you wish to remove the local Docker images as well, run make clean CLEAN_OPTION=full.

More commands Copy Section

The makefile serves as a convenience tool start the local preview. If you need more granular control, the package.json file contains a full list of available commands.

To use these, you will need to intentionally run npm install and npm run prebuild before anything else.

Use npm run coverage to run coverage tests.

Preview environments for unified-docs and dev-portal Copy Section

Unified docs API serves as one of the content APIs for dev-portal (frontend application for DevDot). As a result, when implementing new features, you may need to modify both the backend (this repo) and the frontend (dev-portal).

If you are working on a ticket that requires changes to both the unified docs API and dev-portal, please set custom environment variables for your branch in Vercel to simplify testing instructions.

For example, in Vercel, for your dev-portal branch, you can set the following environment variables:

Vercel will use these values to create deploy previews.

API development Copy Section

Reach out to team #team-web-presence if you need to do local API development

Background Copy Chapter

Project Rationale Copy Section

• Storing documentation in one branch of one repo dramatically simplifies the workflow for contributing documentation.
• Publishing changes to multiple versions can be done in a single PR, as opposed to multiple PRs which is required by the current setup.
• Finding and making the same change across multiple versions is as simple as doing a find-and-replace since all the versioned docs are on the filesystem at the same time.
• Adding a new product is as easy as making a new folder, as opposed to the current process which requires code-changes on the API side and the installation of a GitHub App to monitor for events.
• Sourcing from one branch in one repo eliminates the situation where a missed GitHub event can result in out-of-date documentation. If something goes wrong in the publishing process, simply run it again instead of relying on incoming commit/release events from the GitHub API.
• Since we can make edits to all docs for all products and versions from a single PR, making platform-level changes is dramatically simplified (such as updating to MDX v2, or rewriting URLs).
• Adding new features like content conformance (basically linting for docs) can be done for the entire codebase at once.
• Removes the ability for docs to break the release workflow in product repos.
• Enables us to support fully versioned deployment previews, whereas current previews are limited to the branch being modified.

[Architectural Decision Records](https://github.com/hashicorp/web-unified-docs/tree/main/docs/decisions) Copy Section

Update product repo documentation Copy Chapter

This script helps with product documentation migration to the web-unified-docs repository. When migrating documentation:

1. The web-unified-docs repository becomes the source of truth
2. Original documentation may remain temporarily as a fallback
3. Users should be directed to make future changes in web-unified-docs only

This script automatically adds a prominent notice to all MDX files in the original location, informing contributors where to make future changes.

./scripts/update-mdx-files.sh ~/Desktop/hashicorp/terraform-plugin-framework/website/docs

Example output:

Progress:

Files processed: 135
Files updated: 135
Files with no frontmatter: 0
Files with errors: 0

Completed! All MDX files have been processed.

Run broken link checker locally Copy Chapter

The repository uses a focused broken link monitoring system:

• PR previews: Show broken links in comments for awareness (informational only, don't block PRs)
• Production monitoring: Weekly scans create GitHub issues and Datadog alerts for user-facing problems

The weekly broken-link-check-full workflow generates comprehensive broken link reports. When contributors create PRs that modify content, the link checker shows any broken links in PR comments without blocking development.

For detailed information about the monitoring system, see Broken Link Monitoring Documentation.

UDR Migration Link Check Copy Section

For teams migrating products to UDR (Unified Docs Renderer), use the dedicated migration workflow:

1. Go to Actions → UDR Product Link Check
2. Click "Run workflow" and select your product
3. Review migration-specific broken link analysis in the generated GitHub issue

This workflow provides targeted link checking with migration-focused reporting and prioritization.

Local Testing Copy Section

You can also run the broken link checker locally. The following commands launch a lychee Docker container to check the content directories you specify.

Run the broken link checker on all content.

npm run broken-link

Check a specific directory within content.

npm run broken-link terraform-plugin-framework

Check multiple directories.

npm run broken-link terraform-plugin-framework-log terraform-plugin-mux

Architecture Copy Chapter

The following diagram illustrates the relationships between the unified docs API (this repo), dev-portal, and the existing content API:

graph LR
    subgraph "Content sources (non-migrated)"
        BDY[boundary]
        CSL[consul]
        HCP[hcp-docs]
        NMD[nomad]
        PKR[packer]
        SNT[sentinel]
        TF[terraform]
        TFC[terraform-cdk]
        TFA[terraform-docs-agents]
        TFD[terraform-docs-common]
        VGT[vagrant]
        VLT[vault]
        WPT[waypoint]

        CURALL["/content or /website"]
        BDY & CSL & HCP & NMD & PKR & SNT & TF & TFC & TFA & TFD & VGT & VLT & WPT --> CURALL
    end

    subgraph "Migrated content repo"
        TPF[terraform-plugin-framework]
        TPL[terraform-plugin-log]
        TPM[terraform-plugin-mux]
        TPS[terraform-plugin-sdk]
        TPT[terraform-plugin-testing]
        TFE[terraform-enterprise]

        MIGALL["/content"]
        TPF & TPL & TPM & TPS & TPT & TFE --> MIGALL
    end

    subgraph "APIs"
        CP[Content APIcontent.hashicorp.com]
        UDR[Unified Docs Repositoryweb-unified-docs]
    end

    subgraph "Frontend"
        DP[Dev Portaldev-portal]
    end

    %% BDY & CSL & HCP & NMD & PKR & PTF & SNT & TF & TFC & TFA & TFD & VGT & VLT & WPT --> CP
    %% TPF & TPL & TPM & TPS & TPT --> UDR

    CURALL -->|Current content flow| CP
    MIGALL -->|Migrated content| UDR

    CP -->|Serves most content| DP
    UDR -->|Serves unified/content content| DP

    class TPF,TPL,TPM,TPS,TPT,BDY,CSL,HCP,NMD,PKR,PTF,SNT,TF,TFC,TFA,TFD,VGT,VLT,WPT productRepo

The diagram shows:

• The content API — the existing system that sources product documentation content from product repositories
• The unified docs API — the new system that sources product documentation from this repo's /content directory. The migrated repos will use a directory approach to versioning (rather than the historic branch and tag strategy)
• The Dev Portal — the frontend that serves the main DevDot interface. Dev Portal sources its content from both the existing content API and unified docs API.