Health checks and load balancing with Terraform

Two nested blocks turn a plain record into a traffic-managed one: lb_policy decides which healthy values to return, and health_check_template decides how DTM probes them. Both attach to dtm_record, dtm_override_record, and dtm_auto_record (auto records reject value_weights). lb_policy can also be set on dtm_zone as the zone-wide default that records without their own policy inherit. There is also a record-independent dtm_healthcheck resource (monitor-only).

Load-balancing policy

lb_policy {
  method = "region"
}

Attribute	Description
method	round-robin, failover, latency, region, geo-failover, or weighted. See load-balancing methods.
client_affinity	Pin a client (by source-IP hash) to one value. When true it takes precedence over method: the value is picked by hash, and the method and its options (failover_order, region_preference, value_weights) are not consulted.
affinity_ttl	How long (seconds) the client-to-value mapping holds, up to a 24 hour maximum (larger values are clamped). Only meaningful with client_affinity = true.
failover_order	Explicit ordering for method = "failover". Values not listed fall to the end.
region_preference	Fallback regions, in order, used only when the client’s own region has no healthy value; the first listed region that has one wins. Honoured only when method = "region" and client_affinity is false. It is accepted but ignored under geo-failover.
value_weights	For method = "weighted": map of record value to a percentage (0 to 100; listed weights may sum to at most 100). Unlisted values split the leftover evenly; 0 drains a value. Works on dtm_record and dtm_override_record.

Weighted canary (90/10)

lb_policy {
  method = "weighted"
  value_weights = {
    "10.0.1.10" = 90 # current
    "10.0.1.20" = 10 # canary
  }
}

Set a low TTL on the record: the split is applied per DNS answer, so resolver caching smears it unless answers expire quickly. See known limitations.

Failover with an explicit order

lb_policy {
  method         = "failover"
  failover_order = ["10.0.1.10", "10.0.1.11", "10.0.2.10"]
}

Region preference

Clients get values in their own region; region_preference only says where to send them, in order, when their region has no healthy value:

lb_policy {
  method            = "region"
  region_preference = ["northeurope", "westeurope"]
}

Client affinity

client_affinity pins each client to one value for affinity_ttl seconds. It takes precedence over method, so do not combine it with region_preference (which it would ignore):

lb_policy {
  method          = "round-robin"
  client_affinity = true
  affinity_ttl    = 300
}

Health-check template

When set, DTM creates one health check per value from this template and keeps them in sync as values change.

health_check_template {
  protocol         = "https"
  port             = 443
  path             = "/healthz"
  interval_seconds = 15
  timeout_seconds  = 5
  expected_status  = [200]
}

Attribute	Description
protocol	tcp, http, or https.
port	Target port (1-65535).
path	HTTP/HTTPS path, e.g. /healthz.
interval_seconds	Probe interval. Default 15, minimum 5.
timeout_seconds	Per-probe timeout.
tls_skip_verify	Skip TLS verification (self-signed HTTPS targets).
expected_status	HTTP status codes treated as healthy (default [200]).
expected_body	Substring the response body must contain.
headers	Static request headers sent on every probe (e.g. an Authorization bearer). Header values are secrets: the API returns them redacted, and the provider keeps the values you applied in state.
ca_pem	PEM CA bundle that verifies the target’s TLS certificate (server-cert pin).
tls_server_name	Hostname an https probe sends as SNI and verifies the certificate against; the probe still connects to each value. Unset: a hostname Host header, else the value. (Added in v1.0.5.)

See health checks for tuning guidance on interval and thresholds, and for when an HTTPS backend needs tls_server_name.

Authenticated probes

For auth-gated backends, add one of the oauth2, mtls, or managed_identity sub-blocks (see authenticated health probes for the behaviour). DTM requires HTTPS with verified TLS when credentials are attached: use protocol = "https" and set ca_pem or leave tls_skip_verify off. A template carrying one of those three sub-blocks over plain http is rejected. A static headers token is not covered by that check, so set protocol = "https" for those too. (An admin can relax the certificate-verification half with the dtm_health_probe_policy_config singleton resource. From v1.0.7 it applies to the whole cluster, so keep a single copy of it. The HTTPS requirement always stands.)

From v1.0.4, two more rules apply to the provider’s token:

• A managed_identity sub-block needs an admin token to create or change the resource that carries it, and its resource must be an allowed audience (your own application’s, not Key Vault, Resource Manager, Graph or Storage). See admin-only, and which audiences are allowed.
• A change that moves where a stored secret is sent (a new target, protocol, port or token_url, or a new record value) must carry the real secret. A configuration that holds its secrets normally does. One imported without them still holds the redacted placeholder, so with an editor token such a change is refused until you apply the real secret.

# OAuth2 client-credentials
health_check_template {
  protocol         = "https"
  port             = 443
  path             = "/healthz"
  interval_seconds = 15

  oauth2 {
    token_url     = "https://login.example.com/oauth2/token"
    client_id     = var.probe_client_id
    client_secret = var.probe_client_secret # sensitive
    scope         = "health.read"
  }
}

# Azure Managed Identity - no stored credentials
health_check_template {
  protocol         = "https"
  port             = 443
  path             = "/healthz"
  interval_seconds = 15

  managed_identity {
    resource  = "api://<app-id-uri>"
    # client_id = "<user-assigned-mi-client-id>"  # omit for the VM's system-assigned identity
  }
}

# Mutual TLS
health_check_template {
  protocol         = "https"
  port             = 443
  path             = "/healthz"
  interval_seconds = 15

  mtls {
    cert_pem = file("client.pem")
    key_pem  = file("client.key") # sensitive
  }
}

Caution

client_secret and key_pem are marked sensitive, so Terraform hides them in plan output. Probe credentials (including any secret headers) live in your Terraform state, so protect the state file and keep the source of truth in your own secret store (a Terraform variable, Key Vault, etc.).

Probe secrets are write-only through the API: reads return client_secret, key_pem, and all headers values redacted as ***. The provider preserves the values you applied in state, so a refresh or plan shows no spurious secret drift. To rotate a secret, change the value in your configuration and apply as normal. The one exception is terraform import: an imported resource has no previously applied value, so the redacted *** placeholder sits in state (and shows as a diff) until your first apply restores the real values.

Monitor-only health checks

dtm_healthcheck (role: editor or admin) manages a health check independently of any record. It is monitor-only: the check probes its target and reports status (the Health Checks page and the dtm_health_check_status metric) but never gates a record’s DNS answers. To make answers health-aware, use a health_check_template on the record, as above.

resource "dtm_healthcheck" "web" {
  name              = "web-http"
  protocol          = "http"
  target            = "10.0.1.10:80"
  interval_seconds  = 15
  timeout_seconds   = 5
  failure_threshold = 3
  success_threshold = 2

  http_config {
    method          = "GET"
    path            = "/healthz"
    expected_status = [200]
  }
}

Attribute	Required	Description
name	Yes	Human-readable name.
protocol	Yes	tcp, http, or https.
target	Yes	For tcp, host:port (for example 10.0.0.4:8080): a port-less tcp target is accepted but fails every probe. For http and https, a host or IP with an optional port. No scheme and no path for any protocol: for http/https the scheme comes from protocol and the request path from http_config.path.
interval_seconds	Yes	Probe interval (minimum 5).
timeout_seconds	Yes	Per-probe timeout.
failure_threshold	Yes	Consecutive failures before unhealthy.
success_threshold	Yes	Consecutive successes before healthy again.
region_id	No	Region tag for region-aware filtering.
failure_rate_window_seconds	No	Sliding window for rate-based failure detection; 0 (default) disables it. If you set it, you must also set failure_rate_percent.
failure_rate_percent	No	Percentage of failed probes within the window that flips the check unhealthy; must be set together with a non-zero failure_rate_window_seconds, and supplying only one of the pair is rejected with a 400.
cross_region	No	Allow the check result to drive failover across regions; leave unset to accept the server-side default.
http_config (block)	For HTTP/HTTPS	method, path, expected_status, expected_body, headers, tls_skip_verify, ca_pem, tls_server_name, and the oauth2 / mtls / managed_identity auth blocks. headers values are secrets: returned redacted by the API and preserved in state, exactly as for record probes above.

Authenticated monitor-only checks

http_config carries the same authenticated-probe fields as the record template, field for field: ca_pem plus the oauth2, mtls, and managed_identity blocks. The rules from authenticated probes apply unchanged: at most one bearer method (oauth2 or managed_identity, with mtls composable with either), HTTPS with verified TLS whenever credentials are attached, and the same secret round-trip (redacted reads, state-preserved values, and the terraform import placeholder caveat).

resource "dtm_healthcheck" "secured_api" {
  name              = "secured-api"
  protocol          = "https"
  target            = "api.internal:443"
  interval_seconds  = 30
  timeout_seconds   = 3
  failure_threshold = 3
  success_threshold = 1

  http_config {
    method          = "GET"
    path            = "/healthz"
    expected_status = [200]
    ca_pem          = file("${path.module}/backend-ca.pem")

    oauth2 {
      token_url     = "https://login.microsoftonline.com/<tenant>/oauth2/v2.0/token"
      client_id     = "<app-id>"
      client_secret = var.probe_client_secret # sensitive
      scope         = "api://<app-id>/.default"
    }
  }
}

Managed script checks

Added in v1.0.5. A protocol = "script" check runs a managed script: a dtm_script your signing key approved, pinned by version and hash. It takes a script_config block naming the script, and nothing else: the provider cannot send a free-form command. Script checks need allow_script_health_checks on each node, your signing keys in each node’s configuration, and the admin role.

resource "dtm_script" "orders_db" {
  name        = "orders-db"
  interpreter = "sh" # sh or python3
  body        = file("${path.module}/scripts/orders-db.sh")
  key_id      = "ops-2026"
  # The signature printed by the openssl signing commands (see Managed
  # script health checks), run where the key lives, never on a cluster node.
  signature   = var.orders_db_signature
}

resource "dtm_healthcheck" "orders_db" {
  name              = "orders-db"
  protocol          = "script"
  target            = "10.0.0.5:5432"
  interval_seconds  = 30
  timeout_seconds   = 5
  failure_threshold = 3
  success_threshold = 1

  script_config {
    script_id = dtm_script.orders_db.id
    args      = ["5432"] # sensitive: the API shows *** on read

    # Leave these out to pin the version that is latest when the check is
    # created. Set them to move the check in the same apply that publishes
    # a new body.
    # script_version = dtm_script.orders_db.version
    # script_hash    = dtm_script.orders_db.content_hash
  }
}

• Terraform never holds the signing key. Your pipeline signs the body and passes the signature in. Every node checks it against the script_signing_keys in its own configuration.
• Versions. Each body or interpreter change publishes the script’s next version, and version and content_hash show it. The plan shows the content_hash your pipeline should sign. A signature that is not base64 is refused at plan time; one that does not verify is refused at apply.
• The pin never moves on its own. A new version leaves existing checks where they are, unless they reference dtm_script.<name>.version as above.
• args are sensitive. The API shows them as *** on read; the provider keeps the values you applied, so there is no drift.
• name forces replacement. It is the script’s identity.
• Destroy deletes the script and all its versions. Terraform destroys a referencing dtm_healthcheck first, so a plan that removes both works. Deleting is not revoking: see Stop a script.
• Import: terraform import dtm_script.orders_db <id>. The API does not return a version’s body, key or signature, so set them in configuration to match; the next apply publishes a new version only if the body differs.
• Not in cluster-clone exports. A signature is a decision about the destination’s keys, so exports leave dtm_script out. Declare scripts in the destination workspace.