Decommissioning a cluster

There are two versions of “undo” for a DTM cluster, and they are different jobs. Resetting a node returns it to the state the Marketplace image ships in, keeping the cluster running. Decommissioning removes DTM from your subscription entirely.

The second one has a trap worth knowing about before you start: deleting the virtual machines does not delete everything the cluster held. Four things outlive the VMs, and one of them contains live credentials. The what deleting the VMs does not remove section is the part of this page to read twice.

Reset a node to its shipped state

There is no reset command, because deploying the image is the reset. A node holds no configuration that is not either replicated from its peers or recreated on first boot, so replacing the VM with a fresh one from the Marketplace image gives you a node in its shipped state.

To do that without an outage, use the stable-IP swap in Rolling upgrades. It deploys a replacement node, moves the serving IP across, then removes the old VM, so client DNS settings never change. Point it at the version you are already running rather than a newer one and the result is a reset rather than an upgrade.

Tip

This is also the cleanest recovery from a configuration you no longer trust. Zone and record data returns from the surviving peers automatically, so a node that has drifted can simply be replaced rather than repaired.

Decommission the cluster

Order matters here for the same reason it does when removing a single node. If you delete the nodes while VNets still list their addresses as DNS servers, resolution breaks for every client in those VNets before you have given them anywhere else to ask.

Step 1: move clients off DTM

Repoint the DNS servers setting on every VNet that lists a DTM node address, and remove any resolver forwarding rules that target the cluster. Allow time for the change to reach running VMs, which may mean a reboot or a DHCP lease renewal depending on the guest operating system.

Confirm before continuing that queries are no longer arriving. The per-node query counters described in Observability are the quickest way to see traffic stop.

Step 2: capture anything you still need

Once the VMs are gone, the zone data goes with them. If you might need the configuration later, take a backup first.

Caution

Encrypt it with a passphrase. An unencrypted snapshot is secret-equivalent: it holds live TSIG keys, health-check credentials, and password and token hashes. Treat the file as a credential in its own right, and see step 2 of what deletion leaves behind below.

Step 3: delete the Azure resources

Delete the resources the cluster was deployed from. If DTM was deployed into its own resource group from the Marketplace offer, deleting that resource group removes the nodes, their disks, their network interfaces and the dedicated Key Vault in one operation.

If you deployed with Terraform, running terraform destroy against the DTM configuration is the equivalent, and is preferable, because it also removes the role assignments Terraform created.

Managed disks are encrypted at rest by default and deleting a disk destroys the data on it, so there is no separate wipe step to perform on node storage.

What deleting the VMs does not remove

Four things can survive deleting the virtual machines.

Key Vault secrets, and the retention window

On the Terraform path the initial admin password, the gossip encryption key, the UI TLS bundle, and (when supplied through Terraform) the Entra client secret live in the dedicated Key Vault, with working copies on each node. On the Marketplace path the vault holds only the gossip key and initial admin password: the TLS material exists only on the VMs, and an Entra client secret configured from the UI or API lives in the database on the node where it was configured (so it travels with that node’s backups), not the vault. Deleting the vault does not destroy them immediately. Key Vault soft-delete retains deleted vaults and secrets for a configurable retention period of up to 90 days, and if purge protection is enabled you cannot shorten it. Until that period expires, the secrets remain recoverable by someone holding the right permissions.

Pick one of these deliberately:

• Rotate the secrets before you decommission, so whatever remains recoverable is already worthless. This is the strongest option and the least disruptive, because it happens while the cluster is still running. See Rotating secrets.
• Purge the deleted vault or its secrets explicitly, which is possible only if purge protection is not enabled.
• Accept the retention window, and record that the cluster’s secrets remain recoverable until it expires.

Backups you have downloaded

Any .dtmb snapshot downloaded to a workstation, a file share or a storage account is entirely outside the scope of deleting the cluster. These files hold live TSIG secrets, health-check credentials, and password, refresh-token and service-account hashes.

Danger

Deleting the cluster does nothing to a backup downloaded six months ago. Track those copies down and destroy them deliberately, and remember they may also exist inside backups of the machine you downloaded them to.

Telemetry you exported

If the cluster exported to Azure Monitor, Log Analytics or an OTLP endpoint, that data sits at the destination under its own retention policy. Query logs in particular can contain client IP addresses and queried names. Deleting DTM does not delete them, so adjust retention or purge the data where it landed. See Observability.

Role assignments and identities

A system-assigned managed identity is deleted with its VM, but role assignments made to that identity can be left behind as orphaned entries pointing at a principal that no longer exists. They grant nothing, but they linger in governance reporting. terraform destroy removes the assignments Terraform created; any granted by hand need removing by hand.

Verification checklist

• No VNet lists a DTM node as a DNS server, and no resolver rule forwards to the cluster.
• The resource group, or the Terraform-managed resources, are gone.
• Key Vault is purged, or the secrets were rotated first, or the retention window is recorded and accepted.
• Downloaded backups are accounted for and destroyed.
• Exported telemetry retention has been reviewed at the destination.
• No orphaned role assignments reference the deleted identities.