Skip to content

Known limitations

These are deliberate trade-offs, with the reasoning and the practical mitigation for each. None of them block the common use cases; they are here so nothing surprises you.

DTM steers traffic with DNS, so a client that has cached an answer keeps using it until the TTL expires. Failover time is therefore detection time plus the record TTL. This is inherent to every DNS-based traffic manager (including Azure Traffic Manager).

Mitigation: because DTM serves private zones where you control the resolvers, you can set low record TTLs safely. Combined with a short health-check interval, this gives failover in the low tens of seconds. See load-balancing methods.

The weighted method shapes traffic by returning one value per query in proportion to your weights. Downstream resolvers cache that single answer for the record TTL and clients reuse connections, so the realized split at the application tier is approximate: a 5% canary can receive materially more or less than 5% of real traffic. This is inherent to DNS-based traffic shaping (Azure Traffic Manager’s weighted routing carries the identical caveat).

Mitigation: set a low TTL (5 to 30 seconds) on the weighted record and treat the split as approximate. Selection itself is live per query, since records carrying a load-balancing policy bypass the answer cache; the smearing is entirely downstream.

DTM does not sign zones with DNSSEC.

Why it is acceptable: DTM serves DNS inside a private Azure VNet to network-segmented clients, where the off-path spoofing DNSSEC defends against is outside the threat model. Mitigation: rely on the private-network boundary (NSGs, VNet isolation) and TLS on the management surfaces.

Region-aware routing needs subnet mappings

Section titled “Region-aware routing needs subnet mappings”

The region and geo-failover load-balancing methods identify a client’s region from its source address. Without subnet mappings, DTM assumes every client is in the serving node’s own region, so it cannot distinguish clients that are actually elsewhere and steer them to a nearer region.

Mitigation: create subnet mappings (or use Azure region discovery) before relying on region-aware routing.

A cluster cannot span networks that cannot route to each other

Section titled “A cluster cannot span networks that cannot route to each other”

Replication is peer-to-peer with no leader and no proxy, so every node in a cluster must be able to reach every other node. One cluster spans exactly one routable domain: a VNet, a set of peered VNets, or VNets joined by a transit connection (hub-and-spoke through a firewall or NVA, Virtual WAN, ExpressRoute). A node that cannot reach its peers keeps serving DNS from its own copy while quietly diverging, so the failure mode is silent rather than loud.

Mitigation: decide the cluster boundary before deploying (see deployment). Disconnected estates, such as separate subsidiaries or an unintegrated acquisition, get a separate cluster each rather than one spanning cluster.

Cluster nodes replicate changes among themselves and converge quickly, but they are eventually consistent rather than transactionally consistent. If two administrators edit the same object on different nodes within the same brief replication window, the cluster converges on one of the two edits.

Mitigation: this is vanishingly rare in practice (objects are operator-managed and concurrent edits to the same row are unusual). Keep cluster nodes time-synced (Azure VMs are NTP-synced by default) so ordering is well-behaved.

Readiness guards during bulk writes and very large datasets

Section titled “Readiness guards during bulk writes and very large datasets”

/readyz includes two replication-health components that flip a node to 503 so a load balancer drains it while replication catches up: replication_queue (the replication broadcast queue exceeds its threshold, default 5000) and replication_freshness (no full-state sync has been applied within the staleness window).

A large one-time bulk import can transiently trip the queue guard, and a very large dataset (one whose full-state sync legitimately takes minutes) could formerly hold the freshness guard red even though DNS was serving normally. Two behaviours keep the guards honest:

  • The freshness window auto-scales to the observed sync cadence: five times the larger of the configured push-pull interval and the measured apply interval, capped at 30 minutes. A big dataset that syncs slowly is not falsely marked stale; only a genuine stall (no applies for several normal cycles) trips it.
  • The queue threshold is tunable: set replication.readyz_queue_threshold in the server YAML, or the DTM_REPLICATION_READYZ_QUEUE_THRESHOLD environment variable (0 or unset keeps the default 5000). Raise it for workloads with legitimate write bursts.

Mitigation: none needed in normal operation. A 503 from these components is a deliberate “drain me” signal to your load balancer, not data loss: the DNS data plane keeps answering, and replication reconciles automatically once the burst clears or the next full-state sync lands. See troubleshooting for how to read the /readyz component detail.

Override records: UI writes are admin-only, weights are API/Terraform-only

Section titled “Override records: UI writes are admin-only, weights are API/Terraform-only”

Override records can be created, edited, enabled/disabled, and deleted from the web UI by admin-role users, behind confirmation guardrails that restate the effect before committing; readonly and editor roles see the list read-only in the UI (editors can still write via the REST API). Two things remain API/Terraform-only: explicit per-value weighted percentages (the UI’s weighted mode splits traffic evenly) and, as everywhere, automation-owned state; the Terraform provider stays the recommended steady-state owner. ALIAS records, by contrast, are creatable from the normal record form in the UI without the extra guardrails (UI write controls are admin-only across the board; editors write via the API or Terraform).

Auto-populated records: two accepted trade-offs

Section titled “Auto-populated records: two accepted trade-offs”

A record with auto_populate set may legitimately hold zero values: the service exists, and right now nothing is registered to it. Two standards-correct consequences are worth knowing before you create one:

  • A pre-created record suppresses wildcard synthesis at its name. Per RFC 4592, a name that exists is its own closest encloser, so a *.example.internal wildcard stops covering api.example.internal the moment an auto-populated api record is created, even while it is empty. The name answers NODATA instead of the wildcard’s data. Creating an empty placeholder feels inert; it is not.
  • A secondary loaded by AXFR answers NXDOMAIN where the primary answers NODATA. A BIND zone file is one line per value and has no way to say “this name exists with no data”, so a value-less record contributes nothing to the transfer. DTM-to-DTM replication is unaffected (it ships records, not zone files), and the JSON bundle round-trips these records (health-check template secrets aside, which the export masks). The BIND export leaves a comment line so a human reading the file can see the record exists.

The supported, fully-hardened deployment is the Azure Marketplace image, which is built on Ubuntu 24.04 LTS. The DTM binary also runs on other modern Linux distributions, but only the Marketplace image ships the complete tested baseline.

Mitigation: deploy from the Marketplace image for production. See Deploy.

Access tokens from POST /api/v1/auth/login last 24 hours. Login also returns a refresh token (rotated on each use) that POST /api/v1/auth/refresh exchanges for a fresh access token; a session that goes unused for 7 days expires and needs a new login.

Mitigation: the web UI refreshes its session transparently. Automation should use a service-account token instead of logging in: long-lived (up to 10 years or non-expiring), bound to one role, revocable at any time. One caveat: a service-account token is validated on the node that issued it, so point the automation that uses it at that node.

A bulk import request (POST /api/v1/zones/import) carries at most 20 MB, 50 zones, and 50,000 records per zone, whether the payload is JSON, CSV, or BIND; a bulk request over the body or zone cap is rejected whole and writes nothing, and a JSON or CSV payload over the record cap likewise. One BIND nuance: in a multipart upload, a single zone file over the record cap lands in the response’s parse_errors while the other files still commit. The incremental per-zone import (POST /api/v1/zones/{id}/import, body cap 10 MiB) behaves differently at the record cap: it truncates at 50,000 records, imports the records that parsed, and reports the truncation in the response errors array.

Mitigation: split very large migrations into batches of up to 50 zones per request. See migrating to Hayami DTM.

DTM’s database reuses freed space internally but does not automatically return it to the filesystem, so the file can grow over time.

Mitigation: run compaction periodically (quarterly or when dtm_storage_bbolt_free_pages exceeds 50% of total sustained for 6 hours, the shipped DTMBboltFreelistBloat rule’s hold). It is a per-node admin operation with no downtime during staging.

Single stable IP per region (load balancer in front)

Section titled “Single stable IP per region (load balancer in front)”

The recommended deployment gives clients the DTM nodes’ stable IPs directly, for a zero-hop query path. Some operators prefer a single frontend IP per region behind an Azure Standard Internal Load Balancer. That is supported, with two trade-offs: a small extra network hop on every query, and the load balancer’s TCP-only health probes seeing less than DTM’s own health view.

Mitigation: use direct-to-stable-IP unless your organisation standardises on load-balancer-fronted services. Either way the node IPs stay stable across upgrades.

In-VNet segmentation of DNS and cluster ports is operator-managed

Section titled “In-VNet segmentation of DNS and cluster ports is operator-managed”

The deployment’s NSG narrows the management ports (8443, 8080) to the adminAddressPrefix you choose and enforces it with an explicit deny rule. The DNS (53, 853, and 443 for DoH) and cluster (7946) source prefixes are allow-scoping only: the template ships no matching deny, because a shipped deny that missed one of your client or node subnets would silently break resolution or replication. An in-VNet source outside those prefixes is therefore still admitted by Azure’s default VNet rule.

Mitigation: if you want in-VNet segmentation of DNS or cluster traffic, add your own deny rules below Azure’s default allow-VNet rule, or use subnet-level NSGs. Cluster traffic is authenticated and encrypted at the application layer regardless. See network access.

Active Directory: coexistence, not a domain-controller DNS replacement

Section titled “Active Directory: coexistence, not a domain-controller DNS replacement”

DTM serves the record types and dynamic-update mechanics Active Directory depends on: SRV (authoritative and via RFC 2136 dynamic update), A/AAAA/CNAME/PTR/NS/TXT/MX, RFC 2136 dynamic DNS with optional TSIG, AXFR, reverse zones with automatic PTR, and scavenging of stale dynamic records. What it does not implement is GSS-TSIG (RFC 3645, the Kerberos-authenticated “secure dynamic update” Windows domain controllers and domain-joined clients use by default), and there is no special handling of the AD _msdcs.<forest> delegation. So DCs and domain-joined Windows clients cannot self-register into a DTM zone the way they register into AD-integrated DNS.

Why it is acceptable: GSS-TSIG would bind DTM to AD’s Kerberos auth model and machine accounts, a large surface that cuts against the lightweight, standards-only design. Coexistence delivers the traffic-management value without DTM joining the domain. Mitigation: keep the AD-integrated zones (the domain and _msdcs) authoritative on the DCs and run DTM alongside for the application and traffic-management zones, conditionally forwarding the AD zones to the DCs; non-Windows registrants use DTM’s RFC 2136 + TSIG path directly. Full recipe: Active Directory coexistence.

The cluster encryption key is not rotatable in place

Section titled “The cluster encryption key is not rotatable in place”

The cluster encryption key, generated at first boot and shared by every node, protects node-to-node cluster traffic. There is no API or command that re-keys a running cluster: rotating it means redeploying onto fresh first-boot secrets and restoring data from a backup.

Why it is acceptable: the key never leaves your Key Vault and your nodes, and the traffic it protects flows only between the DTM nodes inside your VNet on port 7946, a path you already restrict at the NSG. Mitigation: guard the Key Vault (least-privilege access, MFA, access logging), keep port 7946 node-to-node only, and if policy or a suspected exposure requires a new key, plan the redeploy with support. See rotating secrets for the full secret-lifecycle picture.

An upgrade swap needs one free address in the node subnet

Section titled “An upgrade swap needs one free address in the node subnet”

DTM upgrades a node by deploying a replacement and moving the node’s stable IP onto it, so the address your clients resolve never changes. Releasing that address from the retiring node needs one unallocated address in the node subnet at that moment, over and above the spare address the replacement was deployed on. A subnet sized exactly to its node count cannot complete the move.

The /24 subnet the deployment templates create makes this a non-issue. It can only be reached on a hand-built minimal subnet (a /29 or smaller, after Azure’s five reserved addresses).

Mitigation: give node subnets headroom, which the shipped templates already do. If a swap does run out, it stops and reports an address-allocation failure naming the cause; free one address in the subnet and retry the upgrade. Nothing is left half-moved by the check itself.

A swap that fails after the address is released has no automatic rollback

Section titled “A swap that fails after the address is released has no automatic rollback”

Between releasing the stable IP from the retiring node and assigning it to the replacement, no resource holds that address. If the upgrade fails inside that window (for example the replacement does not come up), there is no automatic return to the old node, and restarting the old node does not bring the stable IP back with it: it starts on a temporary address instead.

Mitigation: retry the upgrade rather than restarting the old node. The release and the assignment are both idempotent and converge on a retry, which is the supported recovery. Pointing the stable IP back at the retired node is manual Azure work and is only appropriate when the replacement cannot be recovered at all; contact support before going down that path.

Roll back only to a version whose configuration schema matches

Section titled “Roll back only to a version whose configuration schema matches”

A rollback re-renders the node’s configuration from your deployment’s template and hands it to the older binary. DTM parses its configuration strictly, so a setting the older version does not recognise is an error rather than a warning: that is deliberate, because it stops a mistyped security setting from being ignored. The practical effect is that a rollback which crosses a release that added a configuration setting will not bring the node back into service.

The node tells you so rather than failing quietly. It stops before starting the server and reports that the configuration is newer than the version being rolled back to, naming the setting involved.

Mitigation: roll back only to a version whose configuration schema matches the one you are running, and check the rollback section of the rolling upgrade guide before you start. If you need to go back further than that allows, plan it with support.

Zone-transfer interop is primary-side only (no secondary zones)

Section titled “Zone-transfer interop is primary-side only (no secondary zones)”

DTM behaves as a standards-complete primary for downstream DNS servers: outbound AXFR with an IP allowlist and optional per-zone TSIG, and outbound DNS NOTIFY (RFC 1996) on zone changes (axfr.notify_targets in the configuration reference), so mirrors no longer poll SOA serials. What it does not do is act as a secondary: there is no zone type that pulls and serves a read-only replica from an external primary (BIND, Windows DNS, or another vendor), inbound NOTIFY is refused, and IXFR is neither served nor consumed. Stub zones track a delegation and forward at query time; they are not a local replica and do not survive the upstream being down.

Two consequences to plan around: the “run DTM as a secondary first, validate, then cut over” migration ramp is not available (use the parallel-run migration patterns instead), and AD-integrated zones cannot be mirrored locally; coexistence stays forwarding-based, per the Active Directory boundary above. Mitigation: downstream consumers that need a replica (a BIND mirror, ExternalDNS reads) use outbound AXFR plus NOTIFY, which is exactly the standards pairing they expect from a primary. One interop note: each cluster node sends NOTIFY for the writes it accepted itself, so configure mirrors to accept NOTIFY from any cluster node address.

If a limitation is blocking you, email [email protected]. Several of these trade-offs have a clear path to change if there is demand.