Roll out encrypted DNS

Enabling DoT or DoH is one API call; doing it before the listener has a stable certificate hands your resolvers a certificate that changes on every service restart. This tutorial does the rollout in the order that works: stable CA-issued certificate first, encrypted transports second, clients third, and then a sweep to retire every -sk and insecure_skip_verify the self-signed era forced on you.

Scenario

An admin at a company with a working three-node cluster (10.0.250.4 to .6, names dtm1 to dtm3.example.internal). Compliance requires DoT for the Linux estate’s resolvers and DoH for a VDI fleet, both validating against the corporate CA. You can issue certificates from that CA (directly or via Key Vault). Budget about 60 minutes, including one rolling restart. API steps send an admin bearer token as $DTM_TOKEN (how to capture one).

1. Know your two TLS surfaces

DTM terminates TLS in two places with different lifecycles: the web UI (8080) serves a managed two-year certificate, while the API listener (8443), DoT (853), and the dedicated DoH listener (443) share listen.tls material and self-sign afresh at every service start until you configure it. This tutorial touches only the second surface; the full inventory lives in TLS certificates.

2. Issue and place the certificate

Issue one server certificate from the corporate CA with subject alternative names covering every node name and stable IP (dtm1.example.internal, 10.0.250.4, and so on). If the CA publishes through Key Vault, download it yourself onto each node; this is a customer-managed step, DTM has no built-in Key Vault fetch for the API listener certificate (the Key Vault bundle flow you may have read about applies to the web UI certificate only):

az keyvault secret download --vault-name kv-corp-ca \
  --name dtm-api-tls --file /etc/dtm/api-tls/bundle.pem
# split into cert.pem / key.pem, then:
sudo chmod 0600 /etc/dtm/api-tls/key.pem

The key file must be readable by its owner only; the server refuses to start on a group- or world-readable key, and on a missing file it stops with a clear error rather than silently self-signing.

3. Configure and roll it out

On each node, point /etc/dtm/dtm-server.yaml at the files (both keys set together; see the configuration reference):

listen:
  tls:
    cert_file: /etc/dtm/api-tls/cert.pem
    key_file: /etc/dtm/api-tls/key.pem

Caution

There is no configuration reload: the file is read at startup, so this is a restart, and a restarting node stops answering DNS. Roll one node at a time, gating each restart on curl -sk https://<node>:8443/readyz returning 200 before moving on, the same discipline as a rolling upgrade.

4. Verify the certificate is yours, and stable

Compare the served certificate’s serial against the on-disk file, on every node, without any extra restarts (each one costs you a node’s DNS):

openssl s_client -connect 10.0.250.4:8443 </dev/null 2>/dev/null \
  | openssl x509 -noout -serial
openssl x509 -in /etc/dtm/api-tls/cert.pem -noout -serial
# serials match on every node

The rolling restarts you just performed are themselves the stability proof: the serial survived them. If you want a belt-and-braces demonstration, restart one node again and confirm its serial is unchanged.

5. Enable DoT and DoH cluster-wide

Now the payoff of step 2: talk to the API with --cacert, not -sk, and flip both transports with the live runtime toggle:

curl --cacert corp-ca.pem -X PUT https://dtm1.example.internal:8443/api/v1/transport-config \
  -H "Authorization: Bearer $DTM_TOKEN" \
  -H "X-DTM-CSRF: 1" \
  -H "Content-Type: application/json" \
  -d '{"dot_enabled": true, "doh_enabled": true}'

The change applies live and replicates cluster-wide, no restarts; a misspelled field is a 400, never a silent no-op. Toggle mechanics, including why the runtime value wins over the YAML at the next boot, live on DNS over TLS and HTTPS.

Caution

/dns-query is unauthenticated by design on its dedicated listener (443), and DoT opens 853. Before clients arrive, scope both ports in the NSG to exactly the resolver and VDI subnets that should reach them; the dot-doh page spells out the decision.

6. Verify both transports against the corporate CA

# DoT, validating the chain and the SAN
kdig @10.0.250.4 app.example.internal +tls \
  +tls-ca=corp-ca.pem +tls-hostname=dtm1.example.internal

# DoH
kdig @10.0.250.4 app.example.internal +https=/dns-query \
  +tls-ca=corp-ca.pem +tls-hostname=dtm1.example.internal

Both should match plain dig answers exactly. If 853 does not answer on one node, check that node’s logs: a node that cannot load its TLS material leaves DoT down and logs why, rather than serving a broken listener.

7. Point real clients at it

The client half exists on no other page. For a Linux host or estate resolver running systemd-resolved, in /etc/systemd/resolved.conf:

[Resolve]
DNS=10.0.250.4#dtm1.example.internal 10.0.250.5#dtm2.example.internal
DNSOverTLS=yes

The #hostname suffix is what systemd-resolved validates against the certificate’s SAN, so it must match a name you issued in step 2, and the corporate CA must be in the client’s trust store. Restart systemd-resolved and verify end to end:

resolvectl query app.example.internal   # answered, with DNSOverTLS=yes active

Point the VDI fleet’s browser or OS DoH settings at https://dtm1.example.internal/dns-query (the dedicated DoH listener on 443, so no port suffix is needed) per your platform’s policy tooling.

8. De-insecure the estate

The self-signed era left -sk and skip verify scattered through your tooling; retire them now, while the change is fresh:

• Scripts and automation: replace curl -sk with curl --cacert corp-ca.pem.
• Prometheus: set the CA in the scrape job’s tls_config and flip insecure_skip_verify to false (the end state the observability page shows).
• Terraform: the provider can now verify TLS too; drop insecure_skip_verify from the provider block if you had set it.

Verify: scrapes stay green and DoT clients keep resolving after the sweep. Anything that breaks was silently trusting the old certificate.

9. Plan the renewal before you forget

• Calendar the CA certificate’s expiry with margin, and note that renewal is steps 2 to 4 again (place files, rolling restart, serial check).
• The web UI certificate is a separate surface with its own two-year clock and rotation procedure; do not assume this rollout renewed it.
• Update any monitor that pins the API certificate so renewal does not page you as an incident.

Where next

• Wire observability end to end: if your scrape still says insecure_skip_verify: true, close that loop now.
• Production readiness pass: the wider hardening sequence this slots into.
• TLS certificates: the full two-surface reference, including bringing the web UI onto your CA.

Need a hand? Email [email protected].

Last validated: 2026-07-26