LEARN. PRACTICE. GROW.Build a hypothesis before choosing a change
SERVFAIL starts an investigation; it is not a cause name. In the lab JSON, five faults produce that code, but logs distinguish altered data, expiration, future inception, missing signature, and mismatched digest. Record the exact query and answering resolver, then connect the reported cause to observable data and configuration. An Extended DNS Error, when available, adds context; it does not turn the answer into an authenticated channel or remove the need to confirm provenance. In fictional case Mar, only one region fails and the validator’s clock explains the difference. Changing shared keys or dates before that comparison would increase impact. The L3 team should build a hypothesis that a measurement could contradict, identify the component owner, and propose an action with an expected effect.
Coordinate publication and rollover dependencies
A trust change crosses responsibilities. The application may own the address, the DNS operator publish keys and signatures, and another entity maintain DS. In case Ponte, retiring the old key before completing transition left an unmatched reference. The exercise asks for a phased plan: known initial state, material that must coexist, owner of each publication, evidence needed before the next phase, and recovery condition. Do not invent a universal deadline: sets, signatures, caches, and environmental policy belong in the analysis. The key tag helps locate candidates but does not replace digest matching. A newly published key’s self-consistent signature also does not by itself establish the trust point consumers use. Application approval neither executes nor automatically authorizes another owner’s part.
Confirm recovery on the normal path
A repair needs criteria that preserve the control’s intent. In case Lago, two authorities return the same A but only one publishes the complete signed set. Comparison should include data and evidence, followed by queries through affected resolvers. A CD query may help isolate validation, but it is not the final recovery criterion. After correction, also observe cache state and any retained failures before repeating changes. Maintain a timeline of publication, query, and consumer confirmation. If temporarily removing an authority is necessary, its owner assesses capacity, redundancy, delegation, and authorization; the lab does not execute that intervention. The application still needs TLS validation and a representative transaction. Authenticated DNS data do not demonstrate availability, identity, or business outcome of the service using that address.
Prepare a readiness decision in forty minutes
The guide divides work into four ten-minute blocks. First compare the valid control with tampered and write what CD permits you to observe. Then assign causes and owners for Mar, Ponte, and Lago using case data. In the third block, define repair sequence, per-phase evidence, and an acceptable alternative without automatically disabling validation. In the last block, present Cais with a requirements/evidence matrix: local validation already measured, while public chain, rollover, negative proofs, and application still need their own work. The guide has not been performed with participants. Deliver a proceed-or-defer recommendation with rationale and assigned pending actions. Summary: decision quality depends on linking every claim to its evidence. More tests of the same path do not replace requirements that path never exercised.
40-MINUTE GUIDE
0–10: compare valid and tampered; state the meaning and limits of CD and AD.
10–20: assign hypotheses and owners for Mar, Ponte, and Lago.
20–30: define repair, alternative, per-phase criteria, and residual risk.
30–40: present Cais’s readiness matrix and a reasoned decision.
DELIVERABLE
Requirement | existing evidence | missing evidence | owner | acceptance criterion.
Include normal query, trust, time, cache state, and application transaction.
RUN THE PREVIOUS LESSON’S RUNNER
python3 dns-validation.py --unbound /path/unbound --checkconf /path/unbound-checkconf --openssl /path/openssl --dig /path/dig --output dns-validation-evidence.json
Prerequisites: Python 3, Unbound 1.26.1, matching checkconf, OpenSSL 3, and dig.
Recorded runtime: Python 3.13.1, OpenSSL 3.6.1, dig 9.10.6. Account without personal.digrc.
Uses loopback only, generates a temporary key, and does not change system DNS.Ponte published a new DNSKEY without completing the DS change. The recovery plan brings both link owners together and defines how to confirm each phase.
Common pitfalls
Treating SERVFAIL as one cause, retiring keys too early, accepting CD success, ignoring clocks, and confusing a local demonstration with migration acceptance.
Related topics: Change management and rollback · Observability and incidents
Recovery should work under normal trust policy. RUN handover needs owners, evidence per requirement, and explicit limitations.
Reference: RFC 6781: DNSSEC operational practices · DNS RFC 1034/1035 with RFC 2181, 2308, 3596, 4033, 7766 and 8767; dig BIND 9.20