LEARN. PRACTICE. GROW.Define the outcome that needs to change
At a fictional bank, an audit finds that a nightly batch can proceed without complete reconciliation. The team installs a new script and closes the change ticket. This demonstrates administrative action and may document implementation, but it does not yet answer the original finding. Connect the requirement, observed condition, consequence, investigated cause and proposed action. Define what should happen instead: when reconciliation fails, processing should stop or follow an authorized exception, with recording and escalation. Identify the implementation owner and who will assess the outcome. Avoid criteria such as “script delivered” when the risk arises from processing behavior. Criteria should support a conclusion of success, failure or insufficient evidence.
Choose retest scope, environment and period
A correction can work on one node and fail on another, or work during daytime but not at night. The retest plan identifies assets, configurations, implementation version, environment, execution conditions and the relevant period. A staging test can support a design or readiness conclusion but does not automatically demonstrate production behavior. Likewise, evidence collected before implementation does not test the later change. Observation duration and selection depend on the objective, control frequency, risk and assessment method. This exercise establishes no universal two-cycle rule. Record access or timing limitations, how they affect the conclusion and the additional evidence needed. Make those boundaries visible to anyone using the report to decide whether further action remains necessary.
Work with a criteria matrix
The lab provides eleven findings, all with closed tickets, and a fictional matrix of two assets by two cycles. The query starts with required criteria and attaches available evidence, so a criterion lacking an observation does not disappear from the report. A has four current observations satisfying the modeled conditions. D covers only one asset and I covers only the daytime cycle. E uses a test environment, H contains pre-implementation observations and J refers to an old version. Each result exposes a different gap and needs a specific response. The exercise author defined the matrix: a correct query does not establish that the matrix represents every asset or operating condition of a real service.
Retain history and interpret later observations
B had favorable evidence followed by a failure for the same asset and cycle. In the model, the later observation determines current status, while earlier records remain available. This explains change rather than deleting evidence that no longer supports closure. The query uses ROW_NUMBER to select an observation per criterion after filtering environment, version and the cutoff. Equal times use insertion ID as an explicit fictional-data convention. In a real audit, contradictory sources with equal timestamps require investigation; the highest identifier does not establish greater reliability. Nor should the latest success be selected while ignoring later failures. An inconclusive result needs investigation rather than automatic conversion into approval.
Turn output into a bounded conclusion
Run the program and compare the eleven closed tickets with the derived states. A is only eligible for closure review, not an audit opinion issued by software. The assessor still needs to confirm evidence authenticity, relevance, sufficiency and scope, alongside applicable independence requirements. Reviewer names in the lab are data rather than authenticated people. Try the empty-matrix, unreviewed-evidence and additional-asset probes. Each change rolls back through a savepoint to support comparison. Draft a note for D identifying the untested asset, who should provide evidence and how the gap affects the conclusion. The note should distinguish claimed remediation from the outcome actually observed, including what can reasonably be concluded now and what remains unknown.
python3 content/labs/cisa-remediation-followup/run.py
# Compare findings A, B, D, E, H, I and J.
# eligible-for-closure-review is a review candidate, not an audit opinion.
# Production coverage and period sufficiency still require assessment.The installation ticket is closed, but only node-a has valid tests. The follow-up note records claimed implementation and incomplete node-b coverage; it does not present the entire service as remediated.
Common pitfalls
Counting tickets instead of outcomes; testing the wrong version; removing criteria lacking evidence; selecting only favorable observations; treating a review flag as demonstrated independence.
Related topics: Audit evidence · Change control · Post-implementation review
Closure needs a supported conclusion within the relevant scope. An implementation or a closed ticket is part of the evidence, not the whole conclusion.
Reference: Assessing Security and Privacy Controls · CISA outline effective August 1, 2024