LEARN. PRACTICE. GROW.Start with the assertion being assessed
Define the criterion before requesting files. Confirming a current user is authorized differs from confirming all grants during a quarter. The objective determines the population, period, attributes and procedures needed. The NIST approach allows document and configuration examination, interviews and tests appropriate to the objective to be combined. An interview helps explain execution, but an owner’s statement does not automatically replace records or demonstration. Document why the evidence set supports the conclusion and which parts remain unassessed. Do not collect data merely because it is available.
Count and sum are insufficient checks
In a fictional extract, A is worth €100, B €100 and C €300. A file containing A, A and C still has three rows and €500 despite omitting B. Compare identity, multiplicity and relevant attributes alongside totals by scope and currency. A set of distinct IDs can hide repetitions if used alone. Preserve the original and record transformations performed during analysis. A discrepancy may arise from extraction or processing; do not automatically attribute fraud or harm before investigating its origin and consequence.
Select the period using the same time reference
An extract needs explicit time boundaries and a rule for endpoint inclusion. If the period ends before October 1 at 00:00 UTC, an event at September 30 at 23:45 with offset −01:00 is outside it: it corresponds to 00:45 UTC the next day. Keep the original time and conversion used. The offset is record information, not an assumption about a country. If offset or reliable synchronization is missing, document uncertainty and seek corroboration. Sorting text from different local times does not resolve the timeline.
Test the logic that seeks exceptions
Before trusting a script, use a controlled dataset with cases expected to pass and cases expected to fail. If an unapproved change is missed, zero production exceptions do not support effectiveness. Review filters, keys, conditions, empty fields and joins that may discard records. Analyzing the entire available population expands coverage only if that population and procedure are suitable. For a sample, explain selection and generalization limits. Five conveniently selected tickets without exceptions do not authorize concluding that all 800 quarterly changes complied with the control.
Local responsibilities for shared controls
An application may inherit central authentication or supplier support without inheriting every effectiveness condition. Check what the local team must configure, review and monitor. A supplier report may assume periodic customer access review; if it is absent, identify the specific limitation. Public ITAF 5 guidance emphasizes ecosystem dependencies but does not replace normative text or the mandate. Record owner, condition, evidence and scope to prevent two teams from each considering the same obligation the other team’s responsibility. The conclusion should follow the actual division of work.
Artifact approval and effective configuration
A matching hash helps establish identity of the compared file. It does not automatically include external configuration, data or feature flags. If a flag enables export absent from testing, connect that configuration to approval, impact assessment and appropriate tests. In migration, matching totals can also hide offsetting errors: positions of €100 and €300 transformed into €150 and €250 preserve sum and mean but violate per-position agreement. The auditor should follow the defined criterion rather than narrow scope so that a simple check appears sufficient.
Communicate the conclusion without assuming management
Management can accept risk within delegation without making a control effective when it fails the criterion. Separately record the observed condition, consequence, management response and acceptance conditions. Recommending treatment does not require the auditor to execute the change or approve delivery. Protect evidence as well: confirming who approved a deployment normally does not require copying customer balances. Define relevant data, access, retention and authorized disposal. Minimization should preserve identifiers needed for the objective; removing every identity may make authorization verification impossible.
Extract A=100, B=100, C=300 and extract A=100, A=100, C=300 have the same total. The workpaper identifies missing B and excess A multiplicity without concluding fraud.
Common pitfalls
Using count as completeness; filtering approved records to seek missing approval; ignoring offsets; accepting scripts without negative cases; confusing risk acceptance with technical effectiveness.
Related topics: Evidence quality and sampling · Configuration and migration · Mandate and inherited controls
The conclusion needs a correct population, validated procedure and evidence tied to the criterion. Document limits and keep independent assessment distinct from management decisions.
Reference: Assessing Security and Privacy Controls · CISA outline effective August 1, 2024