Validation and indicators | Problem Management: investigate and prevent

Concept and mechanism

A correction provides useful evidence only when observed under conditions capable of exposing the mechanism. Zero incidents during a traffic-free period does not demonstrate resilience to the load that triggered failure. Define criteria before observation: representative operations, volume, duration appropriate to the pattern, healthy indicators, and absence of relevant regressions. Distinguish not observing an error from proving recurrence impossible. Confirm that data collection remains functional; disabling an alert reduces notifications without changing the service. When effects develop slowly, such as a resource leak, a short test may finish before the condition recurs. Validation should reflect these characteristics and state its limits.

Guided application

In an original numerical exercise, the first period has 12 incidents across 6000 runs, or 0.2%; the next has eight across 2000, or 0.4%. The count fell, but the rate per run doubled. Before attributing the difference to the correction, also compare work types, windows, and the incident definition. Rate does not replace severity: a single failure may have a greater consequence. Use a small set of indicators for recurrence, impact, exposure time, and overdue actions, with explicit denominators. Review evidence samples and accept the conclusion only within supported scope. If results contradict the hypothesis, resume investigation and update shared knowledge rather than adjust the indicator to appear favorable.

IN PRACTICE

12/6000 = 0.2%; 8/2000 = 0.4%. Fewer tickets do not establish fewer failures per run.

Common pitfalls

Zero without traffic as proof; counts without denominators; silenced alerts as improvement.

Related topics: Identification and priority · Investigation and evidence · Workarounds and known errors

Take this idea with you

Validate the mechanism and measure outcomes with visible exposure and limitations.

Create account

Reference: Contributing factors test gaps near misses and preventive sharing · Problem management practices 2026-09; ServiceNow Brazil examples with scoped plugins and properties