LEARN. PRACTICE. GROW.1. Define what must happen on the same request
In a fictional fund-instruction service, the criterion requires at least 70% of requests to have a correct result and finish on time. Ten requests are identified from 1 to 10. Requests 1 through 8 have correct results; requests 3 through 10 meet the deadline. Each separate indicator is 80%, but only 3 through 8 satisfy both: six of ten, or 60%. The joint criterion fails. Do not add the two numerators or choose the larger. Link observations to the same identity to calculate the exact intersection. If you knew only the two totals of eight without identities, the number meeting both could be six, seven or eight. The 70% threshold could not then be decided from those totals. Specify whether the requirement is joint per request or consists of two independent targets: the contracts produce different decisions.
2. Measure intervals without counting the same interruption twice
The fictional observation window runs from second 0 to second 600. Service is unavailable from 100 to 160 and from 140 to 200; intervals include their start and exclude their end. The records overlap between 140 and 160. Their union lasts 100 seconds, not 120. If the criterion uses the whole window, 500 of 600 seconds are available, about 83.33%. If another approved definition excludes the first 60 seconds as maintenance, the eligible window becomes 540 while still containing the same 100 unavailable seconds: 440/540, about 81.48%. The difference comes from the defined denominator. Maintenance is not automatically excluded because it is given that name. Clip intervals to the eligible window and apply the agreed rule, including outages starting before or ending after it.
3. Read cumulative counts and bucket boundaries
A set of 100 valid duration observations has 50 values up to 100 ms, 84 up to 200 ms, 96 up to 500 ms and 100 in total. Counts are cumulative: the first 50 are already included in 84. There are 12 observations above 200 and up to 500 ms. For a criterion requiring at least 90% within 300 ms, the data establishes only a range from 84% to 96%. Those 12 values can fall on either side of 300 ms. A uniform estimate would give 88%, but that distribution has not been established. A graph’s displayed estimate does not replace evidence required by an exact criterion. A bucket boundary precisely at 300 ms, or individual values retained through authorized handling, would permit direct counting. Also confirm unit, population and period before using the histogram.
4. Do not calculate the overall percentile by averaging percentiles
This exercise uses nearest rank: sort N values and select the ceiling of p times N, counting positions from 1. A has nine durations, all one second; B has one duration of ten seconds. A’s 90th percentile is one and B’s is ten. Their simple average is 5.5 seconds; weighting by request count gives 1.9. Neither is the overall 90th percentile: after sorting all ten values, position nine still contains one second. The overall 95th percentile is ten seconds because the rounded-up position is ten. The example supplies every value; in actual reports, per-instance percentiles do not retain the complete distribution. For aggregation, plan observations or compatible histograms and retain the method used. These numbers are not production-tail predictions.
5. Calculate what missing data still permits you to conclude
There are 100 eligible requests: 89 are confirmed to meet the criterion, eight are confirmed failures and three have unknown outcomes because telemetry was lost. The target is at least 90%. Without inventing those three outcomes, the actual proportion lies between 89% and 92%. The target could have been met or missed; available evidence is inconclusive. Removing the three from the denominator would produce a different metric. Marking unknown as success invents an observation, and marking it as an observed failure also changes meaning. A policy may treat missing evidence as preventing acceptance; that blocker is then established even though business outcomes remain unknown. Distinguish service state, observation quality and acceptance decision. If the 100 requests were 92 confirmed successes, five confirmed failures and three unknowns, the unknowns would not prevent proving a 90% threshold, although a separate complete-coverage requirement would remain unmet.
6. Relate available precision to the required decision
A fictional device measures 299 ms. This case accepts a maximum absolute error of five milliseconds without any probability distribution. Actual duration can lie between 294 and 304 ms. If the criterion requires up to and including 300 ms, the result is indeterminate. A reading of 294 ms would span 289 to 299, entirely compliant; a reading of 306 would span 301 to 311, entirely noncompliant. Do not interpret this interval as statistical 95% confidence: the stem supplies a deterministic bound accepted for the exercise. In an actual project, the responsible team would need to justify measurement behavior under the conditions used. The analyst must agree with stakeholders whether evidence supports the resolution required by the criterion or whether another method is needed, retaining the business need.
7. Match granularity to the need
A requirement may apply to the whole service, every channel or every closing window. These are not interchangeable ways to state the same thing. If every channel must meet 95%, a low-volume channel at 80% does not comply merely because a larger channel raised the aggregate. If the approved rule is only aggregate, a per-channel condition should not be invented retrospectively to explain rejection either. Record the need justifying the choice and present relevant effects the aggregate may hide. For L3 support, this distinction separates a global result from a failure concentrated in one client group’s integration. If observed exposure justifies changing the requirement, prepare a scoped change and explicit decision. Do not rewrite the metric only after learning which result would be more convenient.
8. Exercise: prepare an acceptance decision
Before reading the solution, write a recommendation stating the condition, calculation, missing evidence and next step. The criterion requires at least 90 of 100 instructions to be correct and complete within 300 ms. There are 98 confirmed correct results. The duration histogram is from section 3; no identity link between the two sets is available yet. Solution: the histogram permits between 84 and 96 on-time instructions but does not determine the set that is both correct and timely. Evidence is insufficient for acceptance. Adding an exact count of 92 within 300 ms can already resolve the decision: among the same 100 requests, 98 + 92 − 100 = 90 is the minimum meeting both conditions. The criterion passes even without the exact intersection. The team later supplies identities and finds 91 in both; this refines the result without being necessary to prove the threshold of 90. If the criterion required 91, separate counts would be insufficient and linking would permit a decision. This conclusion applies to the 100 observed instructions; it neither proves future performance nor dispenses with other acceptance conditions. In the international meeting, explain “the joint criterion is met for the observed population” and state the scope explicitly.
Exercise files
Original Python exercise with editable data, instructions and worked reasoning on intersections, availability, histograms and expansion decisions. Requires Python 3.10 or later.
Eight correct and eight timely requests out of ten do not establish eight that are both correct and timely. Identities may show an intersection of six.
Common pitfalls
Adding overlapping intervals; summing cumulative counts; averaging percentiles; treating unknown data as observed; using an interpolated graph as an exact count; changing granularity after seeing results.
Related topics: Acceptance-criteria planning · Solution evaluation and gap communication
Evidence must retain the detail on which a decision depends. Sometimes established bounds suffice even without an exact value.
References
- PMI-PBA Examination Content Outline · Public five-domain ECO; body copyright 2013/back cover 2017; inspected 2026-10-09
- Histograms and summaries · Current public documentation inspected 2026-10-09; no software version executed
- Monitoring Distributed Systems · Public book chapter inspected 2026-10-09; contextual engineering reference