LEARN. PRACTICE. GROW.From delivery to an observable task
Start by writing what a person should be able to do differently. In a fictional closing process, an operator needs to identify the correct file without switching between three systems while retaining accuracy. Ten delivered dashboards and completed training describe work performed; observing the task helps assess its effect. Define task start, correct completion, and included cases. Also collect failures, abandonment, and requests for assistance. Faster work accompanied by incorrect selections may conflict with the goal. A useful Review discussion examines how the experience changed and what still prevents the intended outcome.
Denominators and opportunity to use
A portal has 200 accounts, but only 40 operators received eligible cases during the week. Thirty used the new workflow. Usage among those with an opportunity is 30/40, or 75%; the proportion across all accounts is 15%. Both calculations are possible, but they answer different questions. Keep the definition beside the indicator and do not change it to favor a result. Usage also establishes neither satisfaction, correctness, nor economic benefit. Observe whether the workflow was mandatory, whether alternatives existed, and whether operators completed their task. When there are no eligible cases, show missing observations rather than an artificial success rate.
The aggregate and its segments
Consider an earlier sample with 81 correct outcomes in 90 simple cases and two in ten complex cases. Later, there are ten correct outcomes in ten simple cases and 27 in 90 complex cases. The aggregate falls from 83% to 37%, although each segment improves: 90% to 100% and 20% to 30%. Case mix changed substantially. Do not discard the total because it represents the observed experience in that collection. Explain the mix change and seek comparable groups before attributing an effect to the solution. Even improvement within comparable groups can coincide with other changes, such as training or triage rules. The numbers guide investigation; they do not automatically isolate a cause.
From recording a hypothesis to learning
A fast deployment can still be followed by a long wait for evidence. In the exercise, the hypothesis is recorded on day zero, code deployed on day two, access opened on day 12, and analysis performed on day 18. If the defined interval ends at analysis, it is 18 days. Waiting for authorization or eligible cases matters when deciding on the next experiment. Clarify measurement boundaries to avoid comparing two engineering days with 18 learning days. A prototype can test naming before full integration but does not prove recovery, security, or production quality. Choose the experiment according to the uncertainty most constraining the decision.
Future costs and benefit conditions
Compare alternatives from the decision you can still make. Include migration, coexistence, operation, and exit where these are relevant future costs. In a simplified model, €48k migration cost and a monthly reduction from €14k to €8k produce €6k net savings and eight months to recover cost after retirement. The calculation excludes discounting, taxes, variability, and prior coexistence; it is not a complete financial forecast. If a consumer still depends on the old system, savings may not start. Make that condition explicit and present sensitivities. With only €4k monthly savings, recovery would take 12 months. None of these results alone confirms that customers receive more value.
Recommendation workshop with challenge
Prepare a one-page recommendation for the next product step. State the goal, observed task, segments, success definition, and future costs. Ask one person to represent APS, another the night user, and another FinOps. APS challenges removing recovery from the Definition of Done; the user identifies missing quarterly cases; FinOps asks when spending actually disappears. Revise the recommendation in response to these objections. PO accountability for Product Backlog management remains when interviews are delegated. For teams on the same product, retain shared quality. End with a bounded decision, assumptions others can challenge, and evidence to gather before expanding investment.
# Fictional undiscounted model; not a financial forecast
migration_kEUR = 48
old_monthly_kEUR = 14
new_monthly_kEUR = 8
net_monthly_saving_kEUR = 6
recovery_months_after_retirement = 8
coexistence_costs = not_includedA service costs €14k per month; its successor costs €8k. With €48k migration cost, simple recovery requires eight months of savings after retiring the old service.
Common pitfalls
Confusing accounts with opportunities to use; hiding case mix; ending learning measurement at deployment; recovering costs with unrealized savings; treating a prototype as production.
Related topics: Discovery, hypotheses, and experiments · EBM and measurement interpretation · Portfolio, investment, and the product operating model
A useful recommendation lets others reconstruct the calculation and challenge assumptions. Transparency about limits improves the next choice.
Reference: The Evidence-Based Management Guide · PSPO II; Scrum Guide November 2020 and EBM Guide May 2024; no public numbered exam revision