Discovery, hypotheses, and experiments | PSPO II: value, experiments, and product strategy

Concept and mechanism

Before expanding investment, identify the uncertainty most constraining the decision. It may concern task understanding, need for a capability, or feasibility of achieving the intended effect. Define a hypothesis, the group to observe, the measure, and comparison conditions. A bounded experiment enables learning without building everything. A prototype used to explore understanding should not be presented as a usable production increment; it has a different purpose and explicit limits. Observing a task may expose difficulties missed by a general question about liking the idea. Retain a clear relationship between the test, observations, and next decision.

Guided application

Interpret results in context. If errors fall during a period with simpler cases, the product change may not explain the entire difference. If eight analysts test a workflow, observations can guide the next hypothesis without statistically representing every user. Do not choose the success definition after seeing which metric improved. An experiment without the expected benefit can be useful by avoiding costly expansion. Compare future options: another hypothesis, a better-bounded experiment, a change of direction, or stopping. Avoid both automatic persistence to justify sunk cost and abandoning the whole product after one unconfirmed hypothesis. Learning requires distinguishing correct experiment execution from hypothesis confirmation.

IN PRACTICE

Six of eight analysts use another tool before the dashboard refreshes. This justifies investigating timing; it does not prove universal product rejection.

Common pitfalls

Stated preference as behavior; small sample as certainty; criterion chosen after data; unsuccessful experiment as useless work.

Related topics: Ordering and stakeholder trade-offs · Forecasts, flow, and release choices

Take this idea with you

An experiment should reduce uncertainty relevant to the next choice.

Create account

Reference: The Evidence-Based Management Guide · PSPO II; Scrum Guide November 2020 and EBM Guide May 2024; no public numbered exam revision