Why it matters to a buyer
A test run without a written hypothesis first tends to drift: the goal metric gets redefined after the fact to whichever number happened to move, and a losing test gets explained away instead of counted as a real, useful result. A buyer should expect every proposed test to come with a hypothesis stated in advance, since that is what keeps a testing program honest and cumulative, each result, win or lose, adds real evidence about what actually works for that specific audience.
How it is measured or applied
A hypothesis is written in a consistent structure: based on evidence, such as a heatmap or session recordings, changing element to new version is expected to affect primary metric, because of a stated reason, and it names in advance the single primary metric that will decide the result. Hypotheses are prioritised by expected impact and ease of implementation, and tests are run to completion against the plan rather than cut short. Each result, whether it confirms or disproves the hypothesis, is recorded so future hypotheses build on what has already been learned rather than repeating past tests.
Sources
- A/B testing, Optimizely
