Why it matters to a buyer
Running a test without setting a minimum detectable effect first is how a team ends up staring at inconclusive results for months: a page with modest traffic simply cannot reliably detect a small lift within a reasonable timeframe, and not knowing that before starting wastes the test window on a change that was never going to reach a trustworthy answer. For a buyer, asking what MDE a proposed test was designed around is a quick way to judge whether a test plan is realistic for the traffic actually available.
How it is measured or applied
MDE is set before a test begins by choosing the smallest lift worth detecting, given the page's current conversion rate, its typical traffic volume, and how long the business can reasonably wait for a result, then using that to calculate the required sample size with a standard A/B test sample size calculator. A smaller MDE requires a much larger sample and longer runtime to detect reliably. It is applied by declining to test changes on pages too low-traffic to reach a sensible MDE within a reasonable window, and instead making a direct, judgement-based improvement on those pages.
Sources
- Sample Size Calculator for A/B testing, Evan Miller
