Minimum Detectable Effect
Minimum detectable effect is the smallest true difference between two variants that a test is capable of reliably detecting.
Every A/B test has a floor below which a real effect exists but the test can't reliably tell it apart from noise. That floor is the minimum detectable effect (MDE), and it depends on the sample size, how variable the underlying metric already is, and the confidence level the test is held to.
A smaller MDE requires more traffic or a longer running time — detecting a subtle difference between two variants takes more evidence than detecting an obvious one. An underpowered test, run with too little traffic or for too short a time, can fail to find a real effect not because the effect isn't there, but because the test wasn't sensitive enough to see it.
MDE is typically set before a test starts, as part of deciding how long to run it and how much traffic it needs. Deciding this in advance keeps a team from stopping a test early just because a result looked promising for a moment.
See it in the product
How NavonaAI implements this, in detail.
Related terms
Looking for another term?
Browse the full glossary, or see what NavonaAI adds back to your store.