3 ms·
Even statistics can lie. Having statistically significant results does not guarantee you that the variation will work better than the control. What it actually
by paraschopra 16y ago
Even statistics can lie. Having statistically significant results does not guarantee you that the variation will work better than the control. What it actually guarantees is that it is HIGHLY likely that variation will work better than the control. So, it ultimately boils down to what are the stakes involved in an A/B test.
- almost 16y agoNot just highly likely, likely to specified probability. Typically 95% or 99%. If you want to be more certain (and have the quantity of data) you can look for 99.9% or greater. I'm not sure how you could see that as statistics "lying" :) EDIT: I just noticed your username and I realise that you probably already understand this :p I still think it's a bit of an odd way of putting it though.