4 ms·
Could you point to the math/protocol that goes from a sample set, to a range "-4 to +34" and of course how one goes from a range to giving a single number? I f
by lifeisstillgood 11y ago
Could you point to the math/protocol that goes from a sample set, to a range "-4 to +34" and of course how one goes from a range to giving a single number?
I feel like the discussion on this thread is missing the underlying "source code"
- tomjen3 11y agoI don't know about the one that goes from a sample set, but the range provided is likely just the 90 or 95% confidence interval, which means you can get the single number by taking the average of the two numbers. Don't do it though, it will tell you a lot less than the two numbers; a range that goes from -50 to +60 means that you have a lot of uncertainty and should probably not deploy, whereas +2 to +6 means you have little uncertainty and should deploy since it is almost certainly a win, if a small one.
- yummyfajitas 11y agoIt's not a confidence interval, it's a credible interval. It's derived from a Bayesian posterior. Confidence intervals are confusing and should rarely be used to communicate statistics to laypeople. (See the links in my above post.) Most of the time, you can tell people the definition of a confidence interval, and they will misinterpret this and think it's a credible interval. This is why we went Bayesian at VWO - we felt it would be easier to change the statistics to match people's thinking than to change people's thinking about statistics.
- yummyfajitas 11y agoHere are slides on the topic and the full math paper. But we dont go from a credible interval to a single number - we just report credible intervals. We find that to be the only honest choice. https://www.chrisstucchio.com/pubs/slides/gilt_bayesian_ab_2015/slides.html https://www.chrisstucchio.com/pubs/slides/gilt_bayesian_ab_2... https://www.chrisstucchio.com/blog/2013/bayesian_analysis_conversion_rates.html https://www.chrisstucchio.com/blog/2013/bayesian_analysis_co... https://cdn2.hubspot.net/hubfs/310840/VWO_SmartStats_technical_whitepaper.pdf https://cdn2.hubspot.net/hubfs/310840/VWO_SmartStats_technic...
- lifeisstillgood 11y agothank you - bedside reading :-)