2 ms·
What does this get us? Okay, so we can more accurately calculate sample sizes, but what about power relative to the "power" of using a Beta prior in the Bayesi
by RA_Fisher 13y ago
What does this get us? Okay, so we can more accurately calculate sample sizes, but what about power relative to the "power" of using a Beta prior in the Bayesian framework?
I use Bayesian methods simply to restrict the test space. It's also relatively simple (computer is doing the math, essentially).
- jmount 13y agoIf somebody proposes a level of improvement they expect in an A/B test (and you have a good prior estimate of the untreated A-rate) they you quickly get specific sample sizes to route to A and B. You also have the form of expression so you can see what is expensive (measuring small differences) and what is cheap (driving up confidence). Frankly a Bayesian article that makes a similar calculation would be at least as good an article (as it would have a much less clumsy treatment of what are you expecting to measure as a distribution prior instead of a point estimate). But I think the merit is how fast you can get from start to finish in such calculations. Edit: I've added some discussion on what you would expect in the Bayesian case as an comment on the blog.