3 ms·
Re: 3, how do you calculate confidence bounds for a bandit test? I was under the impression that traditional AB testing methodologies (e.g. z-tests, p-values, e
by storborg 14y ago
Re: 3, how do you calculate confidence bounds for a bandit test? I was under the impression that traditional AB testing methodologies (e.g. z-tests, p-values, etc) broke down here, because the sample size is variable.
- noelwelsh 14y agoIt's actually simpler in the bandit case, because you're only trying to estimate the average return. You could use a Chernoff-Hoeffding bound for example, but most bandit algorithms are of the upper confidence bound variety and actually calculate a bound as part of the algorithm.