Over the last few years, some people have started to work on a problem that has been described as "pure exploration" in multi-armed bandits [1,2]. The objective

Over the last few years, some people have started to work on a problem that has been described as "pure exploration" in multi-armed bandits [1,2]. The objective in this problem is, roughly speaking, to maximize the rate at which you become certain about whether or not you've correctly identified the best bandit arm (or, more generally, the top n arms). This solves some of the more common complaints about classic MAB algorithms in that, when run for a finite number of trials, the resulting sampling of the arms produces a far more confident decision than either the regret-minimizing policies followed by standard MAB algorithms or the uniform allocation policies typically used in A/B testing. The original article mentions problems in MAB algorithms dealing with delayed feedback. Such issues are largely ameliorated by the use of algorithms related to "Thompson sampling" [3], which induces stochastic trial allocation policies rather than the deterministic policies induced by UCB's selection process. It's definitely possible to develop Thompson-like methods for the exploration-oriented MAB problem, and such methods can rapidly distinguish the best among a rather large set of options, as might be required in applications like MVT (see note below). Note: I'm currently doing academic research in this area and, if anyone's particularly interested, I could share some of the (simulated) empirical data and algorithmic details pertinent to what I said above. [1]: Multi-Bandit Best Arm Identification, Gabillon et. al, NIPS 2011 [2]: PAC Subset Selection in Multi-Armed Bandits, Kalyanakrishnan and Stone, ICML 2012 [3]: An Empirical Evaluation of Thompson Sampling, Chappelle and Li, NIPS 2011