12 ms·
Practical take: Thompson sampling is great. Use it if your main focus is getting things done. I found that most literature about MABs concentrates on various (m
by jwr 8y ago
Practical take: Thompson sampling is great. Use it if your main focus is getting things done. I found that most literature about MABs concentrates on various (mostly not too useful) approaches to MABs instead of on getting stuff done. Which is of course fine if your intent is to study MABs.
Thompson sampling is easy to implement, predictable, stable, and you can leave your experiments running forever without supervision. It has very few drawbacks.
With a combination of two HyperLogLogs and a Bloom filter, you can also have a really nice scalable distributed implementation (HLLs and BFs combine in an idempotent manner, so updates are lock-free), albeit at the cost of low precision early on as the number of samples is low.
Don't use a library. Thompson sampling itself is not difficult (specifically, 6 lines, as I'm looking at my Clojure code), you will need to understand it very well anyway, and most libraries will include lots of superfluous garbage. Most of the work is not with Thompson sampling, but with processing your events, and that is something which is application-specific anyway and can't be abstracted well.
- Matumio 8y agoI agree. I did some research out of curiosity in the past about multi-armed bandits. The optimal solution is quite involved, and not worth the effort of studying in my opinion. Thompson sampling, on the other hand... I'm in love. Clean, simple, straight-forward, and so close to the optimal solution that you will never notice the difference. If you don't understand how to implement it, the stuff that you need to learn (mostly about the beta distribution) is generally useful and well worth studying. Though, after several years now, I'm still looking for an excuse to apply it somewhere. I can't get this method out of my head.
- joshuamorton 8y agoI'm a bit confused, as far as I know, Thompson Sampling is optimal (insofar as it minimizes regret in expectation).
- wolfgangK 8y ago> Thompson sampling itself is not difficult (specifically, 6 lines, as I'm looking at my Clojure code), Going through "A tutorial on Thompson Sampling" and being eager to hone my skills in Clojure, I would be very interested in reading this code if there would be a repository online or a gist you could share with us. Best Regards.