2 ms·

UCB1 is really not that much more complicated than epsilon-greedy. Some slightly sloppy code I wrote a few years ago, maybe 20 lines of code: https://github.com

by j2kun 7y ago

UCB1 is really not that much more complicated than epsilon-greedy. Some slightly sloppy code I wrote a few years ago, maybe 20 lines of code: https://github.com/j2kun/ucb1/blob/master/ucb1.py#L7-L35 https://github.com/j2kun/ucb1/blob/master/ucb1.py#L7-L35 Sure you have to read a bit more to know why it works, but if you write your code well you could plug this in without any extra trouble. It's not like you need a special optimization solver as a dependency.

nestorD 7y ago
And, for not much more effort (computing the variance of your samples), you can use UCB1-tuned [0] which gets rid of the 'c' parameter and tends to be even better. I personnaly think that it should replace UCB1 as a baseline when trying bandit algorithms. [0]: https://homes.di.unimi.it/~cesabian/Pubblicazioni/ml-02.pdf https://homes.di.unimi.it/~cesabian/Pubblicazioni/ml-02.pdf
- j2kun 7y ago
  It's funny, I had read that paper a few times while learning about bandit learning, and I never noticed their version, which funnily enough outperforms vanilla UCB1 in all of their tests!
- 60654 7y ago
  That's a nice paper, thanks for posting this!