4 ms·

Never heard of bandit algorithms before! Or if I did I didn't recognize it as something different from probability. What have people around here used them for?

zdkl 8y ago
This rust project, to manage the number of threads in a monero miner afair. https://github.com/Ragnaroek/mithril https://github.com/Ragnaroek/mithril
- ur-whale 8y ago
  Doesn't alphago use some form of Bandit algorithm in their MonteCarlo code?
  - magoghm 8y ago
    I believe that Monte Carlo Tree Search, used in AlphaGo, does work using bandit algorithms. On top of that AlphaGo uses Reinforcement Learning, which also uses bandit algorithms (in Sutton & Barto's book, "Reinforcement Learning: An Introduction", all of chapter 2 is about multi-armed bandits).
haffi112 8y ago
You can use it when determining the best solution being tested in as few trials as possible. Say you are selling a product and you are AB testing something related to buying the product. When a user visits the site you ideally want to give him the version you are more confident is better. By using a bandit approach you can determine if say option A is currently better (w.r.t. some confidence bounds). After each visit you can update the bounds and after sufficiently many visits you have a winner. The main difference to more traditional AB testing is that the process is more adaptive and less time is wasted on exposing an inferior product to the user.
bochi 8y ago
Bandits are probably one of the most underrated machine learning algorithms. One possible application is recommendation systems. Shameless self promotion. I wrote an article about it: https://towardsdatascience.com/how-not-to-sort-by-popularity-92745397a7ae https://towardsdatascience.com/how-not-to-sort-by-popularity...
mlechha 8y ago
They're probably the most fundamental kind of reinforcement learning algorithms. Understanding bandit algorithms is crucial to developing a good understanding of RL.

zdkl 8y ago

This rust project, to manage the number of threads in a monero miner afair. https://github.com/Ragnaroek/mithril https://github.com/Ragnaroek/mithril

ur-whale 8y ago
Doesn't alphago use some form of Bandit algorithm in their MonteCarlo code?
- magoghm 8y ago
  I believe that Monte Carlo Tree Search, used in AlphaGo, does work using bandit algorithms. On top of that AlphaGo uses Reinforcement Learning, which also uses bandit algorithms (in Sutton & Barto's book, "Reinforcement Learning: An Introduction", all of chapter 2 is about multi-armed bandits).

haffi112 8y ago

You can use it when determining the best solution being tested in as few trials as possible. Say you are selling a product and you are AB testing something related to buying the product. When a user visits the site you ideally want to give him the version you are more confident is better. By using a bandit approach you can determine if say option A is currently better (w.r.t. some confidence bounds). After each visit you can update the bounds and after sufficiently many visits you have a winner. The main difference to more traditional AB testing is that the process is more adaptive and less time is wasted on exposing an inferior product to the user.

bochi 8y ago

Bandits are probably one of the most underrated machine learning algorithms. One possible application is recommendation systems. Shameless self promotion. I wrote an article about it: https://towardsdatascience.com/how-not-to-sort-by-popularity-92745397a7ae https://towardsdatascience.com/how-not-to-sort-by-popularity...

mlechha 8y ago

They're probably the most fundamental kind of reinforcement learning algorithms. Understanding bandit algorithms is crucial to developing a good understanding of RL.