5 ms·

Is it really impossible to adapt Deepmind's approach to, say, poker? I don't see anything fundamentally impossible, but I'm no RL expert.

sanxiyn 9y ago
I would consider a successful application of AlphaZero algorithm to imperfect information games to be a separate achievement, not a done deal. David Silver himself, who led AlphaGo project, wrote in 2016 in "Deep Reinforcement Learning from Self-Play in Imperfect-Information Games" https://arxiv.org/abs/1603.01121 https://arxiv.org/abs/1603.01121 "While many machine learning methods have achieved near-optimal solutions to classical, perfect-information games, these methods fail to converge in imperfect-information games".
- joelthelion 9y ago
  Thanks for that paper, very interesting.
tialaramex 9y ago
Whereas the "correct" way to play chess will consist of knowing always exactly which move to make, most Poker variants will require a statistical model that says things like "In this situation we will fold 4% of the time, call 89% of the time and raise 7% of the time". Then the actual choices you make are random with a specific weighting. So that's hugely different. This truth is why strong poker analysts are always so equivocal. "All options are good here" annoys people but it's often true. For Heads Up (two players) Limit (bet sizes are fixed) Hold Em (a popular modern variant with two hole cards and five community cards) such a strategy was developed and you can view it online, it's essentially unbeatable. Its name is Cepheus.
- seanwilson 9y ago
  > Whereas the "correct" way to play chess will consist of knowing always exactly which move to make, most Poker variants will require a statistical model that says things like "In this situation we will fold 4% of the time, call 89% of the time and raise 7% of the time". Then the actual choices you make are random with a specific weighting. Instead of the output of the neural network being the move to make then, couldn't you make the output be the weight/probability you should fold, call or raise?

sanxiyn 9y ago

I would consider a successful application of AlphaZero algorithm to imperfect information games to be a separate achievement, not a done deal. David Silver himself, who led AlphaGo project, wrote in 2016 in "Deep Reinforcement Learning from Self-Play in Imperfect-Information Games" https://arxiv.org/abs/1603.01121 https://arxiv.org/abs/1603.01121 "While many machine learning methods have achieved near-optimal solutions to classical, perfect-information games, these methods fail to converge in imperfect-information games".

joelthelion 9y ago
Thanks for that paper, very interesting.

tialaramex 9y ago

Whereas the "correct" way to play chess will consist of knowing always exactly which move to make, most Poker variants will require a statistical model that says things like "In this situation we will fold 4% of the time, call 89% of the time and raise 7% of the time". Then the actual choices you make are random with a specific weighting. So that's hugely different. This truth is why strong poker analysts are always so equivocal. "All options are good here" annoys people but it's often true. For Heads Up (two players) Limit (bet sizes are fixed) Hold Em (a popular modern variant with two hole cards and five community cards) such a strategy was developed and you can view it online, it's essentially unbeatable. Its name is Cepheus.

seanwilson 9y ago
> Whereas the "correct" way to play chess will consist of knowing always exactly which move to make, most Poker variants will require a statistical model that says things like "In this situation we will fold 4% of the time, call 89% of the time and raise 7% of the time". Then the actual choices you make are random with a specific weighting. Instead of the output of the neural network being the move to make then, couldn't you make the output be the weight/probability you should fold, call or raise?