4 ms·

I agree, but: > Most board and card games fit this description. Most card games don't, because they are not perfect information games.

joelthelion 9y ago
Is it really impossible to adapt Deepmind's approach to, say, poker? I don't see anything fundamentally impossible, but I'm no RL expert.
- sanxiyn 9y ago
  I would consider a successful application of AlphaZero algorithm to imperfect information games to be a separate achievement, not a done deal. David Silver himself, who led AlphaGo project, wrote in 2016 in "Deep Reinforcement Learning from Self-Play in Imperfect-Information Games" https://arxiv.org/abs/1603.01121 https://arxiv.org/abs/1603.01121 "While many machine learning methods have achieved near-optimal solutions to classical, perfect-information games, these methods fail to converge in imperfect-information games".
  - joelthelion 9y ago
    Thanks for that paper, very interesting.
- tialaramex 9y ago
  Whereas the "correct" way to play chess will consist of knowing always exactly which move to make, most Poker variants will require a statistical model that says things like "In this situation we will fold 4% of the time, call 89% of the time and raise 7% of the time". Then the actual choices you make are random with a specific weighting. So that's hugely different. This truth is why strong poker analysts are always so equivocal. "All options are good here" annoys people but it's often true. For Heads Up (two players) Limit (bet sizes are fixed) Hold Em (a popular modern variant with two hole cards and five community cards) such a strategy was developed and you can view it online, it's essentially unbeatable. Its name is Cepheus.
  - seanwilson 9y ago
    > Whereas the "correct" way to play chess will consist of knowing always exactly which move to make, most Poker variants will require a statistical model that says things like "In this situation we will fold 4% of the time, call 89% of the time and raise 7% of the time". Then the actual choices you make are random with a specific weighting. Instead of the output of the neural network being the move to make then, couldn't you make the output be the weight/probability you should fold, call or raise?
AstralStorm 9y ago
Card games are delayed bounded information games. (With one turn lag to one match lag.) The incomplete information is strictly bounded.

joelthelion 9y ago

Is it really impossible to adapt Deepmind's approach to, say, poker? I don't see anything fundamentally impossible, but I'm no RL expert.

sanxiyn 9y ago
I would consider a successful application of AlphaZero algorithm to imperfect information games to be a separate achievement, not a done deal. David Silver himself, who led AlphaGo project, wrote in 2016 in "Deep Reinforcement Learning from Self-Play in Imperfect-Information Games" https://arxiv.org/abs/1603.01121 https://arxiv.org/abs/1603.01121 "While many machine learning methods have achieved near-optimal solutions to classical, perfect-information games, these methods fail to converge in imperfect-information games".
- joelthelion 9y ago
  Thanks for that paper, very interesting.
tialaramex 9y ago
Whereas the "correct" way to play chess will consist of knowing always exactly which move to make, most Poker variants will require a statistical model that says things like "In this situation we will fold 4% of the time, call 89% of the time and raise 7% of the time". Then the actual choices you make are random with a specific weighting. So that's hugely different. This truth is why strong poker analysts are always so equivocal. "All options are good here" annoys people but it's often true. For Heads Up (two players) Limit (bet sizes are fixed) Hold Em (a popular modern variant with two hole cards and five community cards) such a strategy was developed and you can view it online, it's essentially unbeatable. Its name is Cepheus.
- seanwilson 9y ago
  > Whereas the "correct" way to play chess will consist of knowing always exactly which move to make, most Poker variants will require a statistical model that says things like "In this situation we will fold 4% of the time, call 89% of the time and raise 7% of the time". Then the actual choices you make are random with a specific weighting. Instead of the output of the neural network being the move to make then, couldn't you make the output be the weight/probability you should fold, call or raise?

AstralStorm 9y ago

Card games are delayed bounded information games. (With one turn lag to one match lag.) The incomplete information is strictly bounded.