4 ms·

Multiplayer AlphaZero

53 points by hardmaru 7y ago

jstrong 7y ago
> e.g. equity trading
- narrator 7y ago
  As much as I liked to get excited bout these kinds of papers: "Recomputing the AlphaGo Zero weights will take about 1700 years on commodity hardware." [1] https://github.com/leela-zero/leela-zero https://github.com/leela-zero/leela-zero
  - fnbr 7y ago
    Yeah, but that's an embarassingly parallel task, so you can recreate it with sufficient resources. If there's a sufficiently lucrative application, businesses will happily pay that.
    - dmurray 7y ago
      And it's already been done for the chess version, in a community effort[0]. Well, "recomputing the weights" isn't plausible for a neural network of this size, but computing weights that give the network a similar level of performance. [0] https://lczero.org/ https://lczero.org/
      - narrator 7y ago
        One thing I think that will be critical to the future of software is community efforts to build deep neural nets that match the AI power of the big companies.
  - pixelpoet 7y ago
    There are newer approaches where specialising for the problem domain (Go) gets you a 5-100x speedup: https://github.com/lightvector/KataGo/ https://github.com/lightvector/KataGo/
- make3 7y ago
  Indeed that's a weird thing to say, especially since AFAIK AlphaGo is for (very large) discrete search spaces (where Monte Carlo Tree Search would be used), & equity trading doesn't strike me as a discrete search space.. is it ?
  - dmurray 7y ago
    Yes? At any point in time you can buy or sell a discrete number of shares in a finite universe of stocks. The bigger difference in equity trading is that it's a "game" of hidden information with asymmetric and maybe unknown payoffs. Of course, computers are pretty good at trading stocks, too, but I can say confidently AlphaZero won't be the most useful approach.
    - make3 7y ago
      A part missing in your analysis is that time is involved in the decisions, and that time is continuous, and that time is very important
      - falcor84 7y ago
        Is time really continuous in this case? I have absolutely no idea how it's implemented in practice, but would assume that stock exchange computers use some sort of game loop at a consistent rate.
  - degski 7y ago
    Sound is not discrete either, it does not seem to be a problem on computers, though.
    - make3 7y ago
      That's the feature space, not the decision space
- amitport 7y ago
  Well at least the second author, Tucker Balch, is a computational investing consultant / lecturer. I'm sure they'll continue to work this angle, even if it's not yet there.
gok 7y ago
Interesting results but they really only use toy examples (Tic-Tac-Toe and Connect Four). It would be kind of surprising if it didn't work.
mrleiter 7y ago
Anyone who's interested to see how brilliant AlphaZero played against the best chess computer (then), with solid explanations: https://www.youtube.com/watch?v=lFXJWPhDsSY https://www.youtube.com/watch?v=lFXJWPhDsSY
- dmurray 7y ago
  I'd recommend instead the videos made by Matthew Sadler for chess24[0]. Unlike the guy in your videos, Sadler is a strong chess player and literally wrote the book [1] on Alphazero. The videos are approachable for all levels of chess ability. [0] https://www.youtube.com/playlist?list=PLAwlxGCJB4NchyTBYik8FBbnzLXpCCO79 https://www.youtube.com/playlist?list=PLAwlxGCJB4NchyTBYik8F... [1] https://www.newinchess.com/game-changer https://www.newinchess.com/game-changer https://www.youtube.com/playlist?list=PLAwlxGCJB4NchyTBYik8FBbnzLXpCCO79 https://www.youtube.com/playlist?list=PLAwlxGCJB4NchyTBYik8F...
Mageek 7y ago
Nice and concise paper. All that changed was outputting a vector of predicted game values, as we go from having a zero-sum game with two players (which can be summarized with a single value) to a game with n players. Makes sense. The comparison against MCTS shows strong performance from AlphaZero. Would be curious to see the performance of AlphaZero vs. number of its own rollouts - ie is the probability head output alone already encoding enough information to play well, and how deep does it have to look ahead to determine good play. Finally, for tic-tac-mo and connect 3x3, it should be possible to determine the optimal move. How much training / lookahead is required to achieve that? Does AlphaZero achieve perfect play for these games? The paper's first listed contribution is "an independent reimplementation of DeepMind's AlphaZero algorithm". Maybe I missed it, but I don't see a link to a repo with the implementation.

53 points by hardmaru 7y ago

jstrong 7y ago

> e.g. equity trading

narrator 7y ago
As much as I liked to get excited bout these kinds of papers: "Recomputing the AlphaGo Zero weights will take about 1700 years on commodity hardware." [1] https://github.com/leela-zero/leela-zero https://github.com/leela-zero/leela-zero
- fnbr 7y ago
  Yeah, but that's an embarassingly parallel task, so you can recreate it with sufficient resources. If there's a sufficiently lucrative application, businesses will happily pay that.
  - dmurray 7y ago
    And it's already been done for the chess version, in a community effort[0]. Well, "recomputing the weights" isn't plausible for a neural network of this size, but computing weights that give the network a similar level of performance. [0] https://lczero.org/ https://lczero.org/
    - narrator 7y ago
      One thing I think that will be critical to the future of software is community efforts to build deep neural nets that match the AI power of the big companies.
- pixelpoet 7y ago
  There are newer approaches where specialising for the problem domain (Go) gets you a 5-100x speedup: https://github.com/lightvector/KataGo/ https://github.com/lightvector/KataGo/
make3 7y ago
Indeed that's a weird thing to say, especially since AFAIK AlphaGo is for (very large) discrete search spaces (where Monte Carlo Tree Search would be used), & equity trading doesn't strike me as a discrete search space.. is it ?
- dmurray 7y ago
  Yes? At any point in time you can buy or sell a discrete number of shares in a finite universe of stocks. The bigger difference in equity trading is that it's a "game" of hidden information with asymmetric and maybe unknown payoffs. Of course, computers are pretty good at trading stocks, too, but I can say confidently AlphaZero won't be the most useful approach.
  - make3 7y ago
    A part missing in your analysis is that time is involved in the decisions, and that time is continuous, and that time is very important
    - falcor84 7y ago
      Is time really continuous in this case? I have absolutely no idea how it's implemented in practice, but would assume that stock exchange computers use some sort of game loop at a consistent rate.
- degski 7y ago
  Sound is not discrete either, it does not seem to be a problem on computers, though.
  - make3 7y ago
    That's the feature space, not the decision space
amitport 7y ago
Well at least the second author, Tucker Balch, is a computational investing consultant / lecturer. I'm sure they'll continue to work this angle, even if it's not yet there.

gok 7y ago

Interesting results but they really only use toy examples (Tic-Tac-Toe and Connect Four). It would be kind of surprising if it didn't work.

mrleiter 7y ago

Anyone who's interested to see how brilliant AlphaZero played against the best chess computer (then), with solid explanations: https://www.youtube.com/watch?v=lFXJWPhDsSY https://www.youtube.com/watch?v=lFXJWPhDsSY

dmurray 7y ago
I'd recommend instead the videos made by Matthew Sadler for chess24[0]. Unlike the guy in your videos, Sadler is a strong chess player and literally wrote the book [1] on Alphazero. The videos are approachable for all levels of chess ability. [0] https://www.youtube.com/playlist?list=PLAwlxGCJB4NchyTBYik8FBbnzLXpCCO79 https://www.youtube.com/playlist?list=PLAwlxGCJB4NchyTBYik8F... [1] https://www.newinchess.com/game-changer https://www.newinchess.com/game-changer https://www.youtube.com/playlist?list=PLAwlxGCJB4NchyTBYik8FBbnzLXpCCO79 https://www.youtube.com/playlist?list=PLAwlxGCJB4NchyTBYik8F...

Mageek 7y ago

Nice and concise paper. All that changed was outputting a vector of predicted game values, as we go from having a zero-sum game with two players (which can be summarized with a single value) to a game with n players. Makes sense. The comparison against MCTS shows strong performance from AlphaZero. Would be curious to see the performance of AlphaZero vs. number of its own rollouts - ie is the probability head output alone already encoding enough information to play well, and how deep does it have to look ahead to determine good play. Finally, for tic-tac-mo and connect 3x3, it should be possible to determine the optimal move. How much training / lookahead is required to achieve that? Does AlphaZero achieve perfect play for these games? The paper's first listed contribution is "an independent reimplementation of DeepMind's AlphaZero algorithm". Maybe I missed it, but I don't see a link to a repo with the implementation.