14 ms·

This comment sounds reasonable, but shows that you don't understand the challenge of Go on computers very well. Go does not have openings. It has reasonable c

by btilly 7y ago

This comment sounds reasonable, but shows that you don't understand the challenge of Go on computers very well. Go does not have openings. It has reasonable choices to make, with an insane branching factor, with few moves making much of a difference by themselves. Therefore your database will only extend a few moves, and all of the positions that it winds up with will still be very close to even. So your database confers very little advantage. If you have alpha-go play itself a thousand times, it is unlikely that by move 10 you will wind up with the same board position twice. This is exactly the problem that made Go so hard for computers in the first place. Alpha-beta search is useless within practical limitations of computer hardware. That said in a different game, such as chess, your strategy would work very well indeed. (Which is why all decent computer programs have an opening book.)

dragontamer 7y ago
> Go does not have openings. There are 381 opening moves in Go, but really only 96 because of symmetry. 96 (opening moves) x 380 responses x 379 x 378 x 377 == ~2 Trillion positions after 5 ply. These 2-trillion positions will easily fit in a 16TB hard drive for $400. That's 8-bytes per position, so you probably can get there with more symmetries and some compression applied. ---------- You're thinking too much like a human. There's no Go-openings in the age of Human-Go. But in the age of Cyborg-Go where 16TB hard drives are allowed, we can begin to exhaustively build openings. We even have a super-human AI that can automatically, and algorithmically, explore this opening book. We can build AWS-instances with V100 Tensor cores to use neural-nets to explore all of these positions now. ------------- > If you have alpha-go play itself a thousand times, it is unlikely that by move 10 you will wind up with the same board position twice. Alpha-Go doesn't seem to implement much randomness at all into the moves it plays. The source of randomness is in time-controls (AlphaGo may choose MoveX before 30 seconds of analysis, or MoveY after 30 seconds of analysis), but this is a fairly constrained number of moves. Play alphaGo by itself a thousand times, at precisely the same time MCTS-controls (say: 1-million nodes visited in the MCTS tree), and it will probably play the same game 1000 times in a row. This makes AlphaGo extremely prone to opening database "attacks". Which is why I am using opening books as an easy example for how to beat a particular AlphaGo network. At least, until AlphaGo updates its algorithm for more random play. If the goal is "Beat AlphaGo" in a game, then the opening book construction is far, far simpler. Even with random elements (ex: AlphaGo picks randomly from the top 10 best positions it generates), that is far more constrained than a full 381 x 380 x 379 x ... style opening book.
- btilly 7y ago
  You're thinking too much like a human. No, you're thinking too much like someone who understands chess but not Go. Suppose that we built an opening book with a trillion reasonable courses of action on it. Each one analyzed well. As you have discovered, you will only go a few ply into the game. And all of the positions that you will be directed towards will have only a small edge. Instead put a tiny fraction of the computing power necessary to build this book into self-training. You will get a better internal model and therefore a significantly stronger computer player. (That is how alpha-go was built in the first place.) This option will produce much better results for far less effort, and again leaves no role for a human to do anything useful. The fact that a memorized opening book is useless has nothing to do with human vs computer vs cyborg. It has to do with the characteristics of the game. In chess, it is useful to memorize openings and both computers and humans do it. In Go it is a waste of energy, and it is a waste for both computers and humans.
  - dragontamer 7y ago
    > Instead put a tiny fraction of the computing power necessary to build this book into self-training. Do you believe that AlphaZero could continue to improve dramatically with another 6-months of training? Or if it can improve at another 6-months after that? At some point, the network will reach a local maximum, and it will be unable to improve beyond that. Characterizing AlphaZero's moves through big-data analysis is innately going to become useful as self-training plateaus. Even Google wasn't able to get more than a few months of training in before the plateau. At which point, it will be more reasonable to characterize the weaknesses of the network and build an opening book. Avoid the positions that the network was unable to learn about. Etc. etc. Opening books, at a minimum, would grossly improve AlphaZero's play at competitive levels. Anyone with an opening book of AlphaZero's mistakes will be able to push AlphaZero into a mistaken position.
    - jmcgough 7y ago
      > Anyone with an opening book of AlphaZero's mistakes will be able to push AlphaZero into a mistaken position. This only works if AlphaZero never retrains on previous games.

by btilly 7y ago

dragontamer 7y ago

> Go does not have openings. There are 381 opening moves in Go, but really only 96 because of symmetry. 96 (opening moves) x 380 responses x 379 x 378 x 377 == ~2 Trillion positions after 5 ply. These 2-trillion positions will easily fit in a 16TB hard drive for $400. That's 8-bytes per position, so you probably can get there with more symmetries and some compression applied. ---------- You're thinking too much like a human. There's no Go-openings in the age of Human-Go. But in the age of Cyborg-Go where 16TB hard drives are allowed, we can begin to exhaustively build openings. We even have a super-human AI that can automatically, and algorithmically, explore this opening book. We can build AWS-instances with V100 Tensor cores to use neural-nets to explore all of these positions now. ------------- > If you have alpha-go play itself a thousand times, it is unlikely that by move 10 you will wind up with the same board position twice. Alpha-Go doesn't seem to implement much randomness at all into the moves it plays. The source of randomness is in time-controls (AlphaGo may choose MoveX before 30 seconds of analysis, or MoveY after 30 seconds of analysis), but this is a fairly constrained number of moves. Play alphaGo by itself a thousand times, at precisely the same time MCTS-controls (say: 1-million nodes visited in the MCTS tree), and it will probably play the same game 1000 times in a row. This makes AlphaGo extremely prone to opening database "attacks". Which is why I am using opening books as an easy example for how to beat a particular AlphaGo network. At least, until AlphaGo updates its algorithm for more random play. If the goal is "Beat AlphaGo" in a game, then the opening book construction is far, far simpler. Even with random elements (ex: AlphaGo picks randomly from the top 10 best positions it generates), that is far more constrained than a full 381 x 380 x 379 x ... style opening book.

btilly 7y ago
You're thinking too much like a human. No, you're thinking too much like someone who understands chess but not Go. Suppose that we built an opening book with a trillion reasonable courses of action on it. Each one analyzed well. As you have discovered, you will only go a few ply into the game. And all of the positions that you will be directed towards will have only a small edge. Instead put a tiny fraction of the computing power necessary to build this book into self-training. You will get a better internal model and therefore a significantly stronger computer player. (That is how alpha-go was built in the first place.) This option will produce much better results for far less effort, and again leaves no role for a human to do anything useful. The fact that a memorized opening book is useless has nothing to do with human vs computer vs cyborg. It has to do with the characteristics of the game. In chess, it is useful to memorize openings and both computers and humans do it. In Go it is a waste of energy, and it is a waste for both computers and humans.
- dragontamer 7y ago
  > Instead put a tiny fraction of the computing power necessary to build this book into self-training. Do you believe that AlphaZero could continue to improve dramatically with another 6-months of training? Or if it can improve at another 6-months after that? At some point, the network will reach a local maximum, and it will be unable to improve beyond that. Characterizing AlphaZero's moves through big-data analysis is innately going to become useful as self-training plateaus. Even Google wasn't able to get more than a few months of training in before the plateau. At which point, it will be more reasonable to characterize the weaknesses of the network and build an opening book. Avoid the positions that the network was unable to learn about. Etc. etc. Opening books, at a minimum, would grossly improve AlphaZero's play at competitive levels. Anyone with an opening book of AlphaZero's mistakes will be able to push AlphaZero into a mistaken position.
  - jmcgough 7y ago
    > Anyone with an opening book of AlphaZero's mistakes will be able to push AlphaZero into a mistaken position. This only works if AlphaZero never retrains on previous games.