52 ms·

It could handle 300 moves or so with that many tokens, seems like plenty. I propose that if you gathered enough chess transcripts like this: e4 e5 Nc3 Nc6 f4

It could handle 300 moves or so with that many tokens, seems like plenty. I propose that if you gathered enough chess transcripts like this: e4 e5 Nc3 Nc6 f4 exf4 d4 d5 Bxf4 Bb4 exd5 Qxd5 Kf2 Qh4+ .... And fed them into a blank GPT then it would learn chess like a language and be able to make legal moves most of the time. It would do this by inferring and modelling the board and the rules. It wouldn't be a great player but it would be able to make moves. This is bascially what the Othello paper I linked above is all about. They used GPT-2 I think. Chess is harder but I reckon could be done with a bigger model and more training data.

Retric 4y ago
Sure often playing valid moves is possible, but not only is it moving the goalposts but different approaches actually produce skilled players using neural networks. Anyway, my point was less about the game than how its failure show what’s going on better than it’s successes. The GPT approach is optimized for chat bots and its successes have more to do with exploiting how we approach communication than anything that can turn into AGI.
- codeulike 4y ago
  I'm not moving the goalposts. You said: "It’s not operating from some model of the game" I'm saying: "it would model the rules of chess, if you fed it enough game transcripts"
  - Retric 4y ago
    If it can’t strictly play valid movies then it isn’t modeling the game. Instead it’s modeling something else which is somewhat related to the game. Aka someone playing tick tack toe who moves on top of their opponent’s move isn’t playing tick tack toe.
    - codeulike 4y ago
      If a human had only ever seen Othello moves in notation form and never seen the board and had to infer the rules, they'd probably be about 99.99% accurate. They might make a mistake about 1 move in 1000 - fail to spot something, or encounter some edge case of rules they hadn't been able to infer. That's how accurate GPT-2 was.

Retric 4y ago

Sure often playing valid moves is possible, but not only is it moving the goalposts but different approaches actually produce skilled players using neural networks. Anyway, my point was less about the game than how its failure show what’s going on better than it’s successes. The GPT approach is optimized for chat bots and its successes have more to do with exploiting how we approach communication than anything that can turn into AGI.

codeulike 4y ago
I'm not moving the goalposts. You said: "It’s not operating from some model of the game" I'm saying: "it would model the rules of chess, if you fed it enough game transcripts"
- Retric 4y ago
  If it can’t strictly play valid movies then it isn’t modeling the game. Instead it’s modeling something else which is somewhat related to the game. Aka someone playing tick tack toe who moves on top of their opponent’s move isn’t playing tick tack toe.
  - codeulike 4y ago
    If a human had only ever seen Othello moves in notation form and never seen the board and had to infer the rules, they'd probably be about 99.99% accurate. They might make a mistake about 1 move in 1000 - fail to spot something, or encounter some edge case of rules they hadn't been able to infer. That's how accurate GPT-2 was.