5 ms·

Again, not enough for 1400. 800 maybe.

Jensson 4y ago
We have evidence right here that patterns of moves leads to 1400. To me this is evidence that 1400 level play is very rigid and mostly relies on defined openings and strategies, making it easy for an LLM to find similar games and win using standard continuations. Just like grammar the patterns are too hard for humans to see and encode, but LLMs can encode pretty complex patterns. Domains that are easy to encode as grammars will be really easy for LLMs to solve, and the further from a grammar the harder for it.
- SanderNL 4y ago
  At what point can we just say that understanding “patterns of moves” is understanding chess? It seems you suggest there is more to it, but maybe I am mistaken.
  - Jensson 4y ago
    At least it should make valid moves, that is the minimum level required. It didn't reach that level here. If it never made illegal moves we could talk and see what it does, but until then we can be sure it didn't understand the rules.
    - serverholic 4y ago
      I don’t understand why the threshold is “never”. Isn’t it entirely possible that the AI is learning a model of chess but this model is imperfect? What if AIs don’t fail the same way as humans?
      - Jensson 4y ago
        > What if AIs don’t fail the same way as humans? But it is failing the same way as a human. Humans who remembers patterns and don't learn the logic makes these kind of errors in math or logic all the time. ChatGPT is much better than humans at pattern matching, you see it right here it can pattern match chess moves and win games! But its inability to apply logic to its output instead of just pattern matching is holding it back, as long as that isn't solved it wont be able to perform on the level of humans in many tasks. Chess might be easy enough to solve using just pattern matching and no logic that scaling it up will make it pretty good at chess, but many other topics wont be.
  - meh8881 4y ago
    Ask it to evaluate moves and explain why they are or are not good

We have evidence right here that patterns of moves leads to 1400. To me this is evidence that 1400 level play is very rigid and mostly relies on defined openings and strategies, making it easy for an LLM to find similar games and win using standard continuations. Just like grammar the patterns are too hard for humans to see and encode, but LLMs can encode pretty complex patterns. Domains that are easy to encode as grammars will be really easy for LLMs to solve, and the further from a grammar the harder for it.

SanderNL 4y ago
At what point can we just say that understanding “patterns of moves” is understanding chess? It seems you suggest there is more to it, but maybe I am mistaken.
- Jensson 4y ago
  At least it should make valid moves, that is the minimum level required. It didn't reach that level here. If it never made illegal moves we could talk and see what it does, but until then we can be sure it didn't understand the rules.
  - serverholic 4y ago
    I don’t understand why the threshold is “never”. Isn’t it entirely possible that the AI is learning a model of chess but this model is imperfect? What if AIs don’t fail the same way as humans?
    - Jensson 4y ago
      > What if AIs don’t fail the same way as humans? But it is failing the same way as a human. Humans who remembers patterns and don't learn the logic makes these kind of errors in math or logic all the time. ChatGPT is much better than humans at pattern matching, you see it right here it can pattern match chess moves and win games! But its inability to apply logic to its output instead of just pattern matching is holding it back, as long as that isn't solved it wont be able to perform on the level of humans in many tasks. Chess might be easy enough to solve using just pattern matching and no logic that scaling it up will make it pretty good at chess, but many other topics wont be.
- meh8881 4y ago
  Ask it to evaluate moves and explain why they are or are not good