4 ms·

If the chess grandmaster makes an illegal move, than in that moment, they are not playing by the rules. It's definitional. The GM was, in that moment, operatin

If the chess grandmaster makes an illegal move, than in that moment, they are not playing by the rules. It's definitional. The GM was, in that moment, operating by some other model to evaluate the board state, likely a pattern recognition/statistical model that did not fully evaluate the state of the board. ChatGPT can, today, list all the rules of chess accurately. ChatGPT cannot play chess with any great degree of accuracy. "Knowing" the rules, in so much as the ability to list them out accurately, is not the same as following the rules. There's no reason to believe that humans operate as rules engines in all things, and a great deal of evidence (as with your example of chess GMs making mistakes) that they absolutely are not. That human intelligence is itself partially, and perhaps even largely, statistical in nature.

GaggiX 4y ago
>he is not playing by the rules. It's definitional. Sure but the question is: does the grandmaster know the rules? Because if what you said is true: >A model that "knows" the rules will make no errors, a model that makes any errors does not "know" the rules. Simple as. Then the grandmaster doesn't know the rules. "Simple as".
- nickelpro 4y ago
  The GM is not a game model of chess. The GM is a human. The GM does not evaluate their world according to the rules of chess at all times or in all things. The claim is that OthelloGPT is a game model of Othello. The only input it has is a move list, the only output it produces is a resulting move. Actual game models, such as Stockfish for chess, are incapable of producing invalid moves, it is not in their set of possible outputs. OthelloGPT is not a game model of Othello. It might be a game model of something that is not Othello, but this is hard to tell. The reliance on what humans can do is not a useful comparison here. Humans are also not game models of Othello. Actual game models, programs that model the rules of Othello, are the useful point of comparison. Again, claiming to know the rules, "knowledge" of them, and following those rules, acting according to them, are totally different things. The former is outside the scope, "what is uppercase-k Knowledge" is a question of philosophy. Demonstrably following the rules is the world of game models. If the semantics of putting the word "know" in quotes is what you're hung up on, fine, replace it with whatever pleases you: A model that "follows/encodes/exhibits" the rules will make no errors, a model that makes any errors does not "follow/encode/exhibit" the rules
  - GaggiX 4y ago
    When someone asks: "Does the LLM know the rules?" Then it's a simple binary answer "yes" or "no" depending if the model makes errors, in this case 0.01% so for you it's clearly no, when I ask: "Does the grandmaster know the rules?", then you dodge the question because you realize that what you are saying is not as simple as you want to make it out to be.

GaggiX 4y ago

>he is not playing by the rules. It's definitional. Sure but the question is: does the grandmaster know the rules? Because if what you said is true: >A model that "knows" the rules will make no errors, a model that makes any errors does not "know" the rules. Simple as. Then the grandmaster doesn't know the rules. "Simple as".

nickelpro 4y ago
The GM is not a game model of chess. The GM is a human. The GM does not evaluate their world according to the rules of chess at all times or in all things. The claim is that OthelloGPT is a game model of Othello. The only input it has is a move list, the only output it produces is a resulting move. Actual game models, such as Stockfish for chess, are incapable of producing invalid moves, it is not in their set of possible outputs. OthelloGPT is not a game model of Othello. It might be a game model of something that is not Othello, but this is hard to tell. The reliance on what humans can do is not a useful comparison here. Humans are also not game models of Othello. Actual game models, programs that model the rules of Othello, are the useful point of comparison. Again, claiming to know the rules, "knowledge" of them, and following those rules, acting according to them, are totally different things. The former is outside the scope, "what is uppercase-k Knowledge" is a question of philosophy. Demonstrably following the rules is the world of game models. If the semantics of putting the word "know" in quotes is what you're hung up on, fine, replace it with whatever pleases you: A model that "follows/encodes/exhibits" the rules will make no errors, a model that makes any errors does not "follow/encode/exhibit" the rules
- GaggiX 4y ago
  When someone asks: "Does the LLM know the rules?" Then it's a simple binary answer "yes" or "no" depending if the model makes errors, in this case 0.01% so for you it's clearly no, when I ask: "Does the grandmaster know the rules?", then you dodge the question because you realize that what you are saying is not as simple as you want to make it out to be.