3 ms·

Magnus Carlsen understands chess, a machine designed to simply predict his next move would not necessarily understand chess. This is essentially the Chinese Roo

Magnus Carlsen understands chess, a machine designed to simply predict his next move would not necessarily understand chess. This is essentially the Chinese Room experiment. So I think "word predictor" makes sense here. A word predictor can be really really cool.

munksbeer 6mo ago
What does "understand" even mean here? So many people arguing about this seem to assume they can just use words and everyone must accept that because the words have a certain connotation, their argument must be true. I have no idea how Magnus Carlsen "understands" chess. Neither does anyone else. His brain is giant neural net, taking inputs, sending signals around, and coming out with an output. We think we understand the mechanics of this, but we do not understand exactly why or how sending these signals around produces such good outputs. So to argue you know for certain that an LLM is not intelligent because it is "just" a next token predictor, without knowing if that is how the human brain operates, is thinking too highly of yourself.
- slopinthebag 6mo ago
  I don't have to try and imagine how Magnus Carlsen understands chess, since I also understand chess, and I operate with the assumption that other people are not zombies and possess a similar form of consciousness. My comment works regardless of the skill of the player. Imagine you have never played chess, you have no concept of the rules or how the game is played, yet you've learned the entirety of Stockfish's algorithms and can dutifully run them step by step on a piece of paper when you look at a chess position. You would be the strongest chess player ever, and yet you would have less understanding of the game than even a beginner. Just because you can take an input and produce an intelligent output does not mean there is any sort of underlying understanding. This is really just a modification of Searle's Chinese Room Argument, and one of the most famous refutations of functionalism. https://plato.stanford.edu/entries/chinese-room/ https://plato.stanford.edu/entries/chinese-room/
  - munksbeer 6mo ago
    Again, please can you explain what "understanding" means, without being self-referential.
    - slopinthebag 6mo ago
      No, I am not going to be getting into a debate over definitions of words that we both know the meaning of.
jeremyjh 6mo ago
It’s an incoherent argument, or a meaningless semantic distinction. There is no design of such a machine that does not encode a very deep understanding of the game. Leela Chess Zero does understand chess. She plays at roughly 2300 strength with a search depth of 1 ply - purely on the strength of her gestalt evaluation of the position. Humans have learned a lot about chess from studying her (and AlphaZero’s) games. General, transferable knowledge she developed herself about - for example - the long term value of early rook pawn advances. “Understanding” doesn’t imply anything about personhood or self reflection or awareness.
- slopinthebag 6mo ago
  It's not, and unfortunately you cannot just dismiss perhaps the greatest refutation of functionalism as being incoherent, you have to actually address the argument. Take a person (Fred) with no experience of knowledge of chess. They don't know how the game works, how the pieces move, or any of the rules. They memorise an algorithm, say how Leela does its search and evaluation, and they can then look at a position on a board, run the calculations, and come up with a move. Fred can now play chess really strongly, and simultaneously has no understanding of chess. Now in the original experiment it was a room with a person, and the person used a book to reply in Chinese. But the same idea applies. https://plato.stanford.edu/entries/chinese-room/ https://plato.stanford.edu/entries/chinese-room/
  - jeremyjh 6mo ago
    There is no algorithm that can be memorized. Leela's understanding is in the weights, not neural net algorithms. I'm familiar with the Chinese room argument and I've never accepted it because what it describes isn't real. It imagines some algorithm for which there is no evidence. Show me this process running and then ask me if it understands Chinese. To me this is as philosophically dubious as the notion of p-zombies.
    - slopinthebag 6mo ago
      You can actually do calculations of LLMs and models like Leela on paper or in your head if you had enough time (and patience)! It's basically just a whole lotta matrix multiplication. It's a thought experiment and its validity does not rest on the ability for someone to actually do these calculations in a suitable timespan. The specifics of the algorithm have no relevance. If you did see the process running, when asked would you say it understands Chinese?

munksbeer 6mo ago

What does "understand" even mean here? So many people arguing about this seem to assume they can just use words and everyone must accept that because the words have a certain connotation, their argument must be true. I have no idea how Magnus Carlsen "understands" chess. Neither does anyone else. His brain is giant neural net, taking inputs, sending signals around, and coming out with an output. We think we understand the mechanics of this, but we do not understand exactly why or how sending these signals around produces such good outputs. So to argue you know for certain that an LLM is not intelligent because it is "just" a next token predictor, without knowing if that is how the human brain operates, is thinking too highly of yourself.

slopinthebag 6mo ago
I don't have to try and imagine how Magnus Carlsen understands chess, since I also understand chess, and I operate with the assumption that other people are not zombies and possess a similar form of consciousness. My comment works regardless of the skill of the player. Imagine you have never played chess, you have no concept of the rules or how the game is played, yet you've learned the entirety of Stockfish's algorithms and can dutifully run them step by step on a piece of paper when you look at a chess position. You would be the strongest chess player ever, and yet you would have less understanding of the game than even a beginner. Just because you can take an input and produce an intelligent output does not mean there is any sort of underlying understanding. This is really just a modification of Searle's Chinese Room Argument, and one of the most famous refutations of functionalism. https://plato.stanford.edu/entries/chinese-room/ https://plato.stanford.edu/entries/chinese-room/
- munksbeer 6mo ago
  Again, please can you explain what "understanding" means, without being self-referential.
  - slopinthebag 6mo ago
    No, I am not going to be getting into a debate over definitions of words that we both know the meaning of.

jeremyjh 6mo ago

It’s an incoherent argument, or a meaningless semantic distinction. There is no design of such a machine that does not encode a very deep understanding of the game. Leela Chess Zero does understand chess. She plays at roughly 2300 strength with a search depth of 1 ply - purely on the strength of her gestalt evaluation of the position. Humans have learned a lot about chess from studying her (and AlphaZero’s) games. General, transferable knowledge she developed herself about - for example - the long term value of early rook pawn advances. “Understanding” doesn’t imply anything about personhood or self reflection or awareness.

slopinthebag 6mo ago
It's not, and unfortunately you cannot just dismiss perhaps the greatest refutation of functionalism as being incoherent, you have to actually address the argument. Take a person (Fred) with no experience of knowledge of chess. They don't know how the game works, how the pieces move, or any of the rules. They memorise an algorithm, say how Leela does its search and evaluation, and they can then look at a position on a board, run the calculations, and come up with a move. Fred can now play chess really strongly, and simultaneously has no understanding of chess. Now in the original experiment it was a room with a person, and the person used a book to reply in Chinese. But the same idea applies. https://plato.stanford.edu/entries/chinese-room/ https://plato.stanford.edu/entries/chinese-room/
- jeremyjh 6mo ago
  There is no algorithm that can be memorized. Leela's understanding is in the weights, not neural net algorithms. I'm familiar with the Chinese room argument and I've never accepted it because what it describes isn't real. It imagines some algorithm for which there is no evidence. Show me this process running and then ask me if it understands Chinese. To me this is as philosophically dubious as the notion of p-zombies.
  - slopinthebag 6mo ago
    You can actually do calculations of LLMs and models like Leela on paper or in your head if you had enough time (and patience)! It's basically just a whole lotta matrix multiplication. It's a thought experiment and its validity does not rest on the ability for someone to actually do these calculations in a suitable timespan. The specifics of the algorithm have no relevance. If you did see the process running, when asked would you say it understands Chinese?