4 ms·

Electrical signals are not the same as subjective experiences. While a machine may be able to record and play back these signals for humans to experience, that

Electrical signals are not the same as subjective experiences. While a machine may be able to record and play back these signals for humans to experience, that does not imply that the experiences themselves are recorded nor that the machine has any access to them. A deaf person can use a tape recorder to record and play back a symphony but that does not encode the experience in any way the deaf person could share.

mietek 2y ago
That’s some strong claims, given that philosophers (e.g. Chalmers vs Dennett) can’t even agree whether subjective experiences exist or not.
- chongli 2y ago
  Even if you’re a pure Dennettian functionalist you still commit to a functional difference between signals in transit (or at rest) and signals being processed and interpreted. Holding a cassette tape with a recording of a symphony is not the same as hearing the symphony. Applying this case to AI gives rise to the Chinese Room argument. LLMs’ propensity for hallucinations invite this comparison.
  - mietek 2y ago
    Are LLMs having subjective experiences? Surely not. But if you claim that human subjective experiences are not the result of electrical signals in the brain, then what exactly is your position? Dualism? Personally, I think the Chinese room argument is invalid. In order for the person in the room to respond to any possible query by looking up the query in a book, the book would need to be infinite and therefore impossible as a physical object. Otherwise, if the book is supposed to describe an algorithm for the person to follow in order to compute a response, then that algorithm is the intelligent entity that is capable of understanding, and the person in the room is merely the computational substrate.
    - chongli 2y ago
      The Chinese Room is a perfect analogy for what's going on with LLMs. The book is not infinite, it's flawed. And that's the point: we keep bumping into the rough edges of LLMs with their hallucinations and faulty reasoning because the book can never be complete. Thus we keep getting responses that make us realize the LLM is not intelligent and has no idea what it's saying. The only part where the book analogy falls down has to do with the technical implementation of LLMs, with their tokenization and their vast sets of weights. But that is merely an encoding for the training data. Books can be encoded similarly by using traditional compression algorithms (like LZMA).
      - famouswaffles 2y ago
        >The book is not infinite, it's flawed. Oh and the human book is surely infinite and unflawed right ? >we keep bumping into the rough edges of LLMs with their hallucinations and faulty reasoning Both things humans also do in excess The Chinese Room is nonsensical. Can you point to any part of your brain that understands English ? I guess you are a Chinese Room then.
        chongli 2y ago
        Humans have the ability to admit when they do not know something. We say “sorry, I don’t know, let me get back to you.” LLMs cannot do this. They either have the right answer in the book or they make up nonsense (hallucinate). And they do not even know which one they’re doing!
        famouswaffles 2y ago
        >Humans have the ability to admit when they do not know something. No not really. It's not even rare that a human confidently says and believes something and really has no idea what he/she's talking about. >We say “sorry, I don’t know, let me get back to you.” LLMs cannot do this Yeah they can. And they can do it much better than chance. They just don't do it as well as humans. >And they do not even know which one they’re doing! There's plenty of research that suggests this is the case. https://news.ycombinator.com/item?id=41418486 https://news.ycombinator.com/item?id=41418486

mietek 2y ago

That’s some strong claims, given that philosophers (e.g. Chalmers vs Dennett) can’t even agree whether subjective experiences exist or not.

chongli 2y ago
Even if you’re a pure Dennettian functionalist you still commit to a functional difference between signals in transit (or at rest) and signals being processed and interpreted. Holding a cassette tape with a recording of a symphony is not the same as hearing the symphony. Applying this case to AI gives rise to the Chinese Room argument. LLMs’ propensity for hallucinations invite this comparison.
- mietek 2y ago
  Are LLMs having subjective experiences? Surely not. But if you claim that human subjective experiences are not the result of electrical signals in the brain, then what exactly is your position? Dualism? Personally, I think the Chinese room argument is invalid. In order for the person in the room to respond to any possible query by looking up the query in a book, the book would need to be infinite and therefore impossible as a physical object. Otherwise, if the book is supposed to describe an algorithm for the person to follow in order to compute a response, then that algorithm is the intelligent entity that is capable of understanding, and the person in the room is merely the computational substrate.
  - chongli 2y ago
    The Chinese Room is a perfect analogy for what's going on with LLMs. The book is not infinite, it's flawed. And that's the point: we keep bumping into the rough edges of LLMs with their hallucinations and faulty reasoning because the book can never be complete. Thus we keep getting responses that make us realize the LLM is not intelligent and has no idea what it's saying. The only part where the book analogy falls down has to do with the technical implementation of LLMs, with their tokenization and their vast sets of weights. But that is merely an encoding for the training data. Books can be encoded similarly by using traditional compression algorithms (like LZMA).
    - famouswaffles 2y ago
      >The book is not infinite, it's flawed. Oh and the human book is surely infinite and unflawed right ? >we keep bumping into the rough edges of LLMs with their hallucinations and faulty reasoning Both things humans also do in excess The Chinese Room is nonsensical. Can you point to any part of your brain that understands English ? I guess you are a Chinese Room then.
      - chongli 2y ago
        Humans have the ability to admit when they do not know something. We say “sorry, I don’t know, let me get back to you.” LLMs cannot do this. They either have the right answer in the book or they make up nonsense (hallucinate). And they do not even know which one they’re doing!
        famouswaffles 2y ago
        >Humans have the ability to admit when they do not know something. No not really. It's not even rare that a human confidently says and believes something and really has no idea what he/she's talking about. >We say “sorry, I don’t know, let me get back to you.” LLMs cannot do this Yeah they can. And they can do it much better than chance. They just don't do it as well as humans. >And they do not even know which one they’re doing! There's plenty of research that suggests this is the case. https://news.ycombinator.com/item?id=41418486 https://news.ycombinator.com/item?id=41418486