3 ms·

Non-paywalled obit: https://www.theguardian.com/world/2025/oct/05/john-searle-obituary https://www.theguardian.com/world/2025/oct/05/john-searle-ob... His mos

Non-paywalled obit: https://www.theguardian.com/world/2025/oct/05/john-searle-obituary https://www.theguardian.com/world/2025/oct/05/john-searle-ob... His most famous argument: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

tasty_freeze 1y ago
I find the Chinese room argument to be nearly toothless. The human running around inside the room doing the translation work simply by looking up transformation rules in a huge rulebook may produce an accurate translation, but that human still doesn't know a lick of Chinese. Ergo (they claim) computers might simulate consciousness, but will never be conscious. But is the Searle room, the human is the equivalent of, say, ATP in the human brain. ATP powers my brain while I'm speaking English, but ATP doesn't know how to speak English just like the human in the Searle room doesn't know how to speak Chinese.
- slowmovintarget 1y ago
  There is no translation going on in that thought experiment, though. There is text processing. That is, the man in the room receives Chinese text through a slot in the door. He uses a book of complex instructions that tells him what to do with that text, and he produces more Chinese text as a response according to those instructions. Neither the man, nor the room "understand" Chinese. It is the same for the computer and its software. Jeffery Hinton has sad "but the system understands Chinese." I don't think that's a true statement, because at no point is the "system" dealing with semantic context of the input. It only operates algorithmically on the input, which is distinctly not what people do when they read something. Language, when conveyed between conscious individuals creates a shared model of the world. This can lead to visualizations, associations, emotions, creation of new memories because the meaning is shared. This does not happen with mere syntactic manipulation. That was Searle's argument.
  - randallsquared 1y ago
    > It only operates algorithmically on the input, which is distinctly not what people do when they read something. That's not at all clear! > Language, when conveyed between conscious individuals creates a shared model of the world. This can lead to visualizations, associations, emotions, creation of new memories because the meaning is shared. This does not happen with mere syntactic manipulation. That was Searle's argument. All of that is called into question with some LLM output. It's hard to understand how some of that could be produced without some emergency model of the world.
    - slowmovintarget 1y ago
      In the thought experiment as constructed it is abundantly clear. It's the point. LLM output doesn't call that into question at all. Token production through distance function in high-dimensional vector representation space of language tokens gets you a long way. It doesn't get you understanding. I'll take Penrose's notions that consciousness is not computation any day.
      - Cogito 1y ago
        Out of interest, what do you think it would look like if communicating was algorithmic? I know that it doesn't feel like I am doing anything particularly algorithmic when I communicate but I am not the hommunculus inside me shuffling papers around so how would I know?
        jacquesm 1y ago
        I think it would end inspiration.
        adastra22 1y ago
        Inspiration is what a search algorithm feels like from the inside.

tasty_freeze 1y ago

I find the Chinese room argument to be nearly toothless. The human running around inside the room doing the translation work simply by looking up transformation rules in a huge rulebook may produce an accurate translation, but that human still doesn't know a lick of Chinese. Ergo (they claim) computers might simulate consciousness, but will never be conscious. But is the Searle room, the human is the equivalent of, say, ATP in the human brain. ATP powers my brain while I'm speaking English, but ATP doesn't know how to speak English just like the human in the Searle room doesn't know how to speak Chinese.

slowmovintarget 1y ago
There is no translation going on in that thought experiment, though. There is text processing. That is, the man in the room receives Chinese text through a slot in the door. He uses a book of complex instructions that tells him what to do with that text, and he produces more Chinese text as a response according to those instructions. Neither the man, nor the room "understand" Chinese. It is the same for the computer and its software. Jeffery Hinton has sad "but the system understands Chinese." I don't think that's a true statement, because at no point is the "system" dealing with semantic context of the input. It only operates algorithmically on the input, which is distinctly not what people do when they read something. Language, when conveyed between conscious individuals creates a shared model of the world. This can lead to visualizations, associations, emotions, creation of new memories because the meaning is shared. This does not happen with mere syntactic manipulation. That was Searle's argument.
- randallsquared 1y ago
  > It only operates algorithmically on the input, which is distinctly not what people do when they read something. That's not at all clear! > Language, when conveyed between conscious individuals creates a shared model of the world. This can lead to visualizations, associations, emotions, creation of new memories because the meaning is shared. This does not happen with mere syntactic manipulation. That was Searle's argument. All of that is called into question with some LLM output. It's hard to understand how some of that could be produced without some emergency model of the world.
  - slowmovintarget 1y ago
    In the thought experiment as constructed it is abundantly clear. It's the point. LLM output doesn't call that into question at all. Token production through distance function in high-dimensional vector representation space of language tokens gets you a long way. It doesn't get you understanding. I'll take Penrose's notions that consciousness is not computation any day.
    - Cogito 1y ago
      Out of interest, what do you think it would look like if communicating was algorithmic? I know that it doesn't feel like I am doing anything particularly algorithmic when I communicate but I am not the hommunculus inside me shuffling papers around so how would I know?
      - jacquesm 1y ago
        I think it would end inspiration.
        adastra22 1y ago
        Inspiration is what a search algorithm feels like from the inside.