4 ms·

I recall reading about Searle's Chinese Room argument in Daniel Dennett's "Consciousness Explained" about 25 years ago. Maybe the exposition there was more comp

I recall reading about Searle's Chinese Room argument in Daniel Dennett's "Consciousness Explained" about 25 years ago. Maybe the exposition there was more complete than the version given in this article, but the argument of this article is terrible. The author says that the English speaker who simply manipulates symbols and follows rules will never get the joke written in Chinese, even if the people external to the room understand it and think it was produced by an intelligence that understands the joke. But that contains the assumption that the human is the consciousness in that arrangement, when in fact the human is just the energy source which drives the hardware. One might as well say that computers can never create a 3D drawing because its power supply doesn't understand arithmetic.

dboreham 6y ago
Ugh. I remember listening to Searle on the BBC in the early 1980s, and becoming quite irate, as a 17 year old does, at his mistake in defining the boundary within which intelligence should be found as the operator not the whole room.
- variaga 6y ago
  You can make a pretty good argument that the _rulebook_ is the Chinese-speaking intelligence in the room. "But it's an inanimate object, if the human stops following the rules it will be inert". Yeah, and if your mitochondria stop producing ATP you'll be inert too. Also. People taking Seale's position rarely reckon with just how big that rulebook would be.
  - wrs 6y ago
    Daniel Dennett coined the term “intuition pump” for this, which is a great concept to keep in mind so you don’t get caught by one. Wikipedia: An argument “designed to elicit intuitive but incorrect answers by formulating the description in such a way that important implications of the experiment would be difficult to imagine and tend to be ignored.”
  - 3PS 6y ago
    Scott Aaronson has an excellent take on the Chinese room argument in my opinion, namely that the existence of intelligence can be inferred from asymptotic space complexity. If you have a rulebook which lists every possible sentence and optimal response, the size of that book is clearly exponential in the size of the longest possible input. On the other hand, if you could construct an algorithm which can "understand" Chinese using only polynomial space and runtime, then it seems a lot more intuitively clear that there is something genuinely intelligent happening.
dnautics 6y ago
I think the problem with the Chinese room is that if you have something that can "mechanically" translate via algorithm, maybe that means it already understands. It's like saying a blind person cannot understand anything about the world because they cannot see. But to the extent that the blind person can successfully navigate the environment belies a sufficient understanding of that domain.
wrs 6y ago
The “Berkeley ‘systems argument’”, i.e., the room understands, not the human (why Berkeley gets credit for this obvious counterargument I don’t know) is mentioned in the paper, but not refuted other than by some confusing thought experiment involving a joke that just seems to repeat the original argument.
tialaramex 6y ago
It also mentions Harnad, whose classes I attended many years ago. It's still unclear to me what the point of this type of philosophy is though. Perhaps its practitioners find it entertaining? I haven't the faintest idea why we'd fund this more than say, people who want to make giant stone heads and leave them in the city's parks or people who make funny videos about their cats. It just goes around in circles, finding new ways to make the same (dualist) argument that I considered and rejected as an infant. I don't think we should ban it (or the giant stone head people) but it seems like it could be done independently on their own time without troubling society for funding. Harnad at least had done/ been doing something else interesting and important unrelated to his philosophical musing - rebelling against the publisher monopoly on research output. Though he taught classes about Cognitive Science, he was partly at the University because of his radical thinking of how scholarly communication should work, in the form of the journal BBS (Behavioural and Brain Sciences). This was of course way ahead of its time, the things BBS struggled to do for small numbers of scholarly articles in the 1970s can now be easily replicated online for any topic you can think of.

dboreham 6y ago

Ugh. I remember listening to Searle on the BBC in the early 1980s, and becoming quite irate, as a 17 year old does, at his mistake in defining the boundary within which intelligence should be found as the operator not the whole room.

variaga 6y ago
You can make a pretty good argument that the _rulebook_ is the Chinese-speaking intelligence in the room. "But it's an inanimate object, if the human stops following the rules it will be inert". Yeah, and if your mitochondria stop producing ATP you'll be inert too. Also. People taking Seale's position rarely reckon with just how big that rulebook would be.
- wrs 6y ago
  Daniel Dennett coined the term “intuition pump” for this, which is a great concept to keep in mind so you don’t get caught by one. Wikipedia: An argument “designed to elicit intuitive but incorrect answers by formulating the description in such a way that important implications of the experiment would be difficult to imagine and tend to be ignored.”
- 3PS 6y ago
  Scott Aaronson has an excellent take on the Chinese room argument in my opinion, namely that the existence of intelligence can be inferred from asymptotic space complexity. If you have a rulebook which lists every possible sentence and optimal response, the size of that book is clearly exponential in the size of the longest possible input. On the other hand, if you could construct an algorithm which can "understand" Chinese using only polynomial space and runtime, then it seems a lot more intuitively clear that there is something genuinely intelligent happening.

dnautics 6y ago

I think the problem with the Chinese room is that if you have something that can "mechanically" translate via algorithm, maybe that means it already understands. It's like saying a blind person cannot understand anything about the world because they cannot see. But to the extent that the blind person can successfully navigate the environment belies a sufficient understanding of that domain.

wrs 6y ago

The “Berkeley ‘systems argument’”, i.e., the room understands, not the human (why Berkeley gets credit for this obvious counterargument I don’t know) is mentioned in the paper, but not refuted other than by some confusing thought experiment involving a joke that just seems to repeat the original argument.

tialaramex 6y ago

It also mentions Harnad, whose classes I attended many years ago. It's still unclear to me what the point of this type of philosophy is though. Perhaps its practitioners find it entertaining? I haven't the faintest idea why we'd fund this more than say, people who want to make giant stone heads and leave them in the city's parks or people who make funny videos about their cats. It just goes around in circles, finding new ways to make the same (dualist) argument that I considered and rejected as an infant. I don't think we should ban it (or the giant stone head people) but it seems like it could be done independently on their own time without troubling society for funding. Harnad at least had done/ been doing something else interesting and important unrelated to his philosophical musing - rebelling against the publisher monopoly on research output. Though he taught classes about Cognitive Science, he was partly at the University because of his radical thinking of how scholarly communication should work, in the form of the journal BBS (Behavioural and Brain Sciences). This was of course way ahead of its time, the things BBS struggled to do for small numbers of scholarly articles in the 1970s can now be easily replicated online for any topic you can think of.