17 ms·

> If it takes a room of robots following simple instructions a billion years to act as an AI that reads Chinese, are those robots a collective intelligence? Can

> If it takes a room of robots following simple instructions a billion years to act as an AI that reads Chinese, are those robots a collective intelligence? Can it be scaled up to consciousness? No, because of the article you linked to. Chinese room argues AGAINST the existence of consciousness/intelligence arising from following symbolic instructions. Here is a quote I like by John Searle: > Computational models of consciousness are not sufficient by themselves for consciousness. The computational model for consciousness stands to consciousness in the same way the computational model of anything stands to the domain being modelled. Nobody supposes that the computational model of rainstorms in London will leave us all wet. But they make the mistake of supposing that the computational model of consciousness is somehow conscious. It is the same mistake in both cases.

hackinthebochs 5y ago
Searle's argument misses the mark. A simulated rainstorm wont leave us wet, but a simulation of a disordered state is (or contains) a disordered state. If consciousness is a relational property of a system, then a simulation of such a relational property will instantiate that relational property without qualification. The Chinese room argument can't address this conception of consciousness.
- jimbokun 5y ago
  > If consciousness is a relational property of a system But first you have to establish this is true.
  - hackinthebochs 5y ago
    Not really. The point is that as an argument against computationalism about the mind, Searle's Chinese room can't rule out computationalism if mind is merely a relational property of a system.
- mjburgess 5y ago
  The argument is against this view of consciousness. More intuitively: our words are things which do things in the world. When we acquire them we do so by being embedded in the world. When we use them we use them to change the world. (Largely). If a child in a cupboard reading a textbook out-loud could fool a chemist that the child understood chemistry, it doesn't mean they do. In the end, you need to turn on that bunsen burner. The chinese room needs to be taken super-literally: does the person just doing a "hash-table lookup" on "the right answers to pre-prosed questions" actually understand. To me its incredibly obvious that they do not. This does not establish that, eg. cognition, does not involve computational hastable-like processes. I think it does establish that propontents of a naive computationalism have an incredibly big challenge. The challenge is to specify the environment computationally, since that is the only means by which the computer itself can be sufficiently embedded and actually understand anything. The issue here is that if "computer" isnt just a trivial property it has to include, eg., measurability, determinism, etc. Properties that reality lacks. Reality isnt a computer. Starting with the chinese room one /can/ supplement sufficiently until we reach searle's conclusion.
  - mananaysiempre 5y ago
    ... I’m finding it very difficult to phrase this argument about actually understanding in a way that wouldn’t also entail that chairs made of atoms (and—overwhelmingly—empty space) don’t actually exist, they are mere simulacra of real chairs (which don’t exist anywhere). Which might be a consistent position, but I’d instead say that it is simply a largely useless interpretation of the word actually and as good philosophers we should search for a better one. (See also Anderson’s “More is different”.) Or is there such a phrasing?
    - mjburgess 5y ago
      Approximately, You "actually understand" if in saying words, w, about a situation, s, the reason, r, you say those words counts as as an understanding of s. So if you're asked, "what is the weather today?" and a system replies, "its sunny" -- it only understands if the reason it replied was because its saying "sunny" had come about because it understands sunny sitations; and therefore has a justfiied reason on this occasion to say that this occasion is sunny. If I ask a NN trained on a trillion documents, "do you like what I'm wearing?" it cannot answer. It can say the words, "yes I do!" but it cannot have a reason to say those words. It's just a "weighted average retrievla across a compressed dataset of a trillion documetns". The question isnt "on average, what -- historically -- would a generic person reply to the question: do you like what i'm wearing?" The question is if the systems likes what I'm wearing. Its replies are only "actual replies" if it can see what i'm wearing, has experiences/preferences for taste, has some aesthetic judgement, has a disposition to like/dislike -- etc. No ML system on the planet, in this sense, has any understanding whatsoever. Interpolation over historical data is just a means of compressing history into parameters, ie., its a compressed lookup table. That isn't ever a /reason/, exactly, if a person simply used such a table -- they wouldnt mean what they said
mananaysiempre 5y ago
The Chinese room was originally an argument against the consciousness of automatons, but then EPR was an argument against (fundamental) entanglement, Bell more or less expected his inequality to hold, Michaelson–Morley was interpreted as having found the luminiferous aether, and the Poisson (aka Arago) spot was a reductio ad absurdum of the wave theory of light. All of these things aren’t obsolete, they are still useful pieces of insight, but the state of the art regarding how they should be conceptualized has moved on from what their originators thought. Which is not an argument for any particular intepretation of the Chinese room, it’s only to say that the bare fact that the original author thought of it in a particular way doesn’t mean we should do so.
retrac 5y ago
The Chinese Room is useful as an idea or model, even if you don't agree with the original interpretation. I am fond of the view that the room is potentially conscious, with human or machine worker. The worker should expect to understand the conversation they are mechanically carrying out no more than your individual brain cells ought to expect to understand.

hackinthebochs 5y ago

Searle's argument misses the mark. A simulated rainstorm wont leave us wet, but a simulation of a disordered state is (or contains) a disordered state. If consciousness is a relational property of a system, then a simulation of such a relational property will instantiate that relational property without qualification. The Chinese room argument can't address this conception of consciousness.

jimbokun 5y ago
> If consciousness is a relational property of a system But first you have to establish this is true.
- hackinthebochs 5y ago
  Not really. The point is that as an argument against computationalism about the mind, Searle's Chinese room can't rule out computationalism if mind is merely a relational property of a system.
mjburgess 5y ago
The argument is against this view of consciousness. More intuitively: our words are things which do things in the world. When we acquire them we do so by being embedded in the world. When we use them we use them to change the world. (Largely). If a child in a cupboard reading a textbook out-loud could fool a chemist that the child understood chemistry, it doesn't mean they do. In the end, you need to turn on that bunsen burner. The chinese room needs to be taken super-literally: does the person just doing a "hash-table lookup" on "the right answers to pre-prosed questions" actually understand. To me its incredibly obvious that they do not. This does not establish that, eg. cognition, does not involve computational hastable-like processes. I think it does establish that propontents of a naive computationalism have an incredibly big challenge. The challenge is to specify the environment computationally, since that is the only means by which the computer itself can be sufficiently embedded and actually understand anything. The issue here is that if "computer" isnt just a trivial property it has to include, eg., measurability, determinism, etc. Properties that reality lacks. Reality isnt a computer. Starting with the chinese room one /can/ supplement sufficiently until we reach searle's conclusion.
- mananaysiempre 5y ago
  ... I’m finding it very difficult to phrase this argument about actually understanding in a way that wouldn’t also entail that chairs made of atoms (and—overwhelmingly—empty space) don’t actually exist, they are mere simulacra of real chairs (which don’t exist anywhere). Which might be a consistent position, but I’d instead say that it is simply a largely useless interpretation of the word actually and as good philosophers we should search for a better one. (See also Anderson’s “More is different”.) Or is there such a phrasing?
  - mjburgess 5y ago
    Approximately, You "actually understand" if in saying words, w, about a situation, s, the reason, r, you say those words counts as as an understanding of s. So if you're asked, "what is the weather today?" and a system replies, "its sunny" -- it only understands if the reason it replied was because its saying "sunny" had come about because it understands sunny sitations; and therefore has a justfiied reason on this occasion to say that this occasion is sunny. If I ask a NN trained on a trillion documents, "do you like what I'm wearing?" it cannot answer. It can say the words, "yes I do!" but it cannot have a reason to say those words. It's just a "weighted average retrievla across a compressed dataset of a trillion documetns". The question isnt "on average, what -- historically -- would a generic person reply to the question: do you like what i'm wearing?" The question is if the systems likes what I'm wearing. Its replies are only "actual replies" if it can see what i'm wearing, has experiences/preferences for taste, has some aesthetic judgement, has a disposition to like/dislike -- etc. No ML system on the planet, in this sense, has any understanding whatsoever. Interpolation over historical data is just a means of compressing history into parameters, ie., its a compressed lookup table. That isn't ever a /reason/, exactly, if a person simply used such a table -- they wouldnt mean what they said

mananaysiempre 5y ago

The Chinese room was originally an argument against the consciousness of automatons, but then EPR was an argument against (fundamental) entanglement, Bell more or less expected his inequality to hold, Michaelson–Morley was interpreted as having found the luminiferous aether, and the Poisson (aka Arago) spot was a reductio ad absurdum of the wave theory of light. All of these things aren’t obsolete, they are still useful pieces of insight, but the state of the art regarding how they should be conceptualized has moved on from what their originators thought. Which is not an argument for any particular intepretation of the Chinese room, it’s only to say that the bare fact that the original author thought of it in a particular way doesn’t mean we should do so.

retrac 5y ago

The Chinese Room is useful as an idea or model, even if you don't agree with the original interpretation. I am fond of the view that the room is potentially conscious, with human or machine worker. The worker should expect to understand the conversation they are mechanically carrying out no more than your individual brain cells ought to expect to understand.