3 ms·

The Chinese Room is a really, really bad thought experiment. It's just an attempt at misdirection. It asks you to imagine a computer (one that operates on slips

by emtel 5y ago

The Chinese Room is a really, really bad thought experiment. It's just an attempt at misdirection. It asks you to imagine a computer (one that operates on slips of paper according to rules that are written down in a big book), puts a human inside that computer, and then says that because the human doesn't understand what the computer program is doing, the computer program isn't conscious. But this is just question-begging. The experiment purports to resolve the question "can a computer be conscious", and answers it by showing you a computer and claiming that it is not conscious. Worse, the Chinese Room is straight-up Cartesian dualism. By placing the human inside the computer, Searle is preying on your Cartesian intuition that there must be a theatre of the mind where reality is experienced. And since Searle's homunculus doesn't understand Chinese, nothing can be experienced, therefore no conscious experience is happening in this system. If I'm wrong, and the Chinese Room thought experiment is conclusive, it must be possible to reformulate it without the human. After all, the human does nothing but follow rules in a purely mechanistic fashion. Can anyone offer a reformulation of the experiment, that has no human, that they still feel is convincing?

joshuamorton 5y ago
I don't see how this follows at all. The human is necessary to the argument. The argument, briefly is "a human with a Chinese dictionary and translation instructions doesn't 'know' Chinese, therefore a computer with a Chinese dictionary and translation instructions also doesn't 'know' Chinese." It isn't an argument that no computer can be conscious, but an argument that a convincing simulation of consciousness is not sufficient to prove consciousness.
- emtel 5y ago
  The conscious human is not a necessary part of the system, as he is just blindly following rules. Therefore, his lack of comprehension tells us nothing about the system in question. I am sorry to mostly repeat myself. I am trying to think of another way to make my point, and coming up blank.
  - joshuamorton 5y ago
    > The conscious human is not a necessary part of the system as he is just blindly following rules That...that's the whole point. If the human is just blindly following rules, so is the computer!
    - emtel 5y ago
      Yes, which is why I say that the argument is circular. It assumes its conclusion. Either a computer program can "understand" Chinese or it can't. If it can't, we're done. If it can, then having a non-chinese-speaking human simulate the program won't cause it to stop understanding Chinese. The presence of the human doesn't expose a contradiction in the assumption "a computer program can understand chinese".
    - kbelder 5y ago
      Maybe there's a small human inside the human's brain, not understanding the rules the human is following, and it's only the innermost human that is really conscious? I think Searle's Chinese room argument is absolutely nonsensical, a bit like arguing about philosophical zombies. It's an argument that only makes any sense if you already are committed to mysticism or dualism.
      - atoav 5y ago
        > Maybe there's a small human inside the human's brain, not understanding the rules the human is following, and it's only the innermost human that is really conscious? That would be the materialist idea of there being no soul, consciousess etc — just complex biochemical processes happening in matter. This is however not a satisfying answer usually because it fails to explain the consistent experience of ourselves, the worlds we can imagine, remember or think about, the feelings that haunt us etc. If there is only material, where is our consciousness? Apparently our consciousness somehow emerges from that material, but how would a conscious computer then look? Would there be a difference between a computer that simulates conscious behaviour and one that shows the same behaviour but actually "has" conscious in it somewhere?
antognini 5y ago
You raise good questions, but I would really suggest listening through the lectures because Searle addresses all your points in a lot of detail. (It's a lot of material, but it's well worth it and he's an entertaining lecturer.) It's actually a bit funny that you bring up Cartesian dualism because Searle is pretty clear from the outset that he's of the opinion that Descartes's dualism was the worst thing ever to happen to philosophy. He certainly has no truck with it. The question Searle is raising with the thought experiment is where the understanding of Chinese is happening. It's clear that the understanding is not happening with the homunculus. But if not there, then where is it happening? Some functionalists responded with the "systems response" which is that the system of the room plus the homunculus is what has the understanding even though no element of the system taken individually has understanding. He goes through a pretty detailed reply to the systems response, but the basic argument against it is that it's incoherent. There's no well defined thing that is the system. You can remove the man from the room and put him under a tent in a field. Where does the system end and the rest of the world begin? You are of course correct that the experiment can be reformulated without the human, but that's exactly the point. A computer is mechanistically doing exactly the same thing as the man in the room. But if the man has no understanding and the room has no understanding and the system has no understanding, removing the human from the equation changes nothing. The reason Searle included the human is because saying "computer" invokes a lot of mystery to a lot of people as to what's going on. But a computer is doing nothing different than the man in the room.
- emtel 5y ago
  I've read one of Searle's popular books (Rediscovery of the Mind) and many other shorter writings. I'm aware he rejects dualism - my point is that the Chinese Room argument wouldn't even make sense to someone who had never been contaminated by dualist notions. Searle rejects it, but it has crept back in, imo. His arguments against the "systems response" have to stand or fall on their own. The Chinese room doesn't have anything to add. If the "systems response" fails, then we already know a computer can't be conscious without any additional thought experiments. Lastly, I think he goes much too far in dispelling the mystery of the word "computer". As Dennett and others have pointed out, a working chinese room that displayed clear signs of consciousness would be an artifact far beyond current human comprehension, either in scale or sophistification or both. Searle asks you to imagine a human taking slips of paper and calmly consulting a rule book and producing new slips of paper, and claims that it is "obvious" that this system can't be conscious. Well, I agree, but it's equally obvious that the system couldn't possibly display conscious-seeming behavior. The best defense of the CR that I've heard is that it is only intended to prove that a _purely formal_ system (i.e. GOFAI) can't be conscious. I find this more plausible, but I still don't think the CR adds anything to that argument. And, to reiterate the previous paragraph, I think if you had a formal system that everyone agreed _seemed_ conscious, it would be such a complex artifact that it wouldn't be obviously unconscious in the way that, say, an eliza bot is.
Ataraxiaist 5y ago
I think at some point in the future studying consciousness will be as nonsensical as studying the soul. I think that is all we really mean anyway and it is just wrapped up in different packaging. Soul/spirit/consciousness all the same thing. Chasing the same ghost.

by emtel 5y ago

joshuamorton 5y ago

I don't see how this follows at all. The human is necessary to the argument. The argument, briefly is "a human with a Chinese dictionary and translation instructions doesn't 'know' Chinese, therefore a computer with a Chinese dictionary and translation instructions also doesn't 'know' Chinese." It isn't an argument that no computer can be conscious, but an argument that a convincing simulation of consciousness is not sufficient to prove consciousness.

emtel 5y ago
The conscious human is not a necessary part of the system, as he is just blindly following rules. Therefore, his lack of comprehension tells us nothing about the system in question. I am sorry to mostly repeat myself. I am trying to think of another way to make my point, and coming up blank.
- joshuamorton 5y ago
  > The conscious human is not a necessary part of the system as he is just blindly following rules That...that's the whole point. If the human is just blindly following rules, so is the computer!
  - emtel 5y ago
    Yes, which is why I say that the argument is circular. It assumes its conclusion. Either a computer program can "understand" Chinese or it can't. If it can't, we're done. If it can, then having a non-chinese-speaking human simulate the program won't cause it to stop understanding Chinese. The presence of the human doesn't expose a contradiction in the assumption "a computer program can understand chinese".
  - kbelder 5y ago
    Maybe there's a small human inside the human's brain, not understanding the rules the human is following, and it's only the innermost human that is really conscious? I think Searle's Chinese room argument is absolutely nonsensical, a bit like arguing about philosophical zombies. It's an argument that only makes any sense if you already are committed to mysticism or dualism.
    - atoav 5y ago
      > Maybe there's a small human inside the human's brain, not understanding the rules the human is following, and it's only the innermost human that is really conscious? That would be the materialist idea of there being no soul, consciousess etc — just complex biochemical processes happening in matter. This is however not a satisfying answer usually because it fails to explain the consistent experience of ourselves, the worlds we can imagine, remember or think about, the feelings that haunt us etc. If there is only material, where is our consciousness? Apparently our consciousness somehow emerges from that material, but how would a conscious computer then look? Would there be a difference between a computer that simulates conscious behaviour and one that shows the same behaviour but actually "has" conscious in it somewhere?

antognini 5y ago

You raise good questions, but I would really suggest listening through the lectures because Searle addresses all your points in a lot of detail. (It's a lot of material, but it's well worth it and he's an entertaining lecturer.) It's actually a bit funny that you bring up Cartesian dualism because Searle is pretty clear from the outset that he's of the opinion that Descartes's dualism was the worst thing ever to happen to philosophy. He certainly has no truck with it. The question Searle is raising with the thought experiment is where the understanding of Chinese is happening. It's clear that the understanding is not happening with the homunculus. But if not there, then where is it happening? Some functionalists responded with the "systems response" which is that the system of the room plus the homunculus is what has the understanding even though no element of the system taken individually has understanding. He goes through a pretty detailed reply to the systems response, but the basic argument against it is that it's incoherent. There's no well defined thing that is the system. You can remove the man from the room and put him under a tent in a field. Where does the system end and the rest of the world begin? You are of course correct that the experiment can be reformulated without the human, but that's exactly the point. A computer is mechanistically doing exactly the same thing as the man in the room. But if the man has no understanding and the room has no understanding and the system has no understanding, removing the human from the equation changes nothing. The reason Searle included the human is because saying "computer" invokes a lot of mystery to a lot of people as to what's going on. But a computer is doing nothing different than the man in the room.

emtel 5y ago
I've read one of Searle's popular books (Rediscovery of the Mind) and many other shorter writings. I'm aware he rejects dualism - my point is that the Chinese Room argument wouldn't even make sense to someone who had never been contaminated by dualist notions. Searle rejects it, but it has crept back in, imo. His arguments against the "systems response" have to stand or fall on their own. The Chinese room doesn't have anything to add. If the "systems response" fails, then we already know a computer can't be conscious without any additional thought experiments. Lastly, I think he goes much too far in dispelling the mystery of the word "computer". As Dennett and others have pointed out, a working chinese room that displayed clear signs of consciousness would be an artifact far beyond current human comprehension, either in scale or sophistification or both. Searle asks you to imagine a human taking slips of paper and calmly consulting a rule book and producing new slips of paper, and claims that it is "obvious" that this system can't be conscious. Well, I agree, but it's equally obvious that the system couldn't possibly display conscious-seeming behavior. The best defense of the CR that I've heard is that it is only intended to prove that a _purely formal_ system (i.e. GOFAI) can't be conscious. I find this more plausible, but I still don't think the CR adds anything to that argument. And, to reiterate the previous paragraph, I think if you had a formal system that everyone agreed _seemed_ conscious, it would be such a complex artifact that it wouldn't be obviously unconscious in the way that, say, an eliza bot is.

Ataraxiaist 5y ago

I think at some point in the future studying consciousness will be as nonsensical as studying the soul. I think that is all we really mean anyway and it is just wrapped up in different packaging. Soul/spirit/consciousness all the same thing. Chasing the same ghost.