6 ms·

I have a lot of strong thoughts and feelings when I think deeply about this experiment. But one thing bugs me just about the way it's posed. You're told to just

I have a lot of strong thoughts and feelings when I think deeply about this experiment. But one thing bugs me just about the way it's posed. You're told to just assume you've got a functional database of Chinese symbols; maybe it's just a big shelf of old dusty boxes with helpful headings. And you should just assume you've got this book of instructions; maybe it's so big you need a stepladder to read its big pages. But it is, as you've been instructed, to be just a book. And when you use them together, just assume they do something amazing. Haha, I fooled you dummy, something has to be missing here; the thing it does can't really be THAT amazing, because you were told it was just boxes and books; you can get those anywhere. You're told right away to imagine a set of comprehensible mundane objects, assume they can do this thing in the form you've been given, and as a result, the thing must be mundane and incomplete. I know there's more depth to it than that, but I don't think you need to go that deep to find large flaws.

ForHackernews 8y ago
The analogy is supposed to emphasize that AI proponents (and others) presuppose there's a fully materialistic explanation for consciousness. There's no wizard behind the curtain: it's all just deep neural networks ("boxes and books").
- viraptor 8y ago
  It could be a weaker argument: One can theoretically make a fully materialistic device acting in a way indistinguishable from consciousness. Which still preserves the question: if you can reliably fake knowing Chinese, do you know Chinese?
  - joe_the_user 8y ago
    Yeah, the thing about the argument is it mixes together two close but distinct arguments in a way that makes it harder to argue with either. On the one hand, there's the implication that because it's just books and boxes & whatever, it can't do the impressive feat of speaking Chinese. On the other hand, there's the implication that if it is a "room" or "computer" or something doing the speaking of Chinese, it can't "really" be doing it because it doesn't have the spark of life or a soul or whatever magic one might name. And because the argument has a bit of both these claims to it, it is hard to address since a proponent can go back and forth between them. That this argument was taken seriously by philosophers was rather disheartening to me when I discovered this, when it became obvious there really wasn't any more there than a colorful (but not particular relevant) metaphor.
- int_19h 8y ago
  Yes, and then the analogy is used to refute this notion through a thinly disguised "but it is obvious! [that this isn't real]" fallacy (the fallacy is that it is not at all obvious that the Chinese Room is not sapient).
  - ForHackernews 8y ago
    Yes, that's one way out: claiming that the Books+Person Room-System is itself sentient. You can (and philosophers have!) make that case, but it seems very counter-intuitive to the way we experience conscientious as a unitary, embodied being.
    - joe_the_user 8y ago
      The things can move just through the application of chemicals or air-pressure is counter-intuitive. It seems like you've got a philosophy that is taking a person through a series of reasoning based stringing together back intuitions and ending up in the same place religion takes us.
      - ForHackernews 8y ago
        Sure, I guess. Most of the responses in this thread are computer people trying to skirt around the "hard problem" of consciousness: http://www.consc.net/papers/facing.html http://www.consc.net/papers/facing.html If you want to claim that what your brain does is basically a more complicated version of what your laptop does, then you're free to make that case, but I don't think you honestly believe it yourself. You do recognize your own inner life.
        joe_the_user 8y ago
        Well, while I'd both disagree with "hard problem" arguments and not see them as interesting, at least "hard problem" proponents will say directly there's some magic in humans that a computer can't have. Indeed, the "philosophical zombie" idea shows the magic is so transcendent that no evidence can determine whether a person has the magic (of qualia) or whether they are a dreaded Zombie. In contrast, the Chinese Room argument jumps between "hard problem" metaphysics and the appearance of a materialistic perspective.

ForHackernews 8y ago

The analogy is supposed to emphasize that AI proponents (and others) presuppose there's a fully materialistic explanation for consciousness. There's no wizard behind the curtain: it's all just deep neural networks ("boxes and books").

viraptor 8y ago
It could be a weaker argument: One can theoretically make a fully materialistic device acting in a way indistinguishable from consciousness. Which still preserves the question: if you can reliably fake knowing Chinese, do you know Chinese?
- joe_the_user 8y ago
  Yeah, the thing about the argument is it mixes together two close but distinct arguments in a way that makes it harder to argue with either. On the one hand, there's the implication that because it's just books and boxes & whatever, it can't do the impressive feat of speaking Chinese. On the other hand, there's the implication that if it is a "room" or "computer" or something doing the speaking of Chinese, it can't "really" be doing it because it doesn't have the spark of life or a soul or whatever magic one might name. And because the argument has a bit of both these claims to it, it is hard to address since a proponent can go back and forth between them. That this argument was taken seriously by philosophers was rather disheartening to me when I discovered this, when it became obvious there really wasn't any more there than a colorful (but not particular relevant) metaphor.
int_19h 8y ago
Yes, and then the analogy is used to refute this notion through a thinly disguised "but it is obvious! [that this isn't real]" fallacy (the fallacy is that it is not at all obvious that the Chinese Room is not sapient).
- ForHackernews 8y ago
  Yes, that's one way out: claiming that the Books+Person Room-System is itself sentient. You can (and philosophers have!) make that case, but it seems very counter-intuitive to the way we experience conscientious as a unitary, embodied being.
  - joe_the_user 8y ago
    The things can move just through the application of chemicals or air-pressure is counter-intuitive. It seems like you've got a philosophy that is taking a person through a series of reasoning based stringing together back intuitions and ending up in the same place religion takes us.
    - ForHackernews 8y ago
      Sure, I guess. Most of the responses in this thread are computer people trying to skirt around the "hard problem" of consciousness: http://www.consc.net/papers/facing.html http://www.consc.net/papers/facing.html If you want to claim that what your brain does is basically a more complicated version of what your laptop does, then you're free to make that case, but I don't think you honestly believe it yourself. You do recognize your own inner life.
      - joe_the_user 8y ago
        Well, while I'd both disagree with "hard problem" arguments and not see them as interesting, at least "hard problem" proponents will say directly there's some magic in humans that a computer can't have. Indeed, the "philosophical zombie" idea shows the magic is so transcendent that no evidence can determine whether a person has the magic (of qualia) or whether they are a dreaded Zombie. In contrast, the Chinese Room argument jumps between "hard problem" metaphysics and the appearance of a materialistic perspective.