8 ms·

The Chinese Room Argument

29 points by headalgorithm 8y ago

deleted 8y ago
[deleted]
wrkronmiller 8y ago
As a non-philosopher, this argument seems flawed in that one could argue the computer in the chinese room "understands" chinese. The Turing test, as I see it, would metaphorically ask if someone or something in the chinese room understood chinese and you could argue the answer is yes.
- nathias 8y ago
  not the computer but the room itself
  - wrkronmiller 8y ago
    Perhaps it would have been better to phrase it as "an 'understanding' of chinese exists in the room"
- raincom 8y ago
  You are right, if you add quotes to the word 'understood' in the second sentence. So, it is an issue of what understanding in the context of humans, in what way this is different from what understanding in the context of computers/symbolic language is.
  - gmueckl 8y ago
    This highlights how fuzzy terms like intelligence and understanding are. How exactly is human understanding/reasoning different to that of a dog, a monkey or a computer? How do you value each one of these? Should an animal be valued higher than than a constructed machine that is otherwise identical? What about humans and machines?
    - lottin 8y ago
      Does a light bulb know that it is on or off? That it exists? Normally, in order to answer these questions we would look at the empirical evidence, and the evidence in favour that inanimate objects have self-awareness is completely non-existent. On the other hand, the evidence that animals have self-awareness is overwhelmingly strong. Thus, it is reasonable to conclude that inanimate objects aren't self-aware but that people and animals are. (And of course this also means that is NOT reasonable to conclude otherwise.)
      - ForHackernews 8y ago
        See also: https://coronaradiata.net/2012/12/24/does-a-thermostat-have-consciousness/ https://coronaradiata.net/2012/12/24/does-a-thermostat-have-...
      - gmueckl 8y ago
        We attribute self-awareness (or other attributes) based on the results of behavioral tests that we perform on the test subject. Assume, for the sake of the argument, that I built a machine that exhibits the same kind of self-awareness as an elephant. So it would recognize itself in a mirror test etc... So if it manages to pass all these tests, is it self-aware? According to Ernst Mach, two things that are indistinguishable by observation are actually equal. This is essentially a part of the Chinese Room Argument: how can you draw a distinction between two things, when you cannot observe it?
  - mcguire 8y ago
    The Turing test is an attempt to define intelligence operationally, from the outside, as a black box. Searle's argument is that the test is ineffective, since he is hypothetically satisfying it without "intelligence". Unfortunately, his argument applies to a human being as well, so...
- roenxi 8y ago
  Indeed. In a lot of practical cases with humans we have people who talk a good talk but are unable to put the ideas they verbalise into practice. We don't say "this person doesn't understand [the language]", we say "this person doesn't understand [the thing they are talking about]". It seems reasonable to say the Chinese room has a perfect grasp of Chinese, it just happens to be pretty stupid and probably not reliable. It would be interesting to do a study of a group like judges or politicians and compare them to a hypothetical Chinese Room. It seems likely that they would often need to speak briefly on subjects outside their actual understanding, but they clearly have a deep understanding of the language.
stevebmark 8y ago
This is such an awkward and boring thought experiment that it could only have come from the useless cross product of philosophy and academia. There's a reason no one in applied ML cares about this at all.
- phkahler 8y ago
  This is the best response of all. It hinges on the idea that the program is simple or trivial. That a simple recipe that a human might carry out would be able to pass a Turing test. The entire premise is a strawman.
- curious_yogurt 8y ago
  It seems to me that the argument is not meant to address the value judgements of people doing work in applied ML. The basic idea of the Chinese Room is that the Turing Test is inadequate—such that even if a machine could pass the test, we would not be permitted to infer that such a machine is intelligent. This is significant for the distinction between syntax and semantics, as well as any mechanical account of mind that integrates the results of computational logic. And indeed it does seem like many people do wonder about the nature of machine intelligence and about whether we can give a coherent account of the mind from a mechanical perspective. So perhaps some people doing work in applied ML do not care about these questions; but it does not follow from this that such questions are without qualification uninteresting or worthy of being considered.
  - garquis 8y ago
    Good evaluation. Searle’s Chinese room argument always seems to hit a nerve with many on HN, with lots of outright diamissal. Whether or not it’s wrong, it’s intent is often just misunderstood. If I remember correctly Searle isn’t arguing against the possibility of a computational machine equal in function to a biological “brain”. So no real need to take it as a slight against ml/ai work. Searle’s intrests are in phil of mind and language, so his interest in questions of mind/consciousness and semantic meaning aren’t going to have too much to bear on applied ML.
  - anigbrowl 8y ago
solipsism 8y ago
Edit: misremembered the argument. Ignore me :)
- eridius 8y ago
  It’s not an automated translation machine. The Chinese Room received input in Chinese and produces output in Chinese.
joefourier 8y ago
I don't understand the premise of the Chinese Room argument. The human in the room is a red herring - he's acting as a computer executing a provided program. A computer can be made from integrated circuits, discrete transistors, vacuum tubes, mechanical components, or even a man in a room following simple instructions and writing down the results. The man is not the source of the intelligence, the program is (or rather, the execution of the program in an emergent manner - obviously, if the program does not get executed, it cannot "understand" Chinese). Remove the program and the man is unable to process the Chinese input, just like a modern CPU if you remove a program from memory. The choice of conducting the experiment in Chinese is just another misdirection. The only thing that the experiment is truly positing is, assume a human is executing a program that passes the Turing test rather than a traditional silicon computer.
- curious_yogurt 8y ago
  The Chinese Room is a reductio against Strong AI. Strong AI is the idea that a program that programme that is able, for example, to converse in Chinese thereby understands Chinese. The Room, if successful, drives a wedge between "acting as if one knows Chinese" and "understanding Chinese". This may be applied to the any number of programs. But with such a wedge, we do not actually have Strong AI—the appearance of intelligence (grounded in syntax) is different from actual intelligence (presumably grounded in semantics). In other words, assume strong AI. Strong AI implies machines have understanding. But the Chinese Room argues machines can never have understanding. So, strong AI is not possible. In the background is Turing's attempt to provide an account of intelligence. Turing skirts the issue in his famous paper "Computing Machinery and Intelligence"[1]: instead of defining intelligence, Turing proposes a test. But the Chinese Room argues that even if we had a machine pass this test, we would not thereby be bound to say that the machine was intelligent. [1] https://www.csee.umbc.edu/courses/471/papers/turing.pdf https://www.csee.umbc.edu/courses/471/papers/turing.pdf
  - joefourier 8y ago
    Why would the room being successful reveal anything useful? Why does it matter if a program is executed on pencil and paper or electronically? The thought experiment does not bring any novel insights and just misdirects the reader by focusing on the human. You could execute the Strong AI program on any Turing complete system. Asking if the human understands Chinese is just as absurd as asking if a universal Turing machine understands Chinese given the same program. The machine just follows the instructions given, the philosophical questions reside entirely on the side of the program and the execution of it.
jrootabega 8y ago
I have a lot of strong thoughts and feelings when I think deeply about this experiment. But one thing bugs me just about the way it's posed. You're told to just assume you've got a functional database of Chinese symbols; maybe it's just a big shelf of old dusty boxes with helpful headings. And you should just assume you've got this book of instructions; maybe it's so big you need a stepladder to read its big pages. But it is, as you've been instructed, to be just a book. And when you use them together, just assume they do something amazing. Haha, I fooled you dummy, something has to be missing here; the thing it does can't really be THAT amazing, because you were told it was just boxes and books; you can get those anywhere. You're told right away to imagine a set of comprehensible mundane objects, assume they can do this thing in the form you've been given, and as a result, the thing must be mundane and incomplete. I know there's more depth to it than that, but I don't think you need to go that deep to find large flaws.
- ForHackernews 8y ago
  The analogy is supposed to emphasize that AI proponents (and others) presuppose there's a fully materialistic explanation for consciousness. There's no wizard behind the curtain: it's all just deep neural networks ("boxes and books").
  - viraptor 8y ago
    It could be a weaker argument: One can theoretically make a fully materialistic device acting in a way indistinguishable from consciousness. Which still preserves the question: if you can reliably fake knowing Chinese, do you know Chinese?
    - joe_the_user 8y ago
      Yeah, the thing about the argument is it mixes together two close but distinct arguments in a way that makes it harder to argue with either. On the one hand, there's the implication that because it's just books and boxes & whatever, it can't do the impressive feat of speaking Chinese. On the other hand, there's the implication that if it is a "room" or "computer" or something doing the speaking of Chinese, it can't "really" be doing it because it doesn't have the spark of life or a soul or whatever magic one might name. And because the argument has a bit of both these claims to it, it is hard to address since a proponent can go back and forth between them. That this argument was taken seriously by philosophers was rather disheartening to me when I discovered this, when it became obvious there really wasn't any more there than a colorful (but not particular relevant) metaphor.
svalorzen 8y ago
The entire argument is flawed, as it doesn't actually prove anything at all. Here is the full argument, with all assumptions explicit: Assume properties 'consciousness' and 'intelligence' exist. I'm not going to define them because I can't. Assume that a human has these properties. Now I demonstrate that a machine can't have these properties, since they were never actually defined. Since by assumption a human has them, a human has intelligence and a machine not. The type of ideas that can lead to this reasoning are the same of the people who rejected evolution as thinking that man could never have descended from monkeys. The only think the argument proves is the close-mindedness of its author.
- lottin 8y ago
  > Assume properties 'consciousness' and 'intelligence' exist. I'm not going to define them because I can't. Assume that a human has these properties. But this is not what the argument says, is it? The argument says imagine a man that can't understand Chinese. The assumption here is that some humans can understand Chinese while others can not.
  - svalorzen 8y ago
    It is, because the term 'understanding' is not defined in a useful manner. What does it mean to understand something? If you can ever define it, I can bet I can make a machine satisfy your definition. What the argument does here is to simply avoid the definition, so that it cannot be disproved.
    - hackeraccount 8y ago
      I take the Chinese Room to be a reaction to the Turing Test. The Turing Test also avoids defining understanding but it seems to say that we don't need a definition of understanding - if something appears to understand then it does in fact understand. The Chinese Room attempts to refute that idea by saying that you can get the effect of understanding with a methodology that obviously lacks the gestalt that we associate with understanding a subject. Personally I'm a sketptic of hard A.I. in general. Conscious to me seems like a magic trick - in that the only magical thing about it is that it will disappear once you know how it's done. Which makes it seem like a black box that can only be created if you don't know how or what you're doing.
hashfunktion 8y ago
Not a neurologist so excuse the potential naïveté here but can one also make the argument that we can reduce a Chinese speaker's brain's ability to respond into a series of "manipulating symbols and numerals" as well, albeit, a extremely complicated one?
- ForHackernews 8y ago
  I think this is still an open question. Some people have claimed quantum effects may be at play, but this very controversial: https://www.quantamagazine.org/a-new-spin-on-the-quantum-brain-20161102/ https://www.quantamagazine.org/a-new-spin-on-the-quantum-bra... Most neuroscientists feel strongly that the brain is not (just?) a computer https://aeon.co/essays/your-brain-does-not-process-information-and-it-is-not-a-computer https://aeon.co/essays/your-brain-does-not-process-informati...
  - trashtester 8y ago
    My understanding is that the brain, to the extent that it is a computer, is not anywhere near Turing complete. It is not something more than a very fast (Turing complete) computer, it is something less. The brain makes up for the deficiency by being extremely fast and well optimized for the tasks it needs to do. But as a consequence, we cannot draw the metaphor about the brain being a "computer" too far. The "software" in the brain is specific to the given brain, and not hardware independent. Sure, if we imagine a Turing machine with infinite speed and memory, it might be able to run the "software" from every human brain, but we do not know if it is even possible to build such a computer within our current universe (if we require it to operate at real time). And if it is possible, I would guess that it would be so many orders of magnitude less efficient than a non-Turing complete machine that there would be little reason to build it. Penrose and others try to smuggle in something like dualism through QM, but I don't see why that is necessary at all. I think we just need to throw out most of the "Computer" metaphor that requires Turing completeness, and deal with the brain for what it is.
- mcguire 8y ago
  It's chemistry and physics from there down. Searle's argument is closet dualism.
rsre 8y ago
I feel like this fails at the scope level. Of course excluding the program the computer is unable to understand Chinese. Just like a transistor alone is unable to perform a sum of two integers. Either you analyse the complete system or you are just drawing false parallels. EDIT: Spelling
travisoneill1 8y ago
If you assume that the brain of a person speaking Chinese is fundamentally different than a computer executing an instruction set, then you can also prove it based on that assumption. How is this significant?
nullc 8y ago
I have always considered Searle's chinese room to be an intellectually dishonest argument. It argues that a system (the room) consisting of a non-chinese speaker and a arbitrarily large and complex rules system and a human cannot "understand" chinese when the human clearly doesn't understand chinese. The human is just a distraction. You could instead image a variant system where the room contains two people (and no filing cabinets): non-chinese speaking scribe and a chinese speaker. The scribe laboriously transcribes messages without understanding them and passes them to and from the chinese speaker that clearly does. Obviously the addition of the non-understanding human in that example does not make the system unintelligent. Similarly, the (non-)intelligence of the system in the classic argument is independent of the understanding of the human stuck inside it. There are interesting question to ask about the nature of intelligence, but the Chinese Room argument draws our attention to none of them and simply obfuscates the issue with an irrelevant misdirection.
- nullc 8y ago
  Another way to highlight the folly of The Chinese Room is to consider a different alternative: Imagine you create a system of filing cabnets that encode rules to compute AlphaGo on encrypted inputs. This is _clearly_ conceptually possible (if not practical just due to the sheer cost and size). This system would, as AlphaGo does, be the worlds strongest go player. Even though the human inside may not understand go at all (and certainly wouldn't understand the game they were helping to play, as all the moves would be encrypted). In this form we see The Chinese Room not as as evidence against AI as intended, but if the argument actually held (and wasn't just a form of intellectually low misdirection) it would be very strong evidence that its idea of "understanding" is of no practical use. ... since what would "understanding" even mean if we couldn't even say that the worlds best go player didn't understand the game at all?
  - ForHackernews 8y ago
    I'm happy to say that the world's best Go playing algorithm doesn't understand the game at all. AlphaZero doesn't understand anything because there's no semantic content or metacognition to anything it does. It just takes inputs and optimizes outputs. It's a very fancy probabilistic calculator.
jancsika 8y ago
Kurzweil had a nice end-run around this problem. It doesn't matter whether the machine is sentient/intelligent/understands/etc. It matters whether the machine can convince humans that it is doing these things. There is of course still room for skepticism, as a researcher could uncover new, testable ways in which the machine isn't actually living up to our definition of any of these concepts. But if all that is left to decide the question is the way in which the thing doing the convincing happens to be implemented, then you're in the land of xenophobia to claim that as a determining factor. AFAICT this is one step further than the Turing test which is blind (or perhaps double blind). In this test the human starts interacting with something that is clearly a machine, then comes away convinced that the machine possesses non-machine qualities like sentience, a soul, etc. I don't remember Kurzweil addressing whether the machine itself would argue this on its own behalf, but that seems likely. Edit: clarification

29 points by headalgorithm 8y ago

deleted 8y ago

[deleted]

As a non-philosopher, this argument seems flawed in that one could argue the computer in the chinese room "understands" chinese. The Turing test, as I see it, would metaphorically ask if someone or something in the chinese room understood chinese and you could argue the answer is yes.

nathias 8y ago
not the computer but the room itself
- wrkronmiller 8y ago
  Perhaps it would have been better to phrase it as "an 'understanding' of chinese exists in the room"
raincom 8y ago
You are right, if you add quotes to the word 'understood' in the second sentence. So, it is an issue of what understanding in the context of humans, in what way this is different from what understanding in the context of computers/symbolic language is.
- gmueckl 8y ago
  This highlights how fuzzy terms like intelligence and understanding are. How exactly is human understanding/reasoning different to that of a dog, a monkey or a computer? How do you value each one of these? Should an animal be valued higher than than a constructed machine that is otherwise identical? What about humans and machines?
  - lottin 8y ago
    Does a light bulb know that it is on or off? That it exists? Normally, in order to answer these questions we would look at the empirical evidence, and the evidence in favour that inanimate objects have self-awareness is completely non-existent. On the other hand, the evidence that animals have self-awareness is overwhelmingly strong. Thus, it is reasonable to conclude that inanimate objects aren't self-aware but that people and animals are. (And of course this also means that is NOT reasonable to conclude otherwise.)
    - ForHackernews 8y ago
      See also: https://coronaradiata.net/2012/12/24/does-a-thermostat-have-consciousness/ https://coronaradiata.net/2012/12/24/does-a-thermostat-have-...
    - gmueckl 8y ago
      We attribute self-awareness (or other attributes) based on the results of behavioral tests that we perform on the test subject. Assume, for the sake of the argument, that I built a machine that exhibits the same kind of self-awareness as an elephant. So it would recognize itself in a mirror test etc... So if it manages to pass all these tests, is it self-aware? According to Ernst Mach, two things that are indistinguishable by observation are actually equal. This is essentially a part of the Chinese Room Argument: how can you draw a distinction between two things, when you cannot observe it?
- mcguire 8y ago
  The Turing test is an attempt to define intelligence operationally, from the outside, as a black box. Searle's argument is that the test is ineffective, since he is hypothetically satisfying it without "intelligence". Unfortunately, his argument applies to a human being as well, so...
roenxi 8y ago
Indeed. In a lot of practical cases with humans we have people who talk a good talk but are unable to put the ideas they verbalise into practice. We don't say "this person doesn't understand [the language]", we say "this person doesn't understand [the thing they are talking about]". It seems reasonable to say the Chinese room has a perfect grasp of Chinese, it just happens to be pretty stupid and probably not reliable. It would be interesting to do a study of a group like judges or politicians and compare them to a hypothetical Chinese Room. It seems likely that they would often need to speak briefly on subjects outside their actual understanding, but they clearly have a deep understanding of the language.

stevebmark 8y ago

This is such an awkward and boring thought experiment that it could only have come from the useless cross product of philosophy and academia. There's a reason no one in applied ML cares about this at all.

phkahler 8y ago
This is the best response of all. It hinges on the idea that the program is simple or trivial. That a simple recipe that a human might carry out would be able to pass a Turing test. The entire premise is a strawman.
curious_yogurt 8y ago
It seems to me that the argument is not meant to address the value judgements of people doing work in applied ML. The basic idea of the Chinese Room is that the Turing Test is inadequate—such that even if a machine could pass the test, we would not be permitted to infer that such a machine is intelligent. This is significant for the distinction between syntax and semantics, as well as any mechanical account of mind that integrates the results of computational logic. And indeed it does seem like many people do wonder about the nature of machine intelligence and about whether we can give a coherent account of the mind from a mechanical perspective. So perhaps some people doing work in applied ML do not care about these questions; but it does not follow from this that such questions are without qualification uninteresting or worthy of being considered.
- garquis 8y ago
  Good evaluation. Searle’s Chinese room argument always seems to hit a nerve with many on HN, with lots of outright diamissal. Whether or not it’s wrong, it’s intent is often just misunderstood. If I remember correctly Searle isn’t arguing against the possibility of a computational machine equal in function to a biological “brain”. So no real need to take it as a slight against ml/ai work. Searle’s intrests are in phil of mind and language, so his interest in questions of mind/consciousness and semantic meaning aren’t going to have too much to bear on applied ML.
- anigbrowl 8y ago

solipsism 8y ago

Edit: misremembered the argument. Ignore me :)

eridius 8y ago
It’s not an automated translation machine. The Chinese Room received input in Chinese and produces output in Chinese.

joefourier 8y ago

I don't understand the premise of the Chinese Room argument. The human in the room is a red herring - he's acting as a computer executing a provided program. A computer can be made from integrated circuits, discrete transistors, vacuum tubes, mechanical components, or even a man in a room following simple instructions and writing down the results. The man is not the source of the intelligence, the program is (or rather, the execution of the program in an emergent manner - obviously, if the program does not get executed, it cannot "understand" Chinese). Remove the program and the man is unable to process the Chinese input, just like a modern CPU if you remove a program from memory. The choice of conducting the experiment in Chinese is just another misdirection. The only thing that the experiment is truly positing is, assume a human is executing a program that passes the Turing test rather than a traditional silicon computer.

curious_yogurt 8y ago
The Chinese Room is a reductio against Strong AI. Strong AI is the idea that a program that programme that is able, for example, to converse in Chinese thereby understands Chinese. The Room, if successful, drives a wedge between "acting as if one knows Chinese" and "understanding Chinese". This may be applied to the any number of programs. But with such a wedge, we do not actually have Strong AI—the appearance of intelligence (grounded in syntax) is different from actual intelligence (presumably grounded in semantics). In other words, assume strong AI. Strong AI implies machines have understanding. But the Chinese Room argues machines can never have understanding. So, strong AI is not possible. In the background is Turing's attempt to provide an account of intelligence. Turing skirts the issue in his famous paper "Computing Machinery and Intelligence"[1]: instead of defining intelligence, Turing proposes a test. But the Chinese Room argues that even if we had a machine pass this test, we would not thereby be bound to say that the machine was intelligent. [1] https://www.csee.umbc.edu/courses/471/papers/turing.pdf https://www.csee.umbc.edu/courses/471/papers/turing.pdf
- joefourier 8y ago
  Why would the room being successful reveal anything useful? Why does it matter if a program is executed on pencil and paper or electronically? The thought experiment does not bring any novel insights and just misdirects the reader by focusing on the human. You could execute the Strong AI program on any Turing complete system. Asking if the human understands Chinese is just as absurd as asking if a universal Turing machine understands Chinese given the same program. The machine just follows the instructions given, the philosophical questions reside entirely on the side of the program and the execution of it.

jrootabega 8y ago

I have a lot of strong thoughts and feelings when I think deeply about this experiment. But one thing bugs me just about the way it's posed. You're told to just assume you've got a functional database of Chinese symbols; maybe it's just a big shelf of old dusty boxes with helpful headings. And you should just assume you've got this book of instructions; maybe it's so big you need a stepladder to read its big pages. But it is, as you've been instructed, to be just a book. And when you use them together, just assume they do something amazing. Haha, I fooled you dummy, something has to be missing here; the thing it does can't really be THAT amazing, because you were told it was just boxes and books; you can get those anywhere. You're told right away to imagine a set of comprehensible mundane objects, assume they can do this thing in the form you've been given, and as a result, the thing must be mundane and incomplete. I know there's more depth to it than that, but I don't think you need to go that deep to find large flaws.

ForHackernews 8y ago
The analogy is supposed to emphasize that AI proponents (and others) presuppose there's a fully materialistic explanation for consciousness. There's no wizard behind the curtain: it's all just deep neural networks ("boxes and books").
- viraptor 8y ago
  It could be a weaker argument: One can theoretically make a fully materialistic device acting in a way indistinguishable from consciousness. Which still preserves the question: if you can reliably fake knowing Chinese, do you know Chinese?
  - joe_the_user 8y ago
    Yeah, the thing about the argument is it mixes together two close but distinct arguments in a way that makes it harder to argue with either. On the one hand, there's the implication that because it's just books and boxes & whatever, it can't do the impressive feat of speaking Chinese. On the other hand, there's the implication that if it is a "room" or "computer" or something doing the speaking of Chinese, it can't "really" be doing it because it doesn't have the spark of life or a soul or whatever magic one might name. And because the argument has a bit of both these claims to it, it is hard to address since a proponent can go back and forth between them. That this argument was taken seriously by philosophers was rather disheartening to me when I discovered this, when it became obvious there really wasn't any more there than a colorful (but not particular relevant) metaphor.

svalorzen 8y ago

The entire argument is flawed, as it doesn't actually prove anything at all. Here is the full argument, with all assumptions explicit: Assume properties 'consciousness' and 'intelligence' exist. I'm not going to define them because I can't. Assume that a human has these properties. Now I demonstrate that a machine can't have these properties, since they were never actually defined. Since by assumption a human has them, a human has intelligence and a machine not. The type of ideas that can lead to this reasoning are the same of the people who rejected evolution as thinking that man could never have descended from monkeys. The only think the argument proves is the close-mindedness of its author.

lottin 8y ago
> Assume properties 'consciousness' and 'intelligence' exist. I'm not going to define them because I can't. Assume that a human has these properties. But this is not what the argument says, is it? The argument says imagine a man that can't understand Chinese. The assumption here is that some humans can understand Chinese while others can not.
- svalorzen 8y ago
  It is, because the term 'understanding' is not defined in a useful manner. What does it mean to understand something? If you can ever define it, I can bet I can make a machine satisfy your definition. What the argument does here is to simply avoid the definition, so that it cannot be disproved.
  - hackeraccount 8y ago
    I take the Chinese Room to be a reaction to the Turing Test. The Turing Test also avoids defining understanding but it seems to say that we don't need a definition of understanding - if something appears to understand then it does in fact understand. The Chinese Room attempts to refute that idea by saying that you can get the effect of understanding with a methodology that obviously lacks the gestalt that we associate with understanding a subject. Personally I'm a sketptic of hard A.I. in general. Conscious to me seems like a magic trick - in that the only magical thing about it is that it will disappear once you know how it's done. Which makes it seem like a black box that can only be created if you don't know how or what you're doing.

hashfunktion 8y ago

Not a neurologist so excuse the potential naïveté here but can one also make the argument that we can reduce a Chinese speaker's brain's ability to respond into a series of "manipulating symbols and numerals" as well, albeit, a extremely complicated one?

ForHackernews 8y ago
I think this is still an open question. Some people have claimed quantum effects may be at play, but this very controversial: https://www.quantamagazine.org/a-new-spin-on-the-quantum-brain-20161102/ https://www.quantamagazine.org/a-new-spin-on-the-quantum-bra... Most neuroscientists feel strongly that the brain is not (just?) a computer https://aeon.co/essays/your-brain-does-not-process-information-and-it-is-not-a-computer https://aeon.co/essays/your-brain-does-not-process-informati...
- trashtester 8y ago
  My understanding is that the brain, to the extent that it is a computer, is not anywhere near Turing complete. It is not something more than a very fast (Turing complete) computer, it is something less. The brain makes up for the deficiency by being extremely fast and well optimized for the tasks it needs to do. But as a consequence, we cannot draw the metaphor about the brain being a "computer" too far. The "software" in the brain is specific to the given brain, and not hardware independent. Sure, if we imagine a Turing machine with infinite speed and memory, it might be able to run the "software" from every human brain, but we do not know if it is even possible to build such a computer within our current universe (if we require it to operate at real time). And if it is possible, I would guess that it would be so many orders of magnitude less efficient than a non-Turing complete machine that there would be little reason to build it. Penrose and others try to smuggle in something like dualism through QM, but I don't see why that is necessary at all. I think we just need to throw out most of the "Computer" metaphor that requires Turing completeness, and deal with the brain for what it is.
mcguire 8y ago
It's chemistry and physics from there down. Searle's argument is closet dualism.

rsre 8y ago

I feel like this fails at the scope level. Of course excluding the program the computer is unable to understand Chinese. Just like a transistor alone is unable to perform a sum of two integers. Either you analyse the complete system or you are just drawing false parallels. EDIT: Spelling

travisoneill1 8y ago

If you assume that the brain of a person speaking Chinese is fundamentally different than a computer executing an instruction set, then you can also prove it based on that assumption. How is this significant?

nullc 8y ago

I have always considered Searle's chinese room to be an intellectually dishonest argument. It argues that a system (the room) consisting of a non-chinese speaker and a arbitrarily large and complex rules system and a human cannot "understand" chinese when the human clearly doesn't understand chinese. The human is just a distraction. You could instead image a variant system where the room contains two people (and no filing cabinets): non-chinese speaking scribe and a chinese speaker. The scribe laboriously transcribes messages without understanding them and passes them to and from the chinese speaker that clearly does. Obviously the addition of the non-understanding human in that example does not make the system unintelligent. Similarly, the (non-)intelligence of the system in the classic argument is independent of the understanding of the human stuck inside it. There are interesting question to ask about the nature of intelligence, but the Chinese Room argument draws our attention to none of them and simply obfuscates the issue with an irrelevant misdirection.

nullc 8y ago
Another way to highlight the folly of The Chinese Room is to consider a different alternative: Imagine you create a system of filing cabnets that encode rules to compute AlphaGo on encrypted inputs. This is _clearly_ conceptually possible (if not practical just due to the sheer cost and size). This system would, as AlphaGo does, be the worlds strongest go player. Even though the human inside may not understand go at all (and certainly wouldn't understand the game they were helping to play, as all the moves would be encrypted). In this form we see The Chinese Room not as as evidence against AI as intended, but if the argument actually held (and wasn't just a form of intellectually low misdirection) it would be very strong evidence that its idea of "understanding" is of no practical use. ... since what would "understanding" even mean if we couldn't even say that the worlds best go player didn't understand the game at all?
- ForHackernews 8y ago
  I'm happy to say that the world's best Go playing algorithm doesn't understand the game at all. AlphaZero doesn't understand anything because there's no semantic content or metacognition to anything it does. It just takes inputs and optimizes outputs. It's a very fancy probabilistic calculator.

jancsika 8y ago

Kurzweil had a nice end-run around this problem. It doesn't matter whether the machine is sentient/intelligent/understands/etc. It matters whether the machine can convince humans that it is doing these things. There is of course still room for skepticism, as a researcher could uncover new, testable ways in which the machine isn't actually living up to our definition of any of these concepts. But if all that is left to decide the question is the way in which the thing doing the convincing happens to be implemented, then you're in the land of xenophobia to claim that as a determining factor. AFAICT this is one step further than the Turing test which is blind (or perhaps double blind). In this test the human starts interacting with something that is clearly a machine, then comes away convinced that the machine possesses non-machine qualities like sentience, a soul, etc. I don't remember Kurzweil addressing whether the machine itself would argue this on its own behalf, but that seems likely. Edit: clarification