5 ms·

“I used to think the machines are geniuses,” Ms. Hou, 24, said. “Now I know we’re the reason for their genius.” I recall John Searle's Google Tech Talk, when h

“I used to think the machines are geniuses,” Ms. Hou, 24, said. “Now I know we’re the reason for their genius.” I recall John Searle's Google Tech Talk, when he said outright that AI was nothing more than the automation and repackaging of the output of human intelligence. Most of the room was clearly against him, just a bit short of booing him. Nowadays, the "smart talk" around AI is exactly what he said. For those who may not be aware, John Searle is notorious for his "Chinese Room" argument against hard AI. https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

ThomPete 8y ago
That argument is fundamentally flawed. It correctly points out that the person in the room doesn't know Chinese but it misses the bigger point. Of course, the person in the room doesn't know Chinese just as the individual neuron in the brain doesn't know Chinese. The room or the house (i.e. the feedback loop) is the conscious part. The correct answer is "we don't know". But we do know that we are ourselves the result of emerging complexity built by much simpler structures and that they are built on even simpler structures until in the end some primordial soup. The mistake is isolating any individual element in the feedback loop and claiming that because that doesn't have some "magic" properties the system as a whole can't.
- rjsw 8y ago
  The person in the room will also start to recognize patterns in the input messages and the resulting output messages. It isn't the canonical understanding of Chinese but it is "an" understanding If Searle was correct we could do an MRI on a blind person and detect when their brain switched into "symbol processing mode" if a conversation with them covered any visual concepts.
  - ForHackernews 8y ago
    > If Searle was correct we could do an MRI on a blind person and detect when their brain switched into "symbol processing mode" if a conversation with them covered any visual concepts. I think that is true, at least to some extent: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3667661/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3667661/ particularly for people who weren't blind from birth.
  - mannykannot 8y ago
    The Chinese Room argument fails to make its case, but not for that reason - Searle claims that no such instruction book could be written, and so the experiment could not be performed, meaning that the subject would have no possibility of learning from the outputs generated by performing it. If the experiment could be started, whatever the subject would learn from performing the role would be moot, as merely by getting started, without the subject initially knowing anything of the language in which the questions were posed, it would have refuted Searle. The reason the argument fails to make its case is that Searle does not have an effective response to the 'systems reply': in the paper, his response is a mixture of appeals to intuition, together with the erroneous claim that if the subject memorized the instructions, then the systems reply would not apply.
- stcredzero 8y ago
  That argument is fundamentally flawed. Yes. However, his other observation about AI turns out to be spot on.
  - ThomPete 8y ago
    What observation?
    - thedailymail 8y ago
      I'm guessing the observation "that AI was nothing more than the automation and repackaging of the output of human intelligence" mentioned above
      - mannykannot 8y ago
        That's also how I take it, and there are two different types of AI being discussed here: the 'Chinese Room' paper is about the possibility of conscious machines, while the 'repackaging' comment is about current technology.
      - ThomPete 8y ago
        Ok but i am not sure people in the industry would claim otherwise, also not sure what the point is.
nohat 8y ago
What do you mean the 'smart talk' is exactly what he said? That needs clarification and citation to be a meaningful statement. Inventing the chinese room argument is a point against taking him seriously. It's a very confused argument. The fact that it keeps being trotted out is sad more than anything (not that I'm saying you are doing that).
- stcredzero 8y ago
  What do you mean the 'smart talk' is exactly what he said? It's the idea that AI's achievements have to do with the automation and re-packaging of the output of human intelligence. That was one of the main points of his talk. (To his credit, that was from several years back.) Inventing the chinese room argument is a point against taking him seriously. From a technical point of view, it seems valid to dismiss it as an argument against AI. However, given that the science behind consciousness is incomplete, I'm not sure the entire idea has no merit. In a way, the Chinese Room scenario reveals something about our current cultural biases about what constitutes consciousness. For example, it can be taken as a gedankenexperiment showing we'd have a hard time overcoming our biases if we were confronted with a consciousness operating at a speed 6 orders of magnitude slower than our own. On what empirical basis can we dismiss our intuitions on the matter? How can we be so sure that the logic for our dismissing the argument isn't just a technocrat ideology?
  - nohat 8y ago
    I get that was his claim, but saying that this is now accepted is just not true. Saying that it is 'smart talk' implies that smart people are talking about it? Saying that the value in the Chinese Room argument is in showing how we have difficulty overcoming our biases is a clever inversion, but more a funny debate tactic than serious defense of Searle. Unless you are claiming that Searle doesn't actually believe it and invented it as some sort of long con? I don't think using brain biology and logic over intuition needs to be defended.
    - stcredzero 8y ago
      Saying that it is 'smart talk' implies that smart people are talking about it? I keep reading paraphrases of basically that idea in articles and blog posts. Another phrase which means the same thing refers to getting enough data, or declares that data is king. The latter often implies that what's happening is an automation and repackaging of human intelligence. Saying that the value in the Chinese Room argument is in showing how we have difficulty overcoming our biases is a clever inversion, more a funny debate tactic than serious defense of Searle. It's not even a funny debate tactic, if no one is actually defending the Chinese Room argument. In other words, there is a value in looking at why people are interested in the argument, despite the fact it fails. I don't think using brain biology and logic over intuition needs to be defended. The way that we apply science and logic in the context of our cultural understanding and human biases does need to be examined, however. History shows this quite clearly. I think it's very telling that instead of engaging in an analysis of the intuitions and biases which apply to human beings thinking about consciousness, you are merely interested in the dead-horse beating exhortation of the supremacy of arguments which have already been won. I think an examination of the above situation has tremendous value for people thinking seriously about AI's implications for future society. For example, there is a story about Murray Gell-Mann and how he flummoxed his grad students by being just slightly faster at getting to the next idea in conversations. One day, his grad students played a practical joke on him by enumerating all of the implications of a new line of thinking, ahead of time when he wasn't there, memorizing them, then blurting out the next idea when he met with them later. I also had a professor like that. He never thought things I never thought of, but he was always saying them just as I was formulating the idea, but before I could speak it. I think it's well worth it, to examine exactly what we mean when we think of someone being more or less intelligent. I suspect it will have important implications if and when we do develop a sentient AI.

ThomPete 8y ago

That argument is fundamentally flawed. It correctly points out that the person in the room doesn't know Chinese but it misses the bigger point. Of course, the person in the room doesn't know Chinese just as the individual neuron in the brain doesn't know Chinese. The room or the house (i.e. the feedback loop) is the conscious part. The correct answer is "we don't know". But we do know that we are ourselves the result of emerging complexity built by much simpler structures and that they are built on even simpler structures until in the end some primordial soup. The mistake is isolating any individual element in the feedback loop and claiming that because that doesn't have some "magic" properties the system as a whole can't.

rjsw 8y ago
The person in the room will also start to recognize patterns in the input messages and the resulting output messages. It isn't the canonical understanding of Chinese but it is "an" understanding If Searle was correct we could do an MRI on a blind person and detect when their brain switched into "symbol processing mode" if a conversation with them covered any visual concepts.
- ForHackernews 8y ago
  > If Searle was correct we could do an MRI on a blind person and detect when their brain switched into "symbol processing mode" if a conversation with them covered any visual concepts. I think that is true, at least to some extent: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3667661/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3667661/ particularly for people who weren't blind from birth.
- mannykannot 8y ago
  The Chinese Room argument fails to make its case, but not for that reason - Searle claims that no such instruction book could be written, and so the experiment could not be performed, meaning that the subject would have no possibility of learning from the outputs generated by performing it. If the experiment could be started, whatever the subject would learn from performing the role would be moot, as merely by getting started, without the subject initially knowing anything of the language in which the questions were posed, it would have refuted Searle. The reason the argument fails to make its case is that Searle does not have an effective response to the 'systems reply': in the paper, his response is a mixture of appeals to intuition, together with the erroneous claim that if the subject memorized the instructions, then the systems reply would not apply.
stcredzero 8y ago
That argument is fundamentally flawed. Yes. However, his other observation about AI turns out to be spot on.
- ThomPete 8y ago
  What observation?
  - thedailymail 8y ago
    I'm guessing the observation "that AI was nothing more than the automation and repackaging of the output of human intelligence" mentioned above
    - mannykannot 8y ago
      That's also how I take it, and there are two different types of AI being discussed here: the 'Chinese Room' paper is about the possibility of conscious machines, while the 'repackaging' comment is about current technology.
    - ThomPete 8y ago
      Ok but i am not sure people in the industry would claim otherwise, also not sure what the point is.

nohat 8y ago

What do you mean the 'smart talk' is exactly what he said? That needs clarification and citation to be a meaningful statement. Inventing the chinese room argument is a point against taking him seriously. It's a very confused argument. The fact that it keeps being trotted out is sad more than anything (not that I'm saying you are doing that).

stcredzero 8y ago
What do you mean the 'smart talk' is exactly what he said? It's the idea that AI's achievements have to do with the automation and re-packaging of the output of human intelligence. That was one of the main points of his talk. (To his credit, that was from several years back.) Inventing the chinese room argument is a point against taking him seriously. From a technical point of view, it seems valid to dismiss it as an argument against AI. However, given that the science behind consciousness is incomplete, I'm not sure the entire idea has no merit. In a way, the Chinese Room scenario reveals something about our current cultural biases about what constitutes consciousness. For example, it can be taken as a gedankenexperiment showing we'd have a hard time overcoming our biases if we were confronted with a consciousness operating at a speed 6 orders of magnitude slower than our own. On what empirical basis can we dismiss our intuitions on the matter? How can we be so sure that the logic for our dismissing the argument isn't just a technocrat ideology?
- nohat 8y ago
  I get that was his claim, but saying that this is now accepted is just not true. Saying that it is 'smart talk' implies that smart people are talking about it? Saying that the value in the Chinese Room argument is in showing how we have difficulty overcoming our biases is a clever inversion, but more a funny debate tactic than serious defense of Searle. Unless you are claiming that Searle doesn't actually believe it and invented it as some sort of long con? I don't think using brain biology and logic over intuition needs to be defended.
  - stcredzero 8y ago
    Saying that it is 'smart talk' implies that smart people are talking about it? I keep reading paraphrases of basically that idea in articles and blog posts. Another phrase which means the same thing refers to getting enough data, or declares that data is king. The latter often implies that what's happening is an automation and repackaging of human intelligence. Saying that the value in the Chinese Room argument is in showing how we have difficulty overcoming our biases is a clever inversion, more a funny debate tactic than serious defense of Searle. It's not even a funny debate tactic, if no one is actually defending the Chinese Room argument. In other words, there is a value in looking at why people are interested in the argument, despite the fact it fails. I don't think using brain biology and logic over intuition needs to be defended. The way that we apply science and logic in the context of our cultural understanding and human biases does need to be examined, however. History shows this quite clearly. I think it's very telling that instead of engaging in an analysis of the intuitions and biases which apply to human beings thinking about consciousness, you are merely interested in the dead-horse beating exhortation of the supremacy of arguments which have already been won. I think an examination of the above situation has tremendous value for people thinking seriously about AI's implications for future society. For example, there is a story about Murray Gell-Mann and how he flummoxed his grad students by being just slightly faster at getting to the next idea in conversations. One day, his grad students played a practical joke on him by enumerating all of the implications of a new line of thinking, ahead of time when he wasn't there, memorizing them, then blurting out the next idea when he met with them later. I also had a professor like that. He never thought things I never thought of, but he was always saying them just as I was formulating the idea, but before I could speak it. I think it's well worth it, to examine exactly what we mean when we think of someone being more or less intelligent. I suspect it will have important implications if and when we do develop a sentient AI.