7 ms·

There isn't much new here. Skip ahead to the first audience question from Ray Kurzweil (http://www.youtube.com/watch?v=rHKwIYsPXLg&t=38m51s http://www.youtube.c

• chubot 11y ago
  Yeah honestly I don't get what he is really contributing (and I'm sort of an AI skeptic). In 2000 in undergrad, I recall checking out some of his books from the library because people said he was important, and I learned about the "Chinese Room" argument [1] in class. How is it even an argument? It doesn't illuminate anything, and it's not even clever. It seems like the most facile wrong-headed stab at refutation, by begging the question. As far as I can tell, the argument is, "well you can make this room that manipulates symbols like a computer, and of course it's not conscious, so a computer can't be either"? There are so many problems with this argument I don't even know where to begin. The fact that he appears to think that changing a "computer" to a "room" has persuasive power just makes it all the more antiquated. As if people can't understand the idea that computers "just" manipulate symbols? Changing it to a "room" adds nothing. [1] http://plato.stanford.edu/entries/chinese-room/ http://plato.stanford.edu/entries/chinese-room/
  • drdeca 11y ago
    I thought the idea was that because the only part of the room actually doing things (the person), doesn't understand chinese? I mean, agree with it or not, but I think that's a bit stronger than just, making it seem intuitively worse because its a room instead of "a computer"? I think the important part isn't the swap of "room" for "computer", but instead the swap of "person" for "cpu"?
    • chubot 11y ago
      Yeah but it just makes me more confused? How does that say anything about a computer then? There's no human being who doesn't understand something inside a computer.
      • drdeca 11y ago
        The idea is, roughly, if a person in the place of the cpu does not understand chinese, then the cpu doesn't understand chinese. And because the cpu is the part that does stuff, like the person, then if the system w/ the person doesn't understand chinese, then the computer w/ the cpu doesn't understand chinese. Because there's nothing to make the cpu more understand-y, only things to make it less understand-y, and otherwise the systems are the same.
    • Chathamization 11y ago
      Yeah, but the system would be the person + the lookup tables, not just the person. The problem is, we don't tend to say "does a room with a person in it and several books have this knowledge?" Relying on a system that doesn't tend to get grouped together (there's no term for the system human + book inside room), and having only one animate object (so that people think of the animate object as the system instead of the animate and inanimate objects), as well as asking the question only about the animate part of the system, all seem to suggest that the purpose of the thought experiment is to mislead people. A better example would be saying something like - does this company have the knowledge to make a particular product? We can say that no individual member of the company does, but the company as a whole does.
      • drdeca 11y ago
        I think this is called the "systems response". Which, well there's a whole series of responses back and forth, with different ideas about what is or is not a good response. One idea describes a machine where each state of the program is pre-computed, and the computer steps through the states one by one, but in each state, if the next of the pre-computed states was wrong (i.e. would not be the next step of the program, following from the current state), if (e.g.) a switch was flipped, it would cause the program to be computed correctly despite the pre-computed states being wrong, and if the switch is not flipped, then it would continue along the pre-computer states. If the switch is flipped on, or if its flipped off, and all the pre-computer states are correct, the same things happen, and it does not interact with the switch at all. If all the pre-computed states are nonsense, and the switch is flipped on, then it runs the program correctly, despite the pre-computed states being nonsense. So, suppose that if the pre-computed states are all wrong, and the switch is on, that that counts as conscious. Then, if the pre-computed states are all correct, and the switch is on, would that still be conscious? What if almost all the pre-computed states were wrong, but a few were right? It doesn't seem like there is an obvious cutoff point between "all the pre-computed steps are wrong" and "all the pre-computer steps are right", where there would be a switch between what is conscious. So then, one might conclude that the one where all the pre-computed steps are right, and the switch is on, is just as conscious as the one which has the switch on but all the pre-computed states are wrong. But then what of the one where all the pre-computed states are right, and the switch is off? The switch does not interact with the rest of the stuff unless a pre-computed next step would be wrong, so how could it be that when the switch is on, the one with all the pre-computations is conscious, but when it is off, it isn't? But the one with all the pre-computations correct, and the switch off, is not particularly different from just reading the list of states in a book. If one grants consciousness to that, why not grant it to e.g. fictional characters that "communicate with" the reader? One might come up with some sort of thing like, it depends on how it interacts with the world, and it doesn't make sense for it to have pre-computed steps if it is interacting with the world in new ways, that might be a way out. Or one could argue that it really does matter if the switch is flipped one way or the other, and when you flip it back and forth it switches between being actually conscious and being, basically, a p-zombie. And speaking of which you could say, "well what if the same thing is done with brain states being pre-computed?", etc. etc. I think the Chinese Room problem, while not conclusive, is a useful introduction to these issues?
  • dnautics 11y ago
    I was never satisfied with the chinese room thought experiment. Let's momentarily replace the thing in the chinese room with a human, to parse Searle's notion of "understanding". Searle would argue that a human trained in to emit meaningful chinese characters would still lack understanding. But I think this is backwards and speaks to your identification of Searle begging the question: the only way a human could emit meaningful chinese responses would be if it had an understanding of chinese. Consequently if a machine is outputting meaningful chinese, it too must already understand chinese, and any argument otherwise is kind of a pro-biology bigotry with a shaky underlying logic at best. This then devolves into semantics. Can a person locked in a room really come to "understand" chinese culture, for example, if only non-experiential learning were used as data inputs? I think we have to say the answer is yes. I am a chemist. I have never seen an atomic orbital with my bare eyes, yet I can design chemical reactions that work with my understanding of chemistry. Because I have not experienced an atomic orbital does that mean I do not understand? Even, when I set up my first reaction, I did not have any experience, and knew what I was doing only through what could be described as sophisticated analogy. I would say my understading was low, but it was certainly non-zero. Where does one draw the line?
    • rjsw 11y ago
      I have always felt that the human in the room would start to recognize patterns and develop an "understanding". Their "understanding" may have no basis in reality but I don't see that it is any less valid to them. If Searle is right then we should be able to perform a MRI on a blind person while they are talking to someone and spot the point where their brain switches into "symbol manipulation mode" when the conversation subject becomes something visual.
      • qbrass 11y ago
        The guy in the room can memorize all of the rules and the ledgers and give you the same responses the room did, and if you asked him in his native language if he knew Chinese, he'd honestly tell you no. He could have an entire conversation with you in Chinese, and only know that what you said merits the response he gave. He doesn't if he's telling you directions to the bathroom, or how to perform brain surgery.
  • simonh 11y ago
    It's just obfuscation. In reality the 'room' would have to be the size of a planet and if one person was manipulating the symbols it might take the whole thing the life span of the universe to think 'Hello World'. But by phrasing it as a 'room' with one person in it he makes it look inadequate to the task, and therefore the task impossible.
    • DonaldFisk 11y ago
      Neither the size of the room nor the speed of the computation is important to Searle's argument. You could replace the person in the room with the population of India (except for those who understand Chinese), and pretend to the Chinese speaker that the communication is by surface mail. Or use a bank of supercomputers if Indians aren't fast enough.
      • simonh 11y ago
        Fair enough. In which case Sear's argument is that even fantastically complex, sophisticated information processing systems with billions of moving parts and vast information storage retrieval resources operating over long periods of time cannot be intelligent. If that's what his position boils down to, what does casting it as a single person in a room add to the argument? As Kurzweil asked, how is that different from neurons manipulating neurotransmitter chemistry? Searl doesn't seem to have an answer to that.
        • DonaldFisk 11y ago
          No, his position, as I understand it, is that it cannot be conscious. It certainly can be intelligent. Searle does try to explain why there's a difference. Although the person in the Chinese Room might be conscious of other things, he has no consciousness of understanding the Chinese text he's manipulating, and will readily verify this, and nothing else in the room is conscious. Chinese speakers are conscious of understanding Chinese.
  • alfapla 11y ago
    The Chinese room argument is actually needlessly convoluted. Just imagine a piece of paper on which three words are printed: "I AM SAD". Now is there anyone who believes that this piece of paper is actually feeling sad just because "it says so"? Of course not. Now, suppose we replace this piece of paper with a small tablet computer that changes its displayed "mood" over time according to some algorithm. Now in my opinion it is rather hard to imagine that all of a sudden consciousness will "arise" in the machine like some ethereal ghost and the tablet will actually start experiencing the displayed emotion. Because it's basically still the same piece of paper.
    • deleted 11y ago
      [deleted]
    • gpderetta 11y ago
      The Chinese room argument is actually needlessly convoluted. Just imagine a piece of paper on which I draw a face that looks sad. Now is there anyone who believes that this piece of paper is actually feeling sad just because it looks sad? Of course not. Now, suppose we replace this piece of paper with an organic machine made of cells, blood and neurons which changes its displayed "mood" over time according to some algorithm. Now in my opinion it is rather hard to imagine that all of a sudden consciousness will "arise" in the machine like some ethereal ghost and the organic machine will actually start experiencing the displayed emotion. Because it's basically still the same piece of paper.
    • Scarblac 11y ago
      AND YET, human brains are an implementation of such an algorithm. By all reasoning, we shouldn't be conscious. Yet here I am, I am the one who is seeing what my eyes see and I am distinct from you. Science still has no idea how that happens, as far as I know. So how knows, maybe all computer programs are in fact conscious in some way.
  • dragonwriter 11y ago
    The Chinese Room argument has always seemed to me a powerful illustration of the problem that "consciousness" is so poorly defined as to be not subject to meaningful discussion dressed up as a meaningful argument against AI consciousness. Its always distressed me that some people take it seriously as an argument against AI consciousness; it does a better job of illustrating the incoherence of the fuzzy concept of consciousness on which the argument is based.
  • tobiasSoftware 11y ago
    As a believer in Weak AI the Chinese Room argument really gave me more understanding of my position. His argument is based on the concept that interpretation of symbols is not the same as the understanding that we do. As an example, say that a person learns 1 + 1 = 2. Because that person understands the concept, he can then go apply it to other situations, and figure out that 1 + 2 = 3. Whereas because the Chinese room is just interpreting symbols, so when the computer is asked the question "what is 1 + 1?" and can answer "2" via lookup table, but the person inside the room has gained no understanding of the actual question so he can't then use that knowledge in different circumstances and know without looking up that 1 + 2 = 3. The Chinese Room argument is that because computers can't "learn", everything has to be taught to them directly, whereas humans are able to take knowledge given and apply it to other situations. While some computers can "learn" enough rules to follow patterns, the argument is that computers can't "jump the track" and that humans can.
    • chubot 11y ago
      Yeah I see that, but the problem is that we don't know how humans are conscious, i.e. where meaning rises. If you believe that brains are just atoms, then meaning arises from "dumb physics" somewhere. Another way to think of it is: a fetus, or a sperm, or ova is not conscious. Some might argue that a newborn isn't really conscious. Somewhere along the line it becomes conscious. How does that happen? Where is the line? We have no idea. You can't assert that meaning can't arise from "dumb symbol manipulation" without understanding how meaning arises in the former case. We simply don't know enough to make any sort of judgement. The Chinese room argument is trying to make something out of nothing. We don't know.
  • ecopoesis 11y ago
    I've always thought that the Chinese Room proved just the opposite of what Searle thinks it does. I think of it this way: I have two rooms: one has a person who doesn't speak Chinese in it, but they have reference and books that allow them to translate incoming papers into Chinese perfectly. The second room just someone who speaks Chinese, and can translate anything coming in perfectly. Searle says that AIs are like person in room one: they don't know Chinese. I would argue that is the wrong way to look at things. A better comparison is that an AI is like the system of room 1, which does know Chinese, and from observation is indistinguishable from the system of room 2. What's going on inside (a human with Chinese reference books vs a human who knows Chinese) doesn't matter, it's just internal processing. If it walks like a duck and quacks like a duck, then it's a duck. If a machine claims to be conscious, and I can't tell it apart from another conscious being, who am I to say it isn't conscious?
• cscurmudgeon 11y ago
  I think we are missing the gist of the Chinese room argument here. The correct question to ask: How is a machine manipulating symbols (that someone says is conscious) different from any other complex physical system? Is New York city's complex sewer system conscious. What about the entire world's sewer & plumbing system? Does a machine have to compute some special function to be conscious? Does the speed of computation matter? If so who measures the speed? (let us not bring in general relativity as speed of computation can be different for different observers. Kurzweil et al's definition of consciousness is exactly as silly as Searle saying "My dog has consciousness because I can look at it and conclude that it has consciousness."
  • chubot 11y ago
    Those are good questions, but I don't see how the Chinese room argument helps with any of them. If anything, it confuses things by dragging a fictional/impossible construct into the argument, rather than just using real examples that people actually understand.
    • dang 11y ago
      I thought the point was to construct an example that most people agree wouldn't count as consciousness, then ask how a computer is any different.
      • chubot 11y ago
        Maybe, but there are too many holes in the construct to make it useful IMO. It just provokes endless confused debate rather than illuminating anything. IMO Kurzweil's response is actually spot on, although not that hard to come up with. You could make the argument: in your brain, there are just atoms following the laws of physics. The atoms have no choice in the matter, and know nothing about Chinese, or hunger, love, life goals, etc. Your brain is entirely composed of atoms so you can't be conscious. Obviously "meaning" arises from mindless syntax or mindless physics somewhere in the process. We just don't know where. The Chinese room doesn't bring us any closer to that understanding, and doesn't refute anything.
• dekhn 11y ago
  So basically, the way to convince Searle (not that that is a real goal) is to build a robot automaton which passes the uncanny valley: very responsive eyes. A collection of tricks. Clever responses. Searle would look at that and conclude it had consciousness.