4 ms·

Sorry but I think you got the first part wrong. It is not at all arguing that sentience cannot emerge from non-sentient parts: Searle is a materialist. Some of

• nullc 4y ago
  I've never spoken directly with Searle, but I have with several of his students and all have been emphatic that the Chinese Room Argument is demonstrates that a computer program can't have intelligence. Quoting Searle himself, "The point of the argument is this: if the man in the room does not understand Chinese on the basis of implementing the appropriate program for understanding Chinese then neither does any other digital computer solely on that basis because no computer, qua computer, has anything the man does not have." I think the most succinct description of his error is substituting the (lack of) understanding of a part of the system (the man) for the understanding of the entire system (the rules and file cabinets, etc.). But I'm interested in learning that I'm mistaken. You could turn his position around and say it's not the computer itself that's intelligent when a Chinese room system exhibits intelligence but the program -- and I suppose I'd agree with that, but it's also just semantics, uninteresting, and I don't believe he's ever taken that position. I do agree that "merely replicating a behavior" doesn't prove much, but I don't think the Chinese room speaks to that substantially. It might if it demanded that the room implement only a very simple input to output map, but it doesn't: it allows the room to implement anything a computer program can implement. (A fact I use in my post to point out that the room could (in our land of hypotheticals) implement the molecular dynamics of an entire human being) GPT has structural properties that make it very easy to classify it as an entirely different thing than a human mind. GPT is frozen in time, it cannot have an internal existence due to how its structured. It doesn't even have memory. It cannot learn (unless you include the whole company training it as part of 'it') except in the sense that it can immediately adapt to the output right in front of it, but can't preserve the knowledge. Theoretically if you made it arbitrarily large you could say it was close enough to having memory by always evaluating its complete history, but because its size grows quadratically with its window that isn't practicaly (and might not be possible to train-- it's totally credible that beyond some size these models will lose performance we just haven't gotten there yet). Figuring out how to train these models make good use of 'memory' is an ongoing challenge, since efficient memory isn't differentiable just ordinarily training with memory as part of the process doesn't work. Except by 'thinking out loud' in its output GPT also has a fixed upper bound on the time it can spend thinking any thought which is seemingly unlike a human mind.
  • epups 4y ago
    >Quoting Searle himself, "The point of the argument is this: if the man in the room does not understand Chinese on the basis of implementing the appropriate program for understanding Chinese then neither does any other digital computer solely on that basis because no computer, qua computer, has anything the man does not have." > I think the most succinct description of his error is substituting the (lack of) understanding of a part of the system (the man) for the understanding of the entire system (the rules and file cabinets, etc.). But I'm interested in learning that I'm mistaken. As you know, there have been many replies to this thought experiment, and some of the most interesting ones (to me) go in the direction you went here, ie, where is "understanding" occuring? The most basic version of the Chinese Room does intend to make you see yourself literally as a man who does not understand any Chinese and is just asked to look up symbols in a list. Perhaps that man doesn't understand Chinese, but the room as a whole at least gives the impression that it does. However, I think the most important aspect is not this "intuition pump" as Daniel Dennett calls it. To me, what is key here is that we can all agree that such a Chinese Room, or ChatGPT for that matter, does not necessarily replicate the fundamental mechanisms of human cognition. Then, it follows that other human properties such as awareness or qualia do not necessarily emerge from such cognitive architectures in the same way that it emerges from our brains. To me, Searle's point is ultimately that we don't know enough about the human mind to be able to judge whether it can be replicated artificially. And now that we have almost literally developed a Chinese Room, we can see that clearly. The arguments you bring up in your last paragraph are a great example of that, it's just very hard to conceive that this thing is conscious at all, even though it is capable of producing output that could convince people of that. Regarding Searle's quote that you brought up, I think "solely on that basis" is doing a lot of heavy lifting there, but it does align with what I said previously. He is saying that simply producing intelligent output, like in 1974 translation would represent, does not mean you are reasoning in a human way.
    • IshKebab 4y ago
      > To me, Searle's point is ultimately that we don't know enough about the human mind to be able to judge whether it can be replicated artificially. There's string circumstantial evidence that we do. And really "computers can simulate the physics in the brain" is the null hypothesis. In any case why is the Chinese Room always stated as if it has a clear conclusion rather than "this doesn't really prove anything" if we don't know enough to say either way? > And now that we have almost literally developed a Chinese Room I don't think so. The GPTs are currently still very far from the complexity of the human brain, and they are missing many features that may make a big difference to consciousness - for instance the ability to learn while running. So while it may be fairly easy to say ChatGPT isn't conscious/sentient, that isn't the question. It's whether computers theoretically can't be conscious because consciousness comes from some physical property that they can't reproduce (like quantum microtubule crankery).