4 ms·

> ignoring the fact that is the program that is responsible for the behavior, not the human This is called the systems response to the Chinese Room Argument. [

> ignoring the fact that is the program that is responsible for the behavior, not the human This is called the systems response to the Chinese Room Argument. [0] Searle's response [0] goes something like this: You're missing the point. Suppose the man in the room memorised the program, so that he could answer the questions himself with no need of external aids. Now, the man and the system are the same, and yet by your account, the man doesn't speak Chinese, but the system does. Personally I find the whole argument to bear no clear connection to the questions of consciousness. The question of whether a system consciously understands a problem is muddled, as it isn't clear a priori that problem-solving competence has any connection to consciousness. To use Dennett's term, a system can be competent without comprehension. A pocket calculator is quite unable to explain what it's doing, but is able to perform superhuman number-crunching. The argument may succeed in tying the reader in a knot, but I figure that's just because most people haven't thought much about what conscious comprehension really means. Another problem with the Chinese Room Argument is that, if it really holds up, it ought to hold up just as well against the human species itself. If you're going to attack the idea that consciousness can arise from non-conscious components, where does that leave us? [0] https://en.wikipedia.org/wiki/Chinese_room#Systems_and_virtual_mind_replies:_finding_the_mind https://en.wikipedia.org/wiki/Chinese_room#Systems_and_virtu...

GoblinSlayer 6y ago
If the man acts as a virtual machine that faithfully runs the program, then it's still the program that understands Chinese, it doesn't matter on what machine it runs. In this case there are two minds: the man's mind and program's mind.
- MaxBarraclough 6y ago
  Mind strikes me as a loaded term, being used to mean locus of computation or computation stream, but carrying the implication of consciousness. If you wish to make the case that computation and consciousness are in some sense intertwined, this needs to be done explicitly. The Chinese Room Argument does not do so. (Incidentally I'm of the opinion that they are indeed intertwined.) If a person is manually executing an algorithm (the memorised Chinese Room algorithm, or naive matrix multiplication, or whatever), then we could distinguish between the algorithm and the person executing it. In that sense there are two computation streams at work. That doesn't show that a second consciousness has been brought into being, though. I wouldn't conclude that there are two 'minds'. Your comparison to virtual machines is a good one, it's one that Dennett uses.
  - GoblinSlayer 6y ago
    It's implied that Chinese Room algorithm works in the same way as human mind/brain/consciousness/intelligence/younameit. If the executed algorithm is consciousness, then a second consciousness has been brought into being, same as for first consciousness.
    - MaxBarraclough 6y ago
      > If the executed algorithm is consciousness, then a second consciousness has been brought into being, same as for first consciousness. A lot hinges on If the executed algorithm is consciousness. I can see the sense in that argument though, in that the 'first consciousness' is acting as the computational substrate for the 'second consciousness', the way the physical action of neurons acts as our computational substrate. I'm not convinced this maps to using a card system to have a conversation in a foreign language. It doesn't seem self-evident that doing so should be considered enough to demonstrate consciousness, it's only enough to demonstrate than the algorithm is effective in having a conversation. It's not self-evidently the equivalent of an accurate real-time computer simulation of a specific person, for instance. I think you can make a strong case that such a system would amount to a conscious person and should be treated as such. (It would follow from this that shutting down a holodeck simulation of someone is morally fraught.) The only alternative would be to morally privilege neuron-based architectures over transistor-based architectures, which seems like an uphill philosophical battle. I don't put much stock in the question of whether the card-based algorithm 'understands' the conversation it is having. Is it competent? Yes. Is it conscious? I'm not convinced it is. Asking whether it understands is to conflate these two questions.
      - GoblinSlayer 6y ago
        Competence here is indistinguishability from human in conversation. The most straightforward way to implement it is to make the algorithm have the same structure and work in the same way as human mind. We can say that it understands and is conscious because it's not different from human mind in structure and operation. It's a good part about artificial algorithms that they are transparent and we can show that they aren't only effective in conversation, but have everything to be had behind that conversation. It's due to the latter. Simply put, the algorithm isn't GPT, but AGI. >It's not self-evidently the equivalent of an accurate real-time computer simulation of a specific person, for instance. I think you can make a strong case that such a system would amount to a conscious person and should be treated as such. This was touched recently in art. If you're interested, it's "Sword art online Alicization". The shutdown problem is applicable to AI too. What's new is that the life of virtual people is shown at length and discussed questions as to what identity those people should have, what worldview, religion, philosophy, pride, dignity, justice.
bondarchuk 6y ago
This is not directed at you personally but I find the whole conundrum tiresome and frustrating. Systems reply, Searle's reply, virtual mind reply, and then Searle replies something like "but a virtual mind can not be really conscious". At the end we are just left with the question "do you agree or disagree with functionalism and computationalism" and all the arguments on both sides turn out to be just empty statements of agreement or disagreement.
- MaxBarraclough 6y ago
  We agree. The Chinese Room Argument offers little insight into consciousness. All it really does is to take someone with confused thoughts on how 'understanding' works (specifically the way intellectual competence interacts with consciousness), and to tie them in a knot. It fails to demonstrate anything interesting about consciousness. It certainly doesn't demonstrate that computer systems can never be conscious in the way we can. Nothing in the argument applies any more or less to neurons than to transistors.

GoblinSlayer 6y ago

If the man acts as a virtual machine that faithfully runs the program, then it's still the program that understands Chinese, it doesn't matter on what machine it runs. In this case there are two minds: the man's mind and program's mind.

MaxBarraclough 6y ago
Mind strikes me as a loaded term, being used to mean locus of computation or computation stream, but carrying the implication of consciousness. If you wish to make the case that computation and consciousness are in some sense intertwined, this needs to be done explicitly. The Chinese Room Argument does not do so. (Incidentally I'm of the opinion that they are indeed intertwined.) If a person is manually executing an algorithm (the memorised Chinese Room algorithm, or naive matrix multiplication, or whatever), then we could distinguish between the algorithm and the person executing it. In that sense there are two computation streams at work. That doesn't show that a second consciousness has been brought into being, though. I wouldn't conclude that there are two 'minds'. Your comparison to virtual machines is a good one, it's one that Dennett uses.
- GoblinSlayer 6y ago
  It's implied that Chinese Room algorithm works in the same way as human mind/brain/consciousness/intelligence/younameit. If the executed algorithm is consciousness, then a second consciousness has been brought into being, same as for first consciousness.
  - MaxBarraclough 6y ago
    > If the executed algorithm is consciousness, then a second consciousness has been brought into being, same as for first consciousness. A lot hinges on If the executed algorithm is consciousness. I can see the sense in that argument though, in that the 'first consciousness' is acting as the computational substrate for the 'second consciousness', the way the physical action of neurons acts as our computational substrate. I'm not convinced this maps to using a card system to have a conversation in a foreign language. It doesn't seem self-evident that doing so should be considered enough to demonstrate consciousness, it's only enough to demonstrate than the algorithm is effective in having a conversation. It's not self-evidently the equivalent of an accurate real-time computer simulation of a specific person, for instance. I think you can make a strong case that such a system would amount to a conscious person and should be treated as such. (It would follow from this that shutting down a holodeck simulation of someone is morally fraught.) The only alternative would be to morally privilege neuron-based architectures over transistor-based architectures, which seems like an uphill philosophical battle. I don't put much stock in the question of whether the card-based algorithm 'understands' the conversation it is having. Is it competent? Yes. Is it conscious? I'm not convinced it is. Asking whether it understands is to conflate these two questions.
    - GoblinSlayer 6y ago
      Competence here is indistinguishability from human in conversation. The most straightforward way to implement it is to make the algorithm have the same structure and work in the same way as human mind. We can say that it understands and is conscious because it's not different from human mind in structure and operation. It's a good part about artificial algorithms that they are transparent and we can show that they aren't only effective in conversation, but have everything to be had behind that conversation. It's due to the latter. Simply put, the algorithm isn't GPT, but AGI. >It's not self-evidently the equivalent of an accurate real-time computer simulation of a specific person, for instance. I think you can make a strong case that such a system would amount to a conscious person and should be treated as such. This was touched recently in art. If you're interested, it's "Sword art online Alicization". The shutdown problem is applicable to AI too. What's new is that the life of virtual people is shown at length and discussed questions as to what identity those people should have, what worldview, religion, philosophy, pride, dignity, justice.

bondarchuk 6y ago

This is not directed at you personally but I find the whole conundrum tiresome and frustrating. Systems reply, Searle's reply, virtual mind reply, and then Searle replies something like "but a virtual mind can not be really conscious". At the end we are just left with the question "do you agree or disagree with functionalism and computationalism" and all the arguments on both sides turn out to be just empty statements of agreement or disagreement.

MaxBarraclough 6y ago
We agree. The Chinese Room Argument offers little insight into consciousness. All it really does is to take someone with confused thoughts on how 'understanding' works (specifically the way intellectual competence interacts with consciousness), and to tie them in a knot. It fails to demonstrate anything interesting about consciousness. It certainly doesn't demonstrate that computer systems can never be conscious in the way we can. Nothing in the argument applies any more or less to neurons than to transistors.