4 ms·

I agree that one should take Searle's argument literally, and I doubt there are many people on either side who suppose that the room's operator understands eith

I agree that one should take Searle's argument literally, and I doubt there are many people on either side who suppose that the room's operator understands either the questions being posed to the room or the answers it delivers. The dispute is whether there is anything significant in this assumption - and, more specifically, whether Searle is right in supposing that if Hard AI is possible, then it would follow that the room's operator must understand the questions, their answers, and why they are appropriate answers. You may recognize that, by distinguishing between the operator and the room, I am prefiguring the so-called "systems reply", which is that the system as a whole is demonstrating whatever level of understanding is manifest here, and there is no assumption, in so supposing, that this understanding is manifest in any single component, even if one of those components has the ability to understand some things on its own. By deliberate construction, Searle is using the room's operator to mechanistically implement an algorithm, a task that does not require knowledge of the language in which the questions are posed (in fact, he is using the language barrier to prevent the operator answering the questions himself.) Searle has not presented an argument for the operator needing to do anything more, and it is obvious that people can be trained to perform tasks they do not have any understanding of beyond the specific actions involved. Searle's attempt to refute this reply seems to show that he cannot even conceive it fully, as that response is to modify the scenario such that the operator memorizes the rule book, as if where it is stored makes any difference. Searle's stance here seems to me to be a version of the homunculus fallacy, supposing that our minds are implemented by (as opposed to being) a conscious entity.

mjburgess 5y ago
I think it is significant that the room, even as a system, does not use language. A hashtable lookup cannot be what it is to "act in the world with words". When I say, "do you like what I'm wearing?" -- regardless of what has been written in any book, at any time, I do not want that reply. I may want those words. But the reply I want is to use your judgement of taste and experiences of the world to tell me your opinion. The words dont answer the question if the reason they are used is incorrect. A hashtable lookup can, basically, never be a reason to use words. And hence the Chinese room reveals much much more than simply a circular point. And the systems reply fails to say much much less than it needs to. The systems reply needs to explain "by what system" the system's use of language counts as a use. "by what system" words are born of understanding, not "mere reterival". To do this, you need to specify the whole world, experience, etc. as "computational systems". And thus the systems reply simply fails against this example. It isnt sufficient to say "maybe" you have to say, "and this is how!". And worse, the world is clearly not a computer -- as any non-trivial definition (eg., universal turing machine) cannot simulate properties the world has (non-determinism, chaos, etc.).
- mannykannot 5y ago
  In your first post in this thread you wrote "the Chinese room needs to be taken super-literally" - a sentiment I agreed with - but since then (starting immediately afterwards, in fact) you seem to think that Searle is restricting the capabilities of the room to those of a hash table. I am pretty sure that a close reading of his paper will fail to reveal any such restriction, and, in fact, if it did, that would count against his argument. Similarly, nothing you say, here or elsewhere among these replies, about the limitations of hash tables, supports Searle's argument (or more general arguments against Hard AI.) Nor is Searle arguing that the Chinese room, if it could be implemented, must use language (beyond the capabilities that he does require of it, which is answering questions from an ill-defined domain.) And as far as I can tell, he does not require it to, as you somewhat vaguely put it, "act in the world with words" beyond this. The systems reply does not need to do anything more than show that Searle is making some unargued-for assumptions about what Hard AI would imply - and Searle's response to it so completely misses the point of that reply that he is actually helping make it clear! (For more details, see the second paragraph of my previous post; as far as I can tell, no argument has been raised against the points I made there.) The limitations of Turing machines are similarly beside the point. A computer, such as those involved in the composition and delivery of this reply, is not just a Turing machine (not even a universal one), even though it is capable of implementing one (up to the physical limits of memory). In particular, a computer (as opposed to a Turing machine) can incorporate environmental inputs, including physically-derived entropy, and can use these in simulating physical processes. You are evidently fond of the phrase "[Reality/the world] is not a computer"; that may be so, but rather more definitively, I think, one can say that the mind is an information processor. I understand that it seems inconceivable to you that a mind could be produced by a suitably-programmed computer of any power, and I am aware that I can only offer hand-wavy arguments for that proposition. I think you are mistaken, however, to read Searle's Chinese Room argument as being about, let alone supporting, the broader intuitions you have expressed in your reply to me and to others. Searle, who is, after all, a philosopher of considerable stature, is well aware that you need something more than just one's intuitions to make a respectable argument; that alone would not pass peer review.
- tsimionescu 5y ago
  > And worse, the world is clearly not a computer -- as any non-trivial definition (eg., universal turing machine) cannot simulate properties the world has (non-determinism, chaos, etc.). This is precisely wrong. With the possible exception of consciousness, in our current understanding of physics, the world is a computer. I Bieber some of your intuition fails because you have wormgly assumed this point to be obvious. You're also mixing up the Chinese room experiment with the larger problem of an interactive AI. The Chinese room system is restricted, by definition, to only interact with the word through language. Over time, as it interacts with a person, it may decide that it likes them or not, and its responses may start changing based on this - that is allowed to be part of the algorithm that the human/machine inside the room is applying. For a silly example, the algorithm may say that 'people who use the same word three times in a row are extremely nice', and use this state of 'extremely nice' to modify its future responses once the same word has been recorded three times in a row. This is no different, fundamentally, to me liking people whose face is symmetric, or the same gender as myself - it's not a choice I made, it's just an arbitrary rule of my (genetic) algorithm.

mjburgess 5y ago

I think it is significant that the room, even as a system, does not use language. A hashtable lookup cannot be what it is to "act in the world with words". When I say, "do you like what I'm wearing?" -- regardless of what has been written in any book, at any time, I do not want that reply. I may want those words. But the reply I want is to use your judgement of taste and experiences of the world to tell me your opinion. The words dont answer the question if the reason they are used is incorrect. A hashtable lookup can, basically, never be a reason to use words. And hence the Chinese room reveals much much more than simply a circular point. And the systems reply fails to say much much less than it needs to. The systems reply needs to explain "by what system" the system's use of language counts as a use. "by what system" words are born of understanding, not "mere reterival". To do this, you need to specify the whole world, experience, etc. as "computational systems". And thus the systems reply simply fails against this example. It isnt sufficient to say "maybe" you have to say, "and this is how!". And worse, the world is clearly not a computer -- as any non-trivial definition (eg., universal turing machine) cannot simulate properties the world has (non-determinism, chaos, etc.).

mannykannot 5y ago
In your first post in this thread you wrote "the Chinese room needs to be taken super-literally" - a sentiment I agreed with - but since then (starting immediately afterwards, in fact) you seem to think that Searle is restricting the capabilities of the room to those of a hash table. I am pretty sure that a close reading of his paper will fail to reveal any such restriction, and, in fact, if it did, that would count against his argument. Similarly, nothing you say, here or elsewhere among these replies, about the limitations of hash tables, supports Searle's argument (or more general arguments against Hard AI.) Nor is Searle arguing that the Chinese room, if it could be implemented, must use language (beyond the capabilities that he does require of it, which is answering questions from an ill-defined domain.) And as far as I can tell, he does not require it to, as you somewhat vaguely put it, "act in the world with words" beyond this. The systems reply does not need to do anything more than show that Searle is making some unargued-for assumptions about what Hard AI would imply - and Searle's response to it so completely misses the point of that reply that he is actually helping make it clear! (For more details, see the second paragraph of my previous post; as far as I can tell, no argument has been raised against the points I made there.) The limitations of Turing machines are similarly beside the point. A computer, such as those involved in the composition and delivery of this reply, is not just a Turing machine (not even a universal one), even though it is capable of implementing one (up to the physical limits of memory). In particular, a computer (as opposed to a Turing machine) can incorporate environmental inputs, including physically-derived entropy, and can use these in simulating physical processes. You are evidently fond of the phrase "[Reality/the world] is not a computer"; that may be so, but rather more definitively, I think, one can say that the mind is an information processor. I understand that it seems inconceivable to you that a mind could be produced by a suitably-programmed computer of any power, and I am aware that I can only offer hand-wavy arguments for that proposition. I think you are mistaken, however, to read Searle's Chinese Room argument as being about, let alone supporting, the broader intuitions you have expressed in your reply to me and to others. Searle, who is, after all, a philosopher of considerable stature, is well aware that you need something more than just one's intuitions to make a respectable argument; that alone would not pass peer review.
tsimionescu 5y ago
> And worse, the world is clearly not a computer -- as any non-trivial definition (eg., universal turing machine) cannot simulate properties the world has (non-determinism, chaos, etc.). This is precisely wrong. With the possible exception of consciousness, in our current understanding of physics, the world is a computer. I Bieber some of your intuition fails because you have wormgly assumed this point to be obvious. You're also mixing up the Chinese room experiment with the larger problem of an interactive AI. The Chinese room system is restricted, by definition, to only interact with the word through language. Over time, as it interacts with a person, it may decide that it likes them or not, and its responses may start changing based on this - that is allowed to be part of the algorithm that the human/machine inside the room is applying. For a silly example, the algorithm may say that 'people who use the same word three times in a row are extremely nice', and use this state of 'extremely nice' to modify its future responses once the same word has been recorded three times in a row. This is no different, fundamentally, to me liking people whose face is symmetric, or the same gender as myself - it's not a choice I made, it's just an arbitrary rule of my (genetic) algorithm.