3 ms·

There's no doubt that people pattern match and sometimes say they're fine reflexively. But the point is that the human in the Room can never do anything else o

There's no doubt that people pattern match and sometimes say they're fine reflexively. But the point is that the human in the Room can never do anything else or convey his true feelings, because it doesn't know the correspondence between 好 and a sensation or a sequence of events or a desire to appear polite, merely the correspondence between 好 and the probability of using or not using other tokens later in the conversation (and he has to look that bit up). He is able to discern nothing in your conversation typology below (a), and he doesn't actually know (a), he's simply capable of following non-Chinese instructions to look up a continuation that matches (a). The appearance to an external observer of having some grasp of (b) and (c) is essentially irrelevant to his thought processes, even though he actually has thought processes and the cards with the embedded knowledge of Chinese don't have thought processes. And, no it is still abso-fucking-lutely ludicrous to conclude that just because humans sometimes parrot, they aren't capable of doing anything else[1]. If humans don't always blindly pattern match conversation without any interaction with their actual thought processes, then clearly their ability to understand "how are you" and "good" is not synonymous with the "understanding" of a person holding up 好 because a book suggested he hold that symbol up. Combining the person and the book as a "union" changes nothing, because the actor still has no ability to communicate his actual thoughts in Chinese, and the book's suggested outputs to pattern match Chinese conversation still remain invariant with respect to the actor's thoughts. An actual Chinese speaker could choose to pick the exact same words in conversation as the person in the room, though they would tend to know (b) and some of (c) when making those word choices. But they could communicate other things, intentionally [1]That's the basic fallacy the "synonymous" argument rests on, though I'd also disagree with your assertions about education level. Frankly it's the opposite: ask a young child how they are and they think about whether their emotional state is happy or sad or angry or waaaaaaahh and use whatever facility with language to convey it, and they'll often spontaneously emit their thoughts. A salesperson who's well versed in small talk and positivity and will reflexively, for the 33rd time today, give an assertive "fantastic, and how are yyyyou?" without regard to his actual mood and ask questions structured around on previous interactions (though a tad more strategically than an LLM...).

throwaway09223 4y ago
"But the point is that the human in the Room can never do anything else" I disagree. I think the point is that the union of the human and the library can in fact do all of those things. The fact that the human in isolation can't is as irrelevant as pointing out that the a book in isolation (without the human) can't either. It's a fundamental mistake as to the problem's reasoning. "And, no it is still abso-fucking-lutely ludicrous to conclude that just because humans sometimes parrot, they aren't capable of doing anything else" Why? What evidence do you have that humans aren't the sum of their inputs? What evidence do you have that "understanding" isn't synonymous with "being able to produce a sufficient response?" I think this is a much deeper point than you realize. It is possible that the very nature of consciousness centers around this dynamic; that evolution has produced systems which are able to determine the next appropriate response to their environment. Seriously, think about it.
- notahacker 4y ago
  > I disagree. I think the point is that the union of the human and the library can in fact do all of those things. No, the "union of the human and the library" can communicate only the set of responses a programmer, who is not part of the room, made a prior decision to make available. (The human can also choose to refuse to participate, or hold up random symbols but this fails to communicate anything). If the person following instructions on which mystery symbols to select ends up convincing an external observer they are conversing with an excitable 23 year old lady from Shanghai, that's because the programmer provided continuations including those personal characteristics, not because the union of a bored middle aged non-Chinese bloke and lots and lots of paper understands itself to be an excitable 23 year old lady from Shanghai. Seriously, this is madness. If I follow instructions to open a URL which points to a Hitler speech, it means I understood how to open links, not that the union of me and YouTube understands the imperative of invading Poland! > The fact that the human in isolation can't is as irrelevant as pointing out that the a book in isolation (without the human) can't either. It's a fundamental mistake as to the problem's reasoning. Do you take this approach to other questions of understanding? If somebody passes a non-Turing test by diligently copying the answer sheet, do you insist that the exam result accurately represents the understanding of the union of the copyist and the answer sheet, and people questioning whether the copyist understood what they were writing are quibbling over irrelevances? The reasoning is very simple: if a human can convincingly simulate understanding simply by retrieving answers from storage media, it stands to reason a running program can do so too, perhaps with even less reason to guess what real world phenomena the symbols refer to. An illustrative example of how patterns can be matched without cognisance of the implications of the patterns Inventing a new kind of theoretical abstraction of "union of person and storage media" and insisting that understanding can be shared between a piece of paper and a person who can't read the words on it like a pretty unconvincing way to reject that claim. But hey, maybe the union of me and the words you wrote thinks differently?! > I think this is a much deeper point than you realize. It is possible that the very nature of consciousness centers around this dynamic; that evolution has produced systems which are able to determine the next appropriate response to their environment. It's entirely possible, probable even, the very nature of consciousness centres around ability to respond to an environment. But a biological organism's environment consists of interacting with the physical world via multiple senses, a whole bunch of chemical impulses called emotions and millions of years of evolving to survive in that environment as well as an extremely lossy tokenised abstract representation of some of those inputs used for communication purposes. Irrespective of whether a machine can "understand" in some meaningful sense, it stretches credulity to assert that the "understanding" of a computer program whose inputs consist solely of lossy tokens is similar or "synonymous" to the understanding of the more complex organism that navigates lots of other stuff.

throwaway09223 4y ago

"But the point is that the human in the Room can never do anything else" I disagree. I think the point is that the union of the human and the library can in fact do all of those things. The fact that the human in isolation can't is as irrelevant as pointing out that the a book in isolation (without the human) can't either. It's a fundamental mistake as to the problem's reasoning. "And, no it is still abso-fucking-lutely ludicrous to conclude that just because humans sometimes parrot, they aren't capable of doing anything else" Why? What evidence do you have that humans aren't the sum of their inputs? What evidence do you have that "understanding" isn't synonymous with "being able to produce a sufficient response?" I think this is a much deeper point than you realize. It is possible that the very nature of consciousness centers around this dynamic; that evolution has produced systems which are able to determine the next appropriate response to their environment. Seriously, think about it.

notahacker 4y ago
> I disagree. I think the point is that the union of the human and the library can in fact do all of those things. No, the "union of the human and the library" can communicate only the set of responses a programmer, who is not part of the room, made a prior decision to make available. (The human can also choose to refuse to participate, or hold up random symbols but this fails to communicate anything). If the person following instructions on which mystery symbols to select ends up convincing an external observer they are conversing with an excitable 23 year old lady from Shanghai, that's because the programmer provided continuations including those personal characteristics, not because the union of a bored middle aged non-Chinese bloke and lots and lots of paper understands itself to be an excitable 23 year old lady from Shanghai. Seriously, this is madness. If I follow instructions to open a URL which points to a Hitler speech, it means I understood how to open links, not that the union of me and YouTube understands the imperative of invading Poland! > The fact that the human in isolation can't is as irrelevant as pointing out that the a book in isolation (without the human) can't either. It's a fundamental mistake as to the problem's reasoning. Do you take this approach to other questions of understanding? If somebody passes a non-Turing test by diligently copying the answer sheet, do you insist that the exam result accurately represents the understanding of the union of the copyist and the answer sheet, and people questioning whether the copyist understood what they were writing are quibbling over irrelevances? The reasoning is very simple: if a human can convincingly simulate understanding simply by retrieving answers from storage media, it stands to reason a running program can do so too, perhaps with even less reason to guess what real world phenomena the symbols refer to. An illustrative example of how patterns can be matched without cognisance of the implications of the patterns Inventing a new kind of theoretical abstraction of "union of person and storage media" and insisting that understanding can be shared between a piece of paper and a person who can't read the words on it like a pretty unconvincing way to reject that claim. But hey, maybe the union of me and the words you wrote thinks differently?! > I think this is a much deeper point than you realize. It is possible that the very nature of consciousness centers around this dynamic; that evolution has produced systems which are able to determine the next appropriate response to their environment. It's entirely possible, probable even, the very nature of consciousness centres around ability to respond to an environment. But a biological organism's environment consists of interacting with the physical world via multiple senses, a whole bunch of chemical impulses called emotions and millions of years of evolving to survive in that environment as well as an extremely lossy tokenised abstract representation of some of those inputs used for communication purposes. Irrespective of whether a machine can "understand" in some meaningful sense, it stretches credulity to assert that the "understanding" of a computer program whose inputs consist solely of lossy tokens is similar or "synonymous" to the understanding of the more complex organism that navigates lots of other stuff.