5 ms·

I've always felt the Chinese Room thought experiment was founded on a complete misunderstanding of bilingualism. The only "algorithm" that can effectively trans

I've always felt the Chinese Room thought experiment was founded on a complete misunderstanding of bilingualism. The only "algorithm" that can effectively transform text between English and Chinese would be one that understands the source and target texts like a human. So the Chinese Room is really just a more complicated Turing Test - a computer can only convince an observer that it "understands Chinese" if it is also able to convince an observer that it understands the conversation the same way as a human, in either language.

bilsbie 3y ago
I thought so too but LLM’s sort of made me change my mind.
- willprice89 3y ago
  I think LLM's support my theory. The reason they are good at translation is because they are already good at simulating understanding in all of the languages they translate between.
JadeNB 3y ago
> The only "algorithm" that can effectively transform text between English and Chinese would be one that understands the source and target texts like a human. How do you know?
- pixl97 3y ago
  Or the corollary viewpoint, that if ChatGPT can do this as the previous poster says, then it is a type of AGI.
  - Jensson 3y ago
    Yeah, people used to think that since only humans can do arithmetics it was the peak of intelligence, and computers can do arithmetics better than humans, then AGI was just around the corner. Now people argue that since only humans have complex language it was the peak of intelligence, and now computers can do that language like humans, then AGI is just around the corner.
    - lacrimacida 3y ago
      Well, I think I need to remind you that humans invented arithmetics, humans invented computers, computer languages, artificial intelligence and so on. I think you are underestimating how complex the human thought is now that we have LLMs. They appear to have unlimited progress but they’re eventually bound to what they’re trained on.
      - Jensson 3y ago
        That was my point though. The AGI optimist community always think that when a computer solves a new thing that computers previously didn't solve then AGI is near. Then a decade later we realize that there are still plenty of things that only humans can solve.
  - willprice89 3y ago
    To clarify, I'm using "understand" in a sense similar to the Turing Test. It doesn't matter if the machine actually understands in the same way as humans (or at all), only that it is able to make humans think it understands. If it can convince native speakers of both languages it is human, it has enough "understanding" to perform a good translation.
- willprice89 3y ago
  Because translation isn't a one-to-one transformation. A bilingual speaker doesn't have a single abstract thought that they are then able to express in two different languages. Rather the expressions themselves represent similar thoughts that are grounded in the overall cultural context surrounding the language. Translation attempts to minimize the difference these expressions have on their respective audiences, but doing so requires understanding the text like the target audience. Another way to think about it is that translation is not a question of "Express this English sentence in Chinese", but rather "Rewrite this sentence in the style of (some specific Chinese person)". You'll get a very different result if the target is "physics professors from Beijing" versus "rice farmers from Yunnan".
mrshadowgoose 3y ago
I've always viewed the Chinese Room and similar thought experiments as a distraction from reasoning about the practical implications of machine learning. We don't understand "understanding" in the vast majority of practical ML systems that currently exist, let alone the physical implementation of "understanding" in our own brains. And yet, understanding "understanding" in ML has never been a pre-requisite to economic viability. GPT4 can displace certain low-grade cognitive laborers regardless of whether or not we understand how it works, or whether or not its understanding is "true".
nullc 3y ago
The logical error is that it's confusing a part with the whole. Clearly a single neuron in your brain does not understand English. Yet no one disputes that you understand English. The non-chinese-speaker is just a component of the ChineseRoom system, that it doesn't understand Chinese is no more remarkable than a single neuron of yours not understanding English. It doesn't say anything about the overall system understanding Chinese or not. I think it appeals to people with a more superficial understanding of computing that under appreciates the fact that instructions on paper processed by a universal circuit (or by a human emulating one) are morally equivalent to hardwired logic (if slower). So they disregard the contribution of the room, and focus on the fact that the meat-bot inside doesn't understand Chinese so the room must not either.

bilsbie 3y ago

I thought so too but LLM’s sort of made me change my mind.

willprice89 3y ago
I think LLM's support my theory. The reason they are good at translation is because they are already good at simulating understanding in all of the languages they translate between.

JadeNB 3y ago

> The only "algorithm" that can effectively transform text between English and Chinese would be one that understands the source and target texts like a human. How do you know?

pixl97 3y ago
Or the corollary viewpoint, that if ChatGPT can do this as the previous poster says, then it is a type of AGI.
- Jensson 3y ago
  Yeah, people used to think that since only humans can do arithmetics it was the peak of intelligence, and computers can do arithmetics better than humans, then AGI was just around the corner. Now people argue that since only humans have complex language it was the peak of intelligence, and now computers can do that language like humans, then AGI is just around the corner.
  - lacrimacida 3y ago
    Well, I think I need to remind you that humans invented arithmetics, humans invented computers, computer languages, artificial intelligence and so on. I think you are underestimating how complex the human thought is now that we have LLMs. They appear to have unlimited progress but they’re eventually bound to what they’re trained on.
    - Jensson 3y ago
      That was my point though. The AGI optimist community always think that when a computer solves a new thing that computers previously didn't solve then AGI is near. Then a decade later we realize that there are still plenty of things that only humans can solve.
- willprice89 3y ago
  To clarify, I'm using "understand" in a sense similar to the Turing Test. It doesn't matter if the machine actually understands in the same way as humans (or at all), only that it is able to make humans think it understands. If it can convince native speakers of both languages it is human, it has enough "understanding" to perform a good translation.
willprice89 3y ago
Because translation isn't a one-to-one transformation. A bilingual speaker doesn't have a single abstract thought that they are then able to express in two different languages. Rather the expressions themselves represent similar thoughts that are grounded in the overall cultural context surrounding the language. Translation attempts to minimize the difference these expressions have on their respective audiences, but doing so requires understanding the text like the target audience. Another way to think about it is that translation is not a question of "Express this English sentence in Chinese", but rather "Rewrite this sentence in the style of (some specific Chinese person)". You'll get a very different result if the target is "physics professors from Beijing" versus "rice farmers from Yunnan".

mrshadowgoose 3y ago

I've always viewed the Chinese Room and similar thought experiments as a distraction from reasoning about the practical implications of machine learning. We don't understand "understanding" in the vast majority of practical ML systems that currently exist, let alone the physical implementation of "understanding" in our own brains. And yet, understanding "understanding" in ML has never been a pre-requisite to economic viability. GPT4 can displace certain low-grade cognitive laborers regardless of whether or not we understand how it works, or whether or not its understanding is "true".

nullc 3y ago

The logical error is that it's confusing a part with the whole. Clearly a single neuron in your brain does not understand English. Yet no one disputes that you understand English. The non-chinese-speaker is just a component of the ChineseRoom system, that it doesn't understand Chinese is no more remarkable than a single neuron of yours not understanding English. It doesn't say anything about the overall system understanding Chinese or not. I think it appeals to people with a more superficial understanding of computing that under appreciates the fact that instructions on paper processed by a universal circuit (or by a human emulating one) are morally equivalent to hardwired logic (if slower). So they disregard the contribution of the room, and focus on the fact that the meat-bot inside doesn't understand Chinese so the room must not either.