5 ms·

For me, the strongest argument in this article is “There is a point where it understands is the most parsimonious explanation, and we have clearly passed it”.

by fwlr 4y ago

For me, the strongest argument in this article is “There is a point where it understands is the most parsimonious explanation, and we have clearly passed it”. Those who deny that ChatGPT understands have to move their goalposts every few weeks; OpenAI’s release schedule seems to be slightly faster, so in time it seems even the fastest-moving goalposts will be outrun by the LLMs. One specific flavor of “ChatGPT doesn’t understand things” I see here and elsewhere - no straw man intended - is that humans completing a language task are doing something fundamentally different than LLMs completing the same language task. To take the example from the article and a comment about it in this thread: if a human were to apply English instructions to a question asked in Chinese, the human is understanding the instructions to achieve that. If an LLM were to apply English instructions to a question asked in Chinese, that is because words across languages with similar meanings are tightly connected in its statistical model, so instructions that affect the English words will also affect the Chinese words, purely through statistical means. This is certainly a more sophisticated and nuanced and believable rebuttal than the crude “mere regurgitation” response. But it’s just as dangerous. In the end, the only thing that’s ‘uniquely human’ is being human, everything else is outputs from a black box. Arguments that ‘what’s inside the black box matters’ are risky, because the outputs gradually converge to complete indistinguishability; there’s no bright line to step off that train, you’ll end up claiming only humans can understand because understanding is a thing only humans can do - or worse (as the article describes) denying your own ability to understand, because your brain is a flesh-instantiated statistical approximator of the Platonic understanding process, and the silicon-instantiated statistical approximator of the Platonic understanding process that cannot be allowed to claim to understand differs only in its medium of instantiation.

fwlr 4y ago
It occurred to me after writing that post that understanding is just what the algorithm feels like from the inside. A human understands English instructions in a way that lets them apply those instructions to Chinese language tasks with the same meaning. We might ask, what is this in the physical structure of the human brain? What’s the specific arrangement of neurons and neuronal connections that is doing this? Assume neuroscience has the answer and can give you a picture of the neurons in question and detail their activation pattern. When we use understanding to do our language task, we don’t actually feel our neurons lighting up, we don’t feel an activation pattern rush through our brain. It just feels like we understand, because that’s what the algorithm feels like from the inside. https://www.lesswrong.com/posts/yA4gF5KrboK2m2Xu7/how-an-algorithm-feels-from-inside https://www.lesswrong.com/posts/yA4gF5KrboK2m2Xu7/how-an-alg...
YeGoblynQueenne 4y ago
>> Those who deny that ChatGPT understands have to move their goalposts every few weeks; OpenAI’s release schedule seems to be slightly faster, so in time it seems even the fastest-moving goalposts will be outrun by the LLMs. No, they don't. It doesn't matter how many releases OpenAI makes, there is still no alternative explanation needed for the behaviour of their systems than the fact that their systems are language models trained to reproduce their training corpus. And btw, the people who point that out, like me for example, are not "denying" anything, just like atheists are not "denying" the existence of god, just because there are people who believe in it. It's the people who believe that a language model can "understand" who are _asserting_ that LLMs understand, and it is they who have to explain how. Which so far has not been done. All we have is exclamations of strong belief, and waving of the hands.
- fwlr 4y ago
  Comparing this to atheism is very illuminating, actually. In some ways this topic is very different to atheism/theism. In the ‘belief in god’ debate both sides are trying to take away the other side’s belief; it’s important to atheists to stop people believing in God, while to believers it’s important to stop people disbelieving in God, so there’s a lot of convincing being attempted, people seeking each other out to tell them that they cannot or must not hold a certain belief, and so forth. This topic is very different! If I want to go off and believe LLMs can understand and reason and think, you don’t really care; likewise, when you believe that, although it might be producing a novel rephrasing, the information in that rephrasing is merely reproduced from the training corpus, I don’t really care. (Beyond active engagements in discussion threads, anyhow - not trying to imply either of us don’t care about the discussion we’re having right now!) But in other ways it is very similar to the atheism/theism debate. For a time I used to argue God exists because the idea of God controls and causes very specific behaviors from a great many people; a naive diagram of the situation has this God figure and He’s telling these little people figures to do certain things and they do exactly those things (which is why “pointing out that certain figures are hypocritically not doing those things” feels important and like it might be evidence against), and the atheist argument that you can’t actually detect God’s existence is like erasing the God figure from the diagram: it leaves a conspicuous hole in the diagram, and the hole is shaped like God, and if you were to naively fill in the hole in the diagram to explain what’s causing the people to behave that way, the precise shape of the hole forces you to end up re-drawing God (this argument was very unconvincing to atheists for multiple reasons and I’m not trying to re-litigate it here). It became clear to me that atheists were not interested in this level of proof, it had to be physical existence; some ‘God-stuff’ that you could grab a handful of and hold up to the light and say “here it is”. This topic is very similar! For me, it is enough that the diagram of an LLM’s capabilities has a hole shaped exactly like “it understands” and I’m happy to fill in the obvious blank; for you, there needs be an actual process of understanding somewhere in the code and the weights that we could hold up to the light and say “here is the understanding” (and that is how you can claim that no matter how good LLMs get they will never be understanding - all of their processes are already fully described and explained, there’s nowhere for a process of understanding to hide).

by fwlr 4y ago

fwlr 4y ago

It occurred to me after writing that post that understanding is just what the algorithm feels like from the inside. A human understands English instructions in a way that lets them apply those instructions to Chinese language tasks with the same meaning. We might ask, what is this in the physical structure of the human brain? What’s the specific arrangement of neurons and neuronal connections that is doing this? Assume neuroscience has the answer and can give you a picture of the neurons in question and detail their activation pattern. When we use understanding to do our language task, we don’t actually feel our neurons lighting up, we don’t feel an activation pattern rush through our brain. It just feels like we understand, because that’s what the algorithm feels like from the inside. https://www.lesswrong.com/posts/yA4gF5KrboK2m2Xu7/how-an-algorithm-feels-from-inside https://www.lesswrong.com/posts/yA4gF5KrboK2m2Xu7/how-an-alg...

YeGoblynQueenne 4y ago

>> Those who deny that ChatGPT understands have to move their goalposts every few weeks; OpenAI’s release schedule seems to be slightly faster, so in time it seems even the fastest-moving goalposts will be outrun by the LLMs. No, they don't. It doesn't matter how many releases OpenAI makes, there is still no alternative explanation needed for the behaviour of their systems than the fact that their systems are language models trained to reproduce their training corpus. And btw, the people who point that out, like me for example, are not "denying" anything, just like atheists are not "denying" the existence of god, just because there are people who believe in it. It's the people who believe that a language model can "understand" who are _asserting_ that LLMs understand, and it is they who have to explain how. Which so far has not been done. All we have is exclamations of strong belief, and waving of the hands.

fwlr 4y ago
Comparing this to atheism is very illuminating, actually. In some ways this topic is very different to atheism/theism. In the ‘belief in god’ debate both sides are trying to take away the other side’s belief; it’s important to atheists to stop people believing in God, while to believers it’s important to stop people disbelieving in God, so there’s a lot of convincing being attempted, people seeking each other out to tell them that they cannot or must not hold a certain belief, and so forth. This topic is very different! If I want to go off and believe LLMs can understand and reason and think, you don’t really care; likewise, when you believe that, although it might be producing a novel rephrasing, the information in that rephrasing is merely reproduced from the training corpus, I don’t really care. (Beyond active engagements in discussion threads, anyhow - not trying to imply either of us don’t care about the discussion we’re having right now!) But in other ways it is very similar to the atheism/theism debate. For a time I used to argue God exists because the idea of God controls and causes very specific behaviors from a great many people; a naive diagram of the situation has this God figure and He’s telling these little people figures to do certain things and they do exactly those things (which is why “pointing out that certain figures are hypocritically not doing those things” feels important and like it might be evidence against), and the atheist argument that you can’t actually detect God’s existence is like erasing the God figure from the diagram: it leaves a conspicuous hole in the diagram, and the hole is shaped like God, and if you were to naively fill in the hole in the diagram to explain what’s causing the people to behave that way, the precise shape of the hole forces you to end up re-drawing God (this argument was very unconvincing to atheists for multiple reasons and I’m not trying to re-litigate it here). It became clear to me that atheists were not interested in this level of proof, it had to be physical existence; some ‘God-stuff’ that you could grab a handful of and hold up to the light and say “here it is”. This topic is very similar! For me, it is enough that the diagram of an LLM’s capabilities has a hole shaped exactly like “it understands” and I’m happy to fill in the obvious blank; for you, there needs be an actual process of understanding somewhere in the code and the weights that we could hold up to the light and say “here is the understanding” (and that is how you can claim that no matter how good LLMs get they will never be understanding - all of their processes are already fully described and explained, there’s nowhere for a process of understanding to hide).