3 ms·
It's a language model, so it's good at generating language. Any congruency with knowledge of what the language expresses is purely accidental side effect.
by scotty79 4y ago
It's a language model, so it's good at generating language. Any congruency with knowledge of what the language expresses is purely accidental side effect.
- FeepingCreature 4y agoSurely human language use (GPT-3's training domain) has some connection to reality, however faint.
- scotty79 4y agoYeah. That connection is the source of this accidental side effect.
- FeepingCreature 4y agoThat seems a stretch to call an "accidental side effect" though. It's an effect exactly inasmuch as human language use is inextricable from reality. Put otherwise, GPT doesn't interact with reality directly, but mediated through our language. But we also don't experience reality directly, so it's hard to see how much of a hindrance that would pose. Hell, much of our model of the world is just as rooted in language as GPT's. The part of my model of reality of which I have direct sensory experience is tiny.
- scotty79 4y ago> Put otherwise, GPT doesn't interact with reality directly, but mediated through our language. Yes, but the human language is garbage. So while passing throught that filter without maintaining the highest degree of precision barely any of the actual reality remains. Truth sounds exactly the same as a lie. The actual language that decently captures reality is math. Nothing less is even barely sufficient to produce almost anything but nonsense. Nonsense nicely sounding to human ears but still nonsense.
- MayeulC 4y agoNot necessarily. Training pushes language models towards generalization and "understanding" of concepts. It certainly "knows" some stuff. However it can't reason about it. There have been some articles on HN recently on how accurately LMs build world models.
- scotty79 4y agoIt's an illusion. For any thing it "knows" you can persuade it to claim exactly opposite thing. It just randomly landed on correct thing first just because it seen it more often in the input data. Despite appearing 100% confident about everything it has actual 0% of confidence about anything it says. Although it insists on some things bit longer than on others.
- FeepingCreature 4y ago> For any thing it "knows" you can persuade it to claim exactly opposite thing. Which is actually a novel capability and arises because the network does reinforcement learning over its own context window. It's a strength, not a weakness. Humans can do the same thing. ("Assume that X...") > It just randomly landed on correct thing first just because it seen it more often in the input data. Isn't that just a description of learning? It's true that the network has no idea what is "true". But it's not like we do either, all we do is learning from correlations. We're just better at it.
- MayeulC 4y agoWell, you are right that it may learn statistical associations that we associate/generalize ourselves with/as a model when we probe it. I think the key bit here is that we can influence its associations by inputting the prompt, changing the model that is presented. I'm waay ahead of myself here, but the thought is interesting, and it will likely remain an open question for at least a few months/years.
- lostoldsession 4y agoAt this point I'm unconvinced that humans are much more than a language model. Worse, I'm only saying that half jokingly.
- scotty79 4y agoTry math competition between 15 year old who gets math and Chat GPT. Humans who can't do math might be a pure language models though.
- ilostmyshoes 4y agoHumans are able to connect words to abstract ideas rather than simply what words go next to each other