3 ms·
> GPT does not only model the language patterns that humans intentionally put into text. GPT models all of the patterns that exist in the text. > Humans, when
by AnimalMuppet 4y ago
> GPT does not only model the language patterns that humans intentionally put into text. GPT models all of the patterns that exist in the text.
> Humans, when writing, do "distill" thought into language; but the result is messy: we end up encoding more patterns than we intended to. Our behavior, our choices, our reasons, and the surrounding situation we exist in, all inform our writing. Those are the patterns that get encoded into the text, right alongside language.
Sometimes truly, and sometimes falsely. Take this paragraph, for instance. If I made a typo on the fifth word, and left out the "l", this paragraph would also encode something that is not at all what it is about. A human reader might chuckle at the typo, but would not be confused by the meaning. But for GPT, that will be a pattern in the text, just like all the others. So sometimes it can reveal what was unintended but is still informative, and sometimes it can be misleading.
But it's not just typos. There's very intentional text from advertisers, shill, liars, and propagandists. That is all patterns that get encoded, too. The lies are encoded right along with the truth.
But where I was really going initially was the hallucinations. If I ask GPT who won this year's NASCAR race on Tierra del Fuego, it will find something - some pattern that matches better than other patterns. Who knows, it might find some kid's fanfic posted on Reddit, and will tell me who won in the kid's story. But if there's nothing like that, it will find something. It won't be any kind of a meaningful pattern that it finds, but it will still present it, and will present it with as much certainty as it would if it found a number of news stories that all agreed with each other.
So I'm seeing in your posts a pattern encoded (perhaps falsely) that makes me think that you think that what the text encodes is gospel. It's not. It's much that is golden and much that is dross, and GIGO applies to GPT as much as any other information system.
- thomastjeffery 4y ago> So I'm seeing in your posts a pattern encoded (perhaps falsely) that makes me think that you think that what the text encodes is gospel. On the contrary. I think you have a pretty good description for how the behavior we want and the behavior we don't want exist as the same features. I am trying to restructure the narrative around LLMs to be about the patterns themselves, not what we hope the patterns to be. We can throw around the word, "hallucinate", but that's just another anthropomorphization. Whether or not the pattern followed "makes sense" is a categorization humans attribute after the fact. GPT doesn't do "certainty". It does what it is given.
- AnimalMuppet 4y agoOK. I think I understand, and I think I agree. Thanks for the conversation.
- thomastjeffery 4y agoYou're welcome! I've been finding a certain satisfaction in trying to articulate this perspective. It's challenging because I'm criticizing most of the narrative that exists in the subject, but that narrative also gives a clear illustration of where to go wrong. I think better understanding the more mysterious patterns that exist in our writing might be really useful. Often, the first step in inventing something useful is to explore an unfamiliar domain and recognize what information we are missing.