3 ms·
Yes, they are auto-completers, but they are auto-completers that are layered AND operate in higher dimensional spaces. This throws all intuitions off, and I thi
by snickell 2y ago
Yes, they are auto-completers, but they are auto-completers that are layered AND operate in higher dimensional spaces. This throws all intuitions off, and I think makes it misleading to think of them as "just" auto-completers. That's part of the story, but not the whole of it.
I suspect we are much closer to auto-completers than most of us like to think, but we're also trained+incentivized by culture, education, parenting, socializing, to produce "useful" results.
Maybe part of the problem is in the data set: how much modeling of "how to admit ignorance or uncertainty" are in LLMs training data sets? If you read the internet, all you see is confident replies to other confident replies. Ignorance or non-confidence tends to elicit either a bluff or non-response. If you read technical literature, you see much of the same.
Maybe LLMs are trained on a dataset, and thereby inherit a culture that's accidentally biased toward ignorant confidence. In human conversation, if somebody asks a question and I don't know the answer, I say I don't know. On the internet, I just skip it and leave it for somebody else who thinks they know.
All this is to say: maybe a statistical autocompleter can admit ignorance instead of firing "neural noise" based on barely-there loose associations. Maybe it just needs a stronger pathway toward talking about not knowing when there's not a strong association.
- 6510 2y agoWasn't it that it by design prefers things expressed with certainty?
- HarHarVeryFunny 2y agoOne problem is that the LLM's own knowledge, or lack of it, doesn't follow from any individual training sample(s). Even if there were a bunch of "I don't know X" samples in the training set, that ought to be trumped by one authoritative "X is ..." one. The next problem is that the LLM doesn't know how reliable it's various training sources are (unlike a human who might trust personal experience > textbook > twitter comment), or even which samples come from which source so that it could learn that.
- chaosist 2y agoI have thought a lot about this and I suspect "I don't know" would be devastating to the model. The magic is in the fact that the model can't say "I don't know". The model would have to have arbitrary thresholds and a type of domain/context classification in order to set this arbitrary threshold for the conversation in order to say "I don't know". It would create all these unsolvable boundary conditions. AGI in this context then would be minimizing the need for "I don't know" until it no longer applies. Is that possible? I don't know :) I would defer to Chomsky also on the subject that we are basically nothing like these models when it comes to language and we are not auto-completers.