3 ms·
Question from someone with a surface-level knowledge of how LLMs work: As I understand it, LLMs work, at a very high level, by repeatedly choosing the token wi
by duskwuff 3y ago
Question from someone with a surface-level knowledge of how LLMs work:
As I understand it, LLMs work, at a very high level, by repeatedly choosing the token with the highest probability. Could potential confabulations be flagged by marking low-confidence sequences of text in the output? This seems like it's be particularly good at detecting when the model is making up a name or number, for example.
- Icko 3y agoThat's a pretty good idea, actually. It'll be a bitch and a half to tune the threshold so it's reliable, and doesn't mark normally generated text though.
- dizhn 3y agoAs far as I understand there's a parameter to make the output more "novel" (different/interesting/less predictable or generic) and that works by specifically not using the highest probability.
- mcguire 3y agoI suspect the probability of a statement being false (a confabulation) is unrelated to the probabilities of the words and grammar making up the statement.
- duskwuff 3y agoThat's certainly possible, but I'd want to hear from someone who'd actually tried it before dismissing the concept.