3 ms·
> What would those probabilities mean in the context of these modern LLMs? They would mean understanding the sources of the information they use for inference,
by nine_k 2y ago
> What would those probabilities mean in the context of these modern LLMs?
They would mean understanding the sources of the information they use for inference, and the certainty of steps they make. Consider:
- "This conclusion is supported by 7 widely cited peer-reviewed papers [list follows]" vs "I don't have a good answer, but consider this idea of mine".
- "This crucial conclusion follows strongly from the principle of the excluded middle; its only logical alternative has been just proved false" vs "This conclusion seems a bit more probable in the light of [...], even though its alternatives remain a possibility".
I suspect that following a steep gradient in some key layers or dimensions may mean more certainty, while following an almost-flat gradient may mean the opposite. This likely can be monitored by the inference process, and integrated into a confidence rating somehow.
- bee_rider 2y agoI don’t think I disagree with your general point, but this is fairly different from what the comment above was looking for—confidence values that we can put next to our outputs. I mean, I don’t think such a value (it is definitely possible I’m reading it overly-specifically), like, a numerical value, can generally be assigned to the truthiness of a snippet of general prose. I mean, in your “7 peer reviewed papers” example, part of the point of research (a big part!) is to eventually overturn previous consensus views. So, if we have 6 peer reviewed papers with lots of citations, and one that conclusively debunks the rest of them, there is not a 6/7 chance that any random sentiment pulled out of the pile of text is “true” in terms of physical reality.