6 ms·
Yes, this is achieved through a meta process > By sampling multiple responses from the LLM and considering the one with the highest confidence score, we can ad
by acchow 2y ago
Yes, this is achieved through a meta process
> By sampling multiple responses from the LLM and considering the one with the highest confidence score, we can additionally obtain more accurate responses from the same LLM, without any extra training steps
> Our proposed LLM uncertainty quantification technique, BSDetector, calls the LLM API multiple times with varying prompts and sampling temperature values (see Figure 1). We expend extra computation in order to quantify how trustworthy the original LLM response is
The data is there, but not directly accessible to the transformer. The meta process enables us to extract it
- fnordpiglet 2y agoI’d also note this isn’t confidence in the answer but in the token prediction. As LLMs have no conceptual “understanding,” they likewise have no computable confidence in accuracy of the correctness of their answers as we understand correctness. While certainly token confidence can be a proxy it’s not a substitute.
- adastra22 2y agoGood luck defining “understanding” in a way that lets you say LLMs don’t understand but humans do. At the end of the day we’re just a weighted neural net making seat of the pants confidence predictions too.
- acchow 2y ago> At the end of the day we’re just a weighted neural net making seat of the pants confidence predictions too. We might be. Or we might be something else entirely. Who knows?
- willy_k 2y agoRoger Penrose knows, imo
- adastra22 2y agoPenrose has no fucking clue. Sorry for the language, but direct speech is required here. It’s about as physically realistic as “intelligent design” is as an alternative to Darwinism. And similarly motivated. I would recommend Dennett’s “Consciousness Explained” if you want a more serious take on the subject.
- namaria 2y agoIf mental faculties are emergent properties, what is illogical about propounding that a larger system than us or than Earth itself might have conceived of designs such as we have for our systems?
- adastra22 2y agoWhere did that system come from? What designed it?
- willy_k 2y agoYou have to answer that question for any model of the universe - what came before the Big Bang? Another universe? What before it? And intelligent design is essentially analogous to simulation theory, and answers more questions than it creates (the anthropic principle, for starters). My personal mental model is that the ‘intelligence’ guides quantum collapse and so the progression of the universe is somewhat deterministic but also not really because ‘important’ collapse decisions are guided towards some higher purpose. This model also doesn’t necessarily require an actual intelligence, I imagine that with the quasi omnitemporal aspect of qm, in this model something like love or consciousness could be an optima that the system moves towards, the ‘love’ optimum would be maximum interpersonal quantum entanglement and ‘consciousness’ being maximizing coherent networks. Not that I have any delusions about my theory being the case, it’s just a model I’ve built up over a while and find interesting to think about, but I doubt it bears any weight on reality.
- 2y ago
- adastra22 2y agoWe can put electrodes in the brain and in neurons in Petri dishes, and see with our own eyes their activating behaviors. Our brains are weighted neural networks. This is certain.
- fnordpiglet 2y agoI didn’t say humans can compute how likely their answers are. I said LLM token prediction scores aren’t confidence of accuracy but likelihood of the tokens given the chain of prior token. The assertion that it is some sort of “accuracy” metric is false. The humans can’t either is a whatsboutism
- viraptor 2y ago> I’d also note this isn’t confidence in the answer but in the token prediction. I really don't understand the distinction you're trying to make here. Nor how do you define "computable confidence" - when you ask an LLM to give you a confidence value, it is indeed computed. (It may not be the value you want, but... it exists)
- acchow 2y ago> when you ask an LLM to give you a confidence value, it is indeed computed You mean the output of the transformer? It does not "compute" confidence values. It's still doing token prediction.
- viraptor 2y agoWhat's your example of a "computed" confidence value for an opinion given through text? I don't understand the requirements you have for this concept.
- fnordpiglet 2y agoThe assertion that the token likelihood metric is some sort of accuracy metric is false. There are more traditional AI techniques that compute probabilistic reasoning scores that are in fact likelihoods of accuracy. I’d note you can’t ask an LLM for a confidence value and get any answer that’s not total nonsense. The likelihood scores for the token prediction given prior tokens isn’t directly accessible to the LLM and isn’t intrinsically meaningful regardless in the way people hope it might be. They can quite confidentially produce nonsense with a high likelihood score.