2 ms·
<not this paper approach> This is one of the key prompting in a lot of Enterprise cases. You can currently prompt LLMs to add a confidence score along with the
by amrrs 2y ago
<not this paper approach>
This is one of the key prompting in a lot of Enterprise cases. You can currently prompt LLMs to add a confidence score along with their responses.
Especially when you are using LLMs for downstream NLP tasks.
The confidence score can be a great indicator also for applying a two-tier model approach!
- lyu07282 2y agoyeah but they say in the paper that isn't very accurate, they suggest a specifically fine-tuned model for it, not just prompting it for a score?
- bigfudge 2y agoIn my experience self validation performance and confidence ratings are both poor. I think the problem is that these sorts of formats just aren’t that common in the training data. What does help is to as a series of structured questions pertaining to quality and to aggregate those, but it’s still often not that helpful.
- kromem 2y agoThe challenge is the confidence scores can often be confabulations themselves.
- elicksaur 2y agoWhat is a confidence score like that based on?