2 ms·
"We show how to detect confabulations by developing a quantitative measure of when an input is likely to cause an LLM to generate arbitrary and ungrounded answe
by Animats 2y ago
"We show how to detect confabulations by developing a quantitative measure of when an input is likely to cause an LLM to generate arbitrary and ungrounded answers. ... Intuitively, our method works by sampling several possible answers to each question and clustering them algorithmically into answers that have similar meanings."
That's reasonable for questions with a single objective answer. It probably won't help when multiple, equally valid answers are possible.
However, that's good enough for search engine applications.