Catching "hallucinations" is quite useful for many applications. I'm doing some research in mitigating effects of factual errors in LLM generated answers for pu

Catching "hallucinations" is quite useful for many applications. I'm doing some research in mitigating effects of factual errors in LLM generated answers for public agencies, where giving a factually wrong answer may be illegal. If those could be detected (with sufficient accuracy), the system could simply decline to give an answer and ask the user to contact the agency. Training the models not to give wrong answers (or giving them less) in the first place would of course be even better. Unnecessary complications come also from the use of pre-trained commercial black-box LLMs through APIs, which is (sadly) the way LLMs are used in applications in vast majority of times. These could perhaps be fine tuned through the APIs too, but it tends to be rather fiddly and limited and very expensive to do for large synthetic datasets like would be used here. P.S. I found it quite difficult to figure out from the article how the "semantic entropy" (actually multiple different entropies) is concretely computed. If somebody is interested in this, it's a lot easier to figure out from the code: https://github.com/jlko/semantic_uncertainty/blob/master/semantic_uncertainty/uncertainty/uncertainty_measures/semantic_entropy.py https://github.com/jlko/semantic_uncertainty/blob/master/sem...