> LLMs don’t know when they are hallucinating What makes you think that? The fact that you can ask them to stop hallucinating and that this does reduce halluci

> LLMs don’t know when they are hallucinating What makes you think that? The fact that you can ask them to stop hallucinating and that this does reduce hallucinations seems to me like a strong indication of the opposite? That is what the OP's article is about. > They don’t know when they don’t know something. The bigger models tend to be able to articulate some of their limits? See e.g this recent article [0]. > They know facts but they have no reasoning. Is that really true? They can solve many novel tasks that seem to require some level of reasoning. They do not possess the full range of reasoning capabilities most humans have, but do pretty well on the HellaSwag dataset which aims to test reasoning [1], even when test set leakage was not a big problem yet [2]. I have 10+ years of experience in machine learning, and I don't find your claims obvious at all. In fact, I see more and more indications of the opposite. Also if you look mechanistically at how LLMs work inside, there is nothing stopping self-knowledge, reasoning or hallucination-detection. LLMs are very black box. [0] https://arxiv.org/abs/2305.18153 https://arxiv.org/abs/2305.18153 [1] https://deepgram.com/learn/hellaswag-llm-benchmark-guide https://deepgram.com/learn/hellaswag-llm-benchmark-guide [2] https://paperswithcode.com/sota/sentence-completion-on-hellaswag https://paperswithcode.com/sota/sentence-completion-on-hella...