This is sort of like the compression algorithm "problem." For the overwhelming majority of inputs, compression algorithms don't compress, and it can be proven t

This is sort of like the compression algorithm "problem." For the overwhelming majority of inputs, compression algorithms don't compress, and it can be proven that on average they don't work. But we're not really interested in compressing things on average. What we use compression for amounts to edge cases of highly regularized or repeatable data. Thus the fact that LLMs can be proven in general to hallucinate doesn't necessarily imply that they must hallucinate in the types of situations for which we use them for. The paper itself discusses a number of mitigating strategies -- such as supplementing their training data with current information or using multiple LLMs to vote on the accuracy of the outcome -- only to basically brush them aside and advise not to use LLMs in any sort of critical situation. And that's probably true enough today, but in the future I think these strategies will greatly reduce the severity of these hallucinations. Just as we as human beings have developed strategies to reduce our reliance on pure memory. This reminds me of a deposition I had to give a number of years back. One of the lawyers asked me if I remembered how the plaintiff and I came to discuss a certain accusation leveled at him by the defendant. And I confidently stated, "Sure, he and I used to have various conversations about the issue and one day he the plaintiff brought up this thing that defendant said to him." And the lawyer says, if you want to, you can refer to your phone text log to refresh your memory. Then I looked at my phone, and the truth was that I myself had spoken to the defendant, and she told me the accusation, and then I went and shared it with the plaintiff. So, I basically remembered the situation exactly backwards, i.e., a hallucination, which I was able to repair by referring to real world information instead of just my memory.