This is just another diagonalization against some finite thing, like the no free lunch theorem. An "LLM" in their definition is essentially any finite thing whi

This is just another diagonalization against some finite thing, like the no free lunch theorem. An "LLM" in their definition is essentially any finite thing which predicts the next token. The same result applies to your brain too, for example. Like all such diagonalization results, it is not really relevant for real world considerations. The reason is that it does not matter if your model fails on none, finitely many or infinitely many inputs. In reality the space of possible inputs is equipped with a probability measure, and the size of the hallucinating inputs set w.r.t. that measure is relevant. Diagonalization arguments usually, make no claim to the size of that set, and it is most likely negligible in the real world.