I don't think there is any math showing that it's the models size that limits "fact" storage, to the extent these models store facts. And model size definitely

I don't think there is any math showing that it's the models size that limits "fact" storage, to the extent these models store facts. And model size definitely does not change the fact that all LLMs will write things based on "how" they are trained, not on how much training data they have. Big models will produce nonsense just as readily as small models. To fix that properly we likely need training objective functions that incorporate some notion of correctness of information. But that's easier said than done.