It wouldn't matter if they trained on a quadrillion tokens, or another ten orders of magnitude. There's only so much information you call stuff into a given set

It wouldn't matter if they trained on a quadrillion tokens, or another ten orders of magnitude. There's only so much information you call stuff into a given set of numbers. But once again I am reminded, never make arguments based on information theory. Nobody understands it.