From the paper: "In this work, we continue the scaling line of language modeling improvements and train a 540 billion parameter, densely activated, autoregressi

From the paper: "In this work, we continue the scaling line of language modeling improvements and train a 540 billion parameter, densely activated, autoregressive Transformer on 780 billion tokens of high-quality text." The number of parameters is nearly the number of text tokens. So aren't they simply overfitting a lot?