In this case no, from the data observed, it still looks like we can train with 500T+ tokens before we start seeing overfitting. We are severely under training

In this case no, from the data observed, it still looks like we can train with 500T+ tokens before we start seeing overfitting. We are severely under training. I don't think anyone realized how much training was needed till OpenAI, once OpenAI started observing emergent behaviors they released to show what was possible and set almost a trend of how long to train. Fortunately Meta decided to buck that trend to see what would happen, and what we have seen is smaller models get smarter.