When you’re churning through terrawords of training data and aren’t cross-validating against proportionally-sized hold out sets? It’s not a certainty that you’

When you’re churning through terrawords of training data and aren’t cross-validating against proportionally-sized hold out sets? It’s not a certainty that you’re overfitting but the burden is to show otherwise. Besides, if your corpus is asymptomatically everything, why wouldn’t fitting it perfectly fitting it be the goal? It would probably be more accurate to say that the bias/variance tradeoff loses meaning as the training set goes to infinity. Grapevine is that the big actors are going to 5x their flops at any price in the next three years. What does validation loss even mean at that scale?