I doubt there’ll ever be a time within the next 15-20 years at least where we’ll see a point of diminishing returns of increasing the model size. The reason is

I doubt there’ll ever be a time within the next 15-20 years at least where we’ll see a point of diminishing returns of increasing the model size. The reason is that we are still just scratching the surface on the input size to the model, 128K tokens of text is nothing compared to few frames of a single-stream HD video.