We're in the hype cycle now at the moment, but the capabilities have been taking off for a few years. GPT-3 came out in 2020, and in retrospect, could have been

We're in the hype cycle now at the moment, but the capabilities have been taking off for a few years. GPT-3 came out in 2020, and in retrospect, could have been trained several years before that, given that the 175 billion parameters of GPT-3 turned out to be way too many for its level of performance (as discovered by Chinchilla). I agree that there are a lot of refinements that can be made even with the current LLMs, but it's unclear how far LLMs will continue to improve.