Moore's Law is not really dead. Even if you froze the datasets and software architecture in place right now, LLMs would get much better simply because compute

Moore's Law is not really dead. Even if you froze the datasets and software architecture in place right now, LLMs would get much better simply because compute costs are coming down. The cheaper training is, the more people you have chipping away at little changes and modifications. And you get more hyper specialized models. Also, setting that aside, theres really tons of low hanging fruit to pick.