> But OpenAI has to keep releasing new ever-increasing models to justify it all. There seems to be some renewed interest for smaller, possibly better-designed

> But OpenAI has to keep releasing new ever-increasing models to justify it all. There seems to be some renewed interest for smaller, possibly better-designed LLMs. I don’t know if this really lowers training costs, but it makes inference cheaper. I suspect at some point we’ll have clusters of smaller models, possibly activated when needed like in MoE LLMs, rather than ever-increasing humongous models with 3T parameters.