> Not sure about the size part, if Kimi K3 achieves SOTA performance at 2.8T parameters, western models being >2x that size would be insanely bad in regards to

> Not sure about the size part, if Kimi K3 achieves SOTA performance at 2.8T parameters, western models being >2x that size would be insanely bad in regards to efficiency. I bet they’re all within the same order of magnitude and below 10T and won’t really have a reason to go even that high for the foreseeable future. They're a lot larger e.g. Fable. It is insanely bad. Do you know how much more resources "Western" companies have? Most in China don't have random GPUs to "play with" like every "frontier lab" employee does. > As investors will start squeezing them for profitability, I suspect focusing more on efficiency will be commonplace. They're born lucky though. Efficiency is "free". The next generation hardware e.g. Nvidia claims Blackwell -> Rubin is 10x efficiency (verified by Neoclouds apparently).