The corollary of this is optimistic though. If increasingly large models don't make a big difference, this is hinting at the fact that data quality is a lot mor

The corollary of this is optimistic though. If increasingly large models don't make a big difference, this is hinting at the fact that data quality is a lot more important than the raw quantity of data. This is good news for open source models, because it would be possible to run or train viable models on less expensive hardware, and having billions of dollars in GPUs isn't as much of a moat as they are suggesting.