6 ms·
I just think we're talking about a completely different problem with AI optimization. There's $billions of effort and research that goes into AI optimization. S
by Difwif 3y ago
I just think we're talking about a completely different problem with AI optimization. There's $billions of effort and research that goes into AI optimization. Scaling model size and training set size happens because it's what the research and evidence tells us will improve model performance reliably. If we could reduce any of it for the same performance we would. Top model performance is an arms race and it's happening at every expense. The largest players are all shooting to beat GPT-4 or Claude Opus and achieve AGI (whatever that means).
This is very different than a program that requires zero research breakthroughs to dramatically improve and is simply slow and bloated because people have different priorities.
- bayindirh 3y ago> If we could reduce any of it for the same performance we would. Nope, because all of them are harder than just going bigger. > Top model performance is an arms race and it's happening at every expense. This is also what I said. "Growth (in model performance) is king, so quick and dirty (going bigger) beating harder optimization efforts". > This is very different than a program that... [Snipped for brevity] Again this is what I said by "it keeps development momentum". Yes people have different priorities. Mostly money and fame in this point. So, we don't disagree a bit.