3 ms·
> It's pretty clear that scaling is the winning method (although exactly how to make best use of scale is an open question), but many researchers find it repul
by aleph_minus_one 2y ago
>
It's pretty clear that scaling is the winning method (although exactly how to make best use of scale is an open question), but many researchers find it repulsive.
"Winning" in the sense of "perhaps suitable to build practical applications that make a lot of money" - perhaps (even though I'd claim that whether this is true is still an open question).
On the other hand, "winning" in the sense of "getting a deeper understanding why the method works and having a model that can be analyzed deeply very well", then I would clearly say that scaling is not the winning method.
Scientific research is about truth-seeking, so many researchers are in particular interested in the second interpretation.
- Legend2440 2y agoLimiting yourself to methods that are easy to understand is like looking for your keys under the streetlight. Small-scale methods may be easy to analyze, but they lack the richness and complexity that makes intelligence interesting.
- aleph_minus_one 2y ago> Small-scale methods may be easy to analyze, but they lack the richness and complexity that makes intelligence interesting. I don't disagree. But what a scientist would do after having strong evidence that huge scaling might help is attempting to understand what part of the much larger complexity leads to this qualitative change.