3 ms·
Wasn't the exponential increase in data and compute always part of the scaling hypothesis? That's my memory of it from reading [1] years ago. Most of the field
by Strilanc 2y ago
Wasn't the exponential increase in data and compute always part of the scaling hypothesis? That's my memory of it from reading [1] years ago. Most of the field thought scaling would hurt, openai thought you'd get logarithmic benefit from it, and openai won that bet.
1: https://gwern.net/scaling-hypothesis https://gwern.net/scaling-hypothesis