2 ms·
>Sam Altman) have already publicly acknowledged that the diminishing returns they're seeing in terms of model size are sufficient to effectively declare that 't
by famouswaffles 3y ago
>Sam Altman) have already publicly acknowledged that the diminishing returns they're seeing in terms of model size are sufficient to effectively declare that 'the age of giant models is already over.'
He never said anything about technical diminishing returns. He's saying we're hitting a wall economically.
The Chief Scientist at Open AI thinks there's plenty of ability left to squeeze out.
- somenameforme 3y agoYou can see his comments, in context, here: https://youtu.be/T5cPoNwO7II?t=356 https://youtu.be/T5cPoNwO7II?t=356 Economics was not hinted or implied in any way. Diminishing returns on model size doesn't mean there's nothing left to squeeze out, it just means that what gains are made are going to be in model refinement, rather than going the NVidia vision of a quadrillion weight system and expecting large, or even linear, gains from that hop up in model size.