3 ms·
You can see his comments, in context, here: https://youtu.be/T5cPoNwO7II?t=356 https://youtu.be/T5cPoNwO7II?t=356 Economics was not hinted or implied in any wa
by somenameforme 4y ago
You can see his comments, in context, here: https://youtu.be/T5cPoNwO7II?t=356 https://youtu.be/T5cPoNwO7II?t=356
Economics was not hinted or implied in any way. Diminishing returns on model size doesn't mean there's nothing left to squeeze out, it just means that what gains are made are going to be in model refinement, rather than going the NVidia vision of a quadrillion weight system and expecting large, or even linear, gains from that hop up in model size.