3 ms·
There's some speculation that there are higher horizons to the training, as explained in this video: https://www.youtube.com/watch?v=Nvb_4Jj5kBo https://www.you
by chii 2y ago
There's some speculation that there are higher horizons to the training, as explained in this video: https://www.youtube.com/watch?v=Nvb_4Jj5kBo https://www.youtube.com/watch?v=Nvb_4Jj5kBo
the term for it is "grokking", amusingly. There's some indication that we are actually undertraining by 10x
- Vecr 2y agoI've seen improvement numbers up to 12x, but after that the returns are so diminishing that there's not really a point. 12x on training costs I mean, probably still won't get AGI.