4 ms·
According to this LLaMA still didn't go far enough: https://www.harmdevries.com/post/model-size-vs-compute-overhead/ https://www.harmdevries.com/post/model-size
by anentropic 4y ago
According to this LLaMA still didn't go far enough: https://www.harmdevries.com/post/model-size-vs-compute-overhead/ https://www.harmdevries.com/post/model-size-vs-compute-overh...
- whimsicalism 4y agoYep, it depends on what your goal is.
- cubefox 4y agoThis doesn't say that LLaMA didn't go far enough.
- anentropic 4y agoNot exactly, but it did say they could have gone further than they did without wasting time and energy on infinitesimally small gains though