3 ms·
Technological advances tend to happen in jumps. We are no doubt approaching a local optima right now, but it won't be long until another major advancement prope
by RevEng 2y ago
Technological advances tend to happen in jumps. We are no doubt approaching a local optima right now, but it won't be long until another major advancement propels things forward again. We've seen the same pattern in ML for decades.
- probablybetter 2y agoPlease name me one technological advance of major import in the fundamental transformer kernel space that has occurred in the last decade that has any import at all on today's LLMs. I will wait.
- RevEng 2y agoThe very idea of the Transformer architecture. Surely you've heard of "Attention is all you need".