4 ms·
Which ones are you referring to? Just to make it clear, I see only 1 breakthrough [0]. Everything that happened afterwards is just application of this breakthr
by StrLght 2y ago
Which ones are you referring to?
Just to make it clear, I see only 1 breakthrough [0]. Everything that happened afterwards is just application of this breakthrough with different training sets / to different domains / etc.
[0]: https://en.wikipedia.org/wiki/Attention_Is_All_You_Need https://en.wikipedia.org/wiki/Attention_Is_All_You_Need
- cubefox 2y agoAutoregressive language models, the discovery of the Chinchilla scaling law, MoEs, supervised fine-tuning, RLHF, whatever was used to create OpenAI o1, diffusion models, AlphaGo, AlphaFold, AlphaGeometry, AlphaProof.