2 ms·
We have seen multiple massive AI breakthroughs in the last few years.
by cubefox 2y ago
We have seen multiple massive AI breakthroughs in the last few years.
- Jensson 2y agoThey are the same breakthrough applied to different domains, I don't see them as different. We will need a new breakthrough, not applying the same solution to new things.
- StrLght 2y agoWhich ones are you referring to? Just to make it clear, I see only 1 breakthrough [0]. Everything that happened afterwards is just application of this breakthrough with different training sets / to different domains / etc. [0]: https://en.wikipedia.org/wiki/Attention_Is_All_You_Need https://en.wikipedia.org/wiki/Attention_Is_All_You_Need
- cubefox 2y agoAutoregressive language models, the discovery of the Chinchilla scaling law, MoEs, supervised fine-tuning, RLHF, whatever was used to create OpenAI o1, diffusion models, AlphaGo, AlphaFold, AlphaGeometry, AlphaProof.