4 ms·

Nando de Freitas Research Director at @DeepMind. CIFAR. Previously Prof @UBC & @UniofOxford. made a lot of headlines: https://twitter.com/NandoDF/status/152539

Nando de Freitas Research Director at @DeepMind. CIFAR. Previously Prof @UBC & @UniofOxford. made a lot of headlines: https://twitter.com/NandoDF/status/1525397036325019649 https://twitter.com/NandoDF/status/1525397036325019649 Someone’s opinion article. My opinion: It’s all about scale now! The Game is Over! It’s about making these models bigger, safer, compute efficient, faster at sampling, smarter memory, more modalities, INNOVATIVE DATA, on/offline, … 1/N Solving these scaling challenges is what will deliver AGI. Research focused on these problems, eg S4 for greater memory, is needed. Philosophy about symbols isn’t. Symbols are tools in the world and big nets have no issue creating them and manipulating them 2/n https://twitter.com/NandoDF/status/1525397036325019649 https://twitter.com/NandoDF/status/1525397036325019649

sailingparrot 4y ago
And I agree with Nando’s view, but he is not saying we can just take a transformer model, scale it 10T parameters and get AGI. He is only saying that trying to reach AGI with a « smarter » algorithm is hopeless, what matters is scale, similar to Sutton’a bitter lesson. But we still need to work on getting systems that scale better, that are more compute efficient etc. And no one knows how far we have to scale. So saying AGI will just be « transformer + RL » to me seems ridiculous. Many more breakthroughs are needed.