3 ms·
And I agree with Nando’s view, but he is not saying we can just take a transformer model, scale it 10T parameters and get AGI. He is only saying that trying to
by sailingparrot 4y ago
And I agree with Nando’s view, but he is not saying we can just take a transformer model, scale it 10T parameters and get AGI. He is only saying that trying to reach AGI with a « smarter » algorithm is hopeless, what matters is scale, similar to Sutton’a bitter lesson.
But we still need to work on getting systems that scale better, that are more compute efficient etc. And no one knows how far we have to scale. So saying AGI will just be « transformer + RL » to me seems ridiculous. Many more breakthroughs are needed.