3 ms·
Yes, it does not at all seem like AGI! You’ve explained succinctly why the GPT2 result is maybe not on the road to AGI.
by doctorpangloss 7y ago
Yes, it does not at all seem like AGI! You’ve explained succinctly why the GPT2 result is maybe not on the road to AGI.
- gwern 7y agoLanguage models trained in a predictive fashion achieve ever closer to human compression rates (humans would only get ~0.7-0.8 BPC on Hutter), and are responsible for not just the creepiest & most realistic text generation to date but also setting SOTAs across all language-related AI benchmarks like SuperGLUE, designed to test language understanding and real-world reasoning, or chatbots. If that has nothing whatsoever to do with progress towards AGI, we'll just have to agree to disagree.
- dwohnitmok 7y agoI found Scott Alexander's take on this fairly convincing. Essentially if you master language you get broadly generalizable insights across domains. https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-general-intelligence/ https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-g... If you think of intelligence as distilling the essence of the universe around you and building on top of that, at least the former starts to sound like really really really sophisticated compression.