3 ms·
>Lastly, your original comment veered dangerously close to claiming that models trained to predict text cannot — by definition! — ever acquire any form of reaso
by Night_Thastus 3y ago
>Lastly, your original comment veered dangerously close to claiming that models trained to predict text cannot — by definition! — ever acquire any form of reasoning or understanding of the world.
They can't, and I stand by that. An LLM is always going to be an LLM, it's never magically going to turn into a gAI. Now, some of the lessons learned from developing all these LLMs could be used to develop a gAI, but that's a long way away and would need to be built from the ground up for that purpose by design.
- dwaltrip 3y agoCan you please argue your case? I argue that the model’s weights could contain a rich representation of the world, even if it looks different from that of humans. And that it can use this capability of world modeling to also make new connections. These “world modeling” / “reasoning” abilities seem to have some similarities with that of humans. We see echoes of our own abilities in gpt-4 and the like. But they also have their own peculiarities and limitations, which isn’t surprising in hindsight.