I mean, this misses this huge problem of having a (mental) model of the actual physical world. Sure, that LLM model have seen the word ladder used in conjunctio

I mean, this misses this huge problem of having a (mental) model of the actual physical world. Sure, that LLM model have seen the word ladder used in conjunction with climbing many times, but it sure as hell don’t have any reasonable “idea” on the weight, weight distribution, usability of a physical ladder, ground, toolkit, the whole 3D coordinate system, orientations, gravity, etc. This is the same problem why FSD likely needs AGI, is that plastic bag in the air a reason to emergency brake on a motorway? Sure, it can be special cased but we have so many learned knowledge of the world around us that we don’t even think of as “knowledge”, that current models are just toys, comparatively. (And I don’t mean it in a dismissive way at all, don’t get me wrong! Research is on good track, but there is likely a lot ahead of us)