I am sure humans are just reflexive probabilistic machines but we have one major advantage - we are embodied in a complex environment, both physical and social.

I am sure humans are just reflexive probabilistic machines but we have one major advantage - we are embodied in a complex environment, both physical and social. The environment allows simple reflexive inferences to align to real outcomes / grounds the model. Maybe we need to build an android baby and raise it as part of human society to get similar benefits for AI. As it stands, AI has no "skin in the game" so to speak, it doesn't optimize rewards and has nothing to lose. Predicting the next word has its limits, a model can't design and try new experiments like human agents, a static training dataset is dead while the world is alive. Even AlphaGo was limited while doing imitation learning, but surpassed humans training from scratch by accessing a Go environment (table + opponent). The environment is the real teacher.