How much of that is actually perception though? "Where can I put my feet safely?" "What is the orientation and 6 higher order velocities of my body?" etc. I

How much of that is actually perception though? "Where can I put my feet safely?" "What is the orientation and 6 higher order velocities of my body?" etc. I've been told that a perfectly observable / estimable system is trivially controllable. It's one of the reasons I believe perception is upstream of everything - interaction dynamics alone are nearly impossible to just wave away with models. I don't even work in perception. But I know that everything is fine until you try to go online with perception in the loop. Then you are behind the perception team's debugging nearly all the time.