The core problem with the "just a neural net" idea is how to tell a robot what it is supposed to do; the whole AI currently relays on some sort of external harn

The core problem with the "just a neural net" idea is how to tell a robot what it is supposed to do; the whole AI currently relays on some sort of external harness either training the model or pushing the control algorithm into a desired operation. You may implement something very thin like artificial pain or hunger, but the odds that it will push the system out of a metastable stupor are likely negligible.