Right. I think the kitchen scenario is more constrained than self-driving (for one, you don't have to continuously keep moving to be safe), and the people who I

Right. I think the kitchen scenario is more constrained than self-driving (for one, you don't have to continuously keep moving to be safe), and the people who I think are at the forefront of self-driving are also doing robots, and "perform tasks in an unfamiliar kitchen" is a ML classic. My main assumption for "don't injure humans" is that even small animals routinely manage to avoid doing actions that would be potentially harmful, so it doesn't seem to require full general intelligence. And of course, if we crack an AI that can work on AI better than humans can, that solves - or at any rate obviates - all other AI problems by proxy.