What are the possibilities for adding more high level tasks like "pick up the [arbitrary thing]"? I assume it's 100 times harder to deal with hands and arms in

What are the possibilities for adding more high level tasks like "pick up the [arbitrary thing]"? I assume it's 100 times harder to deal with hands and arms in a generic way. But maybe for grippers with two claws it could be more tractable to just output two force vectors per claw or something for the grasp and another two fir the drop. And maybe the SDJ could do reverse kinematics or something. But one RGB image wouldn't work. So maybe one would need a depth camera.