2 ms·

I scanned the paper and, if I got it right, during training the robot gets "external" information about its world position and speed compared to pre-set waypoin

I scanned the paper and, if I got it right, during training the robot gets "external" information about its world position and speed compared to pre-set waypoints, something that an animal or person wouldn't have access to. What I didn't understand for sure is if this information is also available to the robot while performing the "parkour": I'm pretty sure that it perceives the obstacles only through its depth camera, but how does it determine where it is and where it should go next? Is this still done through waypoints and global position knowledge? A joystick is mentioned for control; is that being used to set waypoints? Or feed the robot a relative direction and speed?

waldarbeiter 3y ago
When the policy is deployed in the real world only the depth camera is used no waypoints etc.. Scan dots and target heading is used in the first Phase of the training to pretrain a policy in simulation. In Phase 2 a policy is trained end-to-end using the pretrained actor network: "First, exteroceptive information is only available in the form of depth images from a front-facing camera instead of scandots. Second, there is no expert to specify waypoints and target directions, these must be inferred from the visible terrain geometry." For policy training in Phase 2 DAgger which is based on Behavior Cloning is used (with the policy from Phase 1 as the expert), they also use some tricks to make sure no actions that are too different from the expert actions are executed during training. In Phase 2 the network learns to extract environment information instead of from the scan dots from the depth camera. Also in Phase 2 they use the pretrained actor network from Phase 1 but the depth embedding must be learned from scratch. This is how I understand it.
- danmaz74 3y ago
  Thank you for your comment. What I don't understand is this: when the robot is in a new environment, how does it know where it's supposed to go? My understanding is that the training teaches the robot how to get to a position, but I didn't see anything about how to choose where to go (in "old AI" parlance, what could have been defined as planning).

waldarbeiter 3y ago

When the policy is deployed in the real world only the depth camera is used no waypoints etc.. Scan dots and target heading is used in the first Phase of the training to pretrain a policy in simulation. In Phase 2 a policy is trained end-to-end using the pretrained actor network: "First, exteroceptive information is only available in the form of depth images from a front-facing camera instead of scandots. Second, there is no expert to specify waypoints and target directions, these must be inferred from the visible terrain geometry." For policy training in Phase 2 DAgger which is based on Behavior Cloning is used (with the policy from Phase 1 as the expert), they also use some tricks to make sure no actions that are too different from the expert actions are executed during training. In Phase 2 the network learns to extract environment information instead of from the scan dots from the depth camera. Also in Phase 2 they use the pretrained actor network from Phase 1 but the depth embedding must be learned from scratch. This is how I understand it.

danmaz74 3y ago
Thank you for your comment. What I don't understand is this: when the robot is in a new environment, how does it know where it's supposed to go? My understanding is that the training teaches the robot how to get to a position, but I didn't see anything about how to choose where to go (in "old AI" parlance, what could have been defined as planning).