3 ms·

I think that's a question of "what is AI?" I understand this drone's obstacle avoidance as it sensing the terrain ahead, feeding it into a NN and deciding what

I think that's a question of "what is AI?" I understand this drone's obstacle avoidance as it sensing the terrain ahead, feeding it into a NN and deciding what the least-dangerous route to take is. No doubt that score can be calculated with an explicit algorithm that says "score a tree as -100 and an empty space as +10" etc. I liken it to a game developer designing an NPC's choice of path. Sure, Doom had enemies attacking you without neural networks, but maybe it's a better way to do it today. (this is all just my understanding rather than actually having worked on this)

SavantIdiot 5y ago
The thing is, I've worked extensively with object detection and segmentation neural nets for the past three years: they would barf all over the place in this scenario. You can't train for enough of the types of terrain obstacles encountered in such a wide-open range of outdoor spaces, and you certainly couldn't do it fast enough with any degree of accuracy, even with a 3080 GPU, let alone an embedded Xavier-class NVidia module. A dinky int8 quantized "tree/no-tree" wouldn't stand a chance. This is why I asked for more details. They seem to indicate training, which today means neural net and not expert systems of yore. But that one be one hell of an object detector!! My question still stands.
- johnthewise 5y ago
  Well, my understanding is that they are not training object detectors or segmentation models. They wouldn't be very useful anyway, you still need to have a 3d understanding of the scene and going from 2d->3d mapping wouldn't cut it. What they are doing instead is using stereo cameras, they are estimating the depth map of the field. Their model takes two rgb images, and produces a depth map. They combine this depth information with sensors on the drone such as acceleration etc and try to predict what an expert drone agent trained on perfect information in simulation would do. They train in simulation, restrict some of the information to the student agent and have it rely on only stereo cameras and sensors like in the world to mimic the 'privileged drone agent'. Computation wise, if you can run the depth estimation network on the hardware, remaining steps(given the depth map and sensor information, predict privileged drone agent path/vector), should be trivial and most likely are shallow networks.

SavantIdiot 5y ago

The thing is, I've worked extensively with object detection and segmentation neural nets for the past three years: they would barf all over the place in this scenario. You can't train for enough of the types of terrain obstacles encountered in such a wide-open range of outdoor spaces, and you certainly couldn't do it fast enough with any degree of accuracy, even with a 3080 GPU, let alone an embedded Xavier-class NVidia module. A dinky int8 quantized "tree/no-tree" wouldn't stand a chance. This is why I asked for more details. They seem to indicate training, which today means neural net and not expert systems of yore. But that one be one hell of an object detector!! My question still stands.

johnthewise 5y ago
Well, my understanding is that they are not training object detectors or segmentation models. They wouldn't be very useful anyway, you still need to have a 3d understanding of the scene and going from 2d->3d mapping wouldn't cut it. What they are doing instead is using stereo cameras, they are estimating the depth map of the field. Their model takes two rgb images, and produces a depth map. They combine this depth information with sensors on the drone such as acceleration etc and try to predict what an expert drone agent trained on perfect information in simulation would do. They train in simulation, restrict some of the information to the student agent and have it rely on only stereo cameras and sensors like in the world to mimic the 'privileged drone agent'. Computation wise, if you can run the depth estimation network on the hardware, remaining steps(given the depth map and sensor information, predict privileged drone agent path/vector), should be trivial and most likely are shallow networks.