5 ms·

It isn't clear why it needed AI. If there is enough compute power to run a neural net, surely there is enough computer power to parse data from sonar or compute

It isn't clear why it needed AI. If there is enough compute power to run a neural net, surely there is enough computer power to parse data from sonar or compute near-term avoidance vectors from bifocal imagery parallax/depth-queuing. Not clear how the AI obviates this.

BrianHenryIE 5y ago
I think that's a question of "what is AI?" I understand this drone's obstacle avoidance as it sensing the terrain ahead, feeding it into a NN and deciding what the least-dangerous route to take is. No doubt that score can be calculated with an explicit algorithm that says "score a tree as -100 and an empty space as +10" etc. I liken it to a game developer designing an NPC's choice of path. Sure, Doom had enemies attacking you without neural networks, but maybe it's a better way to do it today. (this is all just my understanding rather than actually having worked on this)
- SavantIdiot 5y ago
  The thing is, I've worked extensively with object detection and segmentation neural nets for the past three years: they would barf all over the place in this scenario. You can't train for enough of the types of terrain obstacles encountered in such a wide-open range of outdoor spaces, and you certainly couldn't do it fast enough with any degree of accuracy, even with a 3080 GPU, let alone an embedded Xavier-class NVidia module. A dinky int8 quantized "tree/no-tree" wouldn't stand a chance. This is why I asked for more details. They seem to indicate training, which today means neural net and not expert systems of yore. But that one be one hell of an object detector!! My question still stands.
  - johnthewise 5y ago
    Well, my understanding is that they are not training object detectors or segmentation models. They wouldn't be very useful anyway, you still need to have a 3d understanding of the scene and going from 2d->3d mapping wouldn't cut it. What they are doing instead is using stereo cameras, they are estimating the depth map of the field. Their model takes two rgb images, and produces a depth map. They combine this depth information with sensors on the drone such as acceleration etc and try to predict what an expert drone agent trained on perfect information in simulation would do. They train in simulation, restrict some of the information to the student agent and have it rely on only stereo cameras and sensors like in the world to mimic the 'privileged drone agent'. Computation wise, if you can run the depth estimation network on the hardware, remaining steps(given the depth map and sensor information, predict privileged drone agent path/vector), should be trivial and most likely are shallow networks.
YeGoblynQueenne 5y ago
It's not about AI. They trained a neural net to approximate an offline calculation that required more time and processing power than what's available on a drone, then they used the trained neural net to navigate the drone by trying to predict the approximated calculation. Thus, saving on time and processing power and allow the neural net to pilot the drone at the vertiginous speeds of 40 km/h :P (but through trees and all) The article calls the algorithm that performed the offline calculation a "simulated expert", probably because it operated in a simulation. My guess is it was a classical planner. Planning works very well but it can be slow and deliberate because it usually has to solve some kind of NP-complete SAT problem and so on. By approximating it with a neural net they avoid the time-consuming, intractable calculation. The question is how broadly it can generalise.
- SavantIdiot 5y ago
  Ah. I see. It's using a neural net in the sense that it is literally a "generic function approximation" to make decisions on pathfinding based on depth maps (see 'johnthewise' response below). That's interesting: it foregoes the complex pathfinding algorithm for a trained "guess" based on how expert drone pilots handle the same stereoscopic depth map. Fascinating!

BrianHenryIE 5y ago

I think that's a question of "what is AI?" I understand this drone's obstacle avoidance as it sensing the terrain ahead, feeding it into a NN and deciding what the least-dangerous route to take is. No doubt that score can be calculated with an explicit algorithm that says "score a tree as -100 and an empty space as +10" etc. I liken it to a game developer designing an NPC's choice of path. Sure, Doom had enemies attacking you without neural networks, but maybe it's a better way to do it today. (this is all just my understanding rather than actually having worked on this)

SavantIdiot 5y ago
The thing is, I've worked extensively with object detection and segmentation neural nets for the past three years: they would barf all over the place in this scenario. You can't train for enough of the types of terrain obstacles encountered in such a wide-open range of outdoor spaces, and you certainly couldn't do it fast enough with any degree of accuracy, even with a 3080 GPU, let alone an embedded Xavier-class NVidia module. A dinky int8 quantized "tree/no-tree" wouldn't stand a chance. This is why I asked for more details. They seem to indicate training, which today means neural net and not expert systems of yore. But that one be one hell of an object detector!! My question still stands.
- johnthewise 5y ago
  Well, my understanding is that they are not training object detectors or segmentation models. They wouldn't be very useful anyway, you still need to have a 3d understanding of the scene and going from 2d->3d mapping wouldn't cut it. What they are doing instead is using stereo cameras, they are estimating the depth map of the field. Their model takes two rgb images, and produces a depth map. They combine this depth information with sensors on the drone such as acceleration etc and try to predict what an expert drone agent trained on perfect information in simulation would do. They train in simulation, restrict some of the information to the student agent and have it rely on only stereo cameras and sensors like in the world to mimic the 'privileged drone agent'. Computation wise, if you can run the depth estimation network on the hardware, remaining steps(given the depth map and sensor information, predict privileged drone agent path/vector), should be trivial and most likely are shallow networks.

YeGoblynQueenne 5y ago

It's not about AI. They trained a neural net to approximate an offline calculation that required more time and processing power than what's available on a drone, then they used the trained neural net to navigate the drone by trying to predict the approximated calculation. Thus, saving on time and processing power and allow the neural net to pilot the drone at the vertiginous speeds of 40 km/h :P (but through trees and all) The article calls the algorithm that performed the offline calculation a "simulated expert", probably because it operated in a simulation. My guess is it was a classical planner. Planning works very well but it can be slow and deliberate because it usually has to solve some kind of NP-complete SAT problem and so on. By approximating it with a neural net they avoid the time-consuming, intractable calculation. The question is how broadly it can generalise.

SavantIdiot 5y ago
Ah. I see. It's using a neural net in the sense that it is literally a "generic function approximation" to make decisions on pathfinding based on depth maps (see 'johnthewise' response below). That's interesting: it foregoes the complex pathfinding algorithm for a trained "guess" based on how expert drone pilots handle the same stereoscopic depth map. Fascinating!