11 ms·
Drone Uses AI and 11,500 Crashes to Learn How to Fly
- bluetwo 9y agoDoes it really predict 'weather' to move forward or not? I think you mean 'whether'.
- gallerdude 9y agoif (sunlight){ propellor.frontLeft.activate(.9) } else { return 0 }
- blorenz 9y agoDoes it really 'more' forward or not? I think you mean 'move'.
- flaviuspopan 9y ago"...success is simply a continual failure to crash" My programming methodology has been validated at last!
- marsRoverDev 9y ago"There is an art to flying, or rather a knack. The knack lies in learning how to throw yourself at the ground and miss. Clearly, it is this second part, the missing, that presents the difficulties."
- backtoyoujim 9y agoThat describes my view of the wingsuit flying to a T.
- itsdevlin 9y agoWingsuit instructor here. Can confirm.
- alanfalcon 9y agoIs there a more stressful job than instructor for an extreme sport with such a high risk of death or serious injury? (I literally cannot imagine, though I hope I at least overstate the risk in my mental model.)
- zild3d 9y agoevery interview with a wingsuit flyer i've seen has them mention a few friends or even a partner who died doing it
- itsdevlin 9y agoOddly enough (and probably speaking to our mindsets as a community), it was the death of a good buddy of mine that pushed me to finally start base jumping. Here's that story: https://vimeo.com/167054481 https://vimeo.com/167054481
- jdietrich 9y agoJumping out of a plane wearing a wingsuit is reasonably safe. If something goes wrong, you deploy your parachute. If your parachute fails, you deploy your reserve. You have plenty of time to control with a spin or stall. Jumping off a rooftop or a mountain wearing a wingsuit is practically suicidal. If something goes wrong, you die. The margin of error is simply too small. The latter form of wingsuit flying is relatively new and highly controversial, even within the wingsuit and BASE jumping communities.
- hinkley 9y agoAn expert is someone who has made all of the mistakes that can be made in a narrow field. (Not the definition of 'Expert System' I was thinking of, however)
- pbhjpbhj 9y ago.. and learnt how to avoid repeating them? Seems like that's an essential to claiming you have expertise.
- deleted 9y ago[deleted]
- sumoboy 9y agoLearn to fly scenarios vs how tesla would build an autonomous drone is my question for something viable.
- vanjoe 9y agoWhat determines autonomous flying? Couldn't the drone just hover in the middle of the room and not crash? Would that count? Don't you need some sort of increased score for moving around?
- cmurf 9y agoA paper airplane is flying. A "flight" involves takeoff, climbout, leveling off, descent, and landing. So the question is, what does autonomous mean and what is it adding?
- stupidcar 9y agoIn the video it shows that the AI is only able to do one of three things: "go left", "go forward", and "go right". It can't choose to hover in place because that isn't an intent it can express. Although I don't know why it couldn't just keep going around in tight circles? Funnily enough, if you've ever implemented a Pacman clone, this is also how the ghost AI works.
- WillKirkby 9y agoFun fact: all of the Pacman ghosts actually behave differently: http://www.gamasutra.com/view/feature/3938/the_pacman_dossier.php?print=1 http://www.gamasutra.com/view/feature/3938/the_pacman_dossie...
- kragniz 9y agoThis would be a lower level part of the control system, with some higher level part determining its target. I would assume it would be somewhere close to a PID loop being tuned to avoid flipping.
- return0 9y agoCant they use a simulation to do the training and then finetune it using the poor drone?
- jahewson 9y agoThey're training on video so their simulator would have to produce thousands of hours of realistic video. On top of simulating the aerodynamics and performance of the done itself. All to produce something which is going to be inferior to real-world data.
- amelius 9y agoHuman pilots also train using simulators first.
- yathern 9y agoWell because if a human crashes a plane, that human doesn't stick around long enough to learn from that experience and do better next time. Also the cost of a human crashing a plane is a bit more than a drone crashing itself, such that it's probably better to save on planes, and invest in simulators - whereas developing an accurate physics simulator for the purpose of training a drone might take more time/money than just letting it crash, and figure it out itself.
- Eugr 9y agoType rating, yes, but initial training is usually done in a real airplane, at least here in the U.S.
- clort 9y agoDoes not need thousands, the article says they did only 40 hours of flying for this experiment at least
- robotresearcher 9y agoThe third sentence of the fine article: > The gap between simulation and real world remains large especially for perception problems.
- stupidcar 9y agoI know people have trained AIs to do more sophisticated things than this, but something about watching from the drones perspective as it scans its environment and moves around really makes you feel like you're watching a real intelligence at work.
- notahacker 9y agoIt's certainly cool to watch and in area of research which is proving fruitful, but isn't ascribing true intelligence to it just anthropomorphising the point of view of the camera? When I watch an insect essentially using trial and error to try to find its way towards the light on the other side of an open window, intelligence isn't the first thing that springs to mind
- run4yourlives2 9y agoI think it depends on how you define "intelligence". And to be honest, I don't really think we have a good definition for the various things that encompass what it means to be "intelligent".
- hugozap 9y agoIf the insect learns from its errors would that be intelligence?
- sondr3 9y agoI find that in many cases the goalpost of what is intelligence moves further away each time we get somewhere close to what was previously considered intelligence. If you asked someone you could train a computer to fly on it's own and learn from its mistakes, say, ten years ago, they would probably be more inclined to say it's intelligent than we are.
- soared 9y agoSimilar to the rc rally car that learns to drift by driving and crashing itself [1]. I'm collecting articles/videos where you can see machines teaching themselves things like flying/driving/video games [2]. [1] http://spectrum.ieee.org/cars-that-think/transportation/self-driving/autonomous-mini-rally-car-teaches-itself-to-powerslide http://spectrum.ieee.org/cars-that-think/transportation/self... [2] https://www.reddit.com/r/WatchMachinesLearn/ https://www.reddit.com/r/WatchMachinesLearn/
- kctess5 9y agoThat's not exactly what the car in [1] does. They use stochastic optimal control methods, which are more domain specific. They perform forward simulation on a lot of trajectories and effectively pick a good one. They also use localization so control is based more on current position than sensor inputs. The machine learning component is the dynamics model identifications - determining how the car reacts to control inputs. The model is basically a complicated function with a few inputs and outputs, which tend to be smoothly varying, so ml techniques work very well. This is fairly standard in model predictive control since empirical motion models tend to outperform ones that are physics based. Edit: looking at the paper, they apparently use many physics based models of the car as a basis, but then use ml to mix the models together.
- soared 9y agoUnfortunately I don't know enough to see the difference, or even understand most of your comment :). I only read about ml for fun, I don't do anything with it! If understand it, the racecar computes potential paths it could take, while the drone looks at what caused it to fall vs. continue flying?
- kctess5 9y agoNo worries :) the drone basically generates a big dataset of "crashing" and "not crashing" video clips from the camera. It then feeds all that into a convolutional neural net, which can (after training is complete) give control decisions based on the camera which avoid obstacles. This is very "black box" in the sense that it's hard to say exactly how the system is working. The car, on the other hand, uses hand written algorithms to forward simulate various controls. Based on the forward simulations, it can pick controls which are predicted to give good results. Forward simulation relies on a model of how the car reacts to any possible control. However, this model is complicated because of the nonlinear dynamics going on (inertia, wheel slip, etc). Therefore, they use ml techniques to identify the model.
- artursapek 9y agoLike watching a baby learn to walk! We're truly in an exciting age for technology.
- logicallee 9y agoYep! And the best thing is that nothing could possibly go wrong and there is no conceivable universe in which humanity comes to regret the exact way we're building intelligent machines right now. :)
- lucidrains 9y agook, you gotta be a little more subtle with the sarcasm there, lol
- hinkley 9y agoOh come on. If that were even remotely possible, there would be books about it. Maybe even movies. That would make a good summer blockbuster. Luckily, that possibility is too remote even for Hollywood.
- deleted 9y ago[deleted]
- justinpombrio 9y agoRegrets about building small autonomous drones that later end up killing people? That was a Black Mirror episode (albeit not the best episode): https://en.wikipedia.org/wiki/Hated_in_the_Nation_(Black_Mirror) https://en.wikipedia.org/wiki/Hated_in_the_Nation_(Black_Mir...
- lenkite 9y agoIts a pity that it was programmed to split for a decision only between left and right images. It could have avoided the chairs by flying higher if there were top and bottom images. Ideally, the number of decision-point images should be the area of the FOV divided by the drones forward surface area.
- sheeshkebab 9y agoPretty close to 10000 hours to get good at something?
- tgb 9y agoThe article says 40 hours of flying time.
- gwern 9y agoPlus, trivially parallelizable across many drones, and it's not even using normal RL techniques to accelerate learning. Training a drone controller is more or less a solved problem, the interest of this is whether all the crash datapoints are useful.
- udkl 9y agoI don't really believe in the 10,000 hours number. Granted I've never read the book but it 'feels' faux. The amount of time needed to master is more a combination of deep practice, motivation to master & the individual learning rate. 'The talent code' talks about deep practice. Deep practice is the 80 in the 80/20 principle.
- peter303 9y agoSome essays have refuted this hypothesis. Some people have natural talent, i.e. prodigies, which may become proficient in far shorter time.
- Clanan 9y agoThis is how Uber is going to train it's self-driving cars, isn't it?
- mhb 9y agoIt's how they're training their business.
- cr0sh 9y agoThis is interesting. It really shows how deep learning has become almost "Lego-like". 1. Imagine the problem, add a camera or two (or more). 2. Build/use a pre-trained ImageNet model as a starting point (probably using TensorFlow/Keras). 3. Build a dataset, split it into test, train, validation sets. 4. Train the model further. 5. Test and validate the model. Lower the error rate (don't overfit though!). 6. Profit? As far as what language to use, depending on the speed of whatever you're trying to do, Python would likely work fine in the majority of cases. If you need more than that, C/C++ is around the corner. Oh - and OpenCV or some other vision library will probably be used (but just to grab the images, maybe a little pre-processing). You wouldn't have to use this exact pipeline (you could substitute other deep learning libs, other vision libs, other languages, etc) - but the basics are to start with a well-known CNN model, preferably "pre-trained", then apply your own dataset(s) to the task to get it to work better. Not much more tweaking needed, the biggest thing is to get (or be able to synthesize from what you do have) enough data to throw at it (and have a fast enough system to train it in reasonable time). We've seen this approach many, many times; it seems to work well for a ton of domains and problems. Again - very "Lego-like"...
- deleted 9y ago[deleted]
- clickok 9y agoFor the things they showed in the video, deep learning is probably overkill. The main thing is having a robot capable of surviving its initial failures, and then actually collecting the data. I can't comment specifically on airborne drones, but at my lab we've demonstrated that a robot[0] with an extremely low resolution camera and a very simple model is capable of learning to avoid running into walls with about a minute or so of training data. We use reinforcement learning with linear function approximation[1], and even though the robot sees the world through a couple hundred pixels, it's sufficient to discover that walls have a certain color and that if too much of your vision is "wall" you should probably move in a different direction. The advantage of deep learning is that your agent hopefully learns to generalize, and so isn't fooled by changes in brightness/color or room layout. If your task is simpler than that, you could just use some OpenCV filters to extract colors and textures and let the a simple linear model figure out which ones correspond to obstacles. ---- 0. An iRobot Create, basically a Roomba without the vacuum. 1. Incidentally, that means that if you don't require deep reinforcement learning, you can get simple obstacle avoidance up and running with like 5000 data points.
- Para2016 9y agoPretty cool! I was curious, in the article it mentions difficulties navigating through glass environments, could they combine visual information with sonar to avoid crashing into glass and other transparent barriers?
- otto_ortega 9y agoPretty cool. Why did they use an input that does NOT provide any information about depth/distance from objects?
- sqeaky 9y agoI can only guess because I am not on their team, but: They clearly didn't need it. Human pilots clearly don't need it. Extra custom sensors might produce more noise than they are worth. Image processor is a hot topic of research in CNNs.
- analyst74 9y agoHuman pilots do get depth information by either looking, and/or by radar and other instruments.
- skj 9y agoHumans don't get depth information past some distance that is probably pretty small compared to how far away objects are in flight. Past that point it's all contextual.
- djrogers 9y agoHuman pilots don't really use vision to see depth when flying - most things in flight are too difficult to judge distance/size, so one learns to fly without. As for radar, most planes don't have their own radar. About all you're gonna get for anything close to depth is al altimeter.
- robertfw 9y agoto be fair, we're also usually not flying around inside a room
- otto_ortega 9y agoExcept humans derive distance/depth information from their sight (depth perception), that's why we have a pair of eyes and not a single eye. The equivalent on this setting would be adding a second camera. I'm not criticizing, their experiment is pretty cool, I was just wondering why they chose to use only the camera on board.
- fredley 9y ago> There is an art to flying, or rather a knack. The knack lies in learning how to throw yourself at the ground and miss. ... Clearly, it is this second part, the missing, that presents the difficulties.
- lottin 9y agoSomebody should do a cost-benefit analysis of all this machine learning business. For instance, how much did this project cost and what did they get in return? I'm not suggesting it's not worth it, just curious to know how the numbers turn out.
- rch 9y agoAlso, IIRC early neural net research covered this approach with Hannibal & Attila decades ago. Edit: I guess there are some differences -- I think this is what I was remembering: http://people.csail.mit.edu/brooks/papers/AIM-1091.pdf http://people.csail.mit.edu/brooks/papers/AIM-1091.pdf ...but maybe there was further work that is more closely related.
- deleted 9y ago[deleted]
- gene-h 9y agoI wonder how well the learned policy generalizes to other environments. Places like an art gallery, outside, or a cave. Could the network have learned something fundamental about monocular vision? It would also be interesting to see if the learned policy corrects for perturbations. If we tilt the drone by hitting it, will the policy stabilize it again? While this is a really cool result, I suspect that this approach might not be the best way to control UAVs. Dragon flies are ready to fly, avoid obstacles, perch on stuff, hunt down prey right after warming up their wings for the first time. This implies that a good amount of the flight behavior is 'hard-coded.' Although I really can't wait until someone expands upon this approach. So instead of outputting left or right, the network could output 'stick vectors,' which translate to control stick commands. Maybe even have the network take in some sensor data and a 'move in this direction' vector. Add in a pinch of sufficiently fast video processing and we could probably learn how to do fly through an FPV course or do aggressive maneuvers to fly through someone's windows[0] [0]https://www.youtube.com/watch?v=MvRTALJp8DM https://www.youtube.com/watch?v=MvRTALJp8DM
- jon-wood 9y ago> If we tilt the drone by hitting it, will the policy stabilize it again? My understanding of the way this is being done is that the output from the machine learning model is already a simple "left", "right", "straight on", so it's not really responsible for stabilization anyway. That side of things is likely being handled by the drone's control software which takes those inputs, translates those into what angle the propellers need to be at to achieve it, and then translates that into the correct rotor speeds. If you hit the drone the gyroscope will pick up that it's at the wrong inclination, feed that information into the control software, and the control software will adjust rotor speeds to correct.
- udkl 9y agoWhy would they not use a general purpose classifier [1] instead ? Sure, the tagging of objects in the field of view in this model may be unnecessary but you leverage an existing model that should allow the drone to 'think' beyond the current limited "obstruction here". It could at-least have been used as a base model to build upon. [1] https://www.youtube.com/watch?v=_wXHR-lad-Q https://www.youtube.com/watch?v=_wXHR-lad-Q Personally, I'm also looking forward to neural networks modeled after real brains [2] .... but the tech to accurately scan the complex interconnections in larger brains seems far away. [2] http://www.smithsonianmag.com/smart-news/weve-put-worms-mind-lego-robot-body-180953399/ http://www.smithsonianmag.com/smart-news/weve-put-worms-mind...
- mifeng 9y agoWhy would an object classifier matter? If my plane is about to crash, do I really care whether it's a mountain or a building?
- zeroer 9y agoWould you care if it's a cloud or a reflection off glass?
- udkl 9y agoWell you could decide the action based on the type of the object. eg: If it's a human "Bob" : listen for command. If no relevant command, detour & continue to goal. If it's a vehicle in the middle of the road, wait for it to move and then go no with your path if there are no other moving vehicles on a collision trajectory with you. It's definitely a step ahead of what the OP are doing.... but isn't it a more practical approach ? The point I'm trying to make is that training an NN to efficiently detect objects has been a solved problem for quite some time now. We should give more attention to experiments that take it a step ahead.
- MrQuincle 9y agoYou would have really cool behaviour. + Not going through glass (although humans do not always exceed in that themselves) + Going through a fly door curtain + Going through smoke + Not going through a mosquito net. + Not going through a fountain. + Pushing through soft objects. + Pushing a half open door.
- kevin2r 9y agowhat happens if after trained we move the drone to another location, will those learned "abilities" be reused? making easier to fly at the new location.
- rep_movsd 9y agoWould it not be cheaper and faster to simulate a drone and fly it through virtual 3d environments and still learn? Or would the physics be too complex to model well for simulation?
- ansible 9y agoThere are some fairly realistic flight simulator video game for drones. It wouldn't surprise me if someone's also done a GTA-V mod for something like this too.
- alexleegk 9y agoThat's what Sadeghi and Levine [1] do in their work. They train in simulation in a lot of randomly generated scenes. Since the drone is trained with such a diverse set of scenes, the learned policy generalizes to the real world. Also, note that the physics of the simulation doesn't even need to be realistic. Unless you are doing high-speed control or aggressive maneuvers, the challenging part is the perception and not the control. In the paper from OP, the controls are even high-level discrete actions: left, forward, right. [1] https://arxiv.org/abs/1611.04201 https://arxiv.org/abs/1611.04201
- ef4 9y agoIt's not that the physics can't be modeled, but figuring out the appropriate models is probably harder than it looks. Every sensor and actuator has quirks. Does it matter how much your airframe flexes? Does the turbulence caused by some little protrusion matter enough to model? You aren't going to model every individual molecule, but how much detail is enough to be right? The elegant thing about using machine learning is that you don't need to build any models at all. And you can develop the ML technique once and then reuse it to train different hardware configurations, instead of incurring the cost of modeling every one.
- tormeh 9y agoBut you could use such a model to pre-train your NN before you continues training it in the real world.
- monksy 9y agoIn other words, they're just learning not to crash/failing to crash.
- rcarmo 9y agoThis reminded me of the Douglas Adams books, in which Arthur Dent eventually learns how to fly by "throwing himself at the ground and missing". Also, the flight had an almost organic quality to it somehow. Spooky, but cool.
- sc0tfree 9y agoOh, great. Next step: weaponization and AI target analysis.
- albanium 9y agoI don't understand why it didn't learn to fly; using a flight simulator.
- peter303 9y agoImagine if Google taught their self-driving cars that way: 10,000 crashes. I think self-driving was completely procedural back when it started 15 years ago. But now with faster and better understood neural nets, some parts like recognizing objects have been replaced by deep learning.
- bluetwo 9y agoThat's silly. It took me no more than 1,000 crashes to learn to fly my drone.
- averagewall 9y agoBut hiring you to fly their drone doesn't scale. You're not nearly as good at replicating your ability as a computer is, or at working for free.
- bluetwo 9y agoTrue, but I was kidding.
- tormeh 9y agoI think this is how babies learn to walk.
- rodionos 9y agoA coding drone uses AI and 11,500 bug reports to learn how to code. Are we there yet?
- contingencies 9y agoThis really is the beginnings of SkyNet.
- deleted 9y ago[deleted]
- deleted 9y ago[deleted]
- ncomputersorg 9y agoupvoted and copied to https://ncomputers.org/news https://ncomputers.org/news
- folli 9y agoCool! Do I understand correctly, that the splitting into the part where the drone was doing fine and the part where the drone is crashing (i.e. the annotation of the dataset) was still done manually? A similar approach using unsupervised learning would be even cooler...
- jayeshsalvi 9y agoDrone Uses AI and 11,500 casualties to learn How to kill terrorists ... coming soon /s
- mooresedge 9y agoAfter 10,000 civilian casualties, it finally got a terrorist.