10 ms·
High-Speed AI Drone Overtakes World-Champion Drone Racers
- dolmen 3y ago> The races took place between 5 and 13 June 2022
- jtbayly 3y agoVideo?
- saidinesh5 3y agohttps://m.youtube.com/watch?v=fBiataDpGIo https://m.youtube.com/watch?v=fBiataDpGIo I'm guessing it is this one
- javier_e06 3y agoGreat video! Noise was added to the AI decision making. Reminds me the the saying: "Silence among silence is not true silence, silence among noise is true silence" Next challenge: moving gates!
- abduhl 3y ago“Now a group of researchers from the University of Zurich and Intel has set a new milestone with the first autonomous system capable of beating human champions at a physical sport: drone racing.” This is somewhat impressive but let’s be honest here: drone racing is about as much of a physical sport as riding a Peloton.
- gessha 3y agoMight be a mistranslation but I think what they meant was sports situated in the real world as opposed to the digital world. That requires a lot more feature learning and engineering than let’s say chess and go where board state can be more easily encoded.
- gaudat 3y agoRelated: (coarse language) https://m.soundcloud.com/voice-crack-central-69/edp445s-is-jacking-off-a-sport https://m.soundcloud.com/voice-crack-central-69/edp445s-is-j...
- gessha 3y agoThis is not very surprising. Drone control is a pretty challenging skill for a human to learn. Good work from the drone agent team though.
- iAMkenough 3y agoA change in lighting conditions causes it to fail. Seems like they’re a ways away from applying this outside of a controlled environment.
- grecy 3y agoI'd be willing to bet they'll get it racing in the dark much better than humans can...
- bprater 3y agoOnly if it's using non-vision sensors. Plenty of FPV drones fly at night, and it's all down to pilot expertise.
- salynchnew 3y agoThe proposed use cases in the paper like "forest monitoring," etc. seem like uses cases in highly dynamic environments. It seems like the applications will lend themselves more towards human pilots for the time being. Fun experiment and test, though! Very interesting.
- Oarch 3y ago"Real-world applications include environmental monitoring or disaster response." Reminds me of: https://xkcd.com/2128/ https://xkcd.com/2128/
- deleted 3y ago[deleted]
- CamperBob2 3y ago"If you are not currently experiencing a disaster, please remain on the line and one will be created in your immediate vicinity as soon as possible."
- i_am_jl 3y agoOh, we've already arrived at Slaughterbots. https://www.youtube.com/watch?v=O-2tpwW0kmU https://www.youtube.com/watch?v=O-2tpwW0kmU
- RealityVoid 3y agoEverytime I see the Slaughterbots video posted, I roll my eyes. There are currently many weapons capable of VAST destruction, and suddenly _this_ is where we should start drawing the line? If military drones were to become real and as capable as in the video, they would be hard to aquire, maintain, operate and keep hidden. Besides that, yes, I do tend to believe that autonomous flying drones will play a large role in future conflicts. This dude I follow on Reddit thinks this based on what he is seeing going on in Ukraine[1]. I got sold on it as a possibility. [1] https://www.reddit.com/r/CredibleDefense/comments/15uimh0/credibledefense_daily_megathread_august_18_2023/jws1zm6/ https://www.reddit.com/r/CredibleDefense/comments/15uimh0/cr...
- asah 3y agoToday's news: https://globalpulsenews.com/putins-colonel-killed-in-drone-strike-while-mowing-lawn-at-his-russian-residence/ https://globalpulsenews.com/putins-colonel-killed-in-drone-s...
- networkchad 3y ago[dead]
- Teever 3y ago> they would be hard to aquire, maintain, operate and keep hidden. How exactly? Are you expecting some sort of DRM in all commercial EDA and CAD software? Will opensource CAD and EDA software become illegal? Will customs search all packages from Chinese PCB manufacturers for illicit drone designs? Of course a nuclear weapon can kill more people than a drone swarm, but that's besides the point. What exactly does non-proliferation look like for drone swarms? What happens when the cost of a drone swarm drops to $10?
- 3y ago
- supergeek 3y agoExciting! I do a lot of FPV drone racing and it always felt ripe for an AI control to take over. Because the video system is so low quality the course is intentionally very high contrast and almost perfect for a vision system. Along with the relatively limited and constant inputs available. I will say, flying indoors on a relatively simple track like theirs is a lot easier than flying outdoors on more real world tracks. It's a bummer that they didn't fly the standardized 2023 multiGP global qualifier track. Then we could rank their AI against every pilot more objectively. You can see that track design here: https://www.multigp.com/global-qualifier/ https://www.multigp.com/global-qualifier/
- jacquesm 3y agoIt may well be because it simply can't complete that track yet. As soon as it can complete it at all it will probably be near the top performers. But if the situation changes considerably from the training conditions I would still expect it to fail, as well as with relatively minor (but unexpected) changes to the track.
- bri3d 3y agoI think that the idea of using the residual model (trained on high-resolution outside-in pose estimation) to correct the inside-out model is particularly interesting because it might actually be possible to generalize; if the kinematics residuals (basically the delta between simulation training and the real world aerodynamic + inertial behavior of the aircraft) are able to be made general enough to reflect the flight dynamics of the drone, rather than the specific actions on a specific course, this is a promising approach for general purpose flight. It's a really fascinating approach compared to most "AI-guided" drones which use models for vision and pathfinding but a traditional IMU+PID loop for kinematic control.
- jacquesm 3y agoIt reminds me of the way the Mario Brothers game changed overnight: for the longest time it was considered impossible and then suddenly it was unbeatable. But both benefit from the static course, if it generalizes it will be a game changer, otherwise, I'm not that impressed. But there may well be some gold to be found here and I applaud them for making it work this far. Maybe they could purposefully improve their performance in real world situations by making this one harder, for instance by changing the lighting from one run to another, introducing or removing obstacles or moving goals around. That might force the model to come out more general. We've seen similar strategies used with good results in image classification problems. In fact I used them myself when building the lego sorter, as long as everything was always lined up perfect it worked a lot worse then when introducing various complications. During the real world runs those would show up all by themselves anyway and where before they were classified wrong or ended up in the recycling bin for another shot they were suddenly classified right. Of course a setup where you can gather your training data with thousands of images per hour has some advantages over one where if you get it wrong you have to rebuild your drone...
- capableweb 3y agoIt seems to require training on a static environment before it can actually execute and drive the race track, is that true? If so, wonder how long time / how much resources the training would require. I first was under the impression that the entire thing was executed in real-time with onboard sensors and computer, but seems that's not entirely true if I understand it correctly.
- taeric 3y agoOddly, I would think the "limitation" of only using the onboard system for control is an advantage? In that not having the latency of transmission back to a controller has to be measurable? I probably have very bad intuitions on how long it takes to transmit video and such?
- upbeat_general 3y agoI’m not familiar with the specifics latencies for drone racing but hierarchical planning/control is very very common in robotics. It’s natural to say plan a longer term trajectory at 1Hz, a short-term trajectory at 10Hz, and perform control at the 100Hz, obviously with those rates varying based on the system. And since (generally speaking) higher levels of planning require more compute resources, it might make sense to partition the compute and take the latency penalty.
- taeric 3y agoMakes sense in general. In the specific case of these racers, it seems you wouldn't be losing out on much by keeping everything local, though? That said, I'm still mostly curious on how long it would take you to get the sensory data off the drone. My gut is that the video transmission done would blow through most of the time budgets you listed. I'd love to see a good rundown on the relevant latency values involved.
- bri3d 3y agoGood high-resolution specialized video-codec based (H.264 / H.265) digital FPV video systems like Walksnail or DJI hover at around 30ms of latency glass-to-glass. Analog systems used in racing are at around 15ms glass-to-glass, with most latency coming from the camera's image processing, but would be a bit non-standard to train or build a system around without transforming back into the framebuffer/pixel-value domain, which would introduce some slight degree of extra latency. There's a rather unique uncompressed transmission system called HDZero which roughly matches analog latency. Purpose-built low latency uplinks add 5-10ms to get data turned around and back into the flight controller, albeit at fairly low bitrates as they're mostly serial based. So, for realtime control (kinematics), which runs at the multi-kHz rate on racing drones, onboard control is pretty much a must, but for both short-term and long-term planning, offboard control becomes more practical. Most non-FPV drones are already architected this way, with gyro-to-motor PID control performed on a microcontroller running an RTOS, short-term planning information coming in asynchronously from a larger SoC running Linux and ML-type stuff, and long-term control information coming down over the air.
- MR4D 3y agoI give this a year until we see it in Ukraine. Tools like this will change warfare even more than the last 18 months have.
- koheripbal 3y agoSee what exactly? We already see drones.
- partiallypro 3y agoI suppose they mean you won't need a drone pilot, which takes training and takes a soldier off the battlefield in terms of them needing to have on a headset and can't fire a conventional weapon at the same time. I don't think it will come to the war so quickly though, it's the least of the problems at the moment for Ukraine that badly needs long range missiles, mine clearing equipment and F16s.
- m4rtink 3y agoFPV pilots need special training & jamming could interrupt the control link. A fully autonomous drone is a nightmares inducing option that just does not have these downsides. For extra inhuman points you could have it sleep in a box & launch on trigger or timer. Like the hunter seeker from Dune, just without a pilot, lightning fast & with a RPG7 round stripped to it. Oh those rose colored visions of future utopia people had in the past. ;-)
- bodangly 3y agoI am a consultant who developed the companion computer/autopilot and software environment for DRL drones used at the AIRR race sponsored by Lockheed in 2019. UZH was the team to beat and I met Davide and Elia there. They almost pulled it off but sports being what they are had a bit of a heartbreak at the end, and the team from Delft ended up pulling through and winning the $1mm prize. Delft then raced against Gab from DRL and iirc came behind by only 6 seconds. So glad to see this team from UZH continued pushing the envelope and are now beating human champions. If you saw the team and what they managed in under a year, it was clear they were highly talented and human racers had their work cut out for them to stay ahead.
- datahead 3y agoWhat is the general outline of going from a model of a craft (drone, sailboat, etc) to building a sim that can do reinforcement learning over a physical object interaction with its env? I want to start playing with models, sims and collected data for sailboat racing- I know the RL/data science stuff, and I assume a good model of your craft takes time to build, and can be improved with collected data. What are some areas to explore when chaining model -> sim -> RL for performance? I realize this is an extremely complex topic, with several PhDs worth of potential input- if you had to explain to someone technical what it looks like and where to keep digging, what would you say?
- raulns86 3y agoThat's a very interest project. As a amateur sailor I have played with some models for sailing path optimizatio, wich can be somehow related to your problem. I started with a very simple model using sailing polar graphs, these map wind speed and course to maximum speed and usually based on empirical data, I found some DBs with this data, I can check if you are interest. Nevertheless for a better simulation you will at least need a simple model that gives you the forces at least especially if you want to simulate the boat acceleration, heeling and leeway. Simulating the sail foil behaviour would be very computational intesive as the airflow around it can get very complicated especially as the sail changes it's shape with it.
- c420 3y agoRelated: https://news.ycombinator.com/item?id=37324024 https://news.ycombinator.com/item?id=37324024
- johndhi 3y agoCool story
- yieldcrv 3y agogreat milestone achieved, that's quite a leap for the field, from being half as good as the best human pilots to winning by half a second I would definitely like to see the drones outfitted with different sensors that aren't so sensitive to light visible to humans
- yreg 3y ago> This marks the first time that an autonomous mobile robot has achieved world champion level performance in a real world competitive sport. Interesting, is that true? I would have expected that to have happened in some other sports already (especially racing).
- epolanski 3y agoSlightly OT but I don't think that Deep blue's achievement against Kasparov is the milestone they make it to be, as the moves of the computer were still vetted by other grandmasters. There's no doubt that if deep blue would've not beaten Kasparov another computer would've just few years later (I also think algos have made insane improvements since then, it's not just a matter of raw speed) but still that game is not the kind of achievement that it's touted imho.
- jmpman 3y agoWhen’s the next contest when each team of paintball players each get a single AI Drone armed with a paintball gun? Let’s not kid ourselves, that’s a more accurate simulation of where this research is going.
- aaron695 3y ago[dead]
- 05 3y agoThe world champion level results were achieved when using external motion tracking, not onboard camera. So, not as impressive as the title made it look. Here's the video (from 2022) btw: https://www.youtube.com/watch?v=U_o6H7Xlv2A&t=1s https://www.youtube.com/watch?v=U_o6H7Xlv2A&t=1s
- merrington 3y agoI believe the article states differently, > Until very recently, autonomous drones took twice as long as those piloted by humans to fly through a racetrack, unless they relied on an external position-tracking system to precisely control their trajectories. Swift, however, reacts in real time to the data collected by an onboard camera, like the one used by human racers. Its integrated inertial measurement unit measures acceleration and speed while an artificial neural network uses data from the camera to localize the drone in space and detect the gates along the racetrack. This information is fed to a control unit, also based on a deep neural network that chooses the best action to finish the circuit as fast as possible. And here's a video that seems to have been uploaded today - https://www.youtube.com/watch?v=fBiataDpGIo https://www.youtube.com/watch?v=fBiataDpGIo
- 05 3y agoWell, they've muddied the waters well enough. The b-roll in the last segment of the video you linked shows motion tracked drones and the 2022 event that used motion tracking (the balls used for motion tracking are visible on the drone arms). The voiceover tells us the system won against human racers. So, even if they have a vision only run made on the same track that beat human times, that run wasn't shown (but you couldn't tell that if you just watched that video). And from BMSThomas' video, it's obvious their vision only system wasn't ready for prime time in 2022, but they were still working on it.
- BotJunkie 3y agoThis paper is based on the performance at the 2022 event, but it took this long to write it up and get it published. The tracking system was used during all of the races for data collection of both the autonomous and human-piloted drones, which is why the reflective markers are visible. They did do some demonstrations of the drones controlled with the tracking cameras, and they were significantly faster, but the vision-based drones were definitely able to fly faster than the human pilots in some races.
- exabrial 3y ago> Its integrated inertial measurement unit measures acceleration and speed If that’s fed into the neural net, I don’t this is an actual matchup. An accomplishment for sure! But the computer didn’t beat the human. I’d consider this a win when the computer has to send its inputs over the same wireless link that humans use, fed vision inputs with the same latency and quality. Edit: I want to add this is a criticism of the headline, not a criticism of the developers and project managers
- throwawaylinux 3y ago> If that’s fed into the neural net, I don’t this is an actual matchup. I think it is an actual matchup. Is there a prohibition against the human pilot from riding in the aircraft to improve latency?
- ranguna 3y agoThere's a physical prohibition, yes.
- throwawaylinux 3y agoAnd if that's all that's preventing it then it is an actual matchup.
- ranguna 3y agoWelp you could also say that these are enough grounds to add a new rule to the rule book.
- knallfrosch 3y agoI agree with your sentiment. The computer should also fly without any extra components. But I think it's pretty clear that a) this poses a bigger challenge for which the computer isn't ready yet - it barely beat the humans with its advantage. b) we could also reach equality by feeding the humans the measurements from the intertial measurement unit and allowing humans to ride the drones. Whether they can make use of it is their problem. Showing that "equal conditions" aren't really possible. c) The current matchup: "Computer with extra sensor data, humans with lag" presents how the battle will be in real-world applications, like warfare.
- rossjudson 3y agoLooking forward to getting this for my Tesla, and arriving at work in about 3 minutes.
- virgulino 3y agoNature video: https://youtu.be/pq53uCDZelQ https://youtu.be/pq53uCDZelQ UZH video: https://youtu.be/fBiataDpGIo https://youtu.be/fBiataDpGIo
- jachee 3y agoThis is finally taking quadcopter racing and making into actual drone racing!
- system2 3y agoYear 2023, they can even embed a nice video of this.
- wly_cdgr 3y agoNice, very cool, definitely not dangerous at all.
- stevage 3y ago>Real-world applications include environmental monitoring or disaster response. Why do they always feel compelled to write this nonsense? The real-world application is killing people. Nothing else requires anything like this level of performance.
- foxyv 3y agoGotta love the cheeky euphemisms of these companies. In this case, disaster response isn't responding to a disaster, it's a response that causes a disaster. Preferably for people you do not like.
- WithinReason 3y agoFor some reason every time there is a new robotics development the application is "disaster response".
- skyyler 3y agoThese are for disaster response the same way rockets are for space exploration.
- mrexroad 3y ago> why […]? Their world-view (and knowledge) of disaster response was shaped by steady diet of 90’s disaster films — Volcano, Dante’s Peak, Armageddon, Twister, etc.
- tomovo 3y agoI'm guessing faster Street view updates could be next. Park the car, release bunch of drones (permission from the town is no problem, it's good for local businesses), have all nearby streets scanned in a few minutes. Eventually have a base station in each city and do daily (hourly?) updates.
- stevage 3y ago>permission from the town is no problem That may be true in a small number of very pro-technology cities in the US, but I think it's far from the case around the world in general. Btw, I think the point has been missed here a bit. Autonomous drone technology has been available for quite a while, and is used for use cases like the ones you describe (but usually on private property). The new thing here is drone racing - very high speed, high precision flying.
- TheDudeMan 3y agoI would have guessed this happened 5+ years ago.
- anonymousDan 3y agoI love how all the applications mentioned skirt around what must surely be the most lucrative application currently - military.
- deleted 3y ago[deleted]
- RecycledEle 3y ago"Real-world applications include environmental monitoring or disaster response." And killing John Connor. Mostly killing John Connor.