8 ms·
How to Do Real Time Object Detection on Drones
- sarthakjain 8y agoWe at Nanonets have been building a lot of Machine Learning models for drone based image analysis. A frequent request from our customers is how do they use our models to do real time analysis. So we decided to create a how to on building real time object detection using a DJI drone. Also useful if you're just looking to get the video stream to your laptop/computer.
- jayant123 8y agoAwesome, was looking for something like this. Will definitely try it out. What rate (fps) were you able to process the stream at?
- sarthakjain 8y agoDepending on size of model between 30 and 5 fps
- bicubic 8y agoIf you really want this at low latency, you can also take the hdmi output of the dji controller and plug it into a capture card.
- tmkc13 8y agoDoes this work on parrot drones also?
- sarthakjain 8y agoApps like `nanostream` allow you to forward the drone's live stream to any RTMP server, so this should work for Parrot drones too
- sigi45 8y agoSkynet ist thanking you ;)
- funkaster 8y agoTitle seems a little (maybe a lot?) misleading. It's not "on drones" it's actually on a video stream coming from a drone. There's a huge difference: you're doing all of the processing on a desktop/laptop PC. There's some mention at the end of the article on how to attach a PC to the drone, but basically it would be the same thing: just run the processing on a PC. Just by the title, I was expecting a clever way to run a pre-trained model directly on a drone.
- sarthakjain 8y agoIs there a title you would recommend I change it to, to make it more clear?
- deckar01 8y agoReal Time Object Detection on Drone Video Stream
- sarthakjain 8y agoChanged to "How To Do Real Time Object Detection On Drone Video Streams" in the blog. Can't change it on HN.
- funkaster 8y agoyes, that would be a more accurate title.
- MrTonyD 8y agoWow. Combine this with the ability to drop some of the latest nano-sensors (atoms strung together to build dust-sized sensors and batteries and power at scale), and we should be able to monitor absolutely everything everywhere. Let's see anyone even try to object to my rules or procure a weapon! And given some on-board weapons, we would never have to worry about civil unrest again. Only people grateful for the food and job they are allowed to be in our society. Utopia!
- sarthakjain 8y agoHey MrTonyD. I understand where your coming from. But honestly most people are using this for monitoring things like Solar Panels, Wind Turbines, detecting seals and crop prediction. We can't have a sustainable future without a lot of these technologies in place. https://nanonets.com/drone/wind-turbines/ https://nanonets.com/drone/wind-turbines/ https://nanonets.com/drone/solar-panels/ https://nanonets.com/drone/solar-panels/ All new technology will have some deviant use case associated with it. The internet is one of the worst enablers of child pornography, human trafficking and everything disgusting with the world. However it just means you try and control the use of technology not throw the baby out with the bathwater.
- MrTonyD 8y agoWell, I'm 60 now and I've worked in Silicon Valley for over 40 years. So I guess I hear about all the other uses of such technology. I've worked with NSA, CIA, DIA, all the branches of military, and a bunch of people who don't officially exist. So, while there are definitely things I can't say, I will say that the "dust sensors" have been in use for many years, and I'm not describing anything theoretical. We need to think about these things in order to address them. Simply saying that there also exist good uses isn't enough.
- sarthakjain 8y agoGiven your expertise over the many years how would you recommend we control the use of such technology? Maybe my question is based on the assumption that you agree: stopping development of the technology in the first place is not the solution. Hopefully this assumption is correct.
- msadowski 8y agoVery interesting! I really liked the result. Would you mind if I linked to this tutorial in WeeklyRobotics[1]? [1] https://weeklyrobotics.com/ https://weeklyrobotics.com/
- sarthakjain 8y agosure
- bronco21016 8y agoHas anyone attempted to do anything like this for SAR (Search and Resuce)? I”m not sure where one would gather a good dataset for training a model.
- Sunspots 8y agoProbably, I know of at least one project https://liu.se/en/news-item/har-testas-autonom-sjoraddning-i-praktiken https://liu.se/en/news-item/har-testas-autonom-sjoraddning-i...
- manneshiva 8y agoHere are a few more who have tried it or use deep learning for SAR: https://www.planckaero.com/maritimedrone/ https://www.planckaero.com/maritimedrone/ https://flytbase.com/flytstories/search-and-rescue-drone/ https://flytbase.com/flytstories/search-and-rescue-drone/ https://news.developer.nvidia.com/autonomous-search-and-rescue-drones-outperform-humans-at-navigating-forest-trails/ https://news.developer.nvidia.com/autonomous-search-and-resc... Data for training such models for practical use are difficult to get. Often, one has to train a model on a different but related dataset that contains your object of interest and ensure that your model generalizes well. These models are highly sensitive to the data that they are trained on, so it is best if you can gather data which is closer to the actual environment (in which you want to use the model).
- athiercelin 8y agoUsing the MobileSDK (https://developer.dji.com/mobile-sdk/ https://developer.dji.com/mobile-sdk/) you can build this straight on a mobile app. With the OnboardSDK (https://developer.dji.com/onboard-sdk/ https://developer.dji.com/onboard-sdk/) you can run it on the drone directly.
- deleted 8y ago[deleted]
- jvdo 8y agoI tried this on my i7 macbook pro, but my CPU usage went to 700% and basically i never got any moving image. Is this normal? What kind of computer you need to at least run this kind of real time?