5 ms·
Army photogrammetry technique makes 3D aerial maps in minutes
- aerodog 7y agois this substantially different from how we do 3d reconstruction from motion indoors?
- Fission 7y agoIt's approximately the same technique. Outdoor 3D reconstruction is actually easier, because outdoors you have highly varied and detailed scenes, which enables dense feature matching and accurate dense depth map calculations. Also, you probably have decent GPS priors* and a preplanned flight path, which makes localization of each RGB image pretty easy. * This is a big advantage that usually can't be replicated indoors. GPS is more accurate outside, and the significantly larger flight path means that GPS' main issue, low precision, isn't as much of a barrier.
- vlovich123 7y agoGPS is more accurate because the location of the satellites is known to a high degree. If you knew the location of the WiFi access points to the same degree you could get pretty similar accuracy indoors. Apple indoor mapping gets 5-10m with dedicated staff walking the space & inputting their position on a mobile map (wide open spaces is where the biggest challenge is). If more APs adopt WiFi RTT then that's actually going to give you much more GPS-like behavior indoors.
- rebuilder 7y agoI'm not sure how useful a 5-10 metre accuracy is going to be for indoor photogrammetry. 10 metres cubed is probably larger than the majority of indoor photo sets. Now, for a military drone, I think they get better accuracy than 5 metres, but also the distances involved in an outdoor dataset mean that inaccuracies in the initial position estimate are much less significant.
- Leherenn 7y agoYou can get accuracy down to a few centimetres thanks to RTK Gnss. Most professional drones offer that now, though it is usually rather expensive.
- rkangel 7y agoIt's also a question of the precision of timing. The GPS signal is entirely designed around precise timing, whereas Wifi isn't. There are a number of variables like buffer lengths that can be measured around to some extent, but are never going to be as precise.
- vlovich123 7y agoJust using the very coarse RSSI signal Apple was getting 5-20m level accuracy depending on the venue 5 years ago & Google gets ~15m accuracy indoors worldwide (different mapping techniques). You're thinking at the software level though. Look up 802.11mc though - that enables cm-level mapping.
- DanAndersen 7y agoAgreed, the GPS priors are very helpful -- the ability to specify prior known positions/orientations of photos, to some known error tolerances, is one of the power features in Agisoft Metashape Pro, which I wish was available in more photogrammetry software. Being able to know the actual nearby photos a priori and avoid the costly O(N^2) pairwise comparison is a great benefit. Some research I did a few months ago was about trying to find ways to effectively bring in that advantage of outdoor photogrammetry into indoor environments, by piggybacking onto AR HMDs with built-in SLAM: https://www.youtube.com/watch?v=ldrGpGrOaZc https://www.youtube.com/watch?v=ldrGpGrOaZc
- smithmayowa 7y agoThe us government particularly the Us military has been responsible for quite a significant amount of scientific breakthrough, either through their ability to coordinate human resources and material resources towards a particular end point or their large possession of the later. Sadly or Happily capitalism in particular venture capital investments in startups has largely surpassed them regarding the material resources part of the equation in recent times but they still suck significantly at the human coordination aspect of things.
- soylentgraham 7y agoThe trillions of dollars "investment" couldn't have hurt either :)
- 205guy 7y agoYou can do this yourself crudely from any airplane window. Before you get too high, take two pictures of the same target (building or mountain) within 1 or 2 seconds (try both, so 2 pairs of photos, because it depends on your altitude). Then open the 2 pictures in an app where they are overlaid and you can switch between them quickly (I use irfanview on Windows--still after all these years). Each eye will pick up elements of the corresponding right and left photo, giving an approximation of 3D. https://en.wikipedia.org/wiki/Wiggle_stereoscopy https://en.wikipedia.org/wiki/Wiggle_stereoscopy If you got 2 good pictures (good subject at the right altitude and separation) you could put them into a stereoscopic program and create a nice stereogram (either red-blue colored or side-by-side, depending on your preferred viewing device). What the software in the article does is compute all the parallax from multiple images to give essentially a 3D scan of the landscape. It has been theoretically possible for a long time, maybe this is one of the first to do it successfully in practice.
- Nition 7y agoMan, that default example there is terrible. The "street in Cork" is pretty good. That old gif of the T-Rex from The Lost World is always a good example[1] but I suppose it's probably not licenced the right way for Wikipedia. [1] https://i.imgur.com/DDce0hO.gif https://i.imgur.com/DDce0hO.gif
- saagarjha 7y agoThe street in Cork looks strangely toy-like. Kinda like a tilt-shifted photo, but without the tilt-shifting.
- roywiggins 7y agoProbably because you don't get that much parallax in real life, but you would if it were the size of a small model.
- smcl 7y agoReminds me of the bouncing, dancing scenery from Bowser's Kingdom in the Super Mario Oddyssey: https://www.youtube.com/watch?v=PnT8w10euCk&t=57m0s https://www.youtube.com/watch?v=PnT8w10euCk&t=57m0s (should start at 57 minutes 0 seconds)
- m463 7y agothis has been a thing for some time for drones http://opendronemap.org/ http://opendronemap.org/ http://pix4d.com/ http://pix4d.com/ etc
- joshvm 7y agoPhotogrammetry has been around for centuries. Post War, before digital imaging and fast computers, people would manually match features in images to derive the terrain models. https://ibis.geog.ubc.ca/courses/geob373/lectures/Handouts/History_of_Photogrammetry.pdf https://ibis.geog.ubc.ca/courses/geob373/lectures/Handouts/H...
- dmix 7y agoAs the article mentions what's new is a technique to render using only a single photo (instead of using multiple angles) and to do in real-time.
- oeuviz 7y agoPlease quote. My understanding is that they managed to find a way to do it without human supervision almost real time using video (=multiple images)
- bigiain 7y agoI've used this before: https://www.mapsmadeeasy.com/samples https://www.mapsmadeeasy.com/samples
- _trampeltier 7y agoHow can this get a patent today?
- deckar01 7y agoIt's easy: You submit a bullshit patent application, wait to find out what criteria it was rejected for, add more meaningless technical bullshit related to that topic, then repeat the process until the patent office feels they have extracted enough fees from you. You are now free to sue companies for using technology that sounds like your patent.
- nwallin 7y agoOne of the movies the South Park guys made used this technique to get an R rating out of some gratuitously NC 17 movie. They included some over the top 20 minute graphic sex scene and they cut and cut and cut until there was nothing left but a short gross sex scene and the "regular" movie. See also the duck from battle chess.
- debbiedowner 7y agoI was looking for the deep learning. But there is none in the patent and I don't think there is any in the process. "Method 100 automatically processes FMV data for delivering rapid two-dimensional (2-D) and three-dimensional (3-D) geospatial outputs." According to [0041] 'Method 100' is FIG 1? Looking at FIG 1, it talks a lot about clusters and metadata. From [0026] and [0039] and [0064] it seems like a cluster might be a collection of images containing a point of interest. [0089] says they do measure the altitude of the aircraft. Since they have the metadata of camera FOV they can place each image in a 3D world. I think they take a lot of redundant images and they have many views of each object of interest, which is enough to give them a (x,y,z) estimate for their points of interest. "The increased processing speed of the system is achieved by configuration of system components to perform novel geometric calculations rather than by increased processing capability." This gives me the idea that there is a ridiculous amount of data per scene. I suppose when you control your own data collection, the methods can be completely different than what is worked on in academia. Compare with single image depth estimates, or even for depth maps from video for self driving. I guess it's good that deep learning isn't being used here. Now that I think of it, it is not appropriate. I like this part lol: "Method 100 further may be automated using various programming scripts and tools such as a novel Python script generator."
- debbiedowner 7y agoI pursued this a little further out of curiosity and found the inventor's CV: [0] He had a publication on there belonging to "9th NATO Military Sensing Symposium, 2017" from which papers/presentations are made public: [1], surprising! But I didn't find Massaro's paper there. I also searched for his "Videogrammetric Mapping from Unmanned Aerial Systems" but did not find it in ISPRS or elsewhere. A commercially available software that seems to use a similar method: [2] The Techcrunch article is likely a derivative of the earlier Techlink article: [3] [0]: http://mason.gmu.edu/~rmassaro/CV_Massaro_complete_20180816.pdf http://mason.gmu.edu/~rmassaro/CV_Massaro_complete_20180816.... [1]: https://www.sto.nato.int/publications/STO%20Meeting%20Proceedings/Forms/Meeting%20Proceedings%20Document%20Set/docsethomepage.aspx?ID=43014&FolderCTID=0x0120D5200078F9E87043356C409A0D30823AFA16F602008CF184CAB7588E468F5E9FA364E05BA5&List=7e2cc123-6186-4c30-8082-1ba072228ca7&RootFolder=%2Fpublications%2FSTO%20Meeting%20Proceedings%2FSTO%2DMP%2DSET%2D241 https://www.sto.nato.int/publications/STO%20Meeting%20Procee... [2]: https://www.pix4d.com/blog/underwater-mapping-videogrammetry https://www.pix4d.com/blog/underwater-mapping-videogrammetry [3]: https://techlinkcenter.org/new-us-army-software-rapidly-converts-live-drone-video-into-2d-and-3d-maps/ https://techlinkcenter.org/new-us-army-software-rapidly-conv...
- RenRav 7y agoAny good photogrammetry software that can use handheld camera shots? I have a few hundred clay figurines my mother made that I have wanted to digitize in order to free up some space. I always thought 3d scanning required some kind of special scanner so this gives me hope.
- foobarbecue 7y agoYeah, Meshroom! https://github.com/alicevision/meshroom https://github.com/alicevision/meshroom
- greggman2 7y agoThere are plenty of 3d scanning apps for phones. I don't know which one to recommend but at a glance this one looks nice https://www.qlone.pro/ https://www.qlone.pro/
- rebuilder 7y agoMeshroom, as suggested by others. If you do this, I think you'd do well to spend a little time setting up a little "studio" for it. A rotating plate, e.g. a vinyl player to place the items on, plus even lighting, would help a lot. If possible, two cameras on tripod at different heights would be good. You can do it handheld for sure, just make sure to have plenty of light and put the items on a small pedestal so you can get low angles of them without a table getting in the way. But for several hundred objects, it's going to take a while.
- autogol 7y agohttps://www.bbc.com/news/magazine-13359064 https://www.bbc.com/news/magazine-13359064 British intelligence used overlapping photography to create a 3D map of occupied Europe during WW2. In more recent times, the maps were used to identify possible archaeological sites in Scotland.
- fjfaase 7y agoI have been looking for photogrammetry based 3D modeling software. Most existing photogrammetry solutions produce a point cloud, which usually does not result in nice geometry. Often, we as humans know that a certain object as a certain geometry. Adding that knowledge could result in much more accurate models. This type of software should be interactive, using human input to recalculate the 3D positions of certain points. It would be nice if this could be integrated with an existing 3D modeling package, such as Blender.
- mikorym 7y agoThey mention farming applications... but is there any chance of this actually becoming available for farmers?
- dharma1 7y agoYou have been able to do this for a long time. Photogrammetry apps with a good, automated drone pipeline: https://www.dronedeploy.com/ https://www.dronedeploy.com/ https://www.pix4d.com/ https://www.pix4d.com/ Generic photogrammetry: https://www.capturingreality.com/ https://www.capturingreality.com/ https://www.agisoft.com/ https://www.agisoft.com/ https://github.com/alicevision/meshroom https://github.com/alicevision/meshroom I don't really understand what's unique about this patent, Structure from Motion has been around for ages, from video too. Maybe they have some optimisation around the processing, which traditionally is quite compute intensive
- raxxorrax 7y agoStereoscopy is very difficult in computer vision if you want accuracy. There are certainly many unexplored possibilities and strategies to match two images. Something optimized for buildings, trees, vehicles and their geometric shapes is probably likely. What I ask myself is why they do not use a form of projection like lasers. If I were to build a drone, it would have a ton of lasers...
- namibj 7y agoYou want to use the sun as a light source. The power you'd need due to the numerical aperture being very small and thus most illumination not getting back to your drone reaches infeasible levels from about a few hundred meters onwards.
- jefft255 7y agoDrone laser scanning is a thing! It’s just more expensive, heavy and power consuming (so in a sense power consumption increase from two aspects).
- dharma1 7y agoPhotogrammetry isn't really about stereoscopy, it's about building a dense point cloud representation based on many images from different angles, then transforming that to a mesh (and often mapping the textures from photos onto it). You can also do point cloud with lidar from drones, just tends to be more expensive and heavier - https://enterprise.dji.com/news/detail/how-lidar-is-revolutionizing-mapping-and-geospatial-data https://enterprise.dji.com/news/detail/how-lidar-is-revoluti...