I wonder how hard it would be to develop/train a machine learning method that ingests a photo of anywhere outside and maps it to corresponding satellite/aerial

I wonder how hard it would be to develop/train a machine learning method that ingests a photo of anywhere outside and maps it to corresponding satellite/aerial imagery. We are of course very good at object recognition from still images. We are reasonably good at inferring depth from a single still image (monocular depth estimation [0-2]). Combining these, we should be reasonably good at determining relative positioning of identified objects in 3D, and thus computing their overhead 2D positions. Imagine discretizing overhead x/y positions, essentially yielding an image where each pixel corresponds to a 1 meter square looking down, whose color is the inferred identity of what occupies it. For example, in the linked album cover, we could infer that a street occupies a set of pixels with coordinates S = {(x1, y1),...(xN, yN)}; a building occupies pixel set B1 = {(xb1, yb1),...(xbM, ybM)}; another building pixel set B2; etc. Now, we are also reasonably good at object recognition (and MDE) for overhead imagery [3]. This would let us build a giant set of overhead object identities inferred from the entirety of satellite imagery. The tricky part would be efficiently indexing this, such that the overhead coordinate sets inferred from the ground-level image could be quickly (and fuzzily) queried in the satellite data. A brute-force approach would essentially convolve the overhead object coordinates inferred from the ground-level image over every region of the satellite imagery, at every possible angular orientation (and likely several different scales, since MDE is often imperfect at inferring absolute depth), but this would be impractically slow. I would bet that intelligence agencies have had this capability for some time now. [0] https://www.sciencedirect.com/science/article/pii/S0950705123000515 https://www.sciencedirect.com/science/article/pii/S095070512... [1] https://www.nature.com/articles/s41598-022-20909-x https://www.nature.com/articles/s41598-022-20909-x [2] https://keras.io/examples/vision/depth_estimation/ https://keras.io/examples/vision/depth_estimation/ [3] https://www.mdpi.com/2072-4292/12/17/2719 https://www.mdpi.com/2072-4292/12/17/2719