2 ms·
Given that these scenes are produced from existing rasters, I don’t know how you would do that without either data classification or time of day/GI inference. Y
by andrewmcwatters 5y ago
Given that these scenes are produced from existing rasters, I don’t know how you would do that without either data classification or time of day/GI inference. You’d have to do spacial inference to produce the 3D scenes which is more complicated, and then at that point if you’re using the generated raster output as textures, you’d have to normalize them to their albedos and then do global illumination from there.
- srg0 5y agoI do not follow recent CV research anymore, but there were various "3D from a single image" techniques even 10-15 years ago. I suppose some progress has been made. A quick search turns up this paper https://openaccess.thecvf.com/content/CVPR2021/papers/Zhang_Holistic_3D_Scene_Understanding_From_a_Single_Image_With_Implicit_CVPR_2021_paper.pdf https://openaccess.thecvf.com/content/CVPR2021/papers/Zhang_... So it's possible to infer not only depth map, but also an accurate scene graph. Estimating illumination and texture synthesis have also been done in the past. Maybe it is not necessary to go all the way into proper 3D and ray tracing, but it can be possible to use 3D representation as one of the intermediate layers. My point is these generators should embed some kind of domain model to look more realistic.