That is called guided upsampling and we are in fact doing just that to get a better estimate of the scene geometry. Specifically, we estimate the scene geometry

That is called guided upsampling and we are in fact doing just that to get a better estimate of the scene geometry. Specifically, we estimate the scene geometry at a resolution of 256x256 pixels (for a 1:1 aspect ratio, it varies for other resolutions) and then use the original image as a guide to upsample this estimate to 1024x1024 pixels. So yes, it could be used to a greater extent but it needs more work to be able to do this.