PixelCNN++ does this, kinda. Residual blocks that use downsampled versions of the image. Though in that context, it gets used for lossless compression by entrop

PixelCNN++ does this, kinda. Residual blocks that use downsampled versions of the image. Though in that context, it gets used for lossless compression by entropy coding the likelihoods of each pixel. This idea - trying to investigate behavior at different scale parameters - is also related to renormalization. But I think you can go further than this - currently you are just preserving that which remains consistent. You should also somehow store the complement of that information - the difference between the original and what you get after downscaling then upscaling the downscaled version again. I imagine it's going to be mostly the noisy data local to certain regions of the image, some kind of high frequency details. But if you keep track of both, you can make each rescaling bijective, making your 'compression' invertible, hence lossless, in some sense. Whether you get any size savings doing that is another question altogether :^)