5 ms·

I skimmed the paper, but a lot of it was over my head. As someone not very versed in image generation AI, can anyone help me understand? This sentence (which an

I skimmed the paper, but a lot of it was over my head. As someone not very versed in image generation AI, can anyone help me understand? This sentence (which another commenter highlighted) appears to be the key part: > I-LoRA modulates key feature maps to extract intrinsic scene properties such as normals, depth, albedo, and shading, using the models' existing decoders without additional layers, revealing their deep understanding of scene intrinsics. What exactly does "modulates key feature maps to extract intrinsic scene properties" mean? How were these scene property images generated if no additional decoding layers were added?

visarga 3y ago
Say you have a neural network with 1B parameters, you add 5M more parameters spread around (LoRA), and continue training for a while but only the newly added parameters, not the base network. The result is a "modulated" network that would predict scene properties. The interesting thing is that it only takes a few more parameters so it must be that the original network was pretty close already.
- kenjackson 3y ago
  Great explanation! Wish I could upvote more as it took me from head scratching to “that makes total sense”.