I often wonder how much (percentage-wise) the model is 'encoding' certain knowledge, the equivalent being how much does a person know about web development etc.

I often wonder how much (percentage-wise) the model is 'encoding' certain knowledge, the equivalent being how much does a person know about web development etc. A lot of research are using transfer learning in this fashion, which assumes most of or at least a large chunk of trained weights are relevant to a different task they are attempting (in contrast to the model was originally trained for). So if coding these UI requires very little language skill then you are really using a very small part of the model (or am I too naive)?