To me this is very intuitively true. I can't explain why.I always had the intuition that fine-tuning was overrated. One reason perhaps is that examples are "r

To me this is very intuitively true. I can't explain why.I always had the intuition that fine-tuning was overrated. One reason perhaps is that examples are "right there" and thus implicitly weighted much more in relation to the fine-tuned neurons.