3 ms·
How does it compare to partially fine-tuning the model by freezing most of the network beside the last few layers?
by ismailmaj 2y ago
How does it compare to partially fine-tuning the model by freezing most of the network beside the last few layers?
- K0balt 2y agoIdk but if I was guessing, I would guess that that process would be likely to create intruder dimensions in those layers… but hard to say how impactful that would be. Intuitively I would think it would tend to channel a lot of irrelevant outputs towards the semantic space of the new training data, but idk I how well that intuition would hold up to reality.