3 ms·
>why would anyone ever fully fine-tune a model? You're asking it as if it were a rhetorical question, but I think it carries more weight than many people seem
by GaggiX 4y ago
>why would anyone ever fully fine-tune a model?
You're asking it as if it were a rhetorical question, but I think it carries more weight than many people seem to believe.
- arugulum 4y agoTo balance my view a little, it is definitely a valid question to ask "how far can we get with parameter-efficient tuning", and I firmly believe that as models get larger, the answer is "very, very far". That said, I also dislike it when it is carelessly claimed that parameter-efficient tuning is as good as full fine-tuning, without qualifications or nuance.
- deepsquirrelnet 4y agoI agree that it does carry weight. It is not apparent to me that fine tuning should be better, especially since the LoRA method seems like it could be robust against catastrophic forgetting.
- GaggiX 4y agoI 100% agree with you, but I (we) need more evidence.