3 ms·
Do you only train the new dense network or the whole thing?
by syntaxing 4y ago
Do you only train the new dense network or the whole thing?
- informal007 4y agoGenerally, only train the dense network.
- jstx1 4y agoI meant the former (easy, fast, works well very often) but you could do either. If you decide to tune the whole thing, the training cost, time and complexity grow by a lot, and of course you're destroying a lot of information that exists in the model.
- chudi 4y agodepends, but you fine tune the whole thing, check ulmfit paper and fastai libs
- quibono 4y agoStart with the former, see how well it works. Fine tuning the whole thing could potentially take far more work (but could also be worth it - the answer is probably that this depends on the practical use case you've got in mind).