3 ms·
Yes transfer learning is a fairly umbrella term encompassing a lot of different approaches. We tried to give an example of the one most commonly used in NNs alm
by sarthakjain 10y ago
Yes transfer learning is a fairly umbrella term encompassing a lot of different approaches. We tried to give an example of the one most commonly used in NNs almost exclusively with regards to feature extraction. Do you have some resource that lists a variety of transfer learning approaches? Happy to work with you in creating a aggregated list.
- iraphael 10y agoWell for starters, fine-tuning can be done in a variety of different ways. You can pretrain your model with a larger, different dataset, or you can train an autoencoder that learns some useful representation of that larger dataset and use the encoder as a base for fine tuning. Another approach I've seen that was really cool is Model Distillation [0], which is basically the training of a smaller NN with the inputs and outputs of a larger NN (where the output is slightly modified to increase gradients and make training faster). [0] https://arxiv.org/pdf/1503.02531v1.pdf https://arxiv.org/pdf/1503.02531v1.pdf