3 ms·
What are the downsides of transfer learning? How can it fail? And do you just arbitrarily select the "cut off output layer" for the pretrained model when retra
by scottyli 10y ago
What are the downsides of transfer learning? How can it fail?
And do you just arbitrarily select the "cut off output layer" for the pretrained model when retraining with your own data on new layers?
- prats226 10y agoOne way it can fail is that your model might overfit your data if number of parameters you are training are way more than data you are using for training and if regularization techniques are not used. We don't arbitrarily cut-off layers from pretrained model. We check at what layer, output features are not specific to problem for which pretrained model was trained. At this layer, you can stitch a nanonet and build a model for your data
- nl 10y agoTransfer learning works pretty well for image classification related tasks. Some other areas are much more challenging. For example, in natural language processing tasks you will sometimes see some benefit from using pretrained embeddings, but it is very task and model specific. There's some exciting work going on in this area though.
- prats226 10y agoYeah correct. Even with text, there is some exciting work going on. We are in process of building a text based model and should put it up in this week that should work for multiple problems.