4 ms·

It is probably transfer learning has become more of a technique used for training neural network models (such as Adam being one of the most commonly used optimi

by kyloon 10y ago

It is probably transfer learning has become more of a technique used for training neural network models (such as Adam being one of the most commonly used optimizer) whereas most of the excitement stems from new and/or complex neural network architectures rather than the technique or tools that made training those neural networks possible.

prats226 10y ago
True that most excitement in the field has been in finding new architectures and building general purpose AI. However I would like to point out that people often underestimate potential of transfer learning. We have started solving real problems companies are facing right now because of lack of enough data to be able to train an accurate model themselves.
sarthakjain 10y ago
Your right about excitement been generated from new techniques in the research community. Also if you look at Prisma, Google photos or any number of applications that have gained mass popularity. They are just implementations of existing networks and techniques. The paper on style transfer made Prisma possible, commercialization usually has been a few steps behind research.
almondai 10y ago
Transfer learning in ML refers to the general idea of taking a model trained for one domain and applying it to another. However this article seems to only focus on feature mapping, ie. breaking down images into features using hidden layers of ImageNet models. In this case, the pretrained model is only acting as a feature extractor because it is not trained to maximize the embedded distance between the classes you are trying to differentiate.
- sarthakjain 10y ago
  Your right the article doesn't do justice to all of the transfer learning techniques. I cut out large portions of the original draft for brevity. Will write a longer post referencing all possible techniques, hopefully it won't turn into a review paper.
- deleted 10y ago
  [deleted]

by kyloon 10y ago

prats226 10y ago

True that most excitement in the field has been in finding new architectures and building general purpose AI. However I would like to point out that people often underestimate potential of transfer learning. We have started solving real problems companies are facing right now because of lack of enough data to be able to train an accurate model themselves.

sarthakjain 10y ago

Your right about excitement been generated from new techniques in the research community. Also if you look at Prisma, Google photos or any number of applications that have gained mass popularity. They are just implementations of existing networks and techniques. The paper on style transfer made Prisma possible, commercialization usually has been a few steps behind research.

almondai 10y ago

Transfer learning in ML refers to the general idea of taking a model trained for one domain and applying it to another. However this article seems to only focus on feature mapping, ie. breaking down images into features using hidden layers of ImageNet models. In this case, the pretrained model is only acting as a feature extractor because it is not trained to maximize the embedded distance between the classes you are trying to differentiate.

sarthakjain 10y ago
Your right the article doesn't do justice to all of the transfer learning techniques. I cut out large portions of the original draft for brevity. Will write a longer post referencing all possible techniques, hopefully it won't turn into a review paper.
deleted 10y ago
[deleted]