3 ms·
It seems to me you should be able to take a model, and insert or modify layers without having to retrain everything in the model. I believe this is called trans
by mikewarot 5y ago
It seems to me you should be able to take a model, and insert or modify layers without having to retrain everything in the model. I believe this is called transfer learning.
A new layer could be initialized to act as a 1:1 passthrough, then trained independently of all the existing layers, in situ. Once any gains were optimized, you could then train the whole model to realize any improvements.
- throwaway1492 5y agoIt’s amazing to me when anyone who is even familiar with ML can entertain ideas like the above. Is it the result of “library glue” experience with little basic knowledge? I’m genuinely curious how this comes about. It seems lack of basic understanding is rampant in the field.
- mikewarot 5y agoSo you don't think it is feasible?