4 ms·
I skimmed the paper so I may have missed this but is there a benefit to using a shallow network to simulate the deep network? While this is an interesting resul
by fredophile 12y ago
I skimmed the paper so I may have missed this but is there a benefit to using a shallow network to simulate the deep network? While this is an interesting result I'd think that for now if I already have a trained deep network I'd just use it instead of training a shallow network to mimic it.
- Houshalter 12y agoWhile it doesn't have to be shallow, there is an advantage in training a smaller model to mimic a more complicated one. The smaller net uses less computations. Neural nets are often trained to be as large as possible on clusters of GPUs. Machine learning also commonly uses ensembles of dozens of different models. So there is a lot to be gained by compressing it down to a single model.