3 ms·
“You don’t need a fancy model, you can just find a similar example directly from the training data and get similar results”
by peteretep 6y ago
“You don’t need a fancy model, you can just find a similar example directly from the training data and get similar results”
- lumost 6y agoThis has always anecdotally appeared to be the case when investigating the predictions of neural nets. Particularly when it comes time to answer the question “what does this model not handle”
- deleted 6y ago[deleted]
- ironSkillet 6y agoIf you look at how they actually "translate" the fancy model to the simple one, it requires fully fitting the original model (and keeping track of the evolution of gradients over the training). So it wouldn't make training more efficient, but perhaps it would be useful in inference or probing the characteristics of the original model.
- smallnamespace 6y agoDefining ‘similar’ robustly is the meat of the problem, and what we’re finding deep NNs to do well.