4 ms·
Do you have any material you can recommend as an introduction to interpretation of NN systems, or how to design NN-based ML to be good for interpretation? My b
by snovv_crash 6y ago
Do you have any material you can recommend as an introduction to interpretation of NN systems, or how to design NN-based ML to be good for interpretation?
My background is more classic ML and computer vision (features design, geometry, photogrammetry) and I've been very hesitant to use NN in a number of situations precisely because of things like auditability and being able to fix bugs in critical-for-customer edge cases.
- pkage 6y agoR Guidotti et. al[0] wrote a good literature survey on black-box explainers, and contains a summary table on page 20 of the current state of the art. In terms of designing NN-based ML, the above paper has some info and this paper by S Teso[1] is a good place to start looking further (though it is focused on XAL). SENNs are cool, but ultimately most inherenly interpretable models come down to classic ML (decision trees, linear/logreg, etc.) which is limiting compared to the power of NNs. Post-hoc explanations are basically the only option (esp. for DNNs). [0] http://arxiv.org/abs/1802.01933 http://arxiv.org/abs/1802.01933 [1] https://www.semanticscholar.org/paper/Toward-Faithful-Explanatory-Active-Learning-with-Teso/a0564ba7ef9f37c286c55f0703fdbe1cc1937a90 https://www.semanticscholar.org/paper/Toward-Faithful-Explan...