3 ms·
"it is not very good at explaining the process it has gone through to reach such a conclusion". To be fair, generating human-understandable explanations of pre
by nerdponx 8y ago
"it is not very good at explaining the process it has gone through to reach such a conclusion".
To be fair, generating human-understandable explanations of predictions in a complicated nonlinear model is difficult in general. You typically need to come up with some kind of simplification of the model (like LIME [0]), and it's far from perfect or generally-applicable.
[0]: https://homes.cs.washington.edu/~marcotcr/blog/lime/ https://homes.cs.washington.edu/~marcotcr/blog/lime/
- soraki_soladead 8y agoSHAP is more recent and does work with (just about) any ML model: https://github.com/slundberg/shap https://github.com/slundberg/shap There's still more work to do in interpretability but models are rarely opaque black boxes with no way to see inside.