This is plain wrong. https://arxiv.org/abs/1602.04938 https://arxiv.org/abs/1602.04938 A simple counter-example. A really neat library called "eli5" and speci

This is plain wrong. https://arxiv.org/abs/1602.04938 https://arxiv.org/abs/1602.04938 A simple counter-example. A really neat library called "eli5" and specifically, it's white-boxing functionality for NLP and CV applications does a great job of helping one figure out if their model is learning "incorrect" information in its prediction pipeline. The paper I link show why this does a good job of improving user trust in models, which is your second point on human-computer interaction https://eli5.readthedocs.io/en/latest/tutorials/black-box-text-classifiers.html https://eli5.readthedocs.io/en/latest/tutorials/black-box-te... For instance, one can run the same TextExplainer algorithm on a word embedding powered model, which encodes its features in some N dimensional space that doesn't correspond to anything that a human understands directly. TextExplainer can reverse engineer which words were most important for a classifier predicting x sample in y class, when this would otherwise be difficult to do (as is the case for basically all state of the art NLP models) All you need to do is comb through your models erroneous predictions with Textexplainer and you can figure out what "incorrect" information is being learned. I found out this way that punctuation is pretty bad for generalizable NLP, and should be preprocessed out in most circumstances. I can take the really simple vizualizations and show them to managers and suits to justify why a model was wrong somewhere. This is value gained from interpretability This absolutely is an interpretability technique which actually helps people achieve a task they care about. The only cavet of this method is that the white-box model which immitates the black-box model isn't 100% accurate, but its accuracy is a hyper-paramater. PS. If you look into who the developers of eli5 are - it's an agency which takes money from DARPA and works with sigint agencies. Eli5 and text-explainer can work on any type of black-box model. Spooks want to crack open the models. That should be your signal that there's valuable information to be gained from interpretable machine learning.