> This is a good exposition of some formal definitions for 'interpretability' in the context of machine learning, but I am still not really clear on why such a

> This is a good exposition of some formal definitions for 'interpretability' in the context of machine learning, but I am still not really clear on why such a property is necessary or even desirable in the context of high dimensional statistical learning algorithms. Because the models can fail, and we want to know how to prevent them from failing further. In a pure black box model, we know about the test and validation runs, and not much else. When Google thus deploys a model that classifies accounts as toxic or not and then cancels the toxic ones, regardless of how many domains you manage or YouTube followers you have or even whether you have a YouTube TV subscription, you'd prefer knowing why the model chose to give you the axe. You might even prefer a "human in the loop" when the system makes a call but doesn't really have confidence. For certain areas like NLP, sure, it'd be tough. But for CV tasks or many other ML tasks, some form of explanation would be invaluable and much more (human) user-friendly.