3 ms·

With respect, I think there's a fundamental error in this reply. You're making a dogmatic statement about what interpretability is taken to mean, which assumes

With respect, I think there's a fundamental error in this reply. You're making a dogmatic statement about what interpretability is taken to mean, which assumes of course that the vast body of work, both in academia, journalism, and business meetings, holds a consistent view. The point of this paper, however, in part, is to point out that this view is mistaken.

p4wnc6 10y ago
You are incorrect in your claim. The paper does make a good point that under the umbrella term "interpretability" there are varied and conflicting understandings of what, specifically, is meant. That fact has nearly zero relevance to what I'm saying. When I say that people look for mappings from the model to a space of cognitive structures over which they have authority, it could be (and observed in practice always is) a smattering of all the different things talked about in the paper -- authority over which diagnostics represent 'trust,' authority over which visualizations are adequately informative, authority over explaining 'how' an algorithm works, and so on. Though all of these are discordant takes on the specifics of interpretability, as discussed in the paper, they still are unified in the sense that the true intention in appealing to any of them is to map things into a domain where someone wields the authority. It's not about actually caring about trusted diagnostics, or caring about quality visualizations, or caring about communicating the intuition behind a model. Those are just the plausible deniability excuses proffered to try to win the battle over which faction's authority gets center stage. I don't require that anyone have a cross-domain consistent view of the specific definition of interpretability in order for my claims to be a viable explanation, and so it is perfectly compatible with the paper. The paper looks at the on-the-ground specifics used in the arguments over interpretability, meanwhile politics explains the behind-the-scenes motivations of any of those specific choices. The Moral Mazes chapter Dexterity with Symbols is a good place if you want to read on the evidence about how this is carried out in management bureaucracy. It has actually been formally studied in some interesting ways.

p4wnc6 10y ago

You are incorrect in your claim. The paper does make a good point that under the umbrella term "interpretability" there are varied and conflicting understandings of what, specifically, is meant. That fact has nearly zero relevance to what I'm saying. When I say that people look for mappings from the model to a space of cognitive structures over which they have authority, it could be (and observed in practice always is) a smattering of all the different things talked about in the paper -- authority over which diagnostics represent 'trust,' authority over which visualizations are adequately informative, authority over explaining 'how' an algorithm works, and so on. Though all of these are discordant takes on the specifics of interpretability, as discussed in the paper, they still are unified in the sense that the true intention in appealing to any of them is to map things into a domain where someone wields the authority. It's not about actually caring about trusted diagnostics, or caring about quality visualizations, or caring about communicating the intuition behind a model. Those are just the plausible deniability excuses proffered to try to win the battle over which faction's authority gets center stage. I don't require that anyone have a cross-domain consistent view of the specific definition of interpretability in order for my claims to be a viable explanation, and so it is perfectly compatible with the paper. The paper looks at the on-the-ground specifics used in the arguments over interpretability, meanwhile politics explains the behind-the-scenes motivations of any of those specific choices. The Moral Mazes chapter Dexterity with Symbols is a good place if you want to read on the evidence about how this is carried out in management bureaucracy. It has actually been formally studied in some interesting ways.