2 ms·
Interpretability is probably one of the most misunderstood topics in AI. Interpretability is fundamentally not a math or statistics challenge, it's a Human Comp
by lalaland1125 7y ago
Interpretability is probably one of the most misunderstood topics in AI. Interpretability is fundamentally not a math or statistics challenge, it's a Human Computer Interaction (HCI) problem and needs to be studied in a similar manner, with user studies. The fundemental goal of interpretability is to help users achieve some concrete goal such as detecting bias, identifying bad models, and debugging. This is similar how we can better design keyboards to help users achieve faster typing speed. Unfortunately, most interpretability studies don't actually bother testing if their methods actually work and I am highly skeptical that things like "white box models" or salience maps or attention weights or whatever latest methods can actually achieve the aims that interpretability proponents claim they do. I would love to see a simple randomized user study that would emperically show a concrete improvement in something like bias detection or error detection.
- anthony_doan 7y ago> nterpretability is fundamentally not a math or statistics challenge No, I keep on seeing AI/ML/DS people keep on downplaying statistic. Statistic interpret things. The majority of the models out there have a one to one, predictor to response, holding all other predictor constant (linear regression, logistic regression, arima, anova, etc..). Statistic inference is a thing. Inference is interpreting. Descriptive statistic is interpreting. Parsimonious is a thing. Experimental design is a thing. Degree of freedom is a thing in statistic. If you want interpretability do statistic. One of it's tenant is to quantify and live with uncertainty not fitting a curve and lots of coefficient to just predict. Not just classification. It's a reason why biostat or econometric is a thing. Statistic. Even the blog cited statistic papers even though it barely mention statistic models in it. ~~And Rudin is a statistician and contributed a lot in statistic.~~ Wrong person (I'm thinking of Rubin for casuality and missingness) This is not a tribal fight between statistic and ML. This is pointing out that ignoring statistic is a detriment to AI/ML/DS as a field. I predict that 2020 to 2030 statistic will be coming to AI/ML much more so regardless how much people downplay statistic. ~~Seeing on Dr. Rudin is coming over.~~ I've seen other statistician too. Dr. Loh works took decision tree and added ANOVA and Chisqaure to build parsimonious decision tree.