3 ms·
> But mostly I think the effort towards explainability is fundamentally misguided... explainability is a mysterious unmeasurable goal. Do you think people shou
by paulsutter 10y ago
> But mostly I think the effort towards explainability is fundamentally misguided... explainability is a mysterious unmeasurable goal.
Do you think people should stop working on it?
Most interesting things seem difficult to measure, until someone finds a way, and then it seems obvious. An example is search engine quality. At first this might seem too subjective to be measurable. But Google started measuring search quality using a panel of humans, and now everybody does that.
The whole idea of these challenges is to broaden the search, to hope for key insights that by their nature seem elusive at first.
- Houshalter 10y agoWhether or not we can ever come up with a good definition of "explainability", I still think it's misguided. In almost all applications of ML, accuracy matters far, far more. Predicting whether a patient has cancer, for instance, it matters a great deal that you get the highest accuracy possible. Every point in accuracy you lose to "explainability" means people die.
- romaniv 10y agoUntil your model starts misdiagnosing people in real life because your training data wasn't 100% perfect (which it never will be for complex real-life problems). https://vimeo.com/125940125 https://vimeo.com/125940125 The problem with unexplainable AI isn't just the lack of explanations. It's the fact that the entire model is a black box, so you cannot somehow tweak a single part of it without retraining.