4 ms·
> We argue that this consensus, at least as applied to health care, both overstates the benefits and undercounts the drawbacks of requiring black-box algorithms
by 6gvONxR4sf7o 5y ago
> We argue that this consensus, at least as applied to health care, both overstates the benefits and undercounts the drawbacks of requiring black-box algorithms to be explainable.
I wouldn't describe this as a drawback of requiring explainability. I'd describe this as drawbacks of today's models and today's explainability methods. Today's explainability tools are pretty nascent and our usual deep learning models know nothing about causation, which handicaps any "why" question. But I'm still pretty strongly in favor of requiring an interrogable answer to "why did you diagnose that," even if that isn't something we really know how to do right yet.
The challenge seems to be that we know we want "why" answers, but we don't know how to describe/codify what constitutes an acceptable "why" answer.
- potatoman22 5y agoThe doctor comes up with the why, the AI serves as one data point for coming to that solution.
- aabaker99 5y agoI worked on making a more explainable AI in bio for while. Probably a lot of the unexplainable AI that is being discusses is about deep learning. Some of the biggest advances in deep learning can also be said to have made the models more explainable. Take computer vision, for example. We have great success with convolutional neural networks because these move the models away from unexplainable fully connected layers where the mathematicals naturally produces many neurons with small effect sizes and towards detecting larger coherent structures in the convolutional layers. I agree there is more work to be done and making good mathematical decisions when designing the layers seems like a promising way to go.