2 ms·
> Humans can explain themselves. Many ML systems are not designed to provide any explanation, even poor ones. Humans are good at generating sentences that appe
by yorwba 8y ago
> Humans can explain themselves. Many ML systems are not designed to provide any explanation, even poor ones.
Humans are good at generating sentences that appear to explain their decisions, but whether those are their actual reasons is unverifiable. Unless the decision is based on some predecided verbal rule, most such "explanations" are probably pretty useless.
While I agree that most ML systems are not set up to explain themselves any better, the ability to get an explanation is something that can be bolted on after the fact. By providing the model with slightly modified inputs, it's possible to determine which kinds of input would have changed the decision if they had been different, i.e. the value of those inputs being as observed is the reason for the output. One possible implementation of that general idea is "Anchors: High-Precision Model-Agnostic Explanations" https://homes.cs.washington.edu/%7Emarcotcr/aaai18.pdf https://homes.cs.washington.edu/%7Emarcotcr/aaai18.pdf