3 ms·
You can't make a full on model audit on a neural net, but there are architectures such as the Transformer (an attention scheme) that can give a lot of insight i
by visarga 8y ago
You can't make a full on model audit on a neural net, but there are architectures such as the Transformer (an attention scheme) that can give a lot of insight into what the neural net thinks. We can also visualise what inputs maximally activate a deep neuron in a CNN. Not all DL models are truly black boxes.
- AstralStorm 8y agoThe problem is the models have no reflection capabilities unlike people. The explanation is always done by a really different external system and sometimes by an actual intelligence. State models (Markovian) are sometimes able to explain things but not always really, especially in complex cases.
- visarga 8y agoOn the other hand, humans tend first to take a decision and later retrofit an explanation to it, even if it is completely wrong, and we assume the explanation is the reason and not the effect.