> But what would even count as an explanation That sort of gets to the heart of it - saying "it's impossible to explain my algorithm because it is so complicat

> But what would even count as an explanation That sort of gets to the heart of it - saying "it's impossible to explain my algorithm because it is so complicated" no longer cuts it. It's the same as if I said to the FDA "This drug works in our testing for the indicated purpose but we have no idea if there are side effects" they aren't go to say "well, OK then, you can sell it". They'll say "come back when you have tested for all possible side effects and you can precisly define its behavior" Currently Facebook, Google et al, will measure a narrow set of metrics to define the success of their algorithms (primarily relating to $$$) and then declare victory based on empirical optimisation of those metrics. They have to show that not only does the algorithm work for its stated purpose, it doesn't have undesirable side effects. They could do massive testing to demonstrate a huge range of side effects are not present. That would be similar to and probably as expensive as drug development. But an easier way to do that is to transparently explain what the algorithms do so that side effects can be predicted. If that is actually impossible for a given algorithm - well, maybe that algorithm shouldn't be used on the public at all.