The end result of AI systems that make decisions that cannot be explained would be to instantiate these automated practices into de facto law. At some point, i

The end result of AI systems that make decisions that cannot be explained would be to instantiate these automated practices into de facto law. At some point, if a decision process is widely adopted yet cannot explain itself, law becomes arbitrary. Ends justify means - ipso de facto. Power becomes unaccountable. That's the best reason I know to demand accountability (and explicibility) from any system or authority, AI or not.